Configuration

ML Container Creator supports multiple configuration methods with a clear precedence order, from interactive prompts to fully automated CLI usage.

Precedence

Configuration sources are applied in strict precedence order (highest to lowest):

Priority	Source	Description	Example
1	CLI Options	Command-line flags	--deployment-config=http-flask
2	CLI Arguments	Positional arguments	ml-container-creator my-project
3	Environment Variables	Shell environment	export AWS_REGION=us-east-1
4	CLI Config File	--config specified file	--config=production.json
5	Custom Config File	config/mcp.json	Auto-discovered in current directory
6	Package.json Section	"ml-container-creator": {...}	Project-specific defaults
7	Bootstrap Config	~/.ml-container-creator/config.json	Active bootstrap profile (role ARN, region, buckets)
8	Generator Defaults	Built-in defaults	region: "us-east-1"
9	Interactive Prompts	User input (fallback)	CLI prompts

MCP servers are not a numbered level: their results are used as suggestions and choice lists in the prompts (see MCP Servers).

Higher precedence sources override lower ones.

Parameter Reference

Parameters supported by MCC, organized by category. Each can be set via CLI flag, config file key, or (where noted) environment variable. The CLI flags are generated from config/parameter-schema-v2.json — run ml-container-creator --help for the authoritative, current list.

Project

Parameter	CLI Flag	Type	Default	Description
projectName	--project-name	string	—	Name for the generated project (env: ML_PROJECT_NAME)
skipPrompts	--skip-prompts	boolean	false	Skip interactive prompts and use configuration from other sources (env: MCC_SKIP_PROMPTS)
autoPrompt	--auto-prompt	boolean	false	Fill defaults, prompt only for missing required values
config	--config	string	—	Path to JSON configuration file
projectDir	--project-dir	string	—	Output directory path (env: ML_PROJECT_DIR)
force	--force	boolean	false	Overwrite existing output directory without prompting
smart	--smart	boolean	false	Enable smart mode (live AWS API calls for MCP servers)
discover	--discover	boolean	false	Enable discovery mode for MCP servers
noValidate	--no-validate	boolean	false	Skip parameter validation
validateEnvVars	--validate-env-vars	boolean	false	Validate environment variables against schema
validateWithDocker	--validate-with-docker	boolean	false	Validate Dockerfile builds successfully
offline	--offline	boolean	false	Run in offline mode (no network calls)

Model & Server

Parameter	CLI Flag	Type	Default	Description
deploymentConfig	--deployment-config (alias: --backend)	enum	—	Deployment configuration (e.g. http-flask, transformers-vllm, transformers-llama-cpp, triton-fil) (env: ML_DEPLOYMENT_CONFIG)
modelName	--model-name	string	—	Model identifier (hf-org/model, s3://..., registry://...) (env: ML_MODEL_NAME)
framework	--framework	enum: sklearn, xgboost, tensorflow, transformers	—	ML framework (deprecated, use --deploymentConfig instead)
modelFormat	--model-format	string	—	Model serialization format (pkl, joblib, json, model, ubj, keras, h5, SavedModel) (env: ML_MODEL_FORMAT)
modelServer	--model-server	enum: flask, fastapi, vllm, sglang	—	Model server (deprecated, use --deploymentConfig instead)
modelEnv	--model-env	string	[]	Model env var, repeatable (e.g. VLLM_TENSOR_PARALLEL_SIZE=4)
serverEnv	--server-env	string	[]	Server env var, repeatable (e.g. SGLANG_MEM_FRACTION=0.9)
engineFeature	--engine-feature	string	[]	Enable an engine-specific feature declared by the selected engine's plugin, NAME=VALUE, repeatable (e.g. radix_attention=true for SGLang, rolling_batch_backend=vllm for LMI). Resolved to the engine's real env var and validated against the plugin manifest. See Serving Engine Capabilities.

Two different --backend flags

At generation time, --backend <config> is an alias for --deployment-config (e.g. --backend transformers-vllm). It is unrelated to ./do/train --backend / ./do/tune --backend, which pick the training compute (smtj, the default, or hp-eks).

Infrastructure

Parameter	CLI Flag	Type	Default	Description
deploymentTarget	--deployment-target	enum	realtime-inference	Deployment target (deprecated — choose the target at deploy time with ./do/deploy --target <t>; ./do/deploy --help lists the targets). Still accepted at generation, where it seeds DEPLOYMENT_TARGET in do/config (batch-transform also omits do/tune, do/train and do/evaluate) (env: ML_DEPLOYMENT_TARGET)
instanceType	--instance-type	string	—	SageMaker instance type (e.g. ml.g5.xlarge, ml.m5.large) (env: ML_INSTANCE_TYPE)
region	--region	string	us-east-1	AWS region (env: ML_REGION)
roleArn	--role-arn	string	—	IAM role ARN for SageMaker execution (env: ML_ROLE_ARN)

Build

Parameter	CLI Flag	Type	Default	Description
baseImage	--base-image	string	—	Base container image for Dockerfile (env: ML_BASE_IMAGE)
buildTarget	--build-target	string	codebuild	Build target (codebuild) (env: ML_BUILD_TARGET)
codebuildComputeType	--codebuild-compute-type	string	BUILD_GENERAL1_LARGE	CodeBuild compute type (SMALL, MEDIUM, LARGE) (env: ML_CODEBUILD_COMPUTE_TYPE)

Endpoint

Parameter	CLI Flag	Type	Default	Description
endpointInitialInstanceCount	--endpoint-initial-instance-count	integer	1	Number of instances for the endpoint (env: ML_ENDPOINT_INSTANCE_COUNT)
endpointDataCapturePercent	--endpoint-data-capture-percent	integer	0	Data capture percentage for monitoring, 0-100
endpointVariantName	--endpoint-variant-name	string	AllTraffic	Production variant name
endpointVolumeSize	--endpoint-volume-size	integer	—	ML storage volume size in GB

Capacity & Serving

Heterogeneous instance pools are a deploy-time choice, not a generation parameter: ./do/deploy --endpoint-strategy heterogeneous --instance-types ml.g5.xlarge,ml.g6.xlarge.

Parameter	CLI Flag	Type	Default	Description
capacityReservationArn	--capacity-reservation-arn	string	—	FTP/capacity reservation ARN for deploying on reserved capacity.

Inference Component

Parameter	CLI Flag	Type	Default	Description
icGpuCount	--ic-gpu-count	integer	—	GPUs allocated to the inference component (env: ML_IC_GPU_COUNT)
icCopyCount	--ic-copy-count	integer	1	Number of inference component copies (env: ML_IC_COPY_COUNT)
icMemorySize	--ic-memory-size	integer	—	Memory in MB for the inference component (env: ML_IC_MEMORY_SIZE)
icCpuCount	--ic-cpu-count	number	—	vCPUs allocated to the inference component (env: ML_IC_CPU_COUNT)
icModelWeight	--ic-model-weight	number	1	Traffic routing weight, 0-1

LoRA Adapters

Parameter	CLI Flag	Type	Default	Description
enableLora	--enable-lora	boolean	true	Enable LoRA adapter serving (env: ML_ENABLE_LORA). Disable with --enable-lora=false
maxLoras	--max-loras	integer	4	Maximum concurrent LoRA adapters in GPU memory (env: ML_MAX_LORAS). Default comes from the serve engine manifest (capability_map.lora_max.default)
maxLoraRank	--max-lora-rank	integer	16	Maximum LoRA rank (env: ML_MAX_LORA_RANK). One of the engine's allowed values (vLLM: 1, 8, 16, 32, 64, 128, 256, 320, 512); default and allowed set come from the serve engine manifest. On hyperpod-eks / eks, deploy raises it to fit recorded tune adapters

Authentication

Parameter	CLI Flag	Type	Default	Description
hfToken	--hf-token	string	—	Plaintext HuggingFace token (not recommended: baked into the image; do/deploy refuses it without --allow-plaintext-secret). Prefer --hf-token-arn
hfTokenArn	--hf-token-arn	string	—	HuggingFace token ARN from Secrets Manager (env: ML_HF_TOKEN_ARN)
ngcToken	--ngc-token	string	—	NVIDIA NGC token (or $NGC_API_KEY for env var reference)
ngcTokenArn	--ngc-token-arn	string	—	NVIDIA NGC token ARN from Secrets Manager (env: ML_NGC_TOKEN_ARN)

Async Inference

Parameter	CLI Flag	Type	Default	Description
asyncS3OutputPath	--async-s3-output-path	string	—	S3 output path for async results (env: ML_ASYNC_S3_OUTPUT_PATH)
asyncSnsSuccessTopic	--async-sns-success-topic	string	—	SNS topic ARN for success notifications
asyncSnsErrorTopic	--async-sns-error-topic	string	—	SNS topic ARN for error notifications
asyncMaxConcurrent	--async-max-concurrent	integer	1	Max concurrent invocations per instance

Batch Transform

Parameter	CLI Flag	Type	Default	Description
batchInputPath	--batch-input-path	string	—	S3 input path for batch data (env: ML_BATCH_INPUT_PATH)
batchOutputPath	--batch-output-path	string	—	S3 output path for batch results (env: ML_BATCH_OUTPUT_PATH)
batchInstanceCount	--batch-instance-count	integer	1	Number of batch instances
batchSplitType	--batch-split-type	enum: Line, RecordIO, None	Line	Input split type: Line, RecordIO, None
batchStrategy	--batch-strategy	enum: MultiRecord, SingleRecord	MultiRecord	Batch strategy: MultiRecord, SingleRecord
batchJoinSource	--batch-join-source	enum: Input, None	None	Join source: Input, None
batchMaxConcurrent	--batch-max-concurrent	integer	1	Max concurrent transforms per instance
batchMaxPayload	--batch-max-payload	integer	6	Max payload size in MB, 0-100

HyperPod EKS

Parameter	CLI Flag	Type	Default	Description
hyperpodCluster	--hyperpod-cluster	string	—	HyperPod EKS cluster name (env: ML_HYPERPOD_CLUSTER)
hyperpodNamespace	--hyperpod-namespace	string	default	Kubernetes namespace (env: ML_HYPERPOD_NAMESPACE)
hyperpodReplicas	--hyperpod-replicas	integer	1	Number of replicas
fsxVolumeHandle	--fsx-volume-handle	string	—	FSx for Lustre volume handle (env: ML_FSX_VOLUME_HANDLE)

Benchmarking

Parameter	CLI Flag	Type	Default	Description
includeBenchmark	--include-benchmark	boolean	true	Include do/benchmark script in the generated project (env: ML_INCLUDE_BENCHMARK). Disable with --include-benchmark=false

Runtime resolution

Benchmark parameters (concurrency, token counts, streaming, S3 paths) are resolved at runtime by do/benchmark --workload <name> from the workload-picker catalog and the bootstrap profile. The --benchmark-* generation flags (--benchmark-concurrency, --benchmark-input-tokens, ...) only seed the BENCHMARK_* defaults in do/config; a named workload overrides them, and ./do/benchmark --workload manual uses them as-is. See Benchmarking for workload profiles and the full runtime flow.

Testing

Parameter	CLI Flag	Type	Default	Description
includeSample	--include-sample	boolean	true	Include sample model code (env: ML_INCLUDE_SAMPLE)
includeTesting	--include-testing	boolean	true	Include test suite (env: ML_INCLUDE_TESTING)
testTypes	--test-types	string	—	Comma-separated test types (env: ML_TEST_TYPES)

Deployment Configs

The --deployment-config flag bundles the architecture and model server into a single value:

Config	Architecture	Backend	Use Case
http-flask	HTTP	Flask	Traditional ML with Flask server
http-fastapi	HTTP	FastAPI	Traditional ML with FastAPI server
transformers-vllm	Transformers	vLLM	LLM serving with vLLM
transformers-sglang	Transformers	SGLang	LLM serving with SGLang
transformers-tensorrt-llm	Transformers	TensorRT-LLM	LLM serving with TensorRT-LLM
transformers-lmi	Transformers	LMI	LLM serving with Large Model Inference
transformers-djl	Transformers	DJL	LLM serving with Deep Java Library
transformers-llama-cpp	Transformers	llama.cpp	GGUF models, CPU or GPU
triton-fil	Triton	FIL	Tree models (XGBoost, LightGBM) on Triton
triton-onnxruntime	Triton	ONNX Runtime	ONNX models on Triton
triton-tensorflow	Triton	TensorFlow	TensorFlow models on Triton
triton-pytorch	Triton	PyTorch	PyTorch models on Triton
triton-vllm	Triton	vLLM	LLM serving on Triton
triton-tensorrtllm	Triton	TensorRT-LLM	LLM serving on Triton with TensorRT-LLM
triton-python	Triton	Python	Custom Python models on Triton
diffusors-vllm-omni	Diffusors	vLLM Omni	Diffusion/multimodal models

For traditional ML configs (http-flask, http-fastapi), also specify --model-format to set the serialization format for your model.

Marketplace deployments are no longer supported

The marketplace deployment config has been removed. It deployed a pre-built vendor model package and never built a container, which conflicts with this tool's core purpose: bring your own container. Both --deployment-config=marketplace and the marketplace:// model-name prefix are now refused. To serve a model, build and deploy your own image with a HuggingFace model ID, an s3:// artifact, or a registry:// model package.

Serving Engine Capabilities

Each LLM serving engine is a self-describing plugin. The table below is what the shipped plugins declare today — speculative decoding support, the benchmark dimensions you can sweep, and the one capability each engine offers that the others don't (or implement differently).

Enable an engine's signature feature with the dedicated --engine-feature NAME=VALUE flag (repeatable). You name the feature (e.g. radix_attention), and MLCC resolves it to the engine's real env var and validates it against the plugin manifest — an unknown feature for the selected engine, or an invalid value, is rejected with a clear message. In an interactive session you are prompted for exactly the features the selected engine declares (and nothing else). Prefer --engine-feature; the raw --server-env <ENV_VAR>=value form still works if you'd rather set the underlying variable directly.

Engine (--deployment-config)	Speculative decoding	Sweepable dimensions	Signature feature (and how to enable)
vLLM (transformers-vllm)	✓ eagle3, eagle2, eagle, draft-model, ngram, mtp	quantization, tensor-parallel, max-model-len, kv-cache-dtype	— (prefix caching is on by default)
SGLang (transformers-sglang)	✓ eagle3, eagle2, eagle, draft-model, mtp (mtp needs engine ≥ 0.4.0)	quantization, tensor-parallel, context-length, kv-cache-dtype	RadixAttention — automatic KV-cache reuse across requests with shared prefixes (multi-turn chat, RAG, agents). Enable: --engine-feature radix_attention=true (raw: --server-env SGLANG_ENABLE_RADIX_CACHE=true)
TensorRT-LLM (transformers-tensorrt-llm)	✗ (not yet wired — the engine supports it, but MLCC's wrapper can't emit its structured config yet)	quantization, tensor-parallel, max-input-len	—
LMI / DJL (transformers-lmi)	✗ (at the LMI layer)	quantization, tensor-parallel, max-model-len	Pluggable backend — LMI is a meta-engine that delegates to vLLM, TensorRT-LLM, or LMI-Dist. Choose: --engine-feature rolling_batch_backend=vllm (or tensorrt-llm, lmi-dist, auto; raw: --server-env OPTION_ROLLING_BATCH=...)
llama.cpp (transformers-llama-cpp)	✗	max-model-len (CTX_SIZE)	CPU/GPU offload control — serve GGUF models with a tunable CPU/GPU split. Enable: --engine-feature gpu_layers=-1 (full GPU offload; raw: --server-env SM_LLAMA_CPP_N_GPU_LAYERS=-1)
vLLM-Omni (diffusors-vllm-omni)	✗ (diffusion engine)	—	Diffusion / any-to-any multimodal serving

Speculative decoding is configured at deploy time with do/draft, not a generate-time flag — see Speculative Decoding on HyperPod EKS. The "Speculative decoding" column shows the algorithms each engine's plugin manifest declares as supported.

Why vLLM and TensorRT-LLM show no signature feature

RadixAttention and the pluggable rolling-batch backend are genuinely engine-specific — they have no vLLM equivalent (vLLM has block-level prefix caching, not RadixAttention; vLLM is a single engine, not a meta-engine). Each engine declares its distinctive capability in its plugin manifest's engine_features; an engine that declares none simply doesn't have one. See the Serve-Engine Plugins reference for the full capability matrix.

Model Formats

Framework	Supported Formats	Default
sklearn	pkl, joblib	pkl
xgboost	json, model, ubj	json
tensorflow	keras, h5, SavedModel	keras
transformers	N/A (models loaded from HuggingFace Hub)	—

S3 Model URIs

When the model identifier starts with s3://, MCC loads model weights directly from S3 instead of downloading from HuggingFace. This enables faster cold-start for pre-staged models. Example: --model-name s3://my-bucket/models/gemma-4-31b/

Runtime Profile Resolution

Generated projects include do/lib/profile.sh, which loads the active bootstrap profile (~/.ml-container-creator/config.json) into _PROFILE_<key> shell variables at runtime (bash 3.2+, no associative arrays). Values like S3 bucket names, account ID, and region are resolved from the profile — no need to regenerate when switching profiles. Precedence: explicit env var > _PROFILE_<key> > default.

Lifecycle Variables in do/config

The do/tune and do/train scripts persist state to do/config after job completion (TUNE_* and TRAIN_* variables respectively). These track adapter output paths, dataset provenance, and job names — enabling do/adapter --from-tune / --from-train to resolve artifacts without manual S3 URI handling. See Fine-Tuning and Custom Training for the full variable reference.

Additionally, do/deploy and do/benchmark may persist DEPLOYED_INSTANCE_TYPE and BENCHMARK_INSTANCE_TYPE for heterogeneous instance pool endpoints where the actual instance is resolved at runtime.

Configuration Methods

Interactive Mode

The default. Run the generator and answer the prompts:

ml-container-creator

CLI Options

Use command-line flags for non-interactive generation:

ml-container-creator my-project \
  --deployment-config=http-flask \
  --model-format=pkl \
  --instance-type=ml.m5.large \
  --skip-prompts

The project name can also be passed as a positional argument (priority 2 in the precedence chain).

Environment Variables

Set infrastructure parameters via the shell environment:

export ML_INSTANCE_TYPE="ml.g5.2xlarge"
export ML_REGION="us-west-2"
export ML_ROLE_ARN="arn:aws:iam::123456789012:role/SageMakerAIRole"

ml-container-creator --deployment-config=transformers-vllm --skip-prompts

Many parameters support environment variables (listed in the parameter reference above with env: annotations). Infrastructure and model parameters are commonly set via env vars in CI pipelines.

Configuration Files

Three file-based sources are supported, in descending precedence:

CLI config file (--config flag or ML_CONTAINER_CREATOR_CONFIG env var):

ml-container-creator --config=production.json --skip-prompts

Custom config file (config/mcp.json, auto-discovered):

{
  "projectName": "my-ml-project",
  "deploymentConfig": "http-flask",
  "modelFormat": "pkl",
  "includeSample": false,
  "includeTesting": true,
  "buildTarget": "codebuild",
  "instanceType": "ml.m5.large",
  "region": "us-east-1",
  "roleArn": "arn:aws:iam::123456789012:role/SageMakerAIRole"
}

Package.json section (infrastructure and project settings only):

{
  "name": "my-project",
  "ml-container-creator": {
    "region": "us-west-2",
    "roleArn": "arn:aws:iam::123456789012:role/MyProjectRole",
    "projectName": "my-ml-service"
  }
}

CLI Commands

Beyond project generation, MCC provides these subcommands (ml-container-creator --help is the authoritative list):

Command	Description
ml-container-creator bootstrap	Set up modular AWS infrastructure (see Bootstrap)
ml-container-creator mcp	Manage MCP servers (add, list, get, remove, init)
ml-container-creator secrets	Manage secrets in AWS Secrets Manager (create, list, describe)
ml-container-creator hey	Chat with the advisor
ml-container-creator prove	Prove a configuration works end-to-end (generate → deploy → test → clean)
ml-container-creator import	Generate an operational project from a running SageMaker endpoint
ml-container-creator update	Update project configuration fields and regenerate only affected files
ml-container-creator regenerate	Re-run generation from saved parameters with the current generator version
ml-container-creator help	Show all options and examples

Any other first word is treated as the project name, so ml-container-creator generate creates a project called generate.

HuggingFace Authentication

When deploying transformer models, you may need to authenticate with HuggingFace to access private or gated models. Public models like Qwen/Qwen3-4B do not require authentication.

Authentication is required for:

• Private models in your HuggingFace account
• Gated models requiring license agreement (e.g., Llama 3)
• Avoiding rate limits on public models

Providing Your Token

Store the token in AWS Secrets Manager and give MLCC the ARN. The serving container resolves the ARN at startup with the SageMaker execution role, so the value is never baked into the image or written to the SageMaker container Environment.

1. Store the token once:

ml-container-creator secrets create --type hf-token --name production --secret-value hf_***

2. Point MLCC at the ARN, either for every project in the bootstrap profile:

mcc bootstrap add-secret hfToken arn:aws:secretsmanager:us-east-1:123456789012:secret:mlcc/hf-token/production-AbCdEf

or per project at generation time:

ml-container-creator my-project \
  --deployment-config=transformers-vllm \
  --model-name=meta-llama/Llama-3.1-8B-Instruct \
  --hf-token-arn=arn:aws:secretsmanager:us-east-1:123456789012:secret:mlcc/hf-token/production-AbCdEf \
  --skip-prompts

(config file key: "hfTokenArn"; do/config variable: HF_TOKEN_ARN). do/build resolves the ARN but does not pass the value into docker build; the container resolves it at startup. See Secrets Management for the full workflow.

Interactive prompt: when you enter a custom model ID during generation, the CLI lists your existing mlcc/hf-token/* secrets so you can pick one, or leave it empty for public models.

Plaintext tokens (not recommended)

--hf-token (config key hfToken) still exists, but the value is baked into the Dockerfile as ENV HF_TOKEN and written to do/config. Anyone who can pull the image can read it. ./do/deploy refuses a plaintext HF_TOKEN with no HF_TOKEN_ARN (exit 3) unless you pass --allow-plaintext-secret, which is the explicit opt-out.

Note

You cannot use both --hf-token and --hf-token-arn simultaneously. Choose one approach per project.

Security

• Prefer the Secrets Manager ARN path above; rotate the token by updating the secret, with no rebuild.
• Never commit tokens to version control.
• Use read-only tokens with minimal permissions.
• Rotate tokens periodically. Generate new ones at huggingface.co/settings/tokens.

Troubleshooting Authentication

Symptom	Cause	Fix
"Repository not found" or "Access denied"	Invalid token, expired token, or license not accepted	Verify token at huggingface.co; accept model license
do/deploy exits 3 on a plaintext token	HF_TOKEN set with no HF_TOKEN_ARN	Store the token in Secrets Manager and set the ARN (above), or unset HF_TOKEN if the model is public
Container builds but fails at runtime	Model requires auth but no token provided	Store the token in Secrets Manager, set HF_TOKEN_ARN (or mcc bootstrap add-secret hfToken <arn>), then ./do/deploy --force

Validation

The generator validates configuration at multiple levels:

Parameter Validation

The generator validates configuration parameters and provides error messages:

# Invalid deployment config
ml-container-creator --deployment-config=invalid --skip-prompts
# Error: invalid not implemented yet.

# Incompatible model format
ml-container-creator --deployment-config=http-flask --model-format=json --skip-prompts
# Error: Unsupported model format 'json' for http-flask (sklearn supports pkl, joblib)

# Invalid ARN
ml-container-creator --role-arn=invalid-arn --skip-prompts
# Error: Invalid AWS Role ARN format

# Missing required parameter
ml-container-creator --skip-prompts
# Error: Required parameter 'deploymentConfig' is missing

Do not mix incompatible options: traditional ML engines with LLM deployment configs, model formats with transformer configs, or sample models with transformer configs will all produce validation errors.

Schema-Driven Validation

Schema-driven validation checks generated AWS API payloads against the actual AWS service model (service-2.json) files. It catches issues that parameter validation cannot — enum values that AWS has deprecated, type mismatches in nested structures, missing required fields for specific API operations, and cross-cutting consistency problems between your Dockerfile, deploy scripts, and configuration.

Setup

Download the AWS service models into your local schema registry:

ml-container-creator bootstrap sync-schemas

This downloads service models for SageMaker AI, IAM, ECR, and S3 from the AWS SDK source and stores them at ~/.ml-container-creator/schemas/. Re-run periodically to pick up new enum values and API changes.

When Validation Runs

Schema validation runs at two points:

At generation time (non-blocking): After the generator produces deploy scripts, it validates the constructed payloads and prints any issues as warnings. Generation still completes — this is informational.

At pre-deploy time (blocking): Run ./do/validate before deploying to catch all issues, including those introduced by manual edits to do/config after generation. The service-model payload check runs only on SageMaker targets (realtime-inference, async-inference, batch-transform); do/validate also runs the right-sizing, config and (on hyperpod-eks / eks) cluster-contract checks on every target. See Pre-Deploy Validation.

# Run full schema validation
./do/validate

# JSON output for CI integration
./do/validate --format json

# Include smart-mode advisory findings (live AWS lookups)
./do/validate --smart

What It Checks

Check	Example Issue Caught
Enum values	InferenceAmiVersion set to a value AWS no longer accepts
Type mismatches	InitialInstanceCount set to a string instead of integer
Required fields	EndpointConfigName missing from CreateEndpointConfig payload
Pattern constraints	Role ARN not matching arn:aws:iam::\d{12}:role/.+
Range constraints	VolumeSizeInGB below minimum or above maximum
GPU consistency	NumberOfAcceleratorDevicesRequired doesn't match instance GPU count
Tensor parallelism	VLLM_TENSOR_PARALLEL_SIZE != IC GPU count != instance GPUs
CUDA compatibility	Base image requires CUDA 12 but instance only supports CUDA 11
Model source requirements	registry:// model package without a resolvable ARN, or an s3:// artifact path that doesn't exist

Exit Codes

Code	Meaning
0	Validation passed (no errors, may have warnings)
1	Validation failed (one or more errors found)
2	Validation could not run (schema registry missing)

Keeping Schemas Current

The schema registry becomes stale as AWS adds new enum values and instance types. If schemas are older than 30 days, validation prints a warning:

⚠️  Schema registry is 45 days old. Run `ml-container-creator bootstrap sync-schemas` to update.

Suppress this warning with --ignore-staleness if you're working offline.

Catalog Validation

Validate that catalog entries use valid AWS enum values:

npm run validate:catalogs

This checks fields like inferenceAmiVersion in model-servers.json against the SageMaker AI service model's enum set. Run this as a CI gate when updating catalog files.

Skipping Validation

Pass --no-validate to the generator to skip schema validation at generation time:

ml-container-creator my-project --deployment-config=transformers-vllm --no-validate --skip-prompts

Architecture Compatibility

Architecture compatibility validation checks whether your chosen model's model_type (from its HuggingFace config.json) is supported by the selected server version. This catches mismatches early — before you spend time building and deploying a container that won't load the model.

Syncing Architecture Data

The architecture registry (supportedModelTypes in model-servers.json) is populated by the bootstrap post-setup chain (mcp init → sync-architectures → sync-schemas). It runs after ml-container-creator bootstrap add <profile> and again after ml-container-creator bootstrap update; --skip-post-setup skips it. There is no standalone registry command in v2 (ml-container-creator registry … would generate a project named registry).

Generation-Time Warning

When you generate a project, the architecture check runs automatically. If the model type is not in the server's supported list, you'll see an advisory warning:

⚠️  Model architecture "mamba" may not be supported by vllm 0.5.5. Consider a newer server version.

This is advisory only — generation still completes. Some models work via trust_remote_code even when not in the official registry.

do/validate Architecture Findings

The do/validate script includes architecture compatibility as one of its cross-cutting checks. If the model type doesn't match the server's supported list, it reports a medium-confidence warning alongside other validation findings.

---

Deploy Mode

The DEPLOY_MODE variable in do/config controls how the project deploys its container image:

Mode	Description	Generated artifacts
custom-container (default)	Build a custom Docker image, push to ECR, deploy from ECR	Dockerfile, do/build, do/push, do/deploy
dlc-direct	Use a stock AWS DLC image directly — no container build	do/stage, do/deploy (no Dockerfile, no do/build, no do/push)

DLC-Direct Mode (--no-build)

Generate a project that uses a stock Deep Learning Container:

ml-container-creator my-project --no-build \
  --deployment-config transformers-vllm \
  --model-name Qwen/Qwen3-0.6B \
  --instance-type ml.g5.xlarge

This resolves a CUDA-driver-compatible DLC image at generation time and stores it as CONTAINER_IMAGE_URI in do/config. The image is selected based on: - Instance family → fleet driver version (from fleet-drivers.json) - Framework + model server → compatible base images - CUDA driver compatibility filtering

Limitations of DLC-direct mode: - No custom Python packages (what's in the DLC is what you get) - No custom serving code (code/ directory is not generated) - No multi-stage Docker builds - Limited to AWS-published DLC images

Switching to custom-container mode: Re-generate the project without --no-build to get the full Dockerfile + build pipeline.