Skip to content

ML Container Creator

ML Container Creator

Simplify your machine learning deployments on AWS SageMaker

ML Container Creator (MCC) is a Yeoman generator designed to simplify the bring-your-own-container (BYOC) deployment approach for model serving on Amazon SageMaker AI. MCC takes user requirements for ML serving containers and injects those values into templated code and configuration files. These files define the serving infrastructure for containerized model serving workloads.

Important

This tool generates starter code and reference implementations. Before deploying to production, review and customize the generated code for your specific security, performance, operational, and cost requirements.

AWS as a First-Class Deployment Platform

This tool treats Amazon Web Services as the platform of choice for deploying models for cloud-hosting. The container files can be extended and modified to run on your platform of choice, but the repository maintainers prioritize and test deployment on Amazon SageMaker AI real-time managed inference endpoints.

Docker as a First-Class Container Builder

The authors of this codebase recognize there are many container-builders to select from when building containerized workloads. At present, Docker is the container-builder of choice. Using different container-builders, while possible, may result in varying levels of performance.

AI Documentation

This documentation was built with Kiro. It is being reviewed for accuracy and completeness by human-reviewers as part of an initiative to accelerate documentation creation.

Why ML Container Creator?

Deploying machine learning models to production shouldn't be complicated. MCC eliminates the complexity of creating BYOC model deployments, letting you focus on what matters most - your models.

Every generated project includes:

  • ✅ Amazon SageMaker AI-compatible container with health checks and invocation endpoints
  • ✅ Local testing files to validate before deployment
  • ✅ Sample model and training code to illustrate the deployment
  • ✅ Integration with HuggingFace for transformer-based deployments
  • ✅ Deployment scripts for AWS CodeBuild and Amazon SageMaker AI
  • ✅ Predictive ML multi-framework support (sklearn, XGBoost and TensorFlow)
  • ✅ Transformers multi-framework support (vLLM, SGLang, DJL, LMI and TensorRT)

Quick Start

# Install Yeoman
npm install -g yo

# Install the generator
git clone https://github.com/awslabs/ml-container-creator.git
cd ml-container-creator

# Install Dependencies and Link Generator
npm install
npm link

# Generate your project
yo ml-container-creator

The Yeoman generator prompts users for details about the deployment they are building. Answer a few questions about your model, the configuration, and the serving architecture, and get a complete project directory containging model serving, mdoel testing, deployment, and endpoint testing scripts.

Check out Getting Started to learn more about installing, prerequisites, and how to deploy your first model with MCC. Review the User Guide for more detailed walkthroughs and examples of how to use MCC. Study the Developer Guide if you plan to contribute to this project.

Community & Support

Production Considerations

Before Production Deployment

The generated code provides a starting point for SageMaker deployments. Before using in production:

  • Security Review - Review IAM roles, network configurations, and data handling
  • Testing - Thoroughly test with your actual models and data
  • Monitoring - Set up CloudWatch alarms and logging
  • Performance - Load test and optimize for your workload
  • Cost Management - Configure auto-scaling and instance types appropriately
  • Compliance - Ensure the setup meets your organization's requirements

See the AWS Well-Architected Framework for production best practices.

License

This project is licensed under the Apache-2.0 License.

Security

See CONTRIBUTING for information on reporting security issues.

Made with ❤️ by the ML community, for the ML community