AWS Features for Kubeflow
Get to know the benefits of using Kubeflow with AWS service intergrations
Running Kubeflow on AWS gives you the following feature benefits and configuration options:
Note: Beginning with v1.3, development for Kubeflow on AWS can be found in the AWS Labs repository. Previous versions can be found in the Kubeflow manifests repository.
Manage AWS compute environments
- Provision and manage your Amazon Elastic Kubernetes Service (EKS) clusters with eksctl and easily configure multiple compute and GPU node configurations.
- Use AWS-optimized container images, based on AWS Deep Learning Containers, with Kubeflow Notebooks.
CloudWatch Logs and Metrics
- Integrate Kubeflow on AWS with Amazon CloudWatch for persistent logging and metrics on EKS clusters and Kubeflow pods.
- Use AWS ContainerInsights to collect, aggregate, and summarize metrics and logs from your containerized applications and microservices.
Load balancing, certificates, and identity management
- Manage external traffic with AWS Application Load Balancer.
- Get started with TLS authentication using AWS Certificate Manager and AWS Cognito.
AWS database and storage solutions
- Integrate Kubeflow with Amazon Relational Database Service (RDS) for a highly scalable pipelines and metadata store.
- Deploy Kubeflow with integrations for Amazon S3 for an easy-to-use pipeline artifacts store.
- Use Kubeflow with Amazon Elastic File System (EFS) for a simple, scalabale, and serverless storage solution.
- Leverage the Amazon FSx CSI driver to manage Lustre file systems which are optimized for compute-intensive workloads, such as high-performance computing and machine learning. Amazon FSx for Lustre can scale to hundreds of GBps of throughput and millions of IOPS.
Integrate with Amazon SageMaker
- Use Kubeflow on AWS with Amazon SageMaker to create hybrid machine learning workflows.
- Train, tune, and deploy machine learning models in Amazon SageMaker without logging into the SageMaker console using SageMaker Operators for Kubernetes (ACK).
- Create a Kubeflow Pipeline built entirely using SageMaker Components for Kubeflow Pipelines, or integrate individual components into your workflow as needed.
Last modified September 19, 2022: Kubeflow 1.6 docs (#365) (5fdb7b1c)