AI Infra Orchestration¶
Hands-on labs for running AI workloads on Amazon EKS: the GPU Operator, Ray and KubeRay for distributed training, and NVIDIA Dynamo for inference serving and routing.
-
GPU Operator (Private EKS)
Deploy the NVIDIA GPU Operator to a private EKS cluster.
-
Ray + EFA on EKS
Distributed training and serving on EKS with Ray and EFA.
-
Megatron Bridge on KubeRay
LoRA fine-tuning of Qwen3-8B with Megatron Bridge, run as a KubeRay RayJob on a single g6e.12xlarge with EKS Auto Mode.
-
Disaggregated Inference (Dynamo)
Prefill/decode disaggregated serving on EKS with NVIDIA Dynamo and vLLM.
-
KV-Aware Routing (Dynamo)
NVIDIA Dynamo's KV-aware router vs round-robin across 2x g6e.xlarge, with the AWS vLLM DLC, Run:ai Model Streamer from a shared EFS volume, and DCGM dashboards.