Skip to content

AI Infra Orchestration

Hands-on labs for running AI workloads on Amazon EKS: the GPU Operator, Ray and KubeRay for distributed training, and NVIDIA Dynamo for inference serving and routing.

  • GPU Operator (Private EKS)


    Deploy the NVIDIA GPU Operator to a private EKS cluster.

    GPU Operator (Private EKS)

  • Ray + EFA on EKS


    Distributed training and serving on EKS with Ray and EFA.

    Ray + EFA on EKS

  • Megatron Bridge on KubeRay


    LoRA fine-tuning of Qwen3-8B with Megatron Bridge, run as a KubeRay RayJob on a single g6e.12xlarge with EKS Auto Mode.

    Megatron Bridge on KubeRay

  • Disaggregated Inference (Dynamo)


    Prefill/decode disaggregated serving on EKS with NVIDIA Dynamo and vLLM.

    Disaggregated Inference (Dynamo)

  • KV-Aware Routing (Dynamo)


    NVIDIA Dynamo's KV-aware router vs round-robin across 2x g6e.xlarge, with the AWS vLLM DLC, Run:ai Model Streamer from a shared EFS volume, and DCGM dashboards.

    KV-Aware Routing (Dynamo)