Inference

Guides for LLM inference serving on AWS Trainium.

• vLLM on Neuron

  ---

  LLM serving with vLLM Neuron Plugin — architecture, model onboarding, performance tuning

  View Guide

• vLLM + NxDI (Classic)

  ---

  NxDI-based vLLM serving — per-model deployment guides (Llama, Qwen, etc.)

  View Guide