AWS Trainium

Practical guides for running large-scale AI workloads on AWS Trainium — the AWS-designed AI chip. Trainium is a powerful accelerator option you can choose alongside GPUs, built for strong price-performance on both training and inference.

You can run Trainium on self-managed infrastructure — Amazon EKS, Amazon ECS, AWS Batch, or Amazon EC2 — or with Amazon SageMaker.

• Inference

  ---

  Deployment guides by serving framework (vLLM, TGI). OpenAI-compatible APIs, continuous batching, and speculative decoding.

  Inference Guide

• Training

  ---

  Distributed training of large models with NxDT, PyTorch Native, and Optimum Neuron — from pre-training to LoRA fine-tuning.

• NKI Kernels

  ---

  Write and optimize custom kernels that program NeuronCores directly, using Python/NumPy-style tile programming.

  NKI Kernels

• Profiling

  ---

  Performance analysis with Neuron Explorer, covering the framework, NKI, compiler, and runtime layers.

  Profiling Guide

• Advanced Configuration

  ---

  Hardware-level optimization options such as Logical NeuronCore configuration and mixed precision.

• Learning Videos

  ---

  Video content on NeuronCore concepts and architecture.