Skip to content

Learning Path

A step-by-step learning guide for engineers who are new to the AWS Neuron SDK.
It is built on the official documentation and takes you in order, from fundamental concepts to hands-on serving.

graph LR
    A["Level 1<br/>Concepts"] --> B["Level 2<br/>Environment Setup"]
    B --> C["Level 3<br/>LLM Serving"]
    C --> D["Level 4<br/>Monitoring"]

Level 1 — Concepts

Understand what the Neuron SDK is, the Trainium hardware structure, and the overall architecture.

# Topic Link Est. Time
1 Neuron SDK Overview AWS Neuron Documentation 10 min
2 Trainium2 Hardware Architecture — NeuronCore-v3, LNC, memory hierarchy Trainium2 Architecture 15 min
3 NeuronCore-v3 Architecture — Tensor/Vector/Scalar/GPSIMD engines NeuronCore-v3 15 min
4 EC2 Instance Architecture — Trn1/Trn2 topology, NeuronLink Neuron Architecture Guides 15 min

If you have GPU experience

If you have CUDA/NVIDIA GPU experience, reading this through the lens of the differences between Trainium and GPUs will speed up your understanding.
Key differences: NeuronCore instead of CUDA Core, and a software-managed SRAM + HBM hierarchy instead of HBM alone.

Level 2 — Environment Setup & First Run

Connect to a Neuron environment using a DLAMI and train your first model.

# Topic Link Est. Time
5 Quick Start Guide — choosing the Training / Inference / Serving path Get Started 5 min
6 Training Quick Start — training your first model on Trainium Train a Model on Trainium 30 min
7 PyTorch NeuronX Developer Guide — torch-neuronx basics and how XLA works PyTorch NeuronX Programming Guide 20 min

Prerequisites

  • An AWS account (including EC2 access permissions)
  • Basic experience using PyTorch
  • An environment that can connect to EC2 instances over SSH

Level 3 — vLLM Serving

Learn the full workflow for serving LLMs on Neuron.

# Topic Link Est. Time
8 vLLM on Neuron Overview — architecture and NxD Inference integration vLLM on Neuron 10 min
9 Online Serving Quick Start — running an OpenAI-compatible API server Online Serving 15 min
10 Offline Inference — batch inference workflow Offline Serving 10 min

Level 4 — Monitoring & Operations Tools

Learn how to use operations-essential tools such as neuron-top and neuron-monitor.

# Topic Link Est. Time
11 Neuron Top — real-time NeuronCore/memory usage Neuron Top User Guide 10 min
12 Neuron Monitor — JSON metrics collection, Prometheus/Grafana integration Neuron Monitor User Guide 10 min
13 EKS Monitoring — deploying the Neuron Monitor DaemonSet Deploy Neuron Monitor DaemonSet 10 min
14 Hands-on: neuron-monitor + Prometheus + Grafana Tutorial: Monitoring with MNIST 30 min

If you're familiar with nvidia-smi

neuron-top is the counterpart to NVIDIA GPU's nvidia-smi.
It lets you check NeuronCore utilization, memory, and loaded model information in real time.

Additional References

Resource Link
AWS Neuron SDK GitHub aws-neuron/aws-neuron-sdk
Neuron Samples (example code) aws-neuron/aws-neuron-samples
AWS Trainium Getting Started (official) aws.amazon.com/trainium/getting-started
Neuron DLAMI Guide DLAMI User Guide

Version note

All links point to the Neuron SDK latest version. If you need documentation for a specific version, change latest in the URL to the version you want (for example, v2.30.0).