Jetson AGX Orin 32GB
1792-core NVIDIA Ampere architecture GPU · 8-core Arm Cortex-A78AE, 2 MB L2 + 4 MB L3 · 32 GB 256-bit LPDDR5
Manufacturer specifications
- Processor architecture
- NVIDIA Orin SoC, Arm v8.2 64-bit
- CPU
- 8-core Arm Cortex-A78AE, 2 MB L2 + 4 MB L3
- GPU
- 1792-core NVIDIA Ampere architecture with 56 Tensor Cores
- AI performance
- 200 TOPS (INT8, sparse)
- Memory
- 32 GB 256-bit LPDDR5
- Memory bandwidth
- 204.8 GB/s
- Storage
- 64 GB eMMC 5.1 · 2 M.2 NVMe slot(s) on our carrier
- Networking
- 1× Gigabit Ethernet + 1× 10 Gigabit Ethernet
- Power
- 15 W – 40 W (configurable power modes)
- Dimensions
- 100 × 87 mm module; dev kit 110 × 110 × 71.65 mm
- Cooling
- Active thermal solution with fan
- Expansion
- PCIe Gen4 up to 2×8 + 1×4 + 2×1, USB 3.2 Gen2, 6 × 4-lane CSI, 1× 8K display
- Supported frameworks
- JetPack 6
- CUDA 12
- cuDNN
- TensorRT
- TensorRT-LLM (selected models)
- PyTorch
- ONNX Runtime
- DeepStream
- Isaac ROS
- Ollama
- llama.cpp
- MLC
- Docker
Source: NVIDIA product pages. Figures are manufacturer specifications, not our measurements.
Theoretical vs real-world
200 TOPS is a peak figure at INT8, sparse with structured sparsity. Chat-style LLM generation is limited by the 204.8 GB/s memory bus and by how much model fits in 32 GB. Rough rule: tokens/s ≈ bandwidth ÷ model bytes.
Estimates assume ~0.6 bytes/parameter at INT4 and ~70% achieved bandwidth. Real results vary by runtime.
Measured LLM inference
We only publish figures we or NVIDIA have measured. None are published yet for this module; use the estimates above for planning.