NVIDIA Jetson · AGX Orin
Jetson AGX Orin 64GB
2048-core NVIDIA Ampere architecture GPU · 12-core Arm Cortex-A78AE, 3 MB L2 + 6 MB L3 · 64 GB 256-bit LPDDR5
Manufacturer specifications
- Processor architecture
- NVIDIA Orin SoC, Arm v8.2 64-bit
- CPU
- 12-core Arm Cortex-A78AE, 3 MB L2 + 6 MB L3
- GPU
- 2048-core NVIDIA Ampere architecture with 64 Tensor Cores
- AI performance
- 275 TOPS (INT8, sparse)
- Memory
- 64 GB 256-bit LPDDR5
- Memory bandwidth
- 204.8 GB/s
- Storage
- 64 GB eMMC 5.1 · 2 M.2 NVMe slot(s) on our carrier
- Networking
- 1× Gigabit Ethernet + 1× 10 Gigabit Ethernet
- Power
- 15 W – 60 W (configurable power modes)
- Dimensions
- 100 × 87 mm module; dev kit 110 × 110 × 71.65 mm
- Cooling
- Active thermal solution with fan; 60 W MAXN needs full-size heatsink
- Expansion
- PCIe Gen4 up to 2×8 + 1×4 + 2×1, USB 3.2 Gen2, 6 × 4-lane CSI, 1× 8K display
- Supported frameworks
- JetPack 6
- CUDA 12
- cuDNN
- TensorRT
- TensorRT-LLM (selected models)
- PyTorch
- ONNX Runtime
- DeepStream
- Isaac ROS
- Ollama
- llama.cpp
- MLC
- Docker
Source: NVIDIA product pages. Figures are manufacturer specifications, not our measurements.
Theoretical vs real-world
275 TOPS is a peak figure at INT8, sparse with structured sparsity. Chat-style LLM generation is limited by the 204.8 GB/s memory bus and by how much model fits in 64 GB. Rough rule: tokens/s ≈ bandwidth ÷ model bytes.
80
~3B INT4
estimated tokens/s
30
~8B INT4
estimated tokens/s
18
~13B INT4
estimated tokens/s
Estimates assume ~0.6 bytes/parameter at INT4 and ~70% achieved bandwidth. Real results vary by runtime.
Measured LLM inference
| Model | Quantisation | Runtime | Tokens/s |
|---|---|---|---|
| Llama 2 7B Jetson AI Lab community benchmark, indicative | INT4 (q4f16_ft) | MLC, MAXN | 45 |
| Llama 2 13B Jetson AI Lab community benchmark, indicative | INT4 (q4f16_ft) | MLC, MAXN | 25 |
| Llama 2 70B Fits only on 64 GB; indicative | INT4 (q4f16_ft) | MLC, MAXN | 4.9 |