ONNX

Overview

ONNX (onnx/onnx) defines the model format and operator sets; ONNX Runtime (microsoft/onnxruntime) executes it through pluggable execution providers (CPU, CUDA, TensorRT, OpenVINO, CoreML, DirectML, QNN). Checked against onnx 1.23.1 and onnxruntime 1.30.0 (October 2026); the runtime-side snippets below were run on those versions.

Instructions

Install

python -m venv .venv && source .venv/bin/activate
pip install onnx onnxruntime numpy        # CPU runtime
pip install onnxsim onnxoptimizer         # optional graph simplifiers

For NVIDIA GPUs install onnxruntime-gpu instead of onnxruntime. The two packages must never be installed together. onnxruntime-directml (Windows) and onnxruntime-qnn are other variants. The default GPU wheel targets CUDA 12.x with a separately installed cuDNN.

Export from PyTorch