Triton Deployment

NVIDIA Triton Inference Server management.

Quick Start

# Pull Triton image
docker pull nvcr.io/nvidia/tritonserver:24.01-py3

# Run server
docker run --gpus all -p 8000:8000 -p 8001:8001 -p 8002:8002 \
  -v $(pwd)/models:/models \
  nvcr.io/nvidia/tritonserver:24.01-py3 \
  tritonserver --model-repository=/models

Model Repository Structure