Together Dedicated Model Inference

Overview

Dedicated model inference (DMI) serves a model on reserved single-tenant GPUs. It bills per minute per running replica (by hardware, not by token), has no hard rate limits, and uses the same inference API as serverless models.

The resource model has six parts: Project → Model → Config → Endpoint → Deployment → Replica.

• Endpoint — stable inference name; routes traffic across its deployments by weight.
• Deployment — binds one model + one config to an endpoint with an autoscaling policy; runs the replicas.
• Config — immutable published revision describing how the model runs (engine, GPU type and count, optimization profile). The model determines the available configs.
• Replica — one model instance on its own dedicated hardware.