Groq Observability

Overview

Monitor Groq LPU inference for latency, token throughput, rate limit utilization, and cost. Groq's defining advantage is speed (280-560 tok/s), so latency degradation is the highest-priority signal. The API returns rich timing metadata (queue_time, prompt_time, completion_time) and rate limit headers on every response.

Prerequisites

• A Groq account with an API key exported as the GROQ_API_KEY environment variable — the groq-sdk client reads it automatically (new Groq()).
• Node.js with groq-sdk and prom-client installed (npm install groq-sdk prom-client).
• A Prometheus scrape target and (optionally) Grafana for the dashboard panels.

Key Metrics to Track