LLM Observability Integration Skill

This skill covers creating LLMObs plugins that instrument LLM library operations and emit span events. Supported operations: chat completions (streaming and non-streaming), embeddings, agent runs, orchestration (workflows / graphs), tool calls, retrieval (RAG / vector DB).

Read Upstream Source First

LLM libraries iterate fast — six-month-old assumptions about an SDK's response shape, streaming contract, or tool-call format are usually wrong. Before category detection or any plugin work, read the upstream library's source for the installed version (versions/<lib>@<range>/node_modules/<lib>). The category decision tree below depends on facts the source carries (does this package make HTTP calls? does it orchestrate? does it support multiple providers?). See apm-integrations § Read Upstream Source First for the shallow-clone / npm pack shapes.

Core Concepts

1. LLMObsPlugin Base Class

All LLMObs plugins extend LLMObsPlugin. Two methods must be implemented:

• getLLMObsSpanRegisterOptions(ctx) — returns { modelProvider, modelName, kind, name }.
• setLLMObsTags(ctx) — extracts and tags input / output messages, token metrics, and model metadata.

Lifecycle: start(ctx) registers the span and captures context; the wrapped operation runs; asyncEnd(ctx) calls setLLMObsTags(); end(ctx) restores the parent.