LLM Observability Testing Skill

Decide how each instrumented surface gets its responses first

That choice picks the response source and test setup — cassettes for a workflow record nothing, while pure-function tests for a provider-backed call miss the network surface entirely. The operation independently determines its span kind and fields. These are working categories for reasoning; none exists as a code constant.

• LLM client / multi-provider — reaches provider HTTP directly or through a supplied provider package (openai, anthropic, genai, ai, langchain): VCR cassettes or a canned fetch.
• Orchestration — carries workflow or graph state and makes no provider calls of its own (langgraph): no VCR; drive nodes with plain return values.
• Infrastructure — implements a protocol or server (modelcontextprotocol-sdk): run the SDK's own server and client over its in-memory transport.
• Canned fetch instead of a cassette — where the spec supplies the responses itself: google-cloud-vertexai swaps global.fetch per test and stubs Google auth, openai-agents and some ai providers pass a fetch option to the client they construct.