genai-conformance

Installation
SKILL.md

GenAI Conformance

The repo ships a self-contained conformance harness at python/openinference-instrumentation/scripts/conformance/ that exercises OpenInference instrumentors against deterministic mock provider APIs, exports OTLP traces to weaver registry live-check, and prints a console summary of registry attributes seen / missing / advice-level counts. It validates the dual-write logic in _genai_conversion.py that translates OpenInference's native attributes (llm.*, input.*, output.*, openinference.*) into the OTel GenAI semantic conventions (gen_ai.*).

When to Use

  • User asks to run the conformance harness, "test conformance", or "run weaver".
  • User asks to maximize / improve gen_ai.* registry coverage.
  • User wants to extend the dual-write conversion in _genai_conversion.py.
  • User wants to add a new provider, a new test scenario, or a new mock endpoint.
  • User mentions specific gen_ai.* attributes (response.id, system_instructions, tool.call., retrieval., etc.) and whether they're being emitted.

Layout

Installs
1
GitHub Stars
1.0K
First Seen
2 days ago
genai-conformance — arize-ai/openinference