JoelClaw Observability + Logging

Prevent silent failure by default. Observability is not optional polish: it is part of done.

Non-Negotiable Rules

1. Use the canonical event contract only.
   • packages/system-bus/src/observability/otel-event.ts
   • packages/system-bus/src/observability/emit.ts
   • packages/system-bus/src/observability/store.ts
2. Worker/Inngest code emits through emitOtelEvent or emitMeasuredOtelEvent.
3. Gateway code emits through emitGatewayOtel.
4. Internal ingestion goes through POST /observability/emit (packages/system-bus/src/serve.ts), not ad-hoc writes.
5. Never treat console.log as primary observability. Keep structured events as source of truth.
6. High-cardinality values go in metadata, not in facet fields (source, component, level, success).
7. Failures must set success: false with a meaningful error.
8. For warn/error/fatal, verify Convex mirror behavior (rolling window) in addition to Typesense write.
9. In Inngest durable functions, any "emit once" telemetry must live inside step.run(...) to avoid replay duplication after resume.