AI and Agent System Architecture

Overview

Design AI products as governed software systems rather than prompts wrapped in an API. Separate the probabilistic model from the deterministic harness that controls context, tools, memory, policy, evaluation, recovery, and irreversible actions.

Core principle: The model proposes; the harness constrains, verifies, records, and decides what may happen next.

NO PRODUCTION AI FLOW WITHOUT:
1. a measurable task and acceptance rubric;
2. explicit model, data, context, memory, and tool boundaries;
3. a failure and fallback path;
4. offline and online evaluation;
5. traceable cost, latency, safety, and version evidence.

When to Use