Idea Critique

Test one idea the user holds before anyone spends on it. The result is the strongest case for the idea, the strongest case against it, the measurement that would decide between them, and a status tied to evidence.

The division of labor follows a large blind-review study in which LLM-generated research ideas were rated more novel than experts' ideas but slightly less feasible, LLM generation lost diversity as more ideas were produced, and LLM self-evaluation of ideas was unreliable (Si, Yang, and Hashimoto, arXiv 2409.04109). The agent supplies mechanisms, evidence, and attacks. Choosing which ideas to pursue stays with the user.

Scope

An idea is one claim someone could be wrong about: a research hypothesis, a design choice, a mechanism expected to work, a prediction about a model's or agent's behavior, or the key assumption under a plan. Run on the user's request, or when another skill such as design-exploration hands over a hunch the user stated. When a message carries several ideas, critique them one at a time, starting with the one the others depend on.

Route nearby requests elsewhere. A user with no single idea yet who wants the space opened needs design-exploration. A paper's argument belongs to credo-paper-plan when it is installed. A verdict on a built evaluation, or on a result before it informs a decision, belongs to evaluation-audit. Turning an idea that survived critique into a resourced bet belongs to praxis-direction.

Claim restatement

Restate the idea in the user's words as one testable sentence: what changes, on which system or population, measured how, compared with what, and under which conditions. Fill gaps from the conversation and project files before asking. Ask the user to confirm only when two readings would lead to different attacks or a different deciding measurement; otherwise state the reading being tested and continue. A vague magnitude such as "a lot better" gets an explicit reading the user can correct. If no reading is testable, the first finding is what the claim must specify to become testable.

Steelman

Build the version a capable proponent would defend, not the weakest wording. Give the mechanism by which it would work, the supporting evidence with locations, and the conditions under which it most plausibly holds. A repair that keeps the same core bet, such as narrowing the scope or adding a control the idea needs, belongs in the steelman; a different bet is an alternative idea (see Rules). When the steelman is narrower than the stated claim, say what was narrowed. The attack targets the steelman, because defeating a weaker version says nothing about the idea.