Design Exploration

Open the design space for a module before anyone converges on one design, and give the user material for intuition: what the module must hold, where the current approach falls short, which mechanisms exist, and which concepts are worth testing. The work ends in one of two ways. The user picks concepts to test and each goes to the skill that converges or tests it, or the exploration closes lightly with the consequential unknowns and next checks when that is all the work needs.

The agent supplies breadth, mechanisms, and attacks; the user chooses. In a blind comparison involving more than a hundred NLP researchers, LLM-generated research ideas were judged more novel than experts' ideas but slightly less feasible, LLM self-evaluation of ideas was unreliable, and LLM ideas lost diversity as more were generated (Si, Yang, and Hashimoto, arXiv 2409.04109). So during exploration do not narrow to a single recommendation or rank concepts by the agent's own score, and get breadth from different mechanisms rather than from generating more ideas.

Skip a step that existing material already answers, and name that material. Breadth belongs to scanning; when presenting, show the few items that change the user's thinking, not an inventory.

Root the work by its type

Before mechanisms or concepts, settle what the module must achieve in the terms its work type requires. Infer the root from the request, the repository, and prior records.

• Defensive module, such as a spam or fraud classifier or a monitor that must hold against an adaptive adversary: the threat model and the guarantee the module claims, stated so that a violation is observable. Establish the threat model with praxis-direction's failure-mode pass and its threat list rather than writing a new list.
• Attack or red-team agent, such as an automated penetration-testing agent: the objective category, the scope and boundary it acts within, and how success is judged. Judging belongs to evaluation-design and its red-team reference; take the criterion from there rather than inventing one here.
• Research module, whose behavior is learned from data or produced by an agent loop with no adversary in scope, such as a document-retrieval model or a web-navigation agent: the question the work answers and which result would change direction.

A module with two types, such as a defensive classifier that a red-team agent will test, states both roots. A system with neither a learned core nor an adversary is rooted like a research module: the property it must hold and the result that would show it does not.

Do not start from end-user usage scenes, screens, or user stories. They suit product feature work and pull the design toward interface choices before anyone knows whether the core can work. Facts about the setting do belong in the root and are often needed: where the module is deployed, which inputs reach it and who controls them, what it is permitted to do, what each kind of failure costs, and which population it is evaluated on.