alterlab-test-selection-guard
Pass
Audited by Gen Agent Trust Hub on Sep 25, 2026
Risk Level: SAFECOMMAND_EXECUTIONINDIRECT_PROMPT_INJECTION
Full Analysis
- [SAFE]: The skill serves as a discipline-enforcing layer for academic research. It provides structured guidance to prevent 'test-shopping' and ensures statistical assumptions are verified before interpreting results. The logic is transparent and focused on scientific integrity.
- [COMMAND_EXECUTION]: The skill includes a Python script (
scripts/test_picker.py) intended to be run locally viauv run python. This script implements a simple decision tree based on user-provided data characteristics (e.g., number of groups, outcome type). The script only uses the Python standard library (argparse,json,sys) and does not perform any network operations or sensitive file access. - [INDIRECT_PROMPT_INJECTION]: The skill is designed to process untrusted user data regarding experimental designs. It mitigates the risk of manipulation through strict adherence to an 'Iron Law' (no test choice after seeing p-values) and includes a 'Red Flags' list to identify and stop attempts to manipulate the agent into endorsing statistically invalid switches. This design effectively creates a safety boundary around its core logic.
Audit Metadata