alterlab-test-selection-guard

Pass

Audited by Gen Agent Trust Hub on Sep 25, 2026

Risk Level: SAFECOMMAND_EXECUTIONINDIRECT_PROMPT_INJECTION
Full Analysis
  • [SAFE]: The skill serves as a discipline-enforcing layer for academic research. It provides structured guidance to prevent 'test-shopping' and ensures statistical assumptions are verified before interpreting results. The logic is transparent and focused on scientific integrity.
  • [COMMAND_EXECUTION]: The skill includes a Python script (scripts/test_picker.py) intended to be run locally via uv run python. This script implements a simple decision tree based on user-provided data characteristics (e.g., number of groups, outcome type). The script only uses the Python standard library (argparse, json, sys) and does not perform any network operations or sensitive file access.
  • [INDIRECT_PROMPT_INJECTION]: The skill is designed to process untrusted user data regarding experimental designs. It mitigates the risk of manipulation through strict adherence to an 'Iron Law' (no test choice after seeing p-values) and includes a 'Red Flags' list to identify and stop attempts to manipulate the agent into endorsing statistically invalid switches. This design effectively creates a safety boundary around its core logic.
Audit Metadata
Risk Level
SAFE
Analyzed
Sep 25, 2026, 02:36 PM