sample-app-benchmark
Fail
Audited by Gen Agent Trust Hub on Jun 15, 2026
Risk Level: HIGHCOMMAND_EXECUTIONDATA_EXFILTRATIONREMOTE_CODE_EXECUTIONPROMPT_INJECTION
Full Analysis
- [COMMAND_EXECUTION]: The skill instructs the agent to execute shell commands, specifically
codex exec --profile sample_app_eval, and to discover and run arbitrary local test commands and application start scripts from the repository being benchmarked. - [DATA_EXFILTRATION]: The skill identifies and accesses multiple sensitive absolute file paths within a specific user's directory (
/Users/gurusharan/). It also mandates the collection and storage of detailed execution metadata in external eval surfaces, including command transcripts, structured run logs, git diffs, and session token usage data. - [REMOTE_CODE_EXECUTION]: The core workflow involves checking out and executing code from arbitrary and potentially unverified repository branches (challenger branches) during the benchmark process, which allows for the execution of untrusted code in the local environment.
- [PROMPT_INJECTION]: The skill exhibits an indirect prompt injection surface by ingesting and acting upon untrusted data from repository branches and user-supplied task prompts without sanitization or boundary markers.
- Ingestion points: Repository files (e.g.,
AGENTS.md,README.md, and local scripts) and the user-provided benchmark task prompt. - Boundary markers: None identified; the instructions do not specify delimiters or warnings to ignore embedded instructions within the ingested data.
- Capability inventory: Subprocess execution via
codex exec, execution of discovered test/run commands from the repo, and execution of various local Python automation scripts (e.g.,run_sample_app_benchmark_session.py). - Sanitization: None; the skill does not implement validation, escaping, or filtering for the external content it processes.
Recommendations
- AI detected serious security threats
Audit Metadata