continuous-learning-v2
Fail
Audited by Gen Agent Trust Hub on Jul 29, 2026
Risk Level: HIGHCOMMAND_EXECUTIONDATA_EXFILTRATIONEXTERNAL_DOWNLOADSPROMPT_INJECTION
Full Analysis
- [COMMAND_EXECUTION]: The
hooks/observe.shscript is vulnerable to Python code injection. It embeds shell-provided JSON data (containing tool inputs and outputs) directly into a Python block using triple quotes ('''$INPUT_JSON'''). If a tool output contains triple quotes, it can escape the string literal and execute arbitrary Python code during the observation phase. - [DATA_EXFILTRATION]: The skill captures the full input and output of every tool used (including
Bash,Read, andEdit). While the documentation claims data stays local, the background observer agent (agents/start-observer.sh) is designed to send these raw logs—which may contain API keys, environment variables, or private file content—to a cloud-based model provider for pattern analysis. - [EXTERNAL_DOWNLOADS]: The
scripts/instinct-cli.pytool includes animportcommand that fetches content from arbitrary URLs viaurllib. This allows the introduction of unverified behavioral instructions into the agent's learning system from remote sources. - [PROMPT_INJECTION]: The skill is susceptible to indirect prompt injection. It ingests session history (untrusted data) to automatically generate new behavioral rules ("instincts"). Malicious instructions encountered during a web browsing or code review session could be interpreted by the observer agent and converted into persistent instructions that the agent will follow in future sessions.
- Ingestion points: Raw tool logs in
~/.gemini/homunculus/observations.jsonlpopulated byhooks/observe.sh. - Boundary markers: Absent; the background observer agent processes raw session events without sanitization or instruction delimiters.
- Capability inventory: The observer agent can write new instructional files to the
instincts/personal/directory and influence future model behavior. - Sanitization: Absent; the system relies on the model's ability to distinguish between data and instructions within the logs.
Recommendations
- AI detected serious security threats
Audit Metadata