continuous-learning-v2

Fail

Audited by Gen Agent Trust Hub on Jul 29, 2026

Risk Level: HIGHCOMMAND_EXECUTIONDATA_EXFILTRATIONEXTERNAL_DOWNLOADSPROMPT_INJECTION
Full Analysis
  • [COMMAND_EXECUTION]: The hooks/observe.sh script is vulnerable to Python code injection. It embeds shell-provided JSON data (containing tool inputs and outputs) directly into a Python block using triple quotes ('''$INPUT_JSON'''). If a tool output contains triple quotes, it can escape the string literal and execute arbitrary Python code during the observation phase.
  • [DATA_EXFILTRATION]: The skill captures the full input and output of every tool used (including Bash, Read, and Edit). While the documentation claims data stays local, the background observer agent (agents/start-observer.sh) is designed to send these raw logs—which may contain API keys, environment variables, or private file content—to a cloud-based model provider for pattern analysis.
  • [EXTERNAL_DOWNLOADS]: The scripts/instinct-cli.py tool includes an import command that fetches content from arbitrary URLs via urllib. This allows the introduction of unverified behavioral instructions into the agent's learning system from remote sources.
  • [PROMPT_INJECTION]: The skill is susceptible to indirect prompt injection. It ingests session history (untrusted data) to automatically generate new behavioral rules ("instincts"). Malicious instructions encountered during a web browsing or code review session could be interpreted by the observer agent and converted into persistent instructions that the agent will follow in future sessions.
  • Ingestion points: Raw tool logs in ~/.gemini/homunculus/observations.jsonl populated by hooks/observe.sh.
  • Boundary markers: Absent; the background observer agent processes raw session events without sanitization or instruction delimiters.
  • Capability inventory: The observer agent can write new instructional files to the instincts/personal/ directory and influence future model behavior.
  • Sanitization: Absent; the system relies on the model's ability to distinguish between data and instructions within the logs.
Recommendations
  • AI detected serious security threats
Audit Metadata
Risk Level
HIGH
Analyzed
Jul 29, 2026, 08:20 AM