caw-eval

Fail

Audited by Gen Agent Trust Hub on Oct 8, 2026

Risk Level: HIGHCREDENTIALS_UNSAFEINDIRECT_PROMPT_INJECTIONCOMMAND_EXECUTIONEXTERNAL_DOWNLOADSREMOTE_CODE_EXECUTION
Full Analysis
  • [CREDENTIALS_UNSAFE]: The file QUICK_START.md contains a hardcoded LANGFUSE_PUBLIC_KEY and a corresponding LANGFUSE_SECRET_KEY. Storing plaintext secret keys in documentation or source code is a significant security risk, even if intended for a sandbox environment.
  • [INDIRECT_PROMPT_INJECTION]: The evaluation workflow ingests data from external Langfuse datasets which are then used to build prompts for headless agents executing tasks on remote servers.
  • Ingestion points: Dataset items retrieved via scripts/eval_utils.py from Langfuse.
  • Boundary markers: Prompts constructed in scripts/run_eval_cc.py and scripts/run_eval_openclaw.py use simple text separators to frame the user message but lack strong directives instructing the agent to ignore any embedded malicious instructions.
  • Capability inventory: The skill facilitates remote shell execution on GCE instances via gcloud ssh and uses the caw CLI to perform blockchain wallet operations.
  • Sanitization: The skill does not sanitize or escape natural language inputs from the dataset before they are processed by the agent.
  • [COMMAND_EXECUTION]: Orchestration scripts such as scripts/run_eval_cc.py and scripts/run_eval_openclaw.py make extensive use of subprocess.run and asyncio.create_subprocess_exec to execute system commands including gcloud, ssh, tar, and git. This functionality is essential for the skill's management of remote evaluation environments but represents a powerful capability that requires careful control.
  • [EXTERNAL_DOWNLOADS]: Server bootstrap and setup documentation (references/server-setup.md, scripts/bootstrap_cc_server.sh) involve fetching software from external domains:
  • Downloads Node.js from nodesource.com.
  • Fetches the @anthropic-ai/claude-code CLI from the NPM registry.
  • Fetches the caw CLI installer from the official vendor repository github.com/CoboGlobal/.
  • [REMOTE_CODE_EXECUTION]: Setup instructions and scripts utilize patterns for piping remote content directly to a shell for execution:
  • curl -fsSL https://deb.nodesource.com/setup_20.x | sudo -E bash - in references/server-setup.md.
  • curl -fsSL https://raw.githubusercontent.com/CoboGlobal/cobo-mpc-sdk/main/scripts/install.sh | bash in references/server-setup.md.
  • While these URLs point to well-known or vendor-owned services, executing remote scripts directly via a pipe is an inherently risky pattern.
Recommendations
  • HIGH: Downloads and executes remote code from: http://metadata.google.internal/computeMetadata/v1 - DO NOT USE without thorough review
  • AI detected serious security threats
Audit Metadata
Risk Level
HIGH
Analyzed
Oct 8, 2026, 10:30 PM