The Agent Skills Directory

[CREDENTIALS_UNSAFE]: Multiple files in the evaluation directory contain hardcoded credentials and secret fallbacks. Evidence: evals/files/Dockerfile hardcodes NPM_TOKEN and DATABASE_URL credentials; evals/files/deployment.yaml contains a plaintext DB_PASSWORD; evals/files/main.tf stores an RDS password in plain text; evals/files/documents_service.py and evals/files/orders_api.js contain hardcoded JWT secrets. Additionally, evals/files/orders_api.js and evals/files/documents_service.py use jwt.decode without signature verification, allowing authentication bypass.\n- [COMMAND_EXECUTION]: Evaluation components demonstrate unsafe execution of system commands. Evidence: evals/files/agent.py uses subprocess.run with shell=True for a diagnostic tool; evals/files/orders_api.js uses exec to run a PDF conversion utility with unvalidated user-supplied filenames.\n- [PRIVILEGE_ESCALATION]: The evals/files/deployment.yaml Kubernetes manifest grants excessive permissions. Evidence: The deployment uses privileged: true, runAsUser: 0, and mounts the host's root filesystem. Additionally, a ClusterRoleBinding assigns cluster-admin rights to the default service account.\n- [REMOTE_CODE_EXECUTION]: The CI/CD configuration in evals/files/ci.yml is vulnerable to supply chain attacks. Evidence: The use of pull_request_target combined with checking out the head of a PR and running npm install allows an external contributor to execute arbitrary code within the pipeline environment and access repository secrets.\n- [INDIRECT_PROMPT_INJECTION]: The AI agent implementation in evals/files/agent.py presents an injection surface. Evidence: The retrieve_context function ingests customer-provided ticket data from a knowledge base into the context window. The agent is equipped with high-privilege tools including run_sql, run_shell, and send_email without sufficient boundary markers or human-in-the-loop controls for consequential actions.

security-audit