The Agent Skills Directory

[COMMAND_EXECUTION]: The skill implements direct desktop automation and shell execution capabilities.
Evidence: Uses pyautogui for mouse clicking, typing, and key presses in the execute_action method.
Evidence: Utilizes subprocess.run to execute system commands like scrot for screen capture.
Evidence: Integrates Anthropic's bash tool (BetaToolBash20241022) for arbitrary shell command execution.
[INDIRECT_PROMPT_INJECTION]: The architecture describes a vision-based agent susceptible to instructions embedded in the graphical environments it perceives.
Ingestion points: The agent captures screenshots of the desktop environment via pyautogui.screenshot() and scrot, which are then processed by a vision-language model.
Boundary markers: No explicit prompt delimiters or instructions are provided to the model to ignore content found within screenshots.
Capability inventory: High-impact capabilities including full mouse/keyboard control (pyautogui) and shell command execution (subprocess.run, bash tool).
Sanitization: There is no evidence of filtering or validation of the text or visual elements extracted from the captured screens before they influence the agent's reasoning loop.
[SAFE]: The skill provides comprehensive security guidance and implementation patterns for sandboxing computer-use agents.
Isolation: Includes a Dockerfile and docker-compose configuration that uses non-root users, drops capabilities (setcap), and implements strict network isolation (internal: true).
Resource Management: Implements CPU and memory limits, as well as read-only filesystems to minimize the blast radius of potential exploits.

computer-use-agents