The Agent Skills Directory

[COMMAND_EXECUTION]: The skill provides patterns for executing local system actions such as mouse clicks, typing, and keyboard input via the pyautogui library. These actions are triggered dynamically based on the output of an AI model.
[DATA_EXPOSURE]: Implementation patterns involve capturing screenshots of the desktop environment via pyautogui.screenshot(), which are then transmitted to the Anthropic API for analysis. The guide includes patterns for redacting sensitive data in logs to mitigate exposure.
[INDIRECT_PROMPT_INJECTION]: The skill is designed to process untrusted data from web pages and desktop applications, which could contain malicious instructions intended to hijack the agent's behavior.
Ingestion points: references/detailed-guide.md (via capture_screenshot and get_page_snapshot functions).
Boundary markers: The guide suggests using system prompts that restrict output to specific JSON schemas and recommends 'Confirmation Gates' for sensitive actions.
Capability inventory: Desktop interaction via pyautogui, browser control via playwright, and network communication with AI providers.
Sanitization: Includes code for a heuristic injection scanner (scan_for_injection) and logic to mask passwords or financial details in logs.
[EXTERNAL_DOWNLOADS]: The provided Dockerfile and implementation guides reference standard software installation from well-known repositories (Ubuntu, PyPI, and Playwright browsers).

computer-use-agents