The Agent Skills Directory

[INDIRECT_PROMPT_INJECTION]: The agent operates by capturing screenshots and current URLs from a web browser via Playwright and feeding them into the Gemini model. This creates a surface for indirect prompt injection, where an attacker-controlled website could include hidden text or visual instructions intended to hijack the agent's behavior. If an injection is successful, an attacker could potentially bypass the model-driven safety confirmation by instructing the model to mark a malicious action as safe.
Ingestion points: External data enters the system through page.screenshot() and page.url within the build_function_responses function in scripts/computer_use_agent.py.
Boundary markers: The script does not implement delimiters or specific instructions to help the model distinguish between the user's initial prompt and the untrusted content retrieved from the web.
Capability inventory: The agent has extensive control over the browser session, including the ability to navigate to arbitrary URLs (including local file:// or internal network addresses), click_at specific coordinates, type_text_at (which can clear existing input and press Enter), and execute arbitrary key_combination actions.
Sanitization: No sanitization, filtering, or OCR verification is performed on the visual or URL data before it is sent to the model.
[EXTERNAL_DOWNLOADS]: The skill's setup instructions in SKILL.md require the user to execute playwright install chromium. This command downloads and installs browser binaries from official Playwright repositories.

gemini-computer-use