The Agent Skills Directory

UI Interaction and Input Injection: The skill provides tools like invoke, click, send-keys, and set-value to simulate user input. This allows an agent to automate workflows within Windows applications, including the ability to fire system-level hotkeys via the --allow-system-keys flag when using the send-input transport. These features are standard for UI automation tools and allow the agent to drive application logic as intended.
Application Data and Visual Capture: The screenshot, record, and get-value commands enable the agent to capture the visual state of the desktop or read text content from UI elements. The screenshot and record tools include a --capture-screen option that can capture overlays and popups that are not part of the primary application window, which is helpful for full-context automation but allows the agent to see all content currently displayed on the desktop.
Indirect Prompt Injection Surface: The skill exposes a potential surface for indirect prompt injection due to how it processes data from external applications.
Ingestion points: The agent receives untrusted data from the UI of running applications through the inspect, search, get-value, get-property, and wait-for commands in SKILL.md.
Boundary markers: No specific delimiters or "ignore embedded instructions" warnings are documented for the content read from UI elements.
Capability inventory: The skill allows the agent to perform actions based on this data using the invoke, click, send-keys, set-value, drag, touch, and pen commands in SKILL.md.
Sanitization: The documentation does not specify any automated sanitization, escaping, or filtering of the content retrieved from application UI elements before it is processed by the agent.
Sandbox Environment Support: The skill includes explicit support for running and automating applications within an isolated Windows Sandbox using the --on sandbox flag. This allows for testing and automation in a controlled environment, which can mitigate risks to the host system during development.

winapp-ui-automation