agent-desktop
Installation
SKILL.md
agent-desktop
agent-desktop is a CLI that lets an AI agent observe and operate desktop apps through the OS accessibility tree. It is a tool, not an agent. Every command prints one JSON envelope (ok, data or error). The observe, act and verify loop runs in you.
Shell syntax
Examples use a POSIX shell. In PowerShell, quote every ref ('@s8f3k2p9:e1'), because a bare @token is the splatting operator. Use $env:NAME = 'value' instead of export, and | ConvertFrom-Json instead of $(...) and jq. The Windows skill has the PowerShell forms.
First run
- Run
agent-desktop status. It reports the platform, permissions andsupported_surfaces. - Run
agent-desktop skills get platform. It loads the skill for the OS the binary runs on (macOS, Windows or Linux). - For flags and syntax, run
agent-desktop <command> --help. This skill does not repeat flag lists.
The loop
Use find when you know the role or exact name. Otherwise map the app first and drill into one region.