agent-desktop

Installation
SKILL.md

agent-desktop

agent-desktop is a CLI that lets an AI agent observe and operate desktop apps through the OS accessibility tree. It is a tool, not an agent. Every command prints one JSON envelope (ok, data or error). The observe, act and verify loop runs in you.

Shell syntax

Examples use a POSIX shell. In PowerShell, quote every ref ('@s8f3k2p9:e1'), because a bare @token is the splatting operator. Use $env:NAME = 'value' instead of export, and | ConvertFrom-Json instead of $(...) and jq. The Windows skill has the PowerShell forms.

First run

  1. Run agent-desktop status. It reports the platform, permissions and supported_surfaces.
  2. Run agent-desktop skills get platform. It loads the skill for the OS the binary runs on (macOS, Windows or Linux).
  3. For flags and syntax, run agent-desktop <command> --help. This skill does not repeat flag lists.

The loop

Use find when you know the role or exact name. Otherwise map the app first and drill into one region.

Installs
1
GitHub Stars
15.4K
First Seen
Nov 2, 2026