Desktop Control Skill

Control the desktop GUI using semantic element targeting. Find and click UI elements by name instead of coordinates.

Key Features:

• AT-SPI - Primary method using accessibility tree (knows element roles, states, actions)
• OCR Fallback - Tesseract-based text finding when AT-SPI can't find the element
• Wait Utilities - Poll for elements to appear with exponential backoff
• Click Verification - Optional pre-click screenshot verification

Prerequisites

Install dependencies:

bash install.sh