macOS Harness

Every command names the app. One shell call: the actions you are sure of, then one read, joined with &&.

macos-harness get_app_state --app "TextEdit"
macos-harness click --app "TextEdit" --element_index 12 && macos-harness type_text --app "TextEdit" --text "hello world" && macos-harness press_key --app "TextEdit" --key "super+s" && macos-harness get_app_state --app "TextEdit"
macos-harness set_value --app "TextEdit" --element_index 7 --value "report.txt" && macos-harness scroll --app "TextEdit" --element_index 3 --direction down --pages 1 && macos-harness get_app_state --app "TextEdit"

• get_app_state prints one line per thing on screen. The number at the start of a line is its element_index.
• Read, act, then read again before the next step. Older numbers may be wrong. Chain sure steps with the read after them: && stops at the first failure, and one call costs less than three.
• The second read prints only what changed (~ changed, + new). Nothing changed means your action did nothing: try another way, do not read again.
• If a click by number does nothing, open the file on the Screenshot line and click with --x and --y (pixels in that image).
• If the state shows only a menu, click one of its lines, or leave it with press_key and --key Escape.
• type_text types at the text cursor: click the field first. set_value replaces a field's whole text and needs no click.
• Text with quotes or several lines: macos-harness paste --app "TextEdit" <<'EOF', the text, EOF.
• Never use AppleScript, osascript, System Events or other ways to click and type unless the person asks.