Cheatcode_
Feedback

Catalog / Design / Agent desktop

Agent skills · 3 skills

Agent desktop

Listed by its author as agent-desktop

Agent Desktop gives any agent reliable computer use on the desktop. Built with Rust, it sees any app's real UI structure through OS accessibility trees and operates it, refs stay stable and actions…

DoesTerminal and systemCoding and refactoringDesign and UIAutomation and workflowsBrowser and scraping

Source on GitHub ↗
curl -fsSL https://api.cheatcodehq.com/install.sh | sh -s -- agent-desktop --to all

Copies 3 skills into each app's skills folder (Claude Code: ~/.claude/skills), after checking every file against lahfir/agent-desktop at commit b8d538c. Nothing else on your computer changes. To undo it, run the terminal command with --uninstall at the end; that also removes what the app added.

Then start a new session in your AI app and ask for the work in plain words; it picks the skill from its description. In Claude Code you can also type /agent-desktop:agent-desktop.

See it in use

An illustration, not a recording: it was not run here because it operates real apps on your Mac through its accessibility controls, which needs the agent-desktop program and the Accessibility permission. The request and steps follow the package's own instructions, and what it shows last (its instructions) is cut unedited from skills/agent-desktop/references/workflows.md, lines 171 to 224. The weekly note is invented for this example: an update from Dana, support lead at Acme Notes, a fictional note-taking app.
The result
Its instructionsIts instructions: The work as its instructions describe it, from skills/agent-desktop/references/workflows.md, lines 171 to 224, unedited. Not from a recorded session.

The task done in the real app, with each step checked against the app's own state.

The work as its instructions describe it, from skills/agent-desktop/references/workflows.md, lines 171 to 224, unedited. Not from a recorded session.

Open the whole result
Try it

After installing, start a new session and say: Use the agent-desktop skill to open TextEdit, type a short note, and save it to my Documents folder.

What else it can do, and its limits
  • A command line tool your AI app calls to see and operate desktop apps through their accessibility controls
  • Covers clicks, typing, menus, dialogs, scrolling, drag and drop, windows, the clipboard and screenshots
  • Works on macOS 12 or newer today; Windows and Linux are planned, not available
  • Needs a terminal to install it with npm, plus the Accessibility permission and Screen Recording for screenshots
  • Its goal runner skill needs a TypeSafe or OpenRouter key and scripts this package does not include; a library skill is for developers

Illustration Drawn Oct 26, 2026 from the package's own files (skills/agent-desktop/references/workflows.md, skills/agent-desktop/SKILL.md, skills/agent-desktop/references/commands-system.md, skills/agent-desktop/references/commands-interaction.md). Not a recorded session: nothing was run, and what it shows comes from the package itself.

What's inside 3 skills

  • agent-desktopReliable computer use via native OS accessibility trees. Use when an AI agent needs to see and operate desktop applications (click buttons, fill forms, navigate menus, read UI state, toggle checkboxes, scroll, drag, type text, take screenshots, manage windows, use clipboard, manage notifications). Covers 59 command names (55 operational; four held-input names fail closed until daemon ownership exists) across observation, interaction, keyboard/mouse, app lifecycle, notifications (macOS), clipboard, wait, session lifecycle, and a `skills` command that bundles docs straight from the binary. Triggers on: "click button", "fill form", "open app", "read UI", "computer use", "operate desktop", "accessibility tree", "snapshot app", "type into field", "navigate menu", "toggle checkbox", "take screenshot", "desktop automation", "agent-desktop", or any desktop GUI interaction task. Supports the macOS Phase 1 adapter, with Windows and Linux planned against the same core contracts.

    Reliable computer use via native OS accessibility trees. Use when an AI agent needs to see and operate desktop applications (click buttons, fill forms, navigate menus, read UI state, toggle checkboxes, scroll, drag, type text, take screenshots, manage windows, use clipboard, manage notifications). Covers 59 command names (55 operational; four held-input names fail closed until daemon ownership exists) across observation, interaction, keyboard/mouse, app lifecycle, notifications (macOS), clipboard, wait, session lifecycle, and a `skills` command that bundles docs straight from the binary. Triggers on: "click button", "fill form", "open app", "read UI", "computer use", "operate desktop", "accessibility tree", "snapshot app", "type into field", "navigate menu", "toggle checkbox", "take screenshot", "desktop automation", "agent-desktop", or any desktop GUI interaction task. Supports the macOS Phase 1 adapter, with Windows and Linux planned against the same core contracts.

  • agent-desktop-ffiC-ABI bindings over agent-desktop's PlatformAdapter. Consumers (Python ctypes, Swift, Node ffi-napi, Go cgo, C++, Ruby fiddle) link libagent_desktop_ffi.{dylib,so,dll} and call `ad_*` functions directly instead of spawning the CLI binary per call. The canonical observe-act workflow is: ad_init → ad_adapter_create[_with_session] → ad_snapshot → parse @e refs → ad_execute_by_ref → ad_free_string → ad_adapter_destroy.

    C-ABI bindings over agent-desktop's PlatformAdapter. Consumers (Python ctypes, Swift, Node ffi-napi, Go cgo, C++, Ruby fiddle) link libagent_desktop_ffi.{dylib,so,dll} and call `ad_*` functions directly instead of spawning the CLI binary per call. The canonical observe-act workflow is: ad_init → ad_adapter_create[_with_session] → ad_snapshot → parse @e refs → ad_execute_by_ref → ad_free_string → ad_adapter_destroy.

  • jev-desktopDrive a desktop application from one plain-language goal, or resolve one intent at a time, without ever reading the accessibility tree. Use when automating a desktop app and you want to keep a 150-element JSON tree out of your context. Hand `run.mjs` a whole goal and it observes, decides and acts until the goal is met; hand `act.mjs` a single step when you want to keep the plan yourself. Covers which element, which operation, when to look deeper, and when to stop.

    Drive a desktop application from one plain-language goal, or resolve one intent at a time, without ever reading the accessibility tree. Use when automating a desktop app and you want to keep a 150-element JSON tree out of your context. Hand `run.mjs` a whole goal and it observes, decides and acts until the goal is met; hand `act.mjs` a single step when you want to keep the plan yourself. Covers which element, which operation, when to look deeper, and when to stop.

About

Agent Desktop gives any agent reliable computer use on the desktop. Built with Rust, it sees any app's real UI structure through OS accessibility trees and operates it, refs stay stable and actions stay safe to retry, instead of guessing from pixels

Where this comes from. The public repository lahfir/agent-desktop at commit b8d538c, under its Apache-2.0 license, copied without changes. lahfir is not affiliated with Cheatcode and has not claimed this listing yet. New releases are repackaged after review, never synced automatically. If this is your project, write to admin@cheatcodehq.com to claim or remove it.

Something wrong with this listing, or it did not install? Tell us

Versions

v0.0.0-gb8d538cListed from lahfir/agent-desktop at b8d538c · 2026-09-28