Awesome MCP › Desktop Automation
agent-desktop gives an AI agent computer use over native desktop applications on macOS, Windows and Linux without screenshots or pixel coordinates. It reads the operating system accessibility tree (the AX API on macOS, UI Automation on Windows, AT-SPI on Linux) through a Rust core and a per-platform adapter, and returns each element as JSON with a stable ref an agent can act on by clicking, typing, scrolling or toggling, then re-reading to confirm the result; the action loop itself lives in the calling agent, not in agent-desktop. It ships as a CLI, a C library for in-process use, and an MCP server (`agent-desktop mcp`) that exposes all of its operational commands as MCP tools over stdio, so any MCP client can snapshot and drive desktop apps the way a human would, including multi-window apps, system dialogs, menus and notifications. Built-in guides teach an agent the ref-based loop, retry rules and platform differences, and an installable skill package is available through a third-party skill registry. It handles ambiguous targets when several windows are open, stale refs after the UI changes, dense interfaces through skeleton and root-scoped snapshots, and keeps secrets out of process arguments by accepting piped input for typing and setting values. Built by Vercel Labs, it targets developers building computer-use agents that need reliable, non-visual control of real desktop software rather than vision-based screen automation.
https://github.com/vercel-labs/agent-desktop
UI-TARS Desktop is a cross-platform GUI Agent application that uses a vision-language model to enable natural language control of computers with precise visual recognition and real-time feedback, integrating Model Context Protocol tools for advanced multimodal AI-driven automation.
Docker-Android packages Android emulators in Docker with VNC access, ADB control and log sharing for app testing, plus a beta MCP server image that exposes emulator control to AI agents.
Windows-MCP is a lightweight, open-source MCP server that enables seamless integration and automation between AI agents using LLMs and the Windows operating system, supporting a wide range of UI interactions and system controls.
Desktop Commander MCP is an MCP server for Claude that integrates AI with terminal control, file system search, and file editing capabilities, enabling advanced desktop automation and development workflows.
Peekaboo is a macOS-only MCP server and CLI tool that enables AI agents to capture and analyze screenshots with AI vision models for visual question answering and automation.
ios-simulator-mcp is an MCP server enabling AI-driven interaction, control, and inspection of iOS simulators for automated UI testing and quality assurance.
Terminator is an open-source automation tool for Windows that leverages the Model Context Protocol to enable AI-driven desktop and browser automation with high reliability and speed.
CodeRunner is an MCP server that securely executes AI-generated code locally on macOS using Apple's container technology, enabling private and efficient processing of local files with remote LLMs.