Awesome MCPDesktop Automation

bytedance/UI-TARS-desktop

⭐ 38960 TypeScript added to this list on 2025-03-20 repository created 2025-01-19

UI-TARS Desktop is a cross-platform desktop application that serves as a native GUI Agent leveraging the UI-TARS model. It is part of the TARS multimodal AI Agent stack developed by ByteDance, which also includes Agent TARS. The application enables natural language control of computers through a vision-language model that combines precise visual recognition with real-time feedback. This allows users to interact with their computers and browsers in a more intuitive and human-like manner, using natural language commands to perform complex tasks. UI-TARS Desktop supports both local and remote operations, including remote computer and browser control, which are provided as free features requiring no configuration. The project aims to enhance user experience by integrating advanced multimodal large language models (LLMs) and seamless integration with real-world Model Context Protocol (MCP) tools. The application is designed to improve workflow efficiency by enabling AI-driven GUI automation and task completion that mimics human behavior. The project also offers an SDK for building GUI automation agents, facilitating further development and customization. UI-TARS Desktop is continuously updated with new features and improvements, such as redesigned agent UI and support for advanced UI-TARS models like UI-TARS-1.5, which enhance performance and control precision. The project is well-documented with quick start guides, deployment instructions, and community support through Discord, Twitter, and other platforms. Overall, UI-TARS Desktop represents a significant advancement in AI-powered GUI agents, providing a powerful tool for natural language-based computer interaction and automation, aligned with the Model Context Protocol framework.

https://github.com/bytedance/UI-TARS-desktop

agentagent-tarsai-powered-guiautomated-gui-interactionautomationbrowser-operationbrowser-usecommand-line-integrationcomputer-controlcomputer-interactioncomputer-usecross-platformelectronfile-system-integrationgui-agentgui-agentsgui-automationkeyboard-controllarge-language-modelsllmlocal-operatorlocal-processingmcpmcp-servermodel-context-protocolmouse-controlmultimodal-ai-agentnatural-language-controlreal-time-feedbackremote-browser-operatorremote-computer-operatorscreenshotsdkui-tarsui-tars-desktopvisionvision-language-modelvisual-recognitionvitevlm

Also in Desktop Automation

budtmo/docker-android

Docker-Android packages Android emulators in Docker with VNC access, ADB control and log sharing for app testing, plus a beta MCP server image that exposes emulator control to AI agents.

wonderwhy-er/DesktopCommanderMCP

Desktop Commander MCP is an MCP server for Claude that integrates AI with terminal control, file system search, and file editing capabilities, enabling advanced desktop automation and development workflows.

CursorTouch/Windows-MCP

Windows-MCP is a lightweight, open-source MCP server that enables seamless integration and automation between AI agents using LLMs and the Windows operating system, supporting a wide range of UI interactions and system controls.

steipete/Peekaboo

Peekaboo is a macOS-only MCP server and CLI tool that enables AI agents to capture and analyze screenshots with AI vision models for visual question answering and automation.

joshuayoes/ios-simulator-mcp

ios-simulator-mcp is an MCP server enabling AI-driven interaction, control, and inspection of iOS simulators for automated UI testing and quality assurance.

mediar-ai/terminator

Terminator is an open-source automation tool for Windows that leverages the Model Context Protocol to enable AI-driven desktop and browser automation with high reliability and speed.

BandarLabs/coderunner

CodeRunner is an MCP server that securely executes AI-generated code locally on macOS using Apple's container technology, enabling private and efficient processing of local files with remote LLMs.

sbroenne/mcp-server-excel

Windows-only MCP server in C# that drives the real Microsoft Excel application through its COM API, exposing 26 tools and 234 operations for Power Query, DAX, PivotTables, charts, VBA and formatting.