GitHub Copilot is moving beyond code and browser-based workflows.
GitHub has started rolling out Computer Use in public preview, giving Copilot the ability to interact directly with desktop applications on Windows and macOS. The feature can read accessible app content and visual context, click controls, enter and edit text, press keys, scroll, drag items and move between applications to complete multi-step workflows.
The result is a significant expansion of what Copilot can automate. Instead of being limited to applications with APIs, command-line tools or MCP integrations, the AI can now work with software that is controlled primarily through a graphical interface.
Copilot can actually operate desktop software
Computer Use is designed for situations where traditional integrations aren’t available.
GitHub says Copilot can use the feature to interact with legacy software and GUI-only applications, allowing it to move information between different programs and complete workflows that would normally require a person to repeatedly click, type and switch windows.
For example, a user could ask Copilot to work through an expense-report process. Instead of simply explaining what needs to be done, Copilot can navigate the relevant application, enter information and move through the workflow.
Other examples include:
- Reviewing information inside an older desktop application
- Updating a presentation
- Entering information into GUI-only software
- Moving information between multiple applications
- Navigating repetitive office workflows
- Reading and summarizing information displayed in desktop apps
This brings Copilot closer to an AI assistant that can use a computer, rather than simply telling the user how to use one.
The feature works across Windows and macOS
GitHub says Computer Use is currently available in public preview through both the GitHub Copilot app and Copilot CLI on Windows and macOS.
Computer Use is disabled by default. Users have to enable it before Copilot can interact with desktop applications.
GitHub has also built approval controls into the experience. Depending on the user’s settings, Copilot can request permission before controlling an application, while users can choose to allow access for a session or always allow a particular application.
On macOS, the feature requires Accessibility and Screen Recording permissions because Copilot needs operating-system access to interact with applications and inspect visual content when necessary.
That permission model will be particularly important as AI agents move from generating information to taking actions on a user’s computer.
Why this matters for everyday AI agents
The biggest limitation for many AI agents has been access.
If an application exposes an API, MCP server or command-line interface, an agent can often connect to it in a structured way. But countless desktop applications still require users to interact with buttons, menus, forms and other graphical controls.
Computer Use gives Copilot another option.
Rather than waiting for every application developer to build an AI integration, Copilot can interact with software through its existing interface.
GitHub itself notes that direct tools such as APIs, MCP servers, terminal commands and dedicated browser tools are generally more predictable when available. Computer Use is intended for the cases where those options don’t exist.
That distinction is important. Computer Use isn’t necessarily replacing traditional integrations; it fills a gap between them.
GitHub Copilot is becoming a broader agent platform
The desktop-control launch comes as GitHub and Microsoft continue expanding Copilot from a coding assistant into a broader agent platform.
The GitHub Copilot app already provides a desktop workspace for agent-driven development, allowing users to work with issues, branches, files, conversations and pull requests from one place. GitHub describes it as a desktop experience built around the software-development lifecycle rather than simply another coding chat window.
GitHub has also been expanding its model lineup. Current Copilot documentation lists models including GPT-6.1 Sol and Claude Sonnet 5.5, among many other OpenAI and Anthropic models.
Recent Copilot releases have also added GPT-6 Sol, GPT-6 Luna and Claude Opus 5.5 to supported plans.
This means Copilot is increasingly becoming a layer where users can choose different models and give agents increasingly broad access to tools and workflows.
Plugins are becoming part of the bigger picture
GitHub is also making its plugin ecosystem easier to manage.
In August, GitHub introduced Agent Plugins 1.0, allowing compatible plugins to package agent skills and MCP servers into a common installable format. Plugins can be discovered through supported marketplaces and used across compatible agent clients, including the Copilot app, Copilot CLI and VS Code.
For organizations, administrators can control which marketplaces and plugins employees are allowed to use. GitHub has also added managed settings that can govern plugins, marketplaces and approval behavior across Copilot clients.
Microsoft is taking a similar approach with its broader Copilot platform.
Its redesigned Copilot experience now brings Home, Code and Autopilot together, with Office applications integrated into the experience and a new plugin registry designed to provide a unified catalog for Microsoft, partner and custom plugins.
Microsoft is turning Copilot into a work platform
The GitHub development is happening alongside a much larger Copilot strategy.
Microsoft’s new Copilot experience is designed to bring chat, delegated work, coding and persistent agents into a single environment. Its Code experience uses technology from GitHub Copilot to let users build applications, dashboards and workflows using natural-language instructions, while Autopilot is designed to continue working on tasks without requiring a constant prompt from the user.
Microsoft is also adding a plugin registry that brings Microsoft, partner and custom-built plugins into one catalog. Administrators can approve and manage those plugins centrally.
Put together, these developments point toward a future where Copilot isn’t simply an AI chatbot sitting beside your applications.
It is increasingly becoming the interface between the user and the software they work with.
Not everyone wants more AI everywhere
The expansion also highlights a growing divide around AI integration.
While some users see computer-controlling agents as a major productivity improvement, others prefer simpler software with fewer AI features and less automation.
There is already a broader pushback against aggressive AI integration in consumer software, with some users seeking ways to disable or remove AI functionality from their operating systems and applications. Recent reporting has also highlighted Microsoft’s efforts to make some Windows AI experiences more optional following criticism of excessive AI integration.
That makes user controls particularly important as Copilot gains the ability to perform actions rather than merely provide answers.
The difference between suggesting an action and taking the action is significant. Approval prompts, permission controls, managed settings and the ability to disable Computer Use give users and organizations a way to define how much control an AI agent actually receives.
The next step is AI that can use almost any app
GitHub’s Computer Use preview represents an important shift in how AI assistants interact with computers.
Copilot can already generate and modify code, work with repositories, manage pull requests and use different AI models. Now it can also operate desktop applications through their graphical interfaces.
That could make previously disconnected software workflows accessible to AI agents without requiring every application to expose a dedicated AI API.
For developers and businesses, the potential is straightforward: less repetitive clicking and more delegated work.
For users who value simplicity and tighter control over their computers, the same development makes permission and privacy settings increasingly important.
Either way, the era of AI assistants that only answer questions is rapidly giving way to agents that can actually do the work on the desktop.
Stay tuned to TheWinCentral.com for further details related to Microsoft, Windows and AI news.
