- New “Computer Use” feature to Copilot Studio gives AI agents more tools
- Agents can now interact directly with apps or sites
- This can help with data entry, analysis, research and more
Microsoft gives its Copilot Studio AI platform more autonomy to access sites and apps directly – without any human interaction.
The company has revealed a new “computer use” feature for Copilot Studio, which allows the platform to directly interact with desktop apps and sites, such as clicking buttons, selecting menus and even writing in fields on the screen.
This means that companies will be able to build AI agents with a much greater understanding of such systems (essentially everything with a graphic user interface) and treat them as if they were tools, enabling improved functionality everywhere.
“Computer use”
“Copilot Studio is the end-to-end-agent platform designed to help organizations achieve their AI and operational goals,” wrote Charles Lamanna, the company’s vice president of Microsoft’s Business & Industry Copilot, in a blog post announcing the news.
“We will allow you to streamline processes, improve productivity and drive innovation.”
Computer use allows agents to handle tasks even when there is no API available to connect directly to the system and automatically adapt to any changes to the app or site.
Developers can use the tool to build agents across both desktop and browser applications, with Edge, Chrome and Firefox included everyone from the launch.
It works in real time using built -in reasoning tools to prepare solutions to problems independently so that development can continue continuously.
Lamanna made several possible scenarios where computer use could prove particularly convenient for developers, including automation of input of large amounts of data from different sources of a centralized system that performs widespread market survey from different online sources and treatment invoices much faster and more accurate than before.
When the tool runs on Microsoft host infrastructure, users do not need to manage their own servers, and this includes Copilot Studio’s existing safety measures and management frameworks to ensure compliance with industrial standards.
The launch, which will see more attention to Microsoft Build in May 2025, follows a similar message of “actions” in the commercial version of Copilot recently. This system allows for background completion of tasks such as making restaurant reservations, booking event tickets and buying goods from online stores.
