- Microsoft is seriously expanding the powers of the Copilot app – for all Windows 11 PCs
- Copilot Voice will provide a new method of powering up AI with voice input, supported by a global rollout of Copilot Vision
- Broader AI powers are coming, including Copilot Actions that allow AI to interact with the files on your PC
Microsoft teased something big happening today, and now we’ve heard all about its plans to make voice interaction a bigger part of the Windows 11 experience — and how AI functionality will be supercharged in the operating system.
The expectation was that this announcement would be focused on AI, and indeed it is, but the twist is that this isn’t just for Copilot+ PCs. Microsoft promises a “new wave of updates that turns every Windows 11 PC into an AI PC – with Copilot at the center of it all”.
As Microsoft’s Yusuf Mehdi, VP and Consumer Chief Marketing Officer, makes clear in a blog post, new voice commands are coming to Windows 11 in the form of Copilot Voice, supported by a wider global rollout of Copilot Vision – and the ability for the Copilot app to take more actions on your behalf.
Let’s start with the voice element first – all you have to do is say “Hey Copilot” (and definitely not “Hey Cortana”) to invoke the AI. The Copilot microphone appears on your screen and you can issue voice commands or ask the AI a query. Saying “Goodbye” or closing the panel automatically ends the conversation (with audible chimes confirming both the start of a session and the end).
I should note that you have to sign up for this voice feature on the Windows 11 desktop (by enabling it in the Copilot app settings).
Behind Copilot Voice we have the impending global rollout of Copilot Vision, which until now has only been available in the US.
The Vision feature – again, given your opt-in permission – analyzes the contents of your screen and can help guide you on how to use apps or accomplish certain tasks in Windows 11. It can even help troubleshoot when things have gone wrong in the OS, or provide tips in a game. All of this can be facilitated with your voice as Copilot Voice and Vision work together.
All well and good, but what does AI actually do for you in Windows 11? Microsoft further explains that it has some nifty new powers on the way that will be tested soon (preview releases).
The new AI powers
Look at
First of all, the taskbar in Windows 11 will get an Ask Copilot bar, complete with one-click access to Copilot Voice and Vision via their icons (glasses and a microphone) if you enable this. We’re also promised faster and better Windows search with this new functionality (it’s about time – and it especially doesn’t involve giving Copilot access to your files, nothing’s changed in that regard under the hood).
Microsoft’s big move with AI (and agents) coming to testing is the introduction of Copilot Actions to the files on your PC.
If Copilot Actions rings a bell, it’s because this is something Microsoft has previously introduced to the web, meaning AI can go online to help you complete tasks like making a hotel booking or buying tickets to a concert.
In a new ‘experimental mode’ in Copilot Labs for testers, Microsoft will allow AI actions to be performed on files on the local drive in Windows 11. This means you can have AI sort through your latest vacation photos, for example, or pull some specific information from a document.
Just describe what you want to do (using your voice if you prefer) and the AI agent will “attempt to complete it by interacting with desktop and web applications,” Microsoft informs us.
You can leave the AI to it and jump back to check its progress whenever you want. The potential for AI to make mistakes is always there, of course, and Microsoft stresses that “testing this experience in the real world is critical” because of this – and that it will start with a “narrow set of use cases” in a slow and cautious manner.
Microsoft stresses that security will clearly be a key consideration here, defending against new threats such a system will introduce, such as cross-prompt injection attacks that can override instructions to AI agents.
Microsoft says it’s establishing a “set of robust security and privacy principles that must be met to make use of new agent features in Windows,” and that includes limiting agent rights and permissions (to files you allow AI to work with).
In addition to Copilot Actions, Microsoft has a new general purpose AI agent for Windows 11 called Manus that can handle complex tasks, with the software giant explaining that it can create a website for you. Using the Manus AI action in File Explorer, you can gather a set of documents and images into a folder and have Manus build a website based on that material – which sounds seriously impressive.
Microsoft is also bringing in more Copilot connections to link services like Gmail to the Copilot app, so you can, for example, ask the AI about your upcoming appointments logged with Google. The Copilot app will also be able to help tweak your PC settings, as Microsoft has already revealed.
Analysis: A big step forward for AI
All of this is in the works, of course, and not even in testing yet – so much of what we’re talking about here may be some way off.
In particular, the idea of unleashing Copilot to work with your files on the desktop, which Microsoft needs to be very careful about implementing, as noted on the security front. Don’t underestimate what Microsoft is planning here. The idea behind Copilot Actions is that it’s an AI agent that takes on tasks you give it and completes them by “interacting with your apps and files, using vision and advanced reasoning to click, type and scroll like a human would.”
This is partly what Microsoft means by using your mouse and keyboard much less. It’s not just about giving voice commands in Windows 11 or voice queries to the Copilot app, it’s also about the AI actually doing things for you in the operating system: Voice, Vision and Actions.
However, Microsoft certainly emphasizes how important voice input is, noting that the more detail and context you share in an AI query to Copilot, the better the response – but that typing it all out is a pain.
Mehdi notes that because of this, when people use voice, they “engage with Copilot twice as much as when they use text,” and the new wake word (Hey Copilot) is a way to make interacting with Copilot even easier on the desktop.
Mehdi emphasizes, “We believe this shift to conversational input will be as transformative as the mouse and keyboard in unlocking new possibilities on the PC for the widest set of people.”
It’s certainly clear that this is a big moment for Windows 11. So far, when it comes to AI, much of the criticism has been that the capabilities therein are underwhelming – but the powers covered here are more far-reaching and potentially impressive. Like the range of capabilities coming to the Copilot app broadly for all PCs, not just Copilot+ laptops.
