- Microsoft expands Copilot’s vision features
- Soon it will be able to look and interact with Windows as a whole and apps
- You can even use it to learn a Photoshop skill
It’s a big day for Microsoft – it’s the 50th anniversary of one of the most effective companies ever, and it’s the day that copilot is led to the next level. Microsoft’s AI assistant, Copilot, becomes more personal and has a number of new features.
However, Copilot Vision’s expansion is worth paying attention to as it will provide one of the most meaningful intelligent boosts for Windows. We have already seen that Copilot can browse the Internet with you, share what you see with selected partners and give a helping hand.
With the permission of the new copilot for the Windows app, you can let AI see your screen when you are in the basic surgery system and in selected applications. As with the Internet, it mixes multimodal AI with agent capacities, and thanks to its ability to see and understand your screen, it will also be able to interact with applications.
Help on screen
This comes in the form of highlighting options on your screen and even adds an extra marker, but also goes through the required steps through speech. It’s pretty cool.
In a shared demo, we saw copilot give instructions for an editing in Photoshop, essentially go through how to use the tool. This can be a big game election, as it is not only an AI for quick facts; You can integrate copilot into your workflow and really use it how to see fit.
Of course, we need to see exactly how well this is doing, but together with a more personal copilot, the addition of vision for Windows is a big step. It will be opt-in at launch and Microsoft is taking a slow and steady approach.
Vision within the COPILOT for the Windows app is starting to roll out for Windows Insiders next week, and Microsoft says it will be available “more wide afterwards.” It’s not super -specific, so it can be a while before you can actually give it a try if you’re not a Windows insider.
For Android and iOS too
Alongside ‘Deep Research’ features and a new ‘shopping’ feature for copilot, its vision features are also expanding to your Android or iOS phone. Via the Copilot app you may be able to shoot up your camera and essentially let AI have the view for these devices. This means you can point it to a dog and ask for the breed or even a store and ask for reviews of it. You can also upload photos from your camera roll.
This improved visual ability for copilot on mobile is also available right now, just be sure to update the app.
This Mobile Change also brings Copilot into the same game field as Google’s capacities with Astra on Android. The changes to Windows represent a big jump in intelligence that can change the way you use apps on the operating system.
I can’t wait to give Vision a go with a complex task, I try to figure out how to perform in Photoshop or when I need to create a complex formula in Excel.
However, all these changes show Microsoft’s ambition with AI, specifically to get more people to use copilot. None of these are category-defining or brand new in themselves, but they definitely make copilot much more convincing along with Google Gemini or Chatgpt-It has the brain after all, so why not more features?
