- Microsoft's AI assistant can now see in the Copilot Vision preview
- Now out in a limited beta for a select few, Copilot can browse the web in Edge
- It's entirely opt-in, but you'll be able to converse with Copilot about what you're seeing
Similar to how Windows Recall will give AI access to what’s on your screen, Microsoft’s rolling out a similar experience for Copilot in a limited preview. Originally teased alongside the redesign of Microsoft’s AI assistant, Copilot Vision is now rolling out as an early preview for a select few in the United States.
As the name suggests, it will let Copilot see what’s on your screen, specifically within Microsoft’s Edge browser, so it can essentially browse the web with you. This way, you can hit it with topical questions about what you’re seeing – be it an illustration, a video, type-based text, or even handwriting. And no, you won't need to type these out, but you can converse with Copilot with just your voice.
In an announcement post published by Microsoft, Copilot Vision is described as: “When you choose to enable Copilot Vision, it sees the page you’re on, it reads along with you, and you can talk through the problem you’re facing together. Browsing no longer needs to be a lonely experience with just you and all your tabs.”
It seems like it’s almost being pitched as a companion, so if you’re browsing a shopping website like Amazon or Zappos, you could potentially ask for suggestions about what you’re looking at. Moreover, though, if you're browsing restaurant reviews, you could ask if any mention a specific event – like good for anniversaries – or ask for other recommendations. Seeing how it’s built directly into Edge – if you opt-in – Microsoft seemingly hopes that it’ll be called upon quite a bit and prove to be useful.
Microsoft is also sharing three of Copilot Vision being used for shopping, planning a trip to a museum, and gaming. The latter is almost similar to Game Assist in Edge, which reduces the need to alt-tab.
Suppose you’re concerned about privacy with Copilot Vision, which may be similar to Recall, which finally launched in November in beta. In that case, you'll be happy that Microsoft is taking a better approach with this new AI feature. It’s entirely opt-in and is only currently available via Copilot Labs to a limited number of Copilot Pro subscribers in the United States.
Furthermore, in its current build, Copilot Vision is only available for a select set of websites, and it seems Microsoft will only expand on that when it deems it ready to do so. Slow and steady will be the approach for adding more subscribers and expanding the sites it works with; Microsoft explains: “We’re taking our time to gather feedback, iterate on the technology, and will expand access to more Pro subscribers and more websites over time.”
If you’re in the early beta of Copilot Vision, another win is that after the session, the associated data and the context it’s used within will be deleted. So that information does not stick around Copilot, only the response to the tools “are logged to improve our safety systems,” says Microsoft.
If you’re keen to give it a go and are a Copilot Pro subscriber in the United States, you can check via Copilot Labs to see if Copilot Vision for Edge is available as a preview for you.
