The age of the AI PC is here and Microsoft has loads of new Windows features to prove it.
At its annual Build conference, Microsoft’s CEO, Satya Nadella, took the wraps off of all sorts of new and improved AI capabilities that are about to find their way onto a new generation of Copilot+ PCs. Want more ChatGPT? You got it. How about a “second brain” that remembers everything you do on your computer? Turns out you’re in luck.
There’s a deluge of updates to sift through, but we’ve compiled a few that — if you’re leaning into AI as hard as Microsoft is — you pretty much need to know.
8. Recall
One of the most notable upgrades on the new Copilot+ PCs is called Recall — a feature that watches everything you do and lets you search through your history using natural language prompts.
Think of it like a browser history for all your apps, only it takes screenshots, allowing you to ask things like, “Who shared that meme on my Facebook feed?” and have your AI assistant find the exact moment you’re referring to. Recall also runs directly on your device, meaning that your private data doesn’t ever have to go to the cloud.
7. GPT-4o Built Into Windows
Remember those cool demos OpenAI showed off a couple of weeks ago with the flirtatious Scarlett Johansson voice? That same “voice mode” will be central to the new Windows experience. You will soon be able to talk to your PC, and it will understand your words, your tone, the ambient sounds, and even the contents of your screen.
So, whether you are playing Minecraft or attending an HR-mandated web conference (or both), you will be able to enjoy the constant companionship of your AI girlfriend… er, assistant.
6. Small Language Models (SLMs)
Nadella revealed three new “tiny” models during his keynote: Phi-3-small, Phi-3-medium, and Phi-3-vision, which, building on their predecessors, have finally reached a tipping point where they are smart and efficient enough to be used in desktop apps.
What does this mean? Well, to us nerds who have been playing with AI for a while now, maybe not much. But there is a huge market of people who still don’t know how to navigate ChatGPT, and small language models (because they’re more streamlined and can be deployed quicker for more specific purposes) could be just the thing to get more people in on the AI craze.
5. Copilot Runtime
Copilot Runtime is a brand new system-level code library for developers to use, and it’s poised to shake up AI-powered software development in much the same way Win32 revolutionized Windows development back in 1993. Before that time, Windows apps were confusing to build and maintain. Win32 provided a consistent platform across all Windows machines, which made developers’ lives a lot easier, and consequently unleashed a tidal wave of new apps that could do more complex and useful things.
So, brace yourselves. If you thought “AI” was a buzzword in 2024, just wait until it’s used in everything from File Explorer to Sticky Notes.
4. AI Agents
Copilot Studio lets you build complex workflows with intelligent agents. Businesses of all sizes can use those smart helpers to do loads of things: periodically check databases, compose personalized sales emails, respond to customer support requests, and even improve based on feedback.
If you were worried about AI taking jobs, well... this is exactly the type of feature you were afraid of.
3. Cocreator for MS Paint
Cocreator is essentially an add-on to Microsoft Paint, but it introduces some generative AI that could be powerful enough to rival heavyweight art software. You can do loads of AI-assisted stuff like scribble a rough outline and use AI to generate a picture based on your outline. You can even copy the generated art onto your scribbling canvas, and repeat.
It sounds simple, but that’s the beauty of it. Microsoft wants Cocreator to be a force multiplier for human creativity. It’s hard to say without using it ourselves, but this demo looks pretty convincing.
2. Real-time filtering and captioning
Imagine having professional AI filters for all your video-calling apps without having to install any extra software. The new Copilot+ PCs do this automatically thanks to their new neural processing unit (NPU) integrated into Qualcomm’s Snapdragon Elite X and Plus chipsets. Ditto for audio captioning. All audio data that enters the device gets passed through the NPU, allowing you to turn on captions in any language.
This update is long overdue, and hopefully, we can finally be rid of the countless startups pitching you this same functionality for $5 a month.
1. Extensions for Copilot 365
Lest you thought Microsoft was trying to monopolize every AI vertical, Nadella announced Copilot Extensions, which allow customers to provide Copilot with external datasets and tools — similar to how ChatGPT Pro customers can do this when building custom GPTs.
This approach seems like a natural rebuttal to Apple, which traditionally employs a “walled garden” strategy and needs the EU to babysit it just to release a product with USB-C. Microsoft is diving into AI with an open-minded attitude, allowing customers to build products on whatever combination of platforms they prefer. Thank goodness.
