It's a nightmare scenario for Microsoft. The headlining feature of its new Copilot+ PC initiative, which is supposed to drive millions of PC sales over the next couple of years, is under significant fire for being what many say is a major breach of privacy and security on Windows. That feature in question is Windows Recall, a new AI tool designed to remember everything you do on Windows.
On paper, it's a cool idea. As CEO Satya Nadella described it, Windows now has a photographic memory that uses AI to triage and index everything you've ever done on your computer, enabling you to semantically search for things you've seen using natural language. It's a new and improved way of finding things on Windows, and in our testing of the feature, it works really well.
However, for a tool like this to be feasible, trust between the user and the platform is required, a luxury Microsoft doesn't appear to have with its Windows user base right now. Recall operates by taking and storing captures of your screen every few seconds to build a database that the user can later search, with screenshots as visual aids. That database is stored locally on your device and never uploaded to the cloud.
In fact, Microsoft goes so far as to promise that it cannot see the data collected by Windows Recall, that it can't train any of its AI models on your data, and that it definitely can't sell that data to advertisers. All of this is true, but that doesn't mean people believe Microsoft when it says these things. In fact, many have jumped to the conclusion that even if it's true today, it won't be true in the future.
Microsoft eroded user trust on Windows with bad practices that are now biting them back
Microsoft is fully aware that the concept of Windows Recall sounds creepy. I know that the company spent a lot of time internally figuring out how to communicate this feature to the world, but it turns out there's no good way to communicate something like this when your users don't trust you.
Users are describing the feature as literal spyware or malware, and droves of people are proclaiming they will proudly switch to Linux or Mac in the wake of it. Microsoft simply doesn't enjoy the same benefit of the doubt that other tech giants like Apple may have.
Had Apple announced a feature like Recall, there would have been much less backlash, as Apple has done a great job building loyalty and trust with its users, prioritizing polished software experiences, and positioning privacy as a high-level concern for the company.
Microsoft, on the other hand, has done a lot to degrade the Windows user experience over the last few years. Everything from obtrusive advertisements to full-screen popups, ignoring app defaults, forcing a Microsoft Account, and more have eroded the trust relationship between Windows users and Microsoft.
Here is a list of just some of the practices Microsoft has employed on Windows that users do not like:
- Requiring a Microsoft Account to setup Windows 11
- Advertising in system-level interfaces like Start, Widgets, Settings, and File Explorer
- Refusing to acknowledge user app defaults like browsers
- Shoehorning MSN into Windows
- Replacing native Windows apps with slower, uglier web apps
- Full-screen prompts pushing you to setup OneDrive or Microsoft 365
While Microsoft has now addressed some of these issues (thanks to the EU forcing its hand), the damage has already been done. It's clear that Microsoft and Apple prioritize their OS platforms in very different ways. Apple ensures its operating systems are clean, polished, and without bloat. Microsoft, on the other hand, views Windows as a platform that should be making money from its users, filling it with ads and bloatware where it can, sometimes at the expense of user choice and OS polish.
It doesn't bode well for a feature like Windows Recall, which relies on complete trust between the user and the platform. If Microsoft considers Windows quality assurance an afterthought, how can it expect people to trust a feature like Windows Recall?
It's no surprise that users are already assuming that Microsoft will eventually end up collecting that data and using it to shape advertisements for you. That really would be a huge invasion of privacy, and people fully expect Microsoft to do it, and I can't help but feel like it's those bad Windows practices that have led people to this conclusion.
The concept of Windows Recall comes with risks on an open platform like Windows
With Windows being an open platform, a built-in tool designed to collect data about everything you've ever seen is a recipe for disaster. Unlike iOS, iPadOS, and even Android, users and apps have complete access to the entire OS.
While some mitigations exist to ensure users and apps don't mess around with system files on Windows, these can be bypassed. It has been discovered that Windows Recall seemingly stores its data unencrypted, which is a huge security concern for many people. This means that third-party apps could reach in and grab that data to learn everything about you.
Many immediately point to malware, which is certainly a concern. However, even third-party apps that you trust could potentially reach in there to learn about you. Your favorite web browser, video editor, or music streaming app of choice could release an update that begins scraping data from Windows Recall and uploading it to its own backend.
Do you trust Microsoft with a feature like Windows Recall?June 4, 2024
That would, of course, be a huge invasion of privacy, but it would technically be possible, and that's thanks to Windows' open nature and the reported lack of security around stored Windows Recall data. Even your employer could build a tool that's preloaded onto your work laptop that's designed to scrape that data. It's all quite concerning.
The fact that Windows is an open platform means anyone can do anything if they want to. That's a blessing and a curse, and it means an app like Recall, in an unencrypted state, doesn't really work on Windows currently.
On iOS and iPadOS, users are locked out of important system files, and app developers are sandboxed and have no ability to read or modify system files outside of documented APIs. So, if iPadOS had its own version of Recall, that data could be stored unencrypted and still be safe from third-party attackers. It's the same story on Android.
Windows enjoys no such luxury, so Microsoft needs to put extra effort into ensuring Windows Recall is secure. It needs to ensure that only the Windows Recall app can read and understand that data. If that data is unencrypted, anyone can read it. Everything it collects is reportedly stored in a plaintext SQLite database, making it easy to parse information from it.
People won't trust Windows Recall regardless of how the data is stored locally.
There's quite a bit of hysteria over the discovery that Windows Recall stores data unencrypted. It's important to remember that Windows Recall isn't actually out yet, so Microsoft could update Windows Recall before launch to address this, or perhaps even potentially delay the feature to ensure security.
Assuming Microsoft does eventually fix these security concerns, I don't think that's going to change much for people. Many have already assumed the worst; that Windows Recall will eventually be used as a means to sell data to advertisers and train AI models, and that if it's not happening today, it's only a matter of time.
Many are even convinced that Microsoft will attempt to enable Windows Recall on PCs that have chosen not to use it via updates down the line. That's just the sort of company people think Microsoft is like. I think this stems from the fact that people don't understand how Windows Recall works.
Microsoft has built a number of safety features into Windows Recall to ensure that the service can't run secretly in the background. When Windows Recall is enabled, it places a permanent visual indicator icon on the Taskbar to let the user know that Windows Recall is capturing data. This icon cannot be hidden or moved.
It's also important to remember that Microsoft has no monetary incentive to force people to use Windows Recall. The data it collects is of no value to Microsoft, as it can't see any of it. Windows Recall is a selling point for new hardware, built as a means to improve user productivity, not sell advertising. But that's hard for people to believe, and perhaps that's rightly so.
With that in mind, there would be no reason for Microsoft to automatically enable Windows Recall in an update down the line. If it does happen, the user will be able to instantly tell thanks to that that visual indicator and turn it off again.
Microsoft chose to keep Windows Recall a secret, and that hasn't helped things.
Some insider baseball here, but for some reason Microsoft was overly secretive about Windows Recall during development. It didn't want anyone to know about it. If you wanted to test the feature internally, you needed to be accepted into a tented program first, which I understand wasn't easy to get into. When I leaked the existence of Windows Recall (AI Explorer) and Copilot+ PCs (CADMUS) back in December 2023, I heard from sources that the company wasn't pleased.
Microsoft has the Windows Insider Program, yet to maintain secrecy, it chose not to test this feature openly. I can't think of a single feature that would have benefitted from public testing more than Windows Recall. This is the kind of feature that needs to be built in the open so that users can learn to trust you with it.
Had it been tested openly, these security concerns would have definitely been pointed out well ahead of general availability, and likely fixed before mass hysteria could ensue. Of course, the true reason Windows Recall wasn't tested openly was because the company wanted to make it exclusive to new Copilot+ PCs, and you can't really do that if you're testing the feature on existing PCs where it works quite well.
Microsoft also wanted to keep Windows Recall a secret so it could have a big reveal on May 20. Except, it wasn't really much of a big reveal. Many of us in the tech press already knew it was coming, even without being briefed on the feature ahead of time.
You can't have Windows Recall anyway
Ultimately, you can't have Windows Recall anyway. It's a feature reserved exclusively for new PCs shipping under the Copilot+ umbrella, which means if you want to use it, you'll have to buy a new device with a neural processing unit (NPU) that can output 40 TOPS of power first. Your existing Windows 11 PC is not eligible to run Windows Recall and very likely never will be.
That's good news for those who don't want Windows Recall, as it means there's nothing you need to do to avoid it. Just keep using your existing device, and you should be safe from the all-seeing eye that is Windows Recall.
If you do happen to acquire a Copilot+ PC, you can choose not to use Windows Recall. There's some discourse around the feature being potentially enabled by default, but I'm told via sources that this is being reconsidered. I suspect Microsoft will give the user a choice to turn Windows Recall on or off during the setup process on Copilot+ PCs.
If it doesn't, that's just another bad Windows practice to add to the list.
It's a shame because Windows Recall is really good.
I think it's fair to say that a feature can be both insecure and good at what it does. That's Windows Recall for me right now. I've been testing it over the last week, and it's a great tool for finding things you only half remember, or have deleted, accidentally or otherwise.
In fact, it came in clutch for this very article. I had deleted a paragraph earlier in the day as I didn't think it was relevant, only later to realize I could reuse that paragraph elsewhere in the story. On a normal PC, that paragraph is gone, and I'd have to rewrite it from scratch. But with Windows Recall, I was able to go back to that point in time when I originally wrote it, copy it from there, and paste it back into my CMS.
The ability to search for things using natural language is genuinely great, and it works really well for a 1.0 product. But there's a very dark cloud hanging over this feature right now, and a lot of privacy conscious people are simply not going to be able to subscribe to the idea of Windows Recall in its current form.
Microsoft told me at the event where Windows Recall was announced that it plans to rapidly update this feature now that it's shipping. I suspect this means we will see new features and capabilities added to Windows Recall over the coming months, along with updates to ensure the data it collects is secure on the device.
What happens now?
So what happens now? Will Microsoft delay Windows Recall, or maybe even cancel it? Will people ever be able to trust it?
I don't think Microsoft will delay or cancel it. I think Windows Recall will ship on June 18 as was originally announced, with the promise of an update coming shortly after to fix the security concerns people have with it.
With Windows Recall being exclusive to Copilot+ PCs, I imagine the number of PCs that could even be targeted with Recall malware over the next few months will be in the low thousands, which gives Microsoft some time to update Windows Recall with better security before more people adopt Copilot+ devices.
I definitely think Microsoft will make Windows Recall an optional feature that you can choose to enable or disable during Windows setup. It would be a really bad look to have it as opt-out rather than opt-in at this current time.
Windows Recall is a feature that ships as part of Windows 11 version 24H2, which technically won't be generally available until this fall for existing PCs. Even when that happens, Windows Recall won't be enabled on your existing device.
