We've known for some time that Apple is investing heavily in AI, with the first fruits likely to appear in iOS 18 this year and to be shown off at WWDC next month. But while a smarter Siri is both welcome and overdue, there's much more to Apple's AI plans than giving Siri superpowers. Making Siri smarter is just the start of Apple's AI ambitions.
The Verge has been doing a deep dive into Apple's AI research, and the results are fascinating. Apple has found a way to put large language models – the same kind of tech that powers ChatGPT – onto mobile devices rather than relying on the cloud, and it's been working on having Siri listen to you constantly so you don't need to command him, it or her when you need something.
One of the most interesting bits of Apple's AI research is called STEER, which is short for Semantic Turn Extension-Expansion Recognition. The goal of STEER is to improve the conversations you have with your personal digital assistant, enabling it to understand follow-up questions without you having to restate things you've already said or asked and to make sense of more ambiguous queries. It would mean Siri would have a more human-like understanding, engaging in conversations if it can't immediately work out what you're trying to do or find out.
Automated image editing and understanding everything your iPhone or headset sees
There's much more to this than Siri, though. The papers The Verge has reviewed talk about using AI to make sense of all the biometric data collated by wearables such as your Apple Watch and future AirPods, and to use AI to help with the creation and refinement of everything from memoji avatars to more serious art. One tool, MGIE, would enable you to edit images by saying to Siri "make the sky more blue" or "add some rocks". That one is still some way away, however: initial results have been encouraging but the tech isn't quite there yet.
Apple might even enable you to remix music: it's been experimenting with AI separation of voices from instruments in recorded music.
Of all the AI experiments, the most interesting one so far seems to be Ferret. Its job is to understand whatever it is you're doing in order to help you navigate apps, describe what you're pointing your iPhone camera at, interpret and give feedback on what you can see through your Vision Pro and so on.
Right now I'd settle for a Siri that doesn't completely mishear simple instructions. But there's clearly tons of potential here, and we should start seeing some of it delivered when Apple takes the wraps of iOS 18's AI improvements at WWDC next month.
