AI is coming to your iPhone, iPad, and Mac this year, and there's an awful lot to cover.
Apple Intelligence differentiates itself from the competition by trying to keep as much on-device as possible while focusing on privacy when data does go to the cloud. And, as leaked a while back, OpenAI has partnered with Apple, too.
Here's everything we just learned about Apple Intelligence on iOS 18, iPadOS 18 and macOS Sequoia.
ChatGPT integration
Arriving late in the presentation, Apple confirmed it's partnering with OpenAI for ChatGPT 4.0 functionality.
Apple says you'll be notified when data needs to be sent off-device, and that ChatGPT 4.0 will be able to process images you share with it, too, letting you ask questions about documents. You'll also be able to have it write to a brief for you.
Apple says "You control when ChatGPT is used and will be asked before any of your information is shared."
"Anyone can access ChatGPT for free, without creating an account. ChatGPT subscribers can connect accounts to access paid features within these experiences."
Siri
Perhaps the biggest beneficiary of the AI features rolled out as part of WWDC 2024 is Siri, which will now be much smarter - and sports a lovely new icon that you can see above.
Siri will now let users move between text and voice requests, with much better awareness of the current context. It'll also be able to remember things you've requested, so that when you chain requests together you won't need to start over from scratch.
Siri's functionality has also changed, so there's no more screen overlay. Instead, it'll highlight your screen with a pinkish glow when activated, and if it can return a request without opening an app, it'll do so with a smaller window that slides in from the top of the screen.
Siri will also let you change a request, so you no longer have to cancel a timer to reset it because you got the increment wrong.
Genmoji
We're seemingly past Animoji, now, as Apple revealed Generative AI emoji it calls Genmoji.
Ensuring you always have an image that you can send for any given situation, these can appear in Sketched, Illustrated, or Animated versions, and you can use the new Image Playground app to create to your heart's content.
They can even use data from Photos as inspiration, but Apple insists this processing is all done on-device. They can be generated right from the keyboard, too, and developers will be able to work with a new Image Playground API.
Photos
Not only has the Photos app been redesigned, but it's also getting natural language support for search - so you can ask for "cat laying on the sofa" and it'll find it for you.
The app will also allow you to edit images with a Siri request like "make this image 'pop' more", while you can now finally use Clean Up to remove an unwanted bystander from your family photo.
You can also create new "Memory Movies" by entering a description. Apple's example is "all the desserts I ate during the summer last year, set to a pop song", which highlighted a whole host of sweet treats and ties in with Apple Music.
Notes
The Notes app is getting AI features for both text and images. For one, you'll be able to solve mathematic equations with on the fly, while also being able to dump text from another app, or generated by Siri, right in there.
For more visual thinkers, they'll be able to convert rough sketches into AI generated art that can pull context from the space around it. You can also generate one from scratch if you're not artistically gifted.
Apple Intelligence will rewrite your emails to fit a different style once you prompt it, letting you amend your text to sound more professional, or convert it to a poem, if that's what you're looking for.
It'll also summarize emails to help you catch up on what's in your inbox more quickly and move priority emails to the top of your inbox automatically. The menu looks a fair bit like Notion's options, which is interesting.
Smart Reply will let you quickly respond to emails via prompts, such as confirming attendance to an event using a simple "Yes" or "No" and having the email written by Apple Intelligence.
Notifications
A new Reduce Interruptions Focus Mode will ensure all of your devices only show priority notifications, such as time-sensitive ones or those from selected contacts.
They appear at the top of your notifications stack, but they're also summarized so you can quickly glance at them to know what's going on.
Phone
You'll be able to capture audio recordings within the Phone app, and these will automatically transcribe and have a summary generated.
And don't worry - everyone on the call will get an alert to say it's being recorded.
Privacy
The "elephant in the room" with AI is Apple's steadfast commitment to privacy, and the company is promising that user data and requests aren't stored. Your data is only used for your requests, and as many tasks as possible are handled on-device.
For anything that requires processing by the cloud, Apple uses Private Cloud Compute, which are server-based Apple Silicon machines to power through complex requests.
Compatability
As was rumored before the event, many of these features will require the latest iPhone 15 Pro's A17 Pro.
For Mac and iPad, you'll need an M-Series chip inside, but it does go right back to M1. You can reasonably expect newer models to handle these complex tasks more efficiently, though.
