I couldn’t sleep last night. No, I wasn’t up watching Netflix or playing Elden Ring, and no, I wasn’t doomscrolling on Instagram or TikTok. I was busy using the new Visual Look Up feature that Apple announced for iOS 16 at WWDC 2022 to remove the backgrounds from pictures and then copy the cutouts for sharing on Messages or Twitter.
Yes, the feature I’m talking about is the one involving Apple cutting a French Bulldog out of an image and then copying it right into Messages; the one that everyone is freaking out over.
We all know that you can’t always believe what you see in a staged tech demo. But in the case of arguably the most mind-blowing feature Apple announced during its almost two-hour developer keynote, this Photoshop-like background removal magic really works — and it’s mighty impressive!
Apple just calls the feature an expansion to Visual Look Up, its machine learning-based object recognition for identifying stuff within photos that was introduced in iOS 15. From Apple’s iOS 16 announcement press release:
Visual Look Up takes photos further by introducing a new feature that allows users to tap and hold on the subject of an image to lift it from the background and place it in apps like Messages. Visual Look Up also expands to recognize birds, insects, and statues.
After installing the iOS 16 developer beta last night (iOS public beta is coming in July, and the public release this fall), I immediately started poking around Apple’s new mobile operating system. As expected, the developer beta is still rough in some places — battery life is concerning and there’s some bugginess — but the core features work for the most part. I spent an unhealthy amount of time messing with the fonts and widgets for my lock screen and then making new ones for different Focus modes.
But the Visual Look Up cutout feature is truly breaking my brain — and the internet’s. No joke, check it out for yourself:
And this is not a Portrait mode photo where the background is better isolated from the foreground subject. This is just a regular photo I plucked out of my iPhone 13 Pro Max’s Photos app.
The results have been impressing even the harshest Apple critics, including Quinn Nelson, AKA SnazzyQ, on YouTube:
That’s not even the best part. Visual Look Up works on any photo and isn’t limited to pictures taken with an iPhone. Here are some high-resolution photos I imported directly from my full-frame Sony A7R3 camera. These 42-megapixel photos clock in around 23-25MB and my iPhone 13 Pro Max just cut the subjects out instantly. There was no discernible delay in removing the background; just tap the subject, and if it’s recognized, you’ll see an animation isolate it from the background and then you can copy or drag-and-drop it into another app.
Visual Look Up does have its limitations; it can’t identify everything. Naturally, many people on Twitter immediately thought about how this could be disastrous for removing backgrounds for — okay, fine, I’ll just say it — d*ck pics. But Visual Look Up won’t work on it. I’m not saying Apple’s object recognition is bulletproof and an eggplant won’t get through, but I wouldn’t bet on it being the future of sexting. (For journalism, I did try on a personal d*ck pic and, no, it didn’t work.)
How useful is this feature in practice? That’s going to be up to you. I could see it being a super quick and dirty way to create a YouTube thumbnail. I threw this very YouTube-ish photo from a recent vacation in Photoshop on my MacBook Pro after copying it (with iOS’s Universal Clipboard, of course) from my iPhone and whipped this thumbnail in under a minute. It would have taken much longer to remove the background with Photoshop.
Above, the original and the cutout with Visual Look Up. And below, my YouTube-esque thumbnail. Lol.
Maybe this feature is as silly as Memoji. But damn, is it fun as hell.
