Take a trip with us back to WWDC 2021 for a moment. Apple's Craig Federighi is just tying a bow on one of the powerful new features built into the then-forthcoming iOS 15 – Look Up – and as if it were nothing, ups the ante by ending the segment with the reveal of Visual Look Up.
Not only can your iPhone (or iPad) pull out text from your photos, but it can also recognize specific dog breeds, plants, famous works of art, books, and even landmarks, and present you with insightful information on whatever it can identify; truly amazing, right?
Since its introduction, Apple has continued to build on Visual Look Up, adding recognition for birds, insects, spiders and statues in iOS 16 and with iOS 17 offering support within videos, as well as the addition of recipe recommendations.
For the gastronomically challenged or culinarily curious, Visual Look Up on iOS 17 now includes recipe recommendations, based on any Instagram-worthy food shots you happen to have snapped. So how do you actually make use of this new feature? Read on to find out.
How to use Visual Look Up to find a food recipe in iOS 17
Beta software, also referred to as a preview, seed, or release candidate, is a version of the software still under development and not yet released to the public. This software is intended only for installation on development devices registered under your Apple Developer Program membership. Registered development devices can be upgraded to future beta releases and the public version of the software. Attempting to install beta software in an unauthorized manner violates Apple policy and could render your device unusable and necessitate an out-of-warranty repair. Make sure to back up your devices before installing beta software and install only on devices and systems you’re prepared to erase if necessary.
Thankfully, Apple hasn't change the way you access the power of Visual Look Up in iOS 17, and so the method is the same as it has been since iOS 15.
- Snap a picture or record a video of a delicious dish, using your iPhone
- Locate and open your Photos app and look for your newly captured food shot
- Provided Visual Look Up recognizes something it can identify in-frame, the information button (the 'i' icon) – positioned between the favorite 'heart' icon and the delete 'trash can' – at the bottom of the screen will change
- In the case of recipe suggestions, it'll change from the 'i' icon within a circle to a knife and fork icon
- Tapping the standard 'i' icon would usually just bring up detailed information about your shot, like where and when the picture was taken, and which of your phone's cameras was used to take it, but when you tap on the knife and fork icon, the card that's then presented will also include an additional field at the top that will say 'Look Up Food'
- Tapping on this will then pull up an additional card with a list of recipe results – suggested recipes based on what dish or ingredient your phone detects in the photo, including direct links to each recommendation
What kind of food images will and won't work with Visual Look Up in iOS 17?
There are, of course, limitations to this new feature. As already touched on, if Visual Look Up doesn't recognize the dish that you've photographed, the knife and fork icon won't appear and you'll just be able to view the standard information card.
What's more, downloaded images from the internet, messages, or other sources aren't supported, it seems for the feature to work at all, the shot needs to be taken on your iPhone.
Does Visual Look Up in iOS 17 recommend the recipe for the specific food photographed?
Not explicitly. Even when the feature does recognize what you've captured as food, it doesn't bring up a recipe for the specific dish it detects but rather recipes for dishes that the subject in your shot might be, along with related dishes.
A great example of this is represented by the official Apple screenshot featured in the lead image for this story. Visual Look Up identifies that the delicious-looking breakfasts in-frame feature quinoa and so recommends a gluten-free quinoa breakfast, an overnight quinoa porridge, and a quinoa salad with roasted squash, however, none of these recommendations look to be the actual dish photographed.
One more thing... this is just for starters
Although Visual Look Up is already a capable and powerful tool – with recipe recommendations being a great new addition in iOS 17 – it still has a long way to go before it becomes one of those indispensable iOS features.
The fact that the feature is tuned for recommendations is, in and of itself, an admission that Apple wasn't able to train the system to identify the specific dishes photographed (an understandably tall order), but it's clearly the direction in which Apple should take the feature going forward.
In subsequent updates, we'd love to reach the point where Visual Look Up can identify the actual meal photographed and serve up the correct rundown of ingredients and preparation steps, but that's not how it functions in iOS 17, right now.
