With the Apple AI focus set to continue right through to iOS 18 and beyond, and the company continues to work on creating the tools and features required to make that happen. One example of that is a new research paper that details Ferret-UI, a multimodal large language model that can make sense of what's on a user's display.
Large language models are what power chatbots like ChatGPT while a multimodal LLM, or MLLM, can make more sense of things like images and videos. Now, Apple's researchers have unveiled Ferret-UI, an MLLM that potentially can understand what's on an iPhone's display.
