Apple's iPhone has long been known for its sleek design and user-friendly interface, but it has not been at the forefront of artificial intelligence (AI) technology. However, a recent research paper by Cornell University researchers collaborating with Apple sheds light on the tech giant's plans to integrate AI into its smartphones.
The paper, titled 'Ferret-UI: Grounded Mobile UI Understanding with Multimodal LLMs,' introduces a multimodal large language model that can analyze and interpret the elements displayed on a mobile user interface, such as an iPhone screen. This model, known as Ferret-UI, leverages a vast amount of training data to identify icons, text, widgets, and other interface elements, enabling users to interact with their devices more intuitively.
