
- Apple is developing an "onscreen awareness" feature to allow Siri to understand and interact with the content currently displayed on your screen
- Apple will also provide APIs to developers for integrating onscreen awareness into their third-party apps and is currently testing ChatGPT integration, allowing Siri to answer questions based on screenshots
- While not available in the iOS 18.2 beta, "onscreen awareness" may arrive in time for iOS 18.4 in 2025