Introduced on the iPhone 16 lineup alongside iOS 18.2, Visual Intelligence brings on-demand contextual scene analysis directly to the Camera Control hardware button. Powered by Apple Intelligence and private cloud compute, Visual Intelligence allows users to point their iPhone at real-world objects, storefronts, event flyers, or handwritten text to extract details, add calendar events, translate languages, or perform targeted search queries via third-party engines like Google Search and OpenAI ChatGPT.
Because Visual Intelligence relies on the dedicated capacitive Camera Control button, understanding the physical gestures and on-screen workflows unlocks instantaneous contextual search without opening dedicated apps or taking photos that clutter your camera roll.
Requirements and Supported Hardware
To use Visual Intelligence, ensure your hardware and software meet the necessary requirements:
- Device Support: iPhone 16, iPhone 16 Plus, iPhone 16 Pro, or iPhone 16 Pro Max (equipped with the Camera Control button).
- Operating System: iOS 18.2 or later.
- Apple Intelligence Active: Apple Intelligence must be enabled in Settings > Apple Intelligence & Siri, with device language configured to a supported English locale.
Configuring Visual Intelligence and Camera Control Gestures
Visual Intelligence is integrated into the Camera Control sensor by default, but gesture sensitivity can be fine-tuned:
- Open Settings on your iPhone.
- Select Camera and tap Camera Control.
- Ensure Visual Intelligence is toggled on under button actions.
- To adjust actuation pressure, navigate to Settings > Accessibility > Camera Control and choose between Default, Lighter, or Firmer force thresholds.
Triggering and Using Visual Intelligence
Visual Intelligence can be launched instantly from anywhere, including the Lock Screen or while using any application.
Step 1: Activating the Visual Viewfinder
- Press and hold the Camera Control button on the lower-right edge of the device.
- A full-screen analytical viewfinder opens, outlined by an active Apple Intelligence glowing border.
- Point the camera lens directly at the object, document, or storefront you wish to examine.
Step 2: Interacting with Contextual Information
Visual Intelligence automatically categorises what is in frame and displays dynamic action buttons along the lower interface:
- Storefronts and Restaurants: Tap the card to view business hours, customer ratings, menu previews, and contact information pulled directly from Apple Maps.
- Event Posters and Flyers: Tap Add to Calendar. The system extracts the event title, start time, date, and venue location, populating a new event card in Apple Calendar with zero manual typing.
- Printed Text and Documents: Tap Copy Text, Translate, or Read Aloud to process documents, receipts, or signs instantly.
- Animals and Plants: Tap the identification badge to discover botanical species or canine and feline breeds.
Leveraging Third-Party Visual Search: Google and ChatGPT
When contextual information requires broader web knowledge or conversational querying, Visual Intelligence connects securely to external providers:
- In the Visual Intelligence viewfinder, tap the Search or Ask button at the bottom of the screen.
- Select Google Search to perform a reverse image and visual product search via Google Lens algorithms.
- Select ChatGPT to ask complex questions about the scene (for example, pointing the lens at an architectural structure and asking for historical architectural style details or troubleshooting instructions).
- A confirmation sheet appears before sharing imagery externally, maintaining strict user privacy control.
Privacy and Data Safeguards
Apple engineered Visual Intelligence with local-first processing:
- Visual frames captured in the viewfinder are not saved to your Photos library.
- On-device machine learning models handle initial text parsing, calendar event extraction, and landmark detection without transferring image data to Apple servers.
- When third-party integrations (such as ChatGPT) are queried, IP addresses are obscured, and external providers are prevented from training models on submitted requests.