For individuals who are blind, visually impaired, or have low vision, interacting with everyday physical appliances and electronic hardware—such as microwaves, dishwashers, washing machines, office photocopiers, and public vending machines—presents significant accessibility hurdles. While consumer touchscreens often support integrated screen readers, physical appliances are typically governed by smooth touch membranes, identical mechanical buttons, or faint printed labels that lack tactile feedback or braille indicators. In iOS, Apple introduced a pioneering accessibility tool within the native Magnifier application: Point and Speak.
Leveraging the optical camera array, neural machine learning models, and the high-precision LiDAR Scanner found on Pro iPhone models, Point and Speak tracks the user’s fingertip in real time. As the user points at physical buttons, dials, or text labels, the iPhone recognises the text adjacent to the finger and announces it aloud via synthesised speech or haptic feedback. Understanding how to use the Point and Speak feature in the iPhone Magnifier app allows users to interact independently with unlabelled physical controls across their home and workplace. This technical tutorial outlines device hardware requirements, explains how to configure Detection Mode in Magnifier, demonstrates active pointing gestures, and provides optimisation tips for challenging lighting environments.
Hardware Prerequisites and LiDAR Technology Requirements
Because Point and Speak requires precise spatial depth sensing to differentiate the user’s fingertip from the background appliance surface, the feature relies on specific hardware components:
| Component / Specification | Technical Requirement | Functional Role in Point and Speak |
|---|---|---|
| LiDAR Scanner | iPhone 12 Pro, 13 Pro, 14 Pro, 15 Pro, 16 Pro (or Pro Max variants). | Emits invisible light pulses to measure the precise three-dimensional distance between the phone, the user’s hand, and the target object. |
| Operating System | iOS 17, iOS 18, or later. | Contains the real-time Vision framework models and Accessibility Detection Mode. |
| Neural Engine | On-device Apple Silicon Neural Engine. | Performs ultra-low-latency on-device optical character recognition (OCR) and finger tracking without transmitting imagery to cloud servers. |
Adding and Accessing the Magnifier App
The Magnifier utility is a dedicated system application included on all iOS devices. If not already present on your Home Screen or Control Centre:
- Swipe down on the Home Screen to open Spotlight Search, type
Magnifier, and tap the icon (a magnifying glass with a plus sign). - To enable instant access from Control Centre, open Settings > Control Centre, locate Magnifier under the “More Controls” list, and tap the green + icon.
- On supported devices (such as iPhone 15 Pro and iPhone 16 models), you can bind the Action button directly to open Magnifier by navigating to Settings > Action Button > Accessibility > Magnifier.
Configuring Detection Mode and Point and Speak Settings
Before using the feature for the first time, customize the detection preferences to match your feedback needs:
- Launch the Magnifier app.
- Locate the Detection Mode icon (represented by an icon showing four corners surrounding an eye or person) and tap it.
- Tap the Settings gear icon located in the bottom-left corner of the detection viewfinder.
- Under the list of available detectors, locate Point and Speak and ensure it is moved into the active detectors group.
- Tap directly on Point and Speak to open its granular parameters:
- Feedback: Choose between Speech (voice readout), Sounds (auditory pitch cues indicating distance), or Haptics (vibrations).
- Speech Voice: Select your preferred synthesised Siri voice and adjust the speaking rate slider.
- Finger Location: Define where the text should be read relative to your finger: choose between Above finger or Below finger. (Setting it to “Above finger” is recommended so your fingertip does not obscure the text).
- Detection Range: Adjust the distance slider (typically 0.3 metres to 1.0 metre) to prevent the scanner from reading background text on distant walls.
- Tap Done to save your accessibility profile.
Step-by-Step: Using Point and Speak on Physical Appliances
Once configured, follow these practical steps to read appliance controls:
- Hold your iPhone in one hand, approximately 30 to 50 centimetres (12 to 20 inches) away from the target panel or appliance door.
- Ensure the rear camera and LiDAR sensor face directly toward the control panel.
- With your other hand, extend your index finger and place or hover it near the button or label you wish to read.
- Keep your finger steady for a fraction of a second.
- The iPhone detects the fingertip, draws a subtle yellow tracking reticle around it on screen, reads the adjacent printed text, and announces the word aloud through the speaker or connected AirPods (e.g., announcing “Defrost”, “Start”, or “Express 30s”).
- Move your finger across the keypad; the iPhone will announce each successive control as your fingertip reaches it.
Optimising Performance in Challenging Lighting Environments
To ensure high accuracy when scanning metallic, reflective, or dimly lit control panels:
- Activate the Built-In Flashlight: If you are reading buttons in a dark utility room or under kitchen cabinets, tap the Torch icon within the Magnifier interface. A well-illuminated panel dramatically improves OCR contrast.
- Prevent Reflective Glare: High-gloss stainless steel and polished plastic touchpads can reflect overhead lights directly into the camera lens, blinding the OCR model. Tilt the iPhone slightly at an angle (roughly 15 degrees) rather than holding it completely perpendicular to eliminate specular reflections.
- Keep Fingers Clean and Distinct: For optimal LiDAR tracking, ensure your pointing finger is extended cleanly away from the rest of your hand, preventing the computer vision algorithm from confusing knuckles with the fingertip.
Troubleshooting Common Point and Speak Issues
If Point and Speak fails to announce text accurately, consider these troubleshooting checks:
- “Point and Speak is not supported on this device”: If the option is missing or disabled in Magnifier Settings, verify your iPhone model. Standard non-Pro models (such as standard iPhone 14, 15, or 16) lack the hardware LiDAR Scanner required to run spatial fingertip tracking.
- Misreading Unconventional Typefaces: Highly stylised script fonts or low-contrast etched icons (such as faint white text on light silver buttons) can be difficult for the OCR engine. Move the phone slightly closer (within 25 centimetres) to increase character resolution.
- Headphone Latency: If using Bluetooth headphones and audio announcements feel delayed, ensure your wireless audio profile supports low latency or use wired EarPods for instant speech synthesis.
By integrating the Point and Speak feature into daily routines, users with visual impairments can interact with the physical world with enhanced confidence, autonomy, and speed on modern iPhone hardware.