and the distribution of digital products.
Gemini in TalkBack: How Google is trying to revolutionize screen readers
Google announced on Thursday that it’s rolling out new AI and accessibility features to Android and Chrome, enhancing the user experience for people with disabilities. The updates aim to make Android and Chrome more inclusive and user-friendly.
One of the key updates is to TalkBack, Android’s screen reader, which now allows users to ask Gemini about the content of images and their phone screen. This builds on last year’s integration of Gemini’s capabilities into TalkBack, providing AI-generated descriptions for images even when Alt text is not available. Users can now ask questions about images, such as the brand and color of a product in a photo, and get descriptions of their entire phone screen, enabling them to navigate apps more easily.
For instance, if a user receives a photo of a new guitar, they can ask Gemini to describe it and inquire about specific details. Similarly, while shopping in an app, users can ask Gemini about the material of an item or available discounts. These features are designed to make Android more accessible to people who are blind or have low vision.
Image: Google
Google is also updating Expressive Captions, its real-time captions feature that uses AI to capture spoken words and tone. The new duration feature on Expressive Captions will convey the emphasis or tone behind spoken words, such as a drawn-out “amaaazing” or “nooooo.” Additionally, new labels will be introduced to indicate sounds like whistling or throat clearing. This update will be available in English in the U.S., U.K., Canada, and Australia for devices running Android 15 and above.
Chrome is also receiving updates, making it easier to access PDFs. With the introduction of Optical Character Recognition (OCR), Chrome can now recognize scanned PDFs, allowing users to highlight, copy, and search for text, as well as use their screen reader to read them.
Page Zoom on Chrome for Android has been enhanced, enabling users to increase text size without affecting webpage layout. Users can customize their zoom preferences and apply them to specific pages or all pages they visit. This feature can be accessed through the three-dot menu in the top-right corner of Chrome.