New GPT-4 app could be ‘life-changing’ for visually impaired people

The first app to integrate GPT-4’s image recognition capabilities has been described as “life-changing” by visually impaired users.

Danish startup Be My Eyes has applied an artificial intelligence model to a new feature for blind or partially sighted people. Dubbed a “virtual volunteer,” the object recognition tool can answer questions about any image it’s sent.

Imagine, for example, that the user is hungry. They could simply take a photo of an ingredient and request the corresponding recipes.

If they prefer to eat out, they can upload a map image and get restaurant directions. Upon arrival, they can take a photo of the menu and listen to the selections. If they then want to work off those extra calories at the gym, they can use their smartphone camera to find a treadmill.

“I know we’re in the midst of an AI hype cycle right now, but some of our beta testers have used the phrase ‘life-changing’ when describing the product,” Mike Buckley, CEO of By My Eyes, told TNW.

Join us on June 15th and 16th at the TNW Conference in Amsterdam

Get 20% off your ticket now! Limited offer.

“This has the potential to be transformative in empowering the community with unprecedented resources to better navigate the physical environment, address daily needs and achieve greater independence.”

Virtual Volunteer takes advantage of OpenAI’s software update. Unlike previous iterations of the company’s vaunted models, GPT-4 is multimodal, meaning it can parse both images and text as input.

Be My Eyes took the opportunity to test the new feature. Although text-to-image systems are nothing new, the startup had never been convinced of the software’s effectiveness before.

“From too many bugs to not being able to chat, the tools on the market weren’t equipped to address many of our community’s needs,” says Buckley.

“The image recognition offered by GPT-4 is superior, and the analytics and conversational layers powered by OpenAI increase the value and utility exponentially.”

Be My Eyes previously supported users exclusively with volunteers. According to OpenAI: a new feature can generate the same level of context and understanding. But if the user doesn’t get a good response or simply prefers human contact, they can still volunteer.

Be my eyes