Let’s face it: When the artificial intelligence research project OpenAI announced the public release of ChatGPT in 2023, no one expected the quirky chatbot to dominate world headlines for weeks to come.
What is ChatGPT? Using large language modeling (LLM), OpenAI researchers fed a dataset of an estimated 175 billion words of text into ChatGPT, training the chatbot to understand word relationships and answer user queries through hyper-fast next word prediction.
So, what does this have to do with assistive technology? The short answer: Soon, everything.
iPhone users familiar with the Be My Eyes app don’t have to wait. The free app, available on both the App Store and Google Play, connects users who are blind or visually impaired to what its Danish developers describe as “a pair of friendly eyes.” Essentially, Be My Eyes accesses a smartphone’s camera and microphone to connect users to sighted volunteers from around the world, much like a FaceTime call.
Now, through a new feature called Be My AI, the Be My Eyes app is leveraging artificial intelligence to interpret visual content in detail like never before.
Utilizing even more advanced algorithms from OpenAI’s ChatGPT-4, Be My AI provides users with detailed visual descriptions of the world around them whenever and wherever they wish, with no need for human interaction. The results are nothing short of astonishing.
I was admittedly skeptical prior to my first test – identifying the conglomeration of buttons on a remote control – the bane of blind television owners since the day remotes first landed on coffee tables.
Not only did Be My AI accurately describe the labels, shapes, and colors of the buttons – it described their positioning, left to right, top to bottom.
A second test on an unfamiliar Keurig coffee brewer yielded similar results. Be My AI explained that the control panel had a circular interface, described the shapes and colors of the buttons, and read each cup serving setting around the “clockface.”
Outdoors, the app described a building exterior, read signs around the building, told me a middle-aged man was sitting on the front steps looking at his phone, and even peered through glass windows to describe arrangements of plants in the lobby.
Be My AI can tell you what’s in your refrigerator, read expiration dates, and suggest meals using the items on your shelves. In the gym, it can direct you to specific equipment and let you know which machines are free and which are occupied.
Again, astonishing.
Ask Be My AI for more detail and it tirelessly provides it, even describing a room’s ambience as “cozy,” “modern,” or “sophisticated.”
When more help is needed, the app offers an easy to locate “call a volunteer” button, allowing users to quickly connect to a sighted person.
While it’s not exactly clear where all this is leading, what is clear is that sooner than later AI is going to open employment and social opportunities never thought possible for people who are blind.
Be My AI is now available on iOS and Android.
Doug Goist is the program manager, workforce development, at NIB’s talent management enterprise, NSITE. A recognized leader in the field of technology accessibility, Doug has worked with the U.S. Department of Defense, the military services, federal agencies, and private sector partners. In 2013, he served as the technical steering committee representative for the U.S. Agency for International Development on a study of mobile money transfer and handset accessibility in Africa.