Microsoft Seeing AI app not just for the visually impaired, broad range of tools

The Seeing AI free app from Microsoft was originally made to help people in the low-vision community. For people outside of this target market, the possibilities are almost endless, some reminding us of futuristic movies.

One of the number of tools in this free app is an OCR, a document scanner that quickly generates audio from any short text, even magazine pages with black text written over a photo.

Dyslexia, illiteracy, and ESL

This read-document-aloud tool is useful for people with dyslexia (1 in 10 people), people who are illiterate (10% of all people), learners of English as Second Language (over 1 billion people worldwide), and children who are still learning vocabulary (73.7 million children in the US alone).

Anyone who enjoys the audio experience over just reading text and people who prefer to multitask can make use of this tool.

Document scanning

For example, restaurant-goers who have trouble reading menus can use the app to skip, scan, and skim headings whether they be appetizers, drinks, or main courses.

As of 2012, the US Census reported 27% of all households were one-person. Consequently, having documents read aloud can offer a sense of companionship for single adults, much like leaving music or a television on “for company” only with a more specific purpose.  

Remembering people’s faces with facial recognition

Another feature of Seeing AI is the facial recognition tool that not only can learn people’s faces and match those to names but can also give a brief description and age of the person and read the emotions on their faces.

Learning other people’s names and matching those to their faces is difficult to impossible for many people, which is worse for new employees, teachers and professors at the start of the school year, substitute teachers, short-term contract employees, and people with brain injuries.

Dementia, Alzheimers, and Autism

Remembering names with faces is another level of difficulty, made more demanding for those people with prosopagnosia or “face blindness” (2-2.5% of the world’s population) or those with dementia, including Alzheimer’s, (about 50 million worldwide).

Recognizing emotion on people’s faces is a challenge, in particular for children with autism spectrum disorders (1 in 68 children), so the Seeing AI app could assist these kids as well as adults.

The future is here. Pivothead’s SMART glasses, Microsoft’s Seeing AI app, and bluetooth or plugged in earbud or headset mixed with Microsoft’s Azure’s Cognitive Services is a recipe for feeling like you are in a science fiction movie with touch/swipe glasses that “see” what you look at and feed the audio to you discreetly.

According to Microsoft Senior Technical Evangelist Rick Barraza and Software Engineer Rita Zhang on the MS Developer Blog, “Microsoft Cognitive Services are a set of pre-trained and pre-built libraries for specific but common machine learning tasks developed by Microsoft Research. Using these services, developers can easily integrate features like facial, speech, and image recognition into their applications.”

The other tool channels in this app include a scene view describer, printed page capturer, cash currency identifier, and product barcode reader.

Visually impaired

For the people in the low-vision community the Seeing AI app is a tool that increases independence, safety, education, comfort, convenience, and creativity.

For people who need a little help from these same app tools, life can be made more productive, less stressful, and more audible.

Saqib Shaikh lost his sight at age seven and now is a Microsoft software developer integral to the Seeing AI project. In 2016, he gave a video demo, describing his 10 years of work at Microsoft improving people’s lives and said how “this is just the beginning.”