Seeing AI, a Microsoft App for Visually Challenged

Seeing AI is a free app by Microsoft to help narrate things, identify persons and read documents. The app is also known as talking camera app, currently available for iPhone users. Seeing AI is a revolutionary application using artificial intelligence to make life of people with low vision and people with blindness comfortable. Problems this seeing AI app can solve are countless. Starting from recognizing people, with their emotions and calculating the approximate age and distance from camera, reading the printed material, identifying the barcode of products are to name a few. VoiceOver the inbuilt screen reader is compatible with this app, if the user is low vision and does not use voiceover, still an audio output is provided. Seeing AI app is first released for iPhone and is very accessible for voiceover users. We tried our best to cover the interface of the seeing AI app and explain each feature of it in detail. Through this article we also want to share tips and tricks to make optimum utilization of the app.

With a simple user interface, the seeing AI app contains menu as the first item, followed by a quick help button. At the bottom is the channel changer option. The area in between the quick help and channel changer is the main feature. This area of the screen changes depending on the channel the user choose to use. A pause/ resume button and camera button are observed in the main region depending on the channel currently in focus.

Seeing AI home screen

Menu

The first item menu contains the following items. Worth exploring them.

  • Face recognition: Explained in detailed later in the article.
  • Help: In detail help of each channel with short videos.
  • Feedback: This option opens email application allowing the user to share feedback. You can also email your feedback to seeingAI@microsoft.com.
  • Settings: Setting currently have only one option i.e. manage lighting toggle button. This feature allows the camera to adjust the lighting for a clear picture.
  • About: The version of Seeing AI can be found here, current version is V1.0.
  • Close: Close is the first option in the menu allows you to close the menu and return to the home screen.

Seeing AI menu screen

Face Recognition

When you first open face recognition option in Seeing AI app, it has a quick help option and next to it is the camera button. By default the front camera will be on. This face recognition option asks you to take 3 pictures of a person and save with the name. Once you save a person on Seeing AI, the app will recognize the person automatically whenever it identifies the person in future.

Adding picture to face recognition feature.

Later when you open the face recognition feature of the seeing AI app again in the future, the screen will list down the recognizable people and also an add button to add new persons.

Quick help

Quick help in each screen provides information only about that particular channel. A button at the end of the screen that shows the video tutorial is also available. A close button on the top of the screen to return to the home screen.

Seeing AI document channel help screen

Channels in detail

Short Text

Short text is the default channel when the Seeing AI app is opened. Simply point the camera on the text you want to read. Read the text using voiceover or if voiceover is not switched on the content is automatically read out by the seeing AI app. If you move the camera while reading or if the app recognizes more clear content it starts reading from the beginning.

Document

Select the document channel on the seeing AI appp. Point the camera to a printed document. Listen the suggestions to bring the all the edges of the document in the camera viewable region. Once the document is accurately fit, the seeing AI app says hold steady and the picture is automatically taken by the appp. Wait for the processing to be done. After the content of the document is read automatically or is available for voiceover users. Intrusting thing is the format of the content such as headings are also read out by voiceover.

Product

Point the camera to the barcode of the product. The seeing AI app gives audio beeps to help locate barcodes and then scans them to identify products. Perfectly identified the pack, see the screenshot below.

Screenshot showing output from a product

Person

A wonderful and exciting feature in the Seeing AI application is recognizing people around you. While taking the picture the feature also announces the location of the face on the camera allowing the visually challenged user to move the phone to a perfect spot before taking the picture. The feature also recognizes number of faces in the capturing region. If the person is not already saved in the face recognition feature explained above, Seeing AI app says the gender, posture such as sitting or standing, approximate distance away. If the person is already captured in the face recognition feature, it announces the name with which the pictures are saved instead. The application also can capture and informs the color of clothes, hat or glasses. The exciting feature in the channel is that the seeing AI app provides approximate age of the person and the emotion such as looking happy, looking neutral etc.
Don’t get offended if the Seeing AI app recognizes the age a little high, try once again where you have better lighting.

Scene beta

Currently observed as an experimental feature available on the Seeing AI app, this channel describe the scene around you. Take a picture of anything around and wait a few minutes until the music stops. The scene or the things around you will be described. This channel has an intelligent feature of identifying different objects around you and structure them with a sentence. A word “probably” is pre-fixed before the sentence. Definitely a cool feature in the Seeing AI application. Can’t wait to use it everywhere I go.

Currency

This channel is currently not available in the Seeing AI app. Seems like a future feature, it identifies currency bills when paying with cash.

Related Links

Share your experiences with the app in comments below.

Comments are closed.