Accessibility to screen readers (for the visually impaired)

[lederman]

Accessibility to screen readers Screen readers like google talk back dont recognize the canvas and components on it. What can be done about it?

[SteveJG]

Google says this about using TalkBack (a facility to help with user accessibility) on an Android device:

Get started on Android with TalkBack - Android Accessibility Help

and Navigate your device with TalkBack - Android Accessibility Help

@lederman

google talk back dont recognize the canvas and components on it.

I expect the recognition of the Canvas and things displayed on it may be beyond the capabilities of the screen reader. Can TalkBack describe a Map? It probably cannot describe the map. The reason why might have to do with the meta data on the screen in an App Inventor app.

When I tested TalkBack to view a Canvas, the result was 'unlabeled image' . I can't imagine anyone attempting to use this awkward to use tool effectively. As an Accessibility tool it is not very good in my opinion. But if a user needs visual help, it might be useful. This is not an easy tool to use.

I've forwarded this to MIT. They may be able to explain why the tool cannot work with Canvas.

[lederman]

Thank you for your reply,

I assumed that this will be the case were one side will roll the issue to the other side as you can see from GOOGLE reply (attached at the end)

I’m a father to two blind, and in the last 30 years I’m very aware to accessibility issues, I agree that the logic solution is to make the SCREEN READERS able to deal with lack of awareness of million of developers to accessibility issues

To my estimation 90% of the app loaded to GOOGLE STORE are not accessible because of the use of canvas and component on it.

MIT is not just another developer, this is why my expectation from MIT is to raise the flag of accessibility and to set the standards and awareness to this issue.

I’m not a SW person (I’m electrical engineer) but I enjoy very much working on your platform and develop accessible app.

Searching the net I found the following site dealing with the same issue, maybe this can be implemented on your platform

Making Custom Views on Android Accessible

Thanks

GOOGLE reply:

Hi Shmuel,
Thank you for contacting the Google Disability Support team! I'm Simone and I'll be happy to assist you today!

I understand that Talkback doesn't recognize elements in a third party app.
Since Talkback detects the code of the apps, you should contact the developers of that specific app in order to make it accessible. They should implement accessibility codes in their app in order to let Talkback announce their elements.

I hope it's clear.

Have a great day!

Simone
Google Disability Support Team

[SteveJG]

I expect you will get some comments with respect to accessibility issues related to App Inventor from MIT. They have been aware of screen reader issues from both the point of view of visually challenged developers using App Inventor and software designed or usable for blind individuals as apps.

In the meanwhile here are several community discussions regarding blind and visually impaired individuals that might interest you:

Accessibility blinded students

There's work being done on making Blockly (the software that creates App Inventor's blocks-based coding editor) accessible to screen readers, but it's still got a ways to go! Last I heard (Oct 2019) it was still in the testing phases at the University of North Texas. Unfortunately, App Inventor is even a little bit further behind than that because they're running a version of Blockly that's a few years old. So in short, no. At the moment there's not really way to make App Inventor screen-reade…

For the Blind and Visually Impared

What do you have in mind Royce? Two existing components should already help individuals develop an app helping develop something like this: TextToSpeech component to make your app speak and SpreechRecognizer, so the app can be controlled with vocal commands. This is not a tutorial in helping the visually impaired but it could be adapted.: HOW TO: Program the native SpeechRecognizer for Continuous Dictation and to do things Here are some resources to help you learn to use the AI2 tools. A ver…

Wikipedia api with Plain Text Search Results

How to use the Wikipedia api to produce search responses as plain text and display the result in a Label. How to use the Wikipedia api to produce html search responses using the WebViewer is discussed in Making sense of the Wikipedia API - #4 by SteveJG . [wikipediaChristmas] The Wikipedia api with Plain Text Search Results app shows how to display search results as text. [wikiChristmas] The example Making sense of the Wikipedia API - #4 by SteveJG produces a WebViewer html and can not pr…

How do you make an action happen when you slide your finger onto a sprite?

Hi there! I'm trying to make a game for blind people. Right now as a test, I'm making the device vibrate when the person slides their finger onto a square sprite. So the user would start with their finger down at any point on the canvas, then would slide their finger across the screen without lifting it up. When it is anywhere within the boundary of the square, the device should vibrate continuously. I've half-managed it using the 'draggedAnySprite' property, but it continues vibrating even wh…

Ten to fifteen years ago, I worked on several Projects using JAWS and amateur radio with an EE and for people with other disabilities (Windows programming). I am aware of many issues facing those struggling to use modern tools.

Regardless of screen readers, App Inventor developers do have several tools that can help make their apps friendly to people with vision issues. Perhaps MIT will be able to add the necessary tags to the source code to be more user friendly with respect to screen readers; mean while, there are things developers can do just by coding sensibly.

--Steve

[ewpatton]

So about a year ago one of our Masters students explored some of the issues around teaching accessible programming to students. Her thesis resulted in some new properties being added to App Inventor, including AlternateText property for the Image component among others. However, the Canvas was not touched as part of her work.

The main challenge with the Canvas is that it is a virtual view. The actual content is just a single bitmap onto which everything is drawn. This is problematic for two reasons. 1) App Inventor can't really know how to interpret the content of the bitmap (whose to say if I draw 5 circles they are a Venn diagram, an abstract drawing of a copyrighted mouse, the Olympic rings, a dart board, etc.), and 2) the sprites (both Ball and ImageSprite) don't actually exist in the Android view hierarchy.

It seems like item 1 could be addressed by the approach Murielle used in her thesis by adding a developer-provided alternate text for TalkBack. The second is more complicated but there does appear to be a way to provide this functionality via a utility class called ExploreByTouchHelper that helps the user explore the content of the view. However, the name suggest it requires being able to touch the components which might still make things challenging for your particular use case.

Another option in the short term depending on how interactive the Canvas is intended to be, would be to use the Save method to write out the contents as a PNG file and then show it in an Image, where you could use the AlternateText property. However, to know if that's workable we'd need to know a bit more about the contents and purposes of the Canvas in your app.

[lederman]

Hi and thanks,

It is not about my app, it is about 90% of the app in GOOGLE STORE (this is my estimation) that are not accessible because screen readers don’t handle canvas.

I’m aware that canvas and the spirits on it are changing and in some cases very rapidly, but there are a lot of cases were they change very slowly like board games, checkmate, backgammon, card games, diagrams, graphs and so on. Even in cases were the game is changing fast there can be an option of slow speed as a choice.

The accessibility issue is covered by law, it is not a nice thing to have, it’s the law (in most Western countries) look what is done in entertainment – movies are made accessible, TV shows, news

It is the obligation of developers and GOOGLE as the distributor of the applications.

[Sacramento_Rodriguez]

Buenos días : He realizado un lanzamiento de mi app y me aparece que hay problemas con los lectores de pantalla.

Mi app es una sencilla GUIATURISTICA cuyo objetivo es dar a conocer la accesibilidad de los monumentos de la localidad. Tiene etiquetas creadas en CANVA y no pueden leerse con lectores de pantalla.
En principio no habíamos pensado en esta accesibilidad, más bien era dar a conocer las barreras arquitéctonicas.
Me gustaría mejorarla antes de hacer el lanzamiento. En ella han contribuido dos grupos de alumnos que tienen discapacidad motórica e intelectual.
Muchas gracias por las aportaciones que se realizan.
Soy una principiante como desarrolladora.