Brain Storming help (Sign Language)

General Discussion
#1

I'd like to discuss with one or more knowledgeable and patient experts about an app idea I have. I have encountered some deaf in the interior of NE Brazil and looked for an app to teach sign language to them. Books are OK but it's better if you can see the movement. My idea is to have a "dictionary" of a grid with pictures / names of objects. When you click on the picture a short video or animated GIF (for file size) show a person making the symbol.

It's just a picture/video viewer and I though I'd find something I could modify but haven't had any luck yet.

Should I use GIFs or video? Should I try to copy a wad of files to the phone or try to put them in the TinyDB? Though internet is rather widespread, the deaf are usually on the poorer side and (in the interior) don't usually have phones or access.

What are your thots?

#2

The following might be useful Byron:

A rudimentary ASL app is here http://www.appinventor.org/content/howDoYou/timedLists/asl that might give you some ideas.

Another tutorial is here https://nebomusic.net/appinventorlessons/American_Sign_Language_Alphabet_App_Directions_V2.pdf

Here are some more sign language apps from the MIT App Inventor Gallery (you need to be signed in to the Gallery to use these links):

https://gallery.appinventor.mit.edu/?galleryid=6730263276683264

https://gallery.appinventor.mit.edu/?galleryid=4739727427502080

https://gallery.appinventor.mit.edu/?galleryid=5868761312002048

1 Like
#3

Gallery was the first place I looked. I had a hard time finding anything. Couldn't figure it out. I'll try again and start with these. Thanks

#4

Hi Byron

I would show Hands rather than a Person where possible, given that a smart phone screen is relatively small. From a file size perspective, gifs are good. Each animation can be stored as an HTML file and displayed on demand in a Web View component.

Your approach is different to most, which tend to only teach the basic signs. A dictionary of object names could comprise of a text list for each letter, loaded into a List View on demand.

By storing the data separate from the App, it would be easy to add more over time and an App update would not need to involve downloading the data again.

#5

Thanks, Chris. You pretty much have to show the upper torso as many word signs are relative to the body. While I may show the letters, they aren't very useful to someone who can't spell.

Can the Web View work off-line, like from a chache?

#6

Hi

Yes, the HTML and text files can be stored on the phone in the "Application Specific Directory" - gone are the days when we could store data anywhere on the phone, Google have tightened security.

#7

This may be beyond my time and abilities. I'm thinking of paying someone to do it. Anyone interested?

I need help with Text to image (sign language from the SpeechRecognizer)
#8

There may be some alternatives to your app Byron. There are different ways to approach sign language. Perhaps something here will help https://www.deafwebsites.com/sign-language/sign-language-dictionaries.html ?

Brazilian Sign Language https://www.google.com/search?q=brazillian+sign+language&rlz=1C1CHBF_enUS887US887&oq=brazillian+sign+language&aqs=chrome..69i57.7126j0j7&sourceid=chrome&ie=UTF-8

Do what? Have you tried any of the examples already provided that show what is possible with App Inventor? For anyone to help you, it is reasonable you demonstrate exactly what the app should do and perhaps point out how you think it should work. Only YOU know what this app should do.

Here are some sign language apps already on Google Play https://play.google.com/store/apps/details?id=com.jquiz.asl&hl=en_US&gl=US Perhaps they will give you ideas or make you decide to use something that already exists for your project.

@ChrisWard ... you indicated this app could be easily constructed using html. Can you provide him an example to get him started?

#9

Yes, I have studied all the examples given and a half dozen more. Two of them I'm still analyzing to see if I can spin off their idea. Most of the stuff I've seen uses written words and teaches the hand alphabet and letters which is useless to a man who doesn't even know what his name is. It's just squiggly lines. And it's geared towards hearing. Or it's a translator.

I didn't intend to insult or offend you. It isn't for a lack of help on your part and I'm not complaining. It isn't sour grapes. I wrote an app that grew to several thousand blocks but I was in an area where I knew where to start and how to progress. But sometimes one is so ignorant in an area that he doesn't even know where to start. That's my case here. I spent almost all day for a month or more during the lockdown writing the fish app and I can't justify that much time here. Even in Visual Basic I once paid a guy to write the SQL part because I just couldn't wrap my hear around it. Too many variables.

I thought I was being fair by offering to pay someone who could churn it out in a couple of hours and not weasel it out for free while I played video games (which I haven't done simce my boys were teens.) This isn't a "my projects due tomorrow who can do it for me." Sorry if it seemed that way.

#10

I don't know if it's possible to make a real sign language interpreter. I think the only way is to record a person who knows the language, then play the video episodes as needed. I just don't know if such a language is really universal in every country. The easiest to do is spelling with your hands and fingers, but you don't want that. Video files are the most important, the rest can be done together. Besides, some words don't require knowledge of the alphabet, but most do. If someone knows sign language, he can rather spell. Unless I got something wrong. Chris mentioned that this is easily accomplished in html. I think it was about the use of unicode characters in which the letters of the sign language are written. I'll make you an application sample without html.

#11

That was actually concerned with two factors:

  1. Hosting an Animated GIF, how easy it is to create a single HTML template and the result being fairly small files
  2. Easy to load the HTML files from local storage and display immediately
#12

Yes, gifs would be the best. Just where to get such gifs. With the gifs, the rest would be easy.

#13

I would have to make the gifs. Maybe frame grab from a video.

#14

Besides letters, there are also words. e.g. LOVE is a combination of three letters. I don't know how many such words there are. What do you think about it? All spelled or words too? An additional problem may also be the variety of sign languages. There are two types of sign language in my country. Each country also has its own language, which can make it difficult to work when we are from a country other than you.

#15

I guess we won't think of anything better than that anyway.

#16

I will be using Libras which is Brazilian sign language. I know personally 4 teachers who live within an hour. The letters are important for personal names, but I plan to start with word symbols. I will show the words, too, but not emphasize them in the beginner app.

#17

OK. What you want to do is worthwhile. You have lots of issues to deal with. Good luck.

Simple Signing ... took 5 minutes to code. **You certainly can do this @Byron_A **. :slight_smile:

simpleSigning
simpleSigning808×210 31 KB
by linking to handspeak.com
SimpleSigningApp
SimpleSigningApp600×960 106 KB
. Simple emoji used to show objects (cat / dog is the example).
SignLanguage.aia (1.8 KB)

You can do similar things with the sign language dictionaries linked above.

Note the copyright restrictions. It seems you should be allowed to do this for a private app.

1 Like
#18

Ok Ok You win. I think I can do it. I did some experimenting. A 125k mp4 shrunk to a 500k gif. I also realized I don't need a separate thumbnail and viewer, the preview frames can be the object. E.G. a picture of a dog is the first frames for preview and when played, the video plays the pic then the sign (here, a rounded hand biting at the face.)

I think I've got my toe-hold idea to build off of.

Thanks

1 Like