Speech Recognizer

MIT App Inventor Help
,
#1

This is not a problem with the MIT speech recogniser BUT it may well be something that someone on here has encountered, and hopefully resolved.
When I call Speechrecogniser.gettext it triggers the Google speech to text service, which works fine.
My app runs alongside an app that controls an SLR camera and uses a Direct WiFi connection with no internet connection.
The Google Gboard speech typing options allow offline use, and on my mobile running Android One with the latest Android 9 works fine.
My Huawei tablet runs Android 8 and when I set the option to work offline it just states that it can’t find a network so does not work offline
I have contacted Huawei, Google and asked on another forum but had no response in a week.

Has anyone on here got any experience of this please?

Many thanks in advance

#2

hi Dave, I don’t have any experience but do note that Android One phones, being defined by Google, could well have built-in capability that others have to access online. If that hunch is correct, you can’t use the Huwaei for this task.

#3

Huawei is restricted (at least for the moment) to using the Android Open Source Project (AOSP), cutting the company off from critical Google apps and services. This could be part of the issue you describe Dave.

#4

Many thanks for your response, it’s appreciated.
I think when manufacturers install Android One the updates come straight from Google rather than the packaged versions that the big sellers control, which suits me fine; I don’t want all the bloatware, I download what I need over and above the OS. All of the options are there, it’s just that this bit of it doesn’t work; it may be the Android version, the Gboard version for, the Huwaei doctoring of it if any or the device, which is why I asked on here, hoping that someone else could offer some advice, or at least confirm where the problem lies.

#5

The restriction is a but of a grey area, it seems it has been lifted for what are considered non security threats. Huawei announced early last year that the Mediapad T5 10 would get Android 9; I have asked if that is still the case but no response yet. If it happens it may or not resolve the issue. I was hoping that someone else with an Android 8 device, could confirm if it works. I’ve read and tried about everything I can find to no avail. It’s a shame but my mobile isn’t big enough to display what I need but hey ho

#6

Our SpeechRecognizer component uses Android’s default speech recognizer service (whatever that is for your device). There’s no requirement that the service be required to work offline. In fact, most are online but recent improvements in machine learning models for speech recognition have made it easier for them to run on device rather than in the cloud. Unfortunately, I don’t think there is a way to assert to the recognizer that you want one or the other. You may be able to install different apps to provide recognition services locally, but it’s also possible this could be locked down by the vendor. Have you tried unchecking the UseLegacy option to see if that helps at all?

#7

Good day to you and thank you very much for your response and I wasn’t expecting anything more from MIT than what it does now. I have tried the UseLegacy option and it makes no difference.
I did start my post acknowledging that the MIT speech recogniser is not at fault.

I was just hoping that there are some MIT users that may have encountered the same issue, or could try the offline option to shed some light on it.

#8

… To work offline, the device would need to have all of the data normally stored online, which I think is the case with Android One.

What exactly is your goal? What should your App do?

#9

The Gboard keyboard app in offline mode downloads a database of words to the mobile to allow it to work offline. It works great on my Android 9 mobile but not Android 8 tablet, though I am unsure if it’s the Android version that is the problem as this ability has been about for a while now. I’ll dig my old standby mobile out and see what it does on that.

The app is fully working but I thought it would be nice to voice control it too, not essential and may not even work as well but nice to learn these things.

I already had my camera mounted on a tripod in the kitchen trained on bird feeders in the garden. This is remote controlled using Direct WiFi with a downloaded app that can zoom, focus and shoot. I split the screen with my app alongside it that controls stepper motors that rotate and pan a tripod mount holding the camera using an Arduino Uno board via Bluetooth. I can provide links if you’re interested but that’s basically it, a nice little project

#10

Sorry you got a Huawei device Dave. Just to let you know, the MIT SpeechRecognizer component works very well without a WIFI connection on my Samsung Tab A. The tablet uses Speech Recognition when connected to WIFI or with WIFI turned off on the tablet.
It is an Android 8.1. How does the SpeechRecognizer component decide whether to use the online recognizer or not? I haven’t a clue. It just happens.

I suspect the Huawei is just stupid. It just might include the os software to use offline speech recognition.

For reference my Android 4.2.2 can only use SR when connected to a network or WIFI.

#11

Thank you for your response.
Perhaps I haven’t explained it very well:-

The MIT Speech recognition works perfectly, all it does is call the device’s speech recognition service. I only asked on here in the hope someone had a similar experience and could help.

My MIT app controls motors that operate a tilt and pan mechanism that I made, which sits between the tripod head and the camera.

My camera can run an app that allows it to be remote controlled by a mobile app BUT it requires a Direct Wifi connection so disconnects from the router, meaning I have no internet access while that is active.

I run the 2 apps together in split screen mode.

The now default Android Google Gboard keyboard app can accept speech input and by default uses the Google online service. It also has the ability to work offline, if this is selected you choose your language(s) and it downloads the dictionaries to your device and will work without an internet connection. The offline option, even though selected, doesn’t work on my Android 8 Huawei tablet, it just states Network Unavailable so it seems to be trying to use the online service still. I am unsure whether the problem is with the Android version, the Gboard input or the Huawei tablet. I have contacted Google and Huawei, but I’m not holding my breath.

Many thanks for your time

UPDATE - I got my emergency mobile from the car, a Motororla Moto G5, and it has Android 8 with Gboard as the default input device and all set up to work offline. I tried voice speech with WiFi on and it works fine, turned WiFi off and it doesn’t work so it looks as though it’s the Android version rather than the device.

#12

Hi Dave

Perhaps a combination of both. One thing not picked up on before - not every device supports split screen mode.

There is another solution to explore, a tablet running Windows 10 (owch!)

#13

This is only for my use and it’s only feasible to run on a large screen in split screen/floating windows. I don’t intend buying a new device, though I may at some stage get a tablet with Android One.

I wouldn’t entertain a Windows tablet, though I have done some Windows CE mobile development before.

I am hoping that Huawei will stick with their original plan of rolling out Android 9 to the T5 Mediapad 10, I have asked but we will see.

#14

That sounds onerous.
What make and model is that camera?

#15

It’s quite common to connect a mobile device to things using a Direct WiFi connection and you can only connect to one network at a time so it disconnects from your router.

It’s a Sony DSC-HXV400V

#16

From what I see at
https://support.d-imaging.sony.co.jp/www/cscs/pmm/page2.php?area=gb&lang=en&mdl=DSC-HX400V&abil=31100&mode=connect
there are also NFC and QR code connect options.

#17

They are just means of initiating the direct WiFi connection the first time. I don’t have NFC so I used the QR code, but it remembers them after that.

TBH the speech control isn’t as quick or easy as the ball on canvas and edge collided option, which I have working great now. It would have been cool to have got speech working though.

The only slight issue I have is that I’m sure when I first tried edge collided when dragging the event was triggered when the edge of the ball touched the canvas edge but now it’s where my finger is, which is the centre of the ball, weird.

I have you to thank for that idea by the way, so many thanks. It’s nice to use a forum where people take the time to help

#18

Hi Dave

That was actually an enhancement in the latest release :innocent: It brings the ball into line with typical game math code and so easier to understand from that perspective.
You could, instead of a built-in Ball, use your own Sprite.

Edit: Given Steve’s test on his Android 8.1 Tablet, it looks as though the Huawei Tablet might not deliver the offline ability you require, even after the update -you should be able to update it to Android v8.1 while waiting for Android v9. Since you are filming bird feeders, the least noise the better? “Our” birds fly away if we make a sound in the kitchen…

#19

What’s the state of the CenterAtOrigin property? It should be false by default, which should not interfere with the old collision behavior. Do you think you can produce an example app that shows the behavior difference between the two versions?

#20

Many thanks again for a prompt response,

The CenterAtOrigin property is checked; I only started trying the ball on canvas option 5 days ago so I didn’t realise it was a recent option.
I only use the ball on canvas like a finger pad/joystick so don’t animate it in any way, but I was fairly sure that when I dragged the ball the .EdgeReached event triggered when the outer edge of the ball reached the edge of the canvas but I may have been dreaming, is that is what is supposed to happen?
It isn’t being triggered until my finger reaches the edge. I have made the ball and canvas small and it is quite usable and I only want to move my finger a small amount anyway, but it would be better if it worked as I thought it did. I will upload an image of my blocks