What you can do with App Inventor to improve recognition is limited compared to using Android Studio etc. with its extensive libraries. However, there are small things you can do to improve recognition.
Here are some suggestions from a Speech Recognizer tutorial
How to avoid mistakes
- Experiment with the various Parrot option choices until you find something the fits your speeking/dictation style. - A secret to making a Speech Recognizer work is to speak in a strong, steady, clear voice. Even then, the Parrot recognizer WILL make mistakes.
- I show one way to delete words Parrot misspeaks…type the word that must be eliminated, press Delete or use the Delete Last of to delete the last spoken text.
- There are a few grayed out Blocks of code at the top of the AfterGettingText event handler. Restore the Blocks and these or similar blocks can be used when the SpeechRecognizer is stubborn. Yes, sometimes it refuses to provide the word you speak. For example, if you do not want to record 2 when you say 2 but to print out two or to respond with for when it thinks you say four, you make it obey when it mis-understands your pronunciation.
If you haven't taken the tutorial
you might find some hints by doing this tutorial.
What you can do with App Inventor to improve recognition is limited compared to using Android Studio etc. as I said. There are small things you can do to improve recognition.
The advice is basically to learn to talk to your recognizer. 
Speak in a steady, clear voice without background noise. That really does help.
Users who first language is English is their first language do better than those for who it is a second language and have even a slight accent.
Learn to write code to ameliorate some of the 'errors' you get while speaking
When your app if frequently making mistakes with voice recognition, provide a TextBox to provide input in text.