Hello,
I am an engineering undergraduate student, working on a research dissertation which detects objects on the ground and sends alerts. I am using the “Personal Image classifier ” extension in MIT Inventor and ran into some problems.
My idea is when the confidence level of any object detected is over 75% a message notification is sent and also a screenshot will be taken. but my blocks did not function as I thought they would. the app would detect objects I trained it to but no message or screenshot is taken even if the confidence level went over 90%. I attached the blocks I made in hopes that I can get some advice on what I have done incorrectly. I used Taifuns' extension for both screengrabs and notifications here.
Thank you all in advance for any suggestions.
best wishes,
Victor
To debug this, display all your ...conf values in different labels as well as the result of your comparison
Also my simple notification extension works only for older Android devices, let me suggest you to use the Notification Style extension instead
And: there is no need to use the ScreenshotService extension
Taifun
Thank you for the reply taifun.
I am not too sure i understand what you meant, do you mean i should visualize all the numbers on a label so i have a better idea on how i can debug this or is it that i should pull the conf values out of the local block with invisible labels and run the notification blocks separately? This is what i have done in terms of changing the code but i am not too sure what i should do with all the conf values showing. Do you mind further explaining to me please ?
I also want the screenshot function to work as i need a way to log the date/time/evidence(for manual confirmation) when the identified object is present and a jpg screen grab of the app after a registered object is detected feels like the easiest way to do this.
Best wishes,
Victor
This looks good to help debugging your blocks
Taifun
will notification style work on a Huawei device which runs on HarmonyOS? I am not sure how the API levels translate into this OS. If not will there be any workarounds possible?
Please test yourself and let us know if you get it running there
Taifun
hi have tested the screenshot function and found out that the web viewer will not appear in the screenshot, I tried both Taifun's screenshot and component to image extension and the problem occurs, will there be any other extensions that allow screen capturing web viewers with the entire screen?
Try this
I have tried @Taifun 's Screenshot and @Jerin_Jacob 's ComponentToImage extension, both of them can not capture the video in the webviewer.
Here is one solution to capture the video in the webviewer.
This also works with the PersonalImageClassifier extension by MIT.
The captured image will be returned in the event WebViewStringChanged as a base64 string like
data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAUAAAAFCAYAAACNby
the full javascript:
function capture (){
const video = document.querySelector('video');
const canvas = document.createElement('canvas');
const w = video.videoWidth;
const h = video.videoHeight;
canvas.width = w; canvas.height = h;
const ctx = canvas.getContext('2d');
ctx.drawImage(video, 0, 0, w, h);
const imageUrl = canvas.toDataURL("image/png");
window.AppInventor.setWebViewString(imageUrl);
canvas.remove();
}
capture();
ImageClassifierVideoCapture.aia (4.6 MB)