Face extension, testing (1). Facial recognition. Detect what makes your face

Hi friends,

I'm going to try the Face extension, Facemesh.
It's about Facial recognition. Detect what makes your face.
https://mit-cml.github.io/extensions/data/extensions/edu.mit.appinventor.ai.facemesh.aix

  • I will use a mobile with Android 9 and MIT Companion.
  • IMPORTANT: in many Design changes it is necessary to "Reset Connection".
    face24

Here is this tutorial in Spanish:
http://kio4.com/appinventor/299E_extension_Facemesh.htm

5 Likes

1.- Let's check each point.

p299D_puntosFace.aia (2.8 MB)

  • We direct the front camera of our mobile towards us, we observe our face.

  • The extension takes 450 points from our face. Mesh.

  • If we mark the ShowMesh Property we will see the position of each point on our face.

  • They look very small and very close together.

  • So we are going to make an application to see the number of each point in a clearer way.

  • I have only put the even points, since in total there are 450 and if we put all of them they will mix with each other.

  • As we move, these points will change their position x, y

  • Each point has two coordinates, in a table: 1 -> x 2 -> y

  • Some points are located directly such as Chin, Forehead, LeftEyeBottom,...


    face27

  • We can also locate the position of the points by their number, for example point number 2 is the nose.



3 Likes

2.- Points and lips.

p299D_labiosFace.aia (2.8 MB)

  • We can see all the points, the even points or the odd points.

  • If we enable block "FaceUpdated", we will only see the points corresponding to the lips, I have taken these points for the lips:
    416,311,312,313,14,83,82,81,192,79,97,91,180,87,16,317,404,325,309

2 Likes

3.- Look at me!.

p299D_mirame.aia (2.8 MB)

  • This application will draw two blue balls on the eyes of the face (LeftEyeTop and RightEyeTop)

  • It will also draw a yellow ball on the nose, it is point 2.

  • In the VideoUpdated event we are going to check if the position of the nose has changed, if it has not changed it is that our nose, point 2, the yellow ball will not be changing, in that case the mobile will vibrate.

2 Likes

4.- Two examples.

face26

(The images are that page)

2 Likes

5.- Basketball with a nose.

p299D_baloncestoFace.aia (2.8 MB)

  • You must introduce the red ball into the basket with your nose.

  • The red ball is in our nose, we must move the nose, the red ball will move, until we put the red ball in the basket.

  • The path of the ball will be drawn.

  • If we open our mouths, the path of the ball will be erased.

  • The base of our nose is point number 2, that is why I have put it in the variable nose.

  • We could also put the nose with the NoseBottom property, in this case we would have the x, y positions.

3 Likes

6.- Move the red ball over the letters with your nose..

p299D_letrasFace.aia (2.8 MB)

Move the red ball with your nose over the letters BRAVO.
The buttons will turn red.
If you open your mouth it will restart.

2 Likes

7.- Draw a mesh of lines.

p299D_mallaFace.aia (2.8 MB)

Hi @ewpatton,
I think there is a problem here, with the Front Camera it works perfectly, but with the Back Camera when the model is moved to one side, the mesh moves in the opposite direction.

1 Like

I believe this is a known issue. The code mirrors the front facing camera but then doesn't reset when switching to the rear camera.

Perfect extension and I hope the next update is videocall :grin:.

The camera don't start in all aia you give
What do I do wrong

Maybe it's because of the Android version, I'm using Android 9, it works with MIT Companion and installed, what version do you use?

1 Like


I have tried to mirror the points around image width center and it worked for rear camera

2 Likes

8.- Mirror effect correction.

Thanks @khalid_al-saket, your code corrects the mirror effect with the "Back camera".

p299D_mallaFace.aia (2.8 MB)

Great, but can we make a picture of the hand and fingers and by moving the fingers move balls or connect with Arduino and move a robotic arm or something like that

There is no extension for hand recognition.

...That extension would be very interesting:

Hi , Need Help on Face Emotion Recognition.

I am developing Face Emotion Detector application for my final year project, currently iam using "Personal Image Classifier" but its not effective, please any one guide me to do on this effective way.

Using Face recognition extension can i detect face emotion, or any other idea please suggest me ,

Kindest Regards,
Vignesh.

1 Like

Yes @Vigneshwaran_Balu, I too have seen that its not much affective...


But I don't know an much affective way...

Good Luck with your app :+1: :iphone:

1 Like