Face extension, testing (1). Facial recognition. Detect what makes your face

Artificial Intelligence
, , ,
1

Hi friends,

I'm going to try the Face extension, Facemesh.
It's about Facial recognition. Detect what makes your face.
https://mit-cml.github.io/extensions/data/extensions/edu.mit.appinventor.ai.facemesh.aix

  • I will use a mobile with Android 9 and MIT Companion.
  • IMPORTANT: in many Design changes it is necessary to "Reset Connection".
    face24

Here is this tutorial in Spanish:
http://kio4.com/appinventor/299E_extension_Facemesh.htm

5 Likes
2

1.- Let's check each point.

p299D_puntosFace.aia (2.8 MB)

  • We direct the front camera of our mobile towards us, we observe our face.

  • The extension takes 450 points from our face. Mesh.

  • If we mark the ShowMesh Property we will see the position of each point on our face.

  • They look very small and very close together.

  • So we are going to make an application to see the number of each point in a clearer way.

  • I have only put the even points, since in total there are 450 and if we put all of them they will mix with each other.

  • As we move, these points will change their position x, y

  • Each point has two coordinates, in a table: 1 -> x 2 -> y

  • Some points are located directly such as Chin, Forehead, LeftEyeBottom,...

    face6
    face6873×466 104 KB

    face27

  • We can also locate the position of the points by their number, for example point number 2 is the nose.

face16
face16696×629 141 KB

face25
face25695×606 72.1 KB

face18
face18774×806 70.2 KB

3 Likes
3

2.- Points and lips.

p299D_labiosFace.aia (2.8 MB)

  • We can see all the points, the even points or the odd points.

  • If we enable block "FaceUpdated", we will only see the points corresponding to the lips, I have taken these points for the lips:
    416,311,312,313,14,83,82,81,192,79,97,91,180,87,16,317,404,325,309

face20
face20348×552 45.4 KB
face21
face21355×529 150 KB

face22
face22860×2290 210 KB

2 Likes
4

3.- Look at me!.

p299D_mirame.aia (2.8 MB)

face1
face1295×581 116 KB

  • This application will draw two blue balls on the eyes of the face (LeftEyeTop and RightEyeTop)

  • It will also draw a yellow ball on the nose, it is point 2.

  • In the VideoUpdated event we are going to check if the position of the nose has changed, if it has not changed it is that our nose, point 2, the yellow ball will not be changing, in that case the mobile will vibrate.

face4
face4932×1259 144 KB

2 Likes
5

4.- Two examples.

face26

(The images are that page)

2 Likes
6

5.- Basketball with a nose.

p299D_baloncestoFace.aia (2.8 MB)

  • You must introduce the red ball into the basket with your nose.

  • The red ball is in our nose, we must move the nose, the red ball will move, until we put the red ball in the basket.

  • The path of the ball will be drawn.

  • If we open our mouths, the path of the ball will be erased.

  • The base of our nose is point number 2, that is why I have put it in the variable nose.

  • We could also put the nose with the NoseBottom property, in this case we would have the x, y positions.

face11
face11498×616 112 KB

face12
face12719×1168 89.7 KB

3 Likes
7

6.- Move the red ball over the letters with your nose..

p299D_letrasFace.aia (2.8 MB)

Move the red ball with your nose over the letters BRAVO.
The buttons will turn red.
If you open your mouth it will restart.

face13
face13500×678 124 KB

face15
face15769×1456 132 KB

2 Likes
8
9

7.- Draw a mesh of lines.

p299D_mallaFace.aia (2.8 MB)

face33
face33658×671 97.2 KB

Hi @ewpatton,
I think there is a problem here, with the Front Camera it works perfectly, but with the Back Camera when the model is moved to one side, the mesh moves in the opposite direction.

facederecha2
facederecha2738×329 107 KB

1 Like
10

I believe this is a known issue. The code mirrors the front facing camera but then doesn't reset when switching to the rear camera.

11

Perfect extension and I hope the next update is videocall :grin:.

12

The camera don't start in all aia you give
What do I do wrong

13

Maybe it's because of the Android version, I'm using Android 9, it works with MIT Companion and installed, what version do you use?

1 Like
14

blocks
blocks2134×796 112 KB

I have tried to mirror the points around image width center and it worked for rear camera

2 Likes
15

8.- Mirror effect correction.

Thanks @khalid_al-saket, your code corrects the mirror effect with the "Back camera".

p299D_mallaFace.aia (2.8 MB)

16

Great, but can we make a picture of the hand and fingers and by moving the fingers move balls or connect with Arduino and move a robotic arm or something like that

17

There is no extension for hand recognition.

18

...That extension would be very interesting:

19

Hi , Need Help on Face Emotion Recognition.

I am developing Face Emotion Detector application for my final year project, currently iam using "Personal Image Classifier" but its not effective, please any one guide me to do on this effective way.

Using Face recognition extension can i detect face emotion, or any other idea please suggest me ,

Kindest Regards,
Vignesh.

1 Like
20

Yes @Vigneshwaran_Balu, I too have seen that its not much affective...

But I don't know an much affective way...

Good Luck with your app :+1: :iphone:

1 Like