Help me please !(picture)

Artificial Intelligence
1

How can I prevent the user from uploading an image that is offensive or contains violence or bullying on my application?
Love
Malak

2

Hello Malak

I think you have already been advised that your App would require AI (Artificial Intelligence) in order to block such images. That requires a lot of work and a lot of testing, it is not something where the Power Users could help you more, and note - even AI on a desktop computer would not be very accurate, my guess would be around 66% accuracy because of the variation and range of images that qualify.

Let's see what the AI gurus think.

2 Likes
3

Hello malak.
I think the best way is to have IT under review by an admin before uploading IT.
So the admin can decide if it's offensieve, contains violence or bullying.
Cheers
Sketch

1 Like
4

Can you help me with this

5

Can I specify some pictures, for example

6

Currently @Sketch's idea looks the best.

But if you are creating something like a group forum, then you can also appoint moderators (like the ones on this community!)

2 Likes
7

My application is about controlling all social applications where if the user wants to send me an offensive message, he prevents him and I wanted to develop it to pictures as well

8

Maybe this can help you: https://cloud.google.com/vision/docs/detecting-safe-search

9

That is an interesting idea!

But -

  • I don't think it is possible to control what has been posted on some other application by your app ... discussed in the previous thread.
  • It has already been mentioned that to prevent the pictures, the implementation of AI is required ... again not possible.
  • Users would also not like to install an app that can read all their messages and media sent to them.
  • Also, how would you control something like this?
    --- Say user A sends a message to user B.
    --- User B has installed your app to control the messages he receives.
    --- You cannot control what user A sends as he doesn't have your app, and user B receives the messages from the respective platform's server. So your app has no control of the message unless you have access to the servers.

Have you thought about this before? Do you have a workaround?

10

Of course, I thought about this and thought that it should be within the application not to show the offensive messages sent

11

Hi Malak

What you have not thought about is how it could be done. Social Media Platforms have huge arrays of computers (large warehouses) dedicated to finding offensive images and text. Despite their colossal efforts, they need to invest in more computing power and individuals to deliver a better scan of this material.

Given the scale of the problem, you can see there is very little that can be done on a smartphone or similar small devices to filter-out offensive material.

If you are trying to protect Users who receive offensive material, a different approach to your App would be have tools in the App for the User to store this material and create a report with it. The report could be sent to the platform from which it came, or to the Police. In many countries there are charitable organisations whose volunteers collate this type of data to send on the the platform that didn't stop it getting through, or try to prosecute the sender.