Gemini Misused To Create Deepfake Terrorism And Child Abuse Material

JAKARTA Google reported the problem of data misuse of Gemini and its artificial intelligence (AI) model to Australian authorities. The company said it had received hundreds of complaints.

There are more than 250 complaints in various parts of the world stating that Google's AI software is used to make deepfakes with terrorism material. Not only that, Google's AI is also used to create extreme violent content.

In addition, Google also reports that it has received 86 reports of child abuse material manufacturing using Gemini software. Dozens of these reports have been submitted to the Australian Electronic Security Commission.

Although there are hundreds of reports regarding the creation of deepfake content of terrorism and dozens of reports regarding the manufacture of child exploitation and abuse material, it is not clear how many reports Google has checked or verified.

Australia's Electronic Security Commission said that the report shows how much technology risks are increasingly sophisticated. Therefore, Google should increase its AI security uniquely preventing harmful matter from being produced.

Although reports of Gemini abuse are increasing in various countries, a Google spokesperson stated that it will not allow the creation or distribution of harmful content, be it violence, terror, exploitation, or child abuse.

"The number of Gemini users' reports we provide to e-Safety represents the total volume of user reports globally, not confirmed policy violations," the spokesperson said. "We are committed to expanding our efforts to help keep Australians safe online."