French authorities have opened an investigation into the proliferation of sexual deepfake content produced by Grok, an artificial intelligence integrated with the X platform. This step was taken after hundreds of women and teenagers reported that their photos on social media were manipulated into nude images without consent.
The Paris Prosecutor's Office told POLITICO that the investigation includes thousands of non-consensual explicit content created using Grok and widely circulated on X. The case could potentially be punished with a maximum of two years in prison and a maximum fine of 60,000 euros.
Two French lawmakers, Arthur Delaporte and Eric Bothorel, officially reported the case to prosecutors on January 2. Both considered the practice a serious violation of human dignity, especially because it targeted women and minors.
In his letter to prosecutors seen by POLITICO, Delaporte asserted that the AI-based photo manipulation was a form of digital violence that degraded the victim and was carried out without the slightest consent.
The French digital affairs office said three ministers had reported the content, namely Minister of the Economy and Industry Roland Lescure, Minister of State for AI and Digital Affairs Anne Le Hénanff, and Minister of Equality Aurore Bergé. The report was also submitted to the government's online monitoring platform, Pharos, to encourage the rapid removal of content.
The French Commissioner for the Protection of Children, Sarah El Haïry, expressed her anger at the practice and stressed the need for stronger protection for children from the misuse of artificial intelligence.
The case strengthens an investigation that had previously been opened by the French cybercrime unit against X. The investigation originally covered the dissemination of antisemitic content and Holocaust denial by Grok, before being expanded last November.
Until this news was released, X has not given an official response. In a post, the Grok account admitted that there were limited cases where users managed to produce images of underage children with minimal clothing, although it was claimed to be an exception.
Grok said it had implemented safeguards, but was still making improvements to fully block such requests. The European Commission has also not commented on the case.
This case once again highlights the dark side of generative artificial intelligence, when the technology that is touted as revolutionary turns into a tool for mass harassment in the digital space.
The English, Chinese, Japanese, Arabic, and French versions are automatically generated by the AI. So there may still be inaccuracies in translating, please always see Indonesian as our main language. (system supported by DigitalSiber.id)
