JAKARTA - A court in the Netherlands has handed down a rare but harsh verdict against Elon Musk's artificial intelligence technology. In an interim ruling, the Amsterdam court ordered the xAI company and the Grok chatbot to stop the creation and distribution of images that "strip off the clothes" of a person - both adults and children - without explicit consent.
This step is one of the first legal interventions in Europe that directly targets the generative capabilities of AI in creating non-consensual sexual content, an issue that is heating up amid a surge in the use of "nudifier" technology.
The court confirmed that Grok should not produce or distribute images that feature individuals in a nude or semi-nude state without permission. If violated, xAI is threatened with a fine of up to 100,000 euros per day - a figure that makes CFOs tremble.
Not only that, the court also ordered that the Grok service should not be available on the X platform as long as it still violates the provisions.
This case was filed by the non-profit organization Offlimits together with the Victims Support Fund, which has been actively dealing with cases of digital-based sexual violence. In the trial, the plaintiffs showed that Grok was still able to manipulate people's images into sexual content without permission, even after xAI claimed to have tightened the security system.
Offlimits director Robbert Hoving said that responsibility cannot be shifted to users alone. "The burden is on companies to ensure their technology is not misused," he said.
On the other hand, xAI argued that they could not completely prevent abuse by malicious users. They also said they had limited certain features, including only opening access to image generation to paid users. However, the judge considered this step to be insufficient.
This ruling comes amid the European Union's strict oversight of AI technology through the Digital Services Act. In fact, the European Commission has previously opened an investigation into the X platform regarding the risk of illegal content generated by AI, including the manipulation of explicit images.
The European Parliament also joined in raising tensions by supporting a ban on AI applications that can create fake sexual images - signaling that the era of "AI free experimentation" in Europe is beginning to be replaced by the era of "AI closely monitored".
More broadly, this ruling has the potential to be an important precedent for the global AI industry. If this trend continues, tech companies can no longer just say "we just make tools" - because the law is starting to see them as parties who share responsibility for the impact.
Follow VOI Whatsapp Channel
The English, Chinese, Japanese, Arabic, and French versions are automatically generated by the AI. So there may still be inaccuracies in translating, please always see Indonesian as our main language. (system supported by DigitalSiber.id)
