Grok Bikin Panas, Jaksa California Kirim Surat Tegas ke xAI soal Deepfake Seksual

JAKARTA - The California Attorney General's Office has officially sent a cease-and-desist order to xAI, Elon Musk's artificial intelligence company. This was done following the alleged use of the Grok chatbot to create sexual deepfake content without consent, including those involving women and minors.

The firm action was announced on Friday local time, just days after California authorities opened an investigation into xAI. The government judged the practice to be illegal and potentially facilitating the spread of child sexual abuse material (CSAM).

"Today, I am sending a cease-and-desist letter to xAI, demanding that the company immediately cease the creation and distribution of non-consensual deepfake intimate images and child sexual abuse material," California Attorney General Rob Bonta said in a statement. "The creation of this kind of material is illegal. I fully expect xAI to comply immediately. California has zero tolerance for CSAM."

According to the Attorney General's office, xAI is suspected of not only negligence, but "facilitating large-scale production" of non-consensual nude images. The material, they said, has been used to harass women and girls on various internet platforms. Authorities have given xAI a five-day deadline to prove that they have taken concrete steps to stop the practice.

The main highlight is the "spicy mode" feature in Grok, which is explicitly designed to produce adult content. This feature is said to open a big gap for the creation of sexual deepfakes without permission. Although xAI claims to have tightened restrictions on image editing features since last Wednesday, the move was not considered sufficient by the California authorities.

The Grok problem is no longer local in scale. Japan, Canada, and the UK are reported to have opened similar investigations. Meanwhile, Malaysia and Indonesia chose a quick move by temporarily blocking the platform. With this global escalation, the pressure on xAI is felt, not only from US state regulators.

X's safety account previously tried to distance itself from the illegal activities of users. "Anyone who uses or orders Grok to create illegal content will receive the same consequences as if they uploaded illegal content," the account wrote. However, this statement does not seem to have been enough to dampen criticism.

Media efforts to seek an official response from xAI also came to a dead end. TechCrunch also reported that their request for comment was only answered by an automated email that read, "Legacy Media Lies." The California Attorney General's Office itself stated that it was ready to provide additional context regarding this case.

The Grok case adds to a long list of ethical issues in the era of free generative AI. The surge in non-consensual sexual content is a cross-platform problem, not just X's. This issue has even attracted the attention of the US Congress. On Thursday, January 15, a number of lawmakers sent a letter to executives of major technology companies such as X, Reddit, Snap, TikTok, Alphabet, and Meta, questioning their strategies to curb the proliferation of sexual deepfakes.

For regulators, the message is clear: AI innovation can go fast, but the legal brakes must still work. Otherwise, the technology that is supposed to be smart turns into a tool for mass harassment.