Apple Quietly Threatens to Remove Grok from App Store Related to Vulgar Deepfake Content

Grok was once threatened with being removed from the App Store (photo: dok. unsplash)

JAKARTA - When Grok was under pressure from various countries at the beginning of this year, Apple was involved in the problem. In fact, the company gave Grok a threat.

As a result of the advanced editing capabilities offered by Grok, Apple threatened to remove the chatbot based on Artificial Intelligence (AI) from the App Store. According to Apple, Grok's capabilities have violated its app store guidelines.

As many parties have found on the X platform, Grok can be used to edit photos while maintaining the authenticity of the subject in the photo. This leads to a high number of sexual deepfakes produced by the xAI-made chatbot.

Based on an exclusive letter obtained by NBC News, Apple is personally pressuring the X and Grok teams to improve their content moderation system. Apple found that Grok's security was so weak that it was widely abused to strip subjects of photos without consent.

X sempat mengirimkan pembaruan aplikasi untuk menanggapi teguran tersebut, tetapi ditolak oleh Apple karena perubahannya dianggap tidak signifikan. Apple meminta perbaikan secara mendalam untuk melindungi pengguna dari konten berbahaya, terutama yang menargetkan wanita dan anak di bawah umur.

Berikut isi surat yang Apple berikan ke senator AS mengenai penolakan mereka terhadap kemampuan pengeditan di Grok.

"We rejected Grok's submission and told the developer that additional changes to fix the violations would be required, or the app could be removed from the App Store."

This threat of deletion explains why xAI is tightening its rules for using its tool amid the editing controversy. The changes include restricting access and banning edits involving photos of real people.

Even though the latest update was finally approved, the latest report shows that Grok still has a security gap. Some users are reportedly still able to trick the system into generating images of women with very open clothing.