Indonesia Urges Grok AI to Address Wave of Nonconsensual Sexual Deepfakes

TEMPO.CO, Jakarta - The Ministry of Communication and Digital (Komdigi) is looking into AI misuse allegations against social media platform X's chatbot Grok. Users have been known to request Elon Musk's AI assistant to edit people's photos without their consent, which leads to harassment. 

The Director General of Digital Space Oversight at Komdigi, Alexander Sabar, said initial findings found that there were no explicit and adequate settings at Grok to prevent the AI chatbot from generating sexual imagery based on non-consenting individuals. He said, "This poses a serious risk of privacy violations and the infringement of individuals' image rights," in a written statement on Wednesday, January 7, 2026.

Alexander mentioned that this phenomenon is not only related to digital manipulation, but also to the deprivation of individual control over their visual identity, which may result in psychological, social, and reputational harm. Komdigi, he said, is working with Electronic System Providers (PSE) to ensure the existence of effective protection mechanisms for their users.

This coordination focuses on improving content moderation systems, preventing the creation of inappropriate deepfakes, and implementing rapid procedures to address reports of privacy violations and image rights infringement. "Each PSE must ensure that the technologies they provide do not become a means of privacy violation, sexual exploitation, or the degradation of someone's dignity," Alexander said.

He reminded that the obligation to comply with Indonesian laws and regulations is inherent to all PSE operating in Indonesia. Indonesia is authorized to impose administrative sanctions and even terminate access to Grok services and the X platform if non-compliance is detected.

"We urge all parties to use artificial intelligence technology responsibly. The digital space is not lawless; it involves the privacy and image rights of every citizen that must be respected and protected," Alexander stated.

How Did It Start?

A recent report from Copyleaks revealed how Grok's AI technology, which was intended to promote creativity, has been wielded as a crude weapon to produce mass-scale sexual harassment materials within days. Based on Copyleaks' analysis, this harmful trend originated from a few adult content creators' experiments on X a few days before the new year.

Initially, they prompted Grok to generate "sexualized imagery of themselves" to promote their content. But the situation immediately snowballed when other users began issuing similar prompts on unconsenting individuals, most of whom women.

Quickly, Grok transitioned from a tool for consensual self-representation to a "scaled nonconsensual image generation." Hundreds, and possibly thousands, of photos of women, from celebrities like Millie Bobby Brown and K-pop idols to ordinary X users, were prompted to become highly sexualized images.

The main problem lies in the inadequacy of barriers within the Grok system. Unlike competing platforms that generally reject sensitive manipulation commands, Grok instead obediently responded to various requests.

Users found that they could easily input commands such as "change outfits" or "adjust poses" on photos that were not sexual in nature, altering them to become highly sexualized. 

This phenomenon spread like a virus on X. Grok was inundated with altered photos that degrade women's dignity. Copyleaks, through observational review, recorded that the occurrence rate of non-consensual sexual images was an average of one image per minute in the content stream they observed, triggering an extraordinary public outrage.

CEO of Copyleaks, Alon Yamin, emphasized that the impact of unauthorized alterations of photos is highly personal and damaging. The Grok case is concrete evidence that with the growing sophistication and ease of access to generative AI, failures in implementing security standards also occur more frequently.

Critics also highlighted that this incident is a direct consequence of technological advancement without simultaneous innovation in safeguarding vulnerable groups. Grok's developer, xAI, has acknowledged violations from their technology, claiming that the company is working to address the issue. 

Disclaimer: If you or someone you know is experiencing or has experienced or witnessed sexual assault, harassment, and/or gender-based violence, please contact the nearest authorities and organizations. If you’re in Indonesia, you can call Pulih Foundation at (021) 78842580, Komnas Perempuan at (021) 80605399, or the Ministry of Women’s Empowerment and Child Protection’s SAPA Service 129 at 08111 129 129. For immediate dangers, call the police at Call Center 110.

Read: Grok AI Under Fire as Users Exploit Platform to Sexualize Photos

Click here to get the latest news updates from Tempo on Google News