California authorities have announced an investigation into the output of Elon Musk’s Grok.
The state’s top lawyer said Grok, an AI tool and image generator made by Musk’s company xAI, makes it easier to harass women and girls with deepfake images on XAI and elsewhere online.
“The reports detailing non-consensual, sexually explicit material produced by xAI and posted online in recent weeks are shocking,” Rob Bonta, California’s attorney general, said in a statement. “I urge XAI to take immediate action to ensure this does not escalate further.”
Bonta’s office is investigating whether and how xAI violated state law.
On the issue, California Governor Gavin Newsom called for an investigation into “Grok’s disgusting dissemination of child porn on this website.”
A tweet from their official account read, “XAI’s decision to create and host a breeding ground for predators to spread non-consensually sexually explicit AI deepfakes, including digitally undressed images of children, is abhorrent.”
The same day, Musk denied that Grok was being used to spread nude photos of minors. He wrote on X: “I am not aware of any nude underage images made by Grok. Virtually zero.”
About two weeks ago, the AI tool itself said it had generated “images depicting minors in minimal clothing” when asked by users.
In recent weeks, Bonta said, there has been a flood of reports that Grok users are taking photos of women or children found online and using xAI bots to virtually disarm them.
According to the Attorney General’s Office, Grok’s image creation model features xAI promoted as a “spicy mode” for creating and editing sexual content, including images.
Last week, an analysis of more than 20,000 Grok-generated images by Paris nonprofit AI Forensics found that more than half depicted “individuals in minimal attire” – most of them women, and 2% appeared to be under the age of 18.
According to Bonta, the images generated by Grok are being used to harass public figures as well as specific social media users.
Three Democratic US senators last week called on Apple and Google to remove X and Grok’s apps from their app stores in response to the flood of sexual images. Both the tech giants have remained silent in response.
xAI has faced a global backlash over erotic deepfake images. Indonesia on Saturday became the first country to completely block access to Grok, followed by neighboring Malaysia on Sunday. India said on Sunday that X had removed thousands of posts and hundreds of user accounts in response to its complaints.
Britain’s Ofcom media regulator said on Monday it was launching an investigation into whether X failed to follow UK law on sexual images. France’s commissioner for children, Sarah El Heri, said on Tuesday that she had sent the images produced by Grok to French prosecutors, Arcom media regulator and the European Union.
The European Commission, which acts as the EU’s digital watchdog, has ordered Ax to retain all internal documents and data related to Grok until the end of 2026 in response to the outcry.
