Grok, Elon Musk’s AI tool, has shut down its image generation function for most users following widespread outcry about its use to create sexually explicit and violent images.
The move comes after Musk was threatened with fines, regulatory action and reports of a possible ban on X in the UK.
This tool was used to manipulate images of women to remove their clothes and place them in erotic positions. The practice of doing so except to make payments to customers has been discontinued.
Posting on Musk’s social media network X, Grok said: “Image creation and editing is currently limited to paying subscribers.”
This means that most users of the platform cannot create images using Grok. Those who have their full details and credit card information stored by X, so can be identified if the function is misused.
The image creation capabilities of the public @grok account have been heavily restricted. However, there is also a separate Grok app, which does not share images publicly, on which non-paying users have still reported being able to generate erotic images of women and children.
Research conducted by the Guardian revealed that it was used to make indecent videos of women without their consent, as well as images of women being shot and killed.
Musk is facing the threat of regulatory action from around the world after Grok was used to create non-consensual sexual imagery.
British Prime Minister Keir Starmer on Wednesday threatened to take strict action against the social media company.
He demanded that
Hinting that an actual ban on X should be considered, Starmer said that communications regulator Ofcom “has our full support to take action on this”.
Under the UK’s Online Safety Act, Ofcom has the power to seek a court order to block a website or app in the UK in serious cases. It can also impose a fine of up to 10% of a company’s global turnover.
Starmer said: “This is illegal. We will not tolerate it. I have asked for all options to be put on the table. This is disgusting. The X needs to get its act together and take this content down. We will take action because it is simply not tolerable.”
Thousands of sexually explicit images of women have been created without their consent in the past two weeks, following an update to the Grok image creation feature in late December. Musk has faced repeated public calls to remove or restrict the feature, but so far the social media app has not taken action.
Jess Asato, a Labor MP who is campaigning for better regulation of pornography, said: “While this is a step forward to remove universal access to Grok’s disgusting nudifying features, it still means that paying users can take images of women for lewdness and brutality without their consent. Paying to put semen, pill holes or bikinis on women is still digital sexual assault and XAI needs to ban this feature.” Should be disabled forever.”
Some of the most objectionable content is being created on the X platform, which is integrated with Grok through the Grok Imagine tool.
Research by Paris-based non-profit organization AI Forensics found nearly 800 images and videos created by the Grok Imagine app contained obscene and sexually violent content. Paul Bouchaud, a researcher at AI Forensics, said: “These are completely pornographic videos and look professional.”
A photorealistic AI video seen by the NGO shows a woman tattooed with the slogan “Do not resuscitate” with a knife between her legs. Other images and videos included erotic images, images of undressing women, suggestive poses, and videos depicting full nudity and sexual acts.
“Overall, the material is significantly more revealing than the bikini trend previously seen on X,” Bouchaud said.
X has been contacted for comment.
