X’s Grok chatbot has not stopped accepting requests from users to undress women and, in some cases, obvious minor For AI-generated bikinis. According to some reports, the flood of AI-generated images includes more extreme content that potentially violates laws against non-consensual intimate imagery (NCII) and child sexual abuse material (CSAM). Even in the US, where X owner Elon Musk has close ties to the government, some lawmakers are criticizing the platform – though overt action is still lacking.
Several international regulators have spoken out against the race to strip Grok. UK communications regulator Ofcom said in a statement It had “immediately contacted X and XAI to understand what steps they have taken to comply with their legal duties to protect users in the UK” and will quickly “assess any potential compliance issues that require investigation.” European Commission spokesperson Thomas Regnier said at a press conference Grok’s outputs were “illegal” and “terrible”. IT Ministry of India threatened to strip X’s legal immunity for user-generated posts unless it promptly submits details of actions taken to prevent illegal content. from regulators Australia, Brazil, France and Malaysia are also watching developments.
Tech platforms in the US are largely protected from liability for their users’ posts under Section 230 of the Communications Decency Act, but even Sen. Ron Wyden (D-OR), co-author of the 1996 law, said the rule should not protect a company’s own AI output. “Given that the Trump administration is moving to protect pedophiles, states should take action to hold Musk and Ax accountable,” Wyden. written on bluesky,
Some images created by Grok may also violate the Take It Down Act. Under that law, the DOJ now has the authority to try to impose criminal penalties against individuals who also publish AI-facilitated NCII, while platforms that fail to promptly remove flagged content could be targeted by the Federal Trade Commission beginning in mid-May.
Grok’s mass production of sexual imagery seems to be exactly the kind of thing the Take It Down Act was designed to deal with. “X needs to change this,” said Senator Amy Klobuchar (D-MN), the bill’s lead sponsor. written on stage“If they don’t, my bipartisan Take It Down Act will soon require them to,” Phoebe Keller, a spokeswoman for Klobuchar’s co-sponsor, Senator Ted Cruz (R-TX), declined to comment on the reporting about Groke,
Some lawmakers are calling for new targeted legislation. Representative Jake Auchincloss (D-MA) called Groke’s behavior “bizarre” in a statement and said his proposal, the Deepfake Liability Act, “would make hosting sexual deepfakes of women and children a board-level problem for Musk and (Meta CEO Mark) Zuckerberg.”
But other lawmakers insist that enforcers already have the tools to deal with grok’s actions. “Attorney General (Pam) Bondi has a simple choice: protect the President’s Big Tech friends or protect America’s young people,” Senator Richard Blumenthal (D-CT) said in a statement.
“It is unacceptable that software used by the federal government is vulnerable to such heinous and illegal use”
Representative Madeleine Dean (D-PA), who helped lead the House version of the Take It Down Act, said in a statement that she was “appalled and dismayed by reports that Elon Musk’s Grok chatbot has flooded the Internet with AI-generated explicit images of women and children.” Dean called on Bondi and FTC Chairman Andrew Ferguson “to launch an immediate investigation into Grok and XAI to protect our children, to ensure this never happens again, and to bring these criminals to justice.” Nearly eight months after signing the Take It Down Act, he said, “It is unacceptable that software used by the federal government is vulnerable to such heinous and illegal uses.”
But critics of the Take It Down Act – Including Cyber Civil Rights Initiative (CCRI)which has long pushed for criminalizing NCIS proliferation — warning for months that Donald Trump’s administration could use the law to punish its enemies while applying it casually against allies like Musk and Xi. Trump’s FTC has been largely silent on the recent Ax controversy. The agency did not respond The VergeRequest for comment. However, Justice Department spokeswoman Natalie Baldassarre said in a statement that it “takes AI-generated child sexual abuse material extremely seriously and will aggressively prosecute any creator or owner of CSAM.”
In the absence of federal action in the US, state attorneys general can still investigate companies for actions that could harm their own residents. It is not yet public whether any such investigation is ongoing. California Department of Justice spokeswoman Alyssa Perez would not confirm or deny the existence of any potential or ongoing investigation, but wrote that Attorney General Rob Bonta “is deeply concerned about the harms of chatbots and is committed to ensuring AI safety, especially when it comes to protecting California’s children.” He said Bonta has been “very involved” in such efforts, including ” support state legislation Its goal is to protect children from AI companion chatbots and by connecting directly with AI companies.” California Law refuses Production and distribution of material depicting minors engaging in or simulating sexual conduct, including AI-generated depictions.
New Mexico Attorney General Raul Torrez, whose office has filed several major lawsuits against major tech companies, is another possible candidate for action. “We are extremely concerned by recent reports of various AI platforms, including Grok, lacking basic safeguards to ensure that their users do not violate the dignity and privacy rights of others, especially children,” Torrez said in a statement. “Like social media, we intend to aggressively police this area and use every tool at our disposal to hold technology companies accountable for the harm these products cause.” Geoff Bergen, a spokesman for New York Attorney General Letitia James, said his office is also reviewing the Grok incidents.
“Elon Musk is laughing at the people being victimized by his platform”
At the same time, the Trump administration and some Republican allies in Congress are pushing to prevent states from enacting their own laws regulating the use of AI through recent executive orders and several unsuccessful attempts to codify the restrictions into law. “While the White House works with Republicans to prevent states from regulating AI, Groke is churning out sexualized images of women and children,” Frank Pallone (D-NJ), ranking member of the House Energy and Commerce Committee, said in a statement. “Let’s be clear, Elon Musk is laughing at people being victimized from his platform and President Trump decided to invite him to dinner. Protecting victims is clearly not a priority for either of them.”
At least one Republican criticized the spread of X’s images, though his solution included legislating Trump’s AI executive order into law. “No AI chatbot should be distributing this harmful content, and the company must take immediate action to strengthen its oversight and ensure that Grok cannot violate its terms of service by creating these images,” Senator Marsha Blackburn (R-TN), co-author of the Kids Online Safety Act, said in a statement. Blackburn has previewed his own legislation that he says would codify Trump’s executive order by creating a federal framework for AI legislation, Trump called America AI Act“This is why Congress must act to pass legislation protecting children online,”
