Makers of AI chatbots that put children at risk will face massive fines or have their services blocked in the UK under changes to the law to be announced by Keir Starmer on Monday.
Ministers are planning to “crack down on vile illegal content created by AI”, fueled by a public outcry in the UK last month after Elon Musk’s ex stopped his Grok AI tool from creating racy images of real people.
With more and more children using chatbots for everything from help with their homework to mental health support, the government said it would “move swiftly to close the legal loophole and force all AI chatbot providers to comply with the Unlawful Content duties in the Online Protection Act or face consequences for breaking the law”.
Starmer also plans to speed up new restrictions on children’s use of social media if MPs agree to a potential under-16 ban following a public consultation. This means that any changes to children’s use of social media, which could include other measures like restricting infinite scrolling, could happen as soon as this summer.
But the Conservatives dismissed claims the government was acting quickly as “more smoke and mirrors” because the consultation has not yet started.
Shadow Education Secretary Laura Trott said, “Claiming they are taking ‘urgent action’ is simply not credible when their so-called urgent advice does not exist.” “Labour has repeatedly said it has no view on whether under-16s should be prevented from accessing social media. This is not good enough. I am clear that we should prevent under-16s from accessing these platforms.”
The move comes after online regulator Ofcom admitted that it did not have the powers to take action against Grok as images and videos created by chatbots without searching the internet are not within the scope of existing laws unless it falls into the category of pornography. The change to bring AI chatbots under the Online Protection Act could happen within a few weeks, although the flaws have been known about for more than two years.
“Technology is advancing really quickly and the law has to follow suit,” Starmer said. “The action we took on Grok sends a clear message that no platform deserves a free hand. Today we are closing the loopholes that put children at risk, and laying the groundwork for further action.”
Companies that breach the Online Security Act could face penalties of up to 10% of global revenue and regulators could apply to the courts to block their connections in the UK.
If AI chatbots are used specifically as search engines, to produce pornography, or to operate in user-to-user contexts, they are already covered by the Act. But they can be used to create content that encourages people to harm themselves or take their lives without facing sanction, or even generate child sexual abuse material. This is the loophole the government says it wants to close.
The NSPCC’s chief executive, Chris Sherwood, said that young people were contacting its helpline reporting the harm caused by AI chatbots and that they did not trust tech companies to design them safely.
In one case, a 14-year-old girl who spoke to an AI chatbot about her eating habits and physical deformities was given false information. In others, he has seen “young people who are self-harming, even as they are given more self-harming material”.
“Social media has brought great benefits to young people, but also caused great harm,” Sherwood said. “AI is going to be on steroids if we’re not careful.”
OpenAI, the $500 billion San Francisco startup behind ChatGPT, one of the UK’s most popular chatbots, and xAI, which creates Grok, were contacted for comment.
Since 16-year-old Adam Raine of California took his own life, his family alleges, “after months of encouragement from ChatGPT”, OpenAI has introduced parental controls and age-prediction technology to restrict access to potentially harmful content.
The government is also consulting on forcing social media platforms to make it impossible for users to send and receive nude photos of children – a practice that is already illegal.
Liz Kendall, the technology secretary, said: “We will not wait for families to take the necessary action, so we will tighten the rules on AI chatbots and we are laying the groundwork so we can act at speed on the results of the consultation with young people and on social media.”
The Molly Rose Foundation, which was founded by the father of 14-year-old Molly Russell, who committed suicide after viewing harmful material online, called the moves “a welcome downpayment”. But it called on the Prime Minister to commit to a new Online Safety Act “that strengthens regulation and makes clear that product safety and the well-being of children is the cost of doing business in the UK”.
