Criminals have used Grok, Elon Musk’s AI, to create child sexual abuse imagery, the Internet Watch Foundation (IWF) has reported.
For days, the IWF has been receiving reports from internet users that Grok had created child abuse images, but that content hadn’t crossed the threshold into illegal content.
Now, it has, says the IWF.
It is one of the few organisations around the world that is allowed to proactively track down child abuse material and found the material in a dark web forum.
The users sharing the material boasted about how they’d used Grok to create it.
“Following reports that the AI chatbot Grok has generated sexual imagery of children, we can confirm our analysts have discovered criminal imagery of children aged between 11 and 13 which appears to have been created using the tool,” said Ngaire Alexander, head of hotline at the IWF.
X and xAI, both owned by billionaire Elon Musk, have come under fire in recent days after numerous users, mainly women, posted saying they’d seen AI-generated sexual images of themselves on X.
Robots with human-type capabilities are coming this year, says Nvidia CEO
Ofcom makes ‘urgent contact’ with X over concerns Grok AI can generate ‘sexualised images of children’
Is the AI bubble about to burst? If so the consequences could be dire
Since the start of the new year, X users – mainly women – have reported that accounts have used Grok to generate images of them without clothing.
There are also several cases where Grok has created sexualised images of children, according to analysis by news agency Reuters.
Ms Alexander added: “The imagery we have seen so far is not on X itself, but a dark web forum where users claim they have used Grok Imagine to create the imagery, which includes sexualised and topless imagery of girls.
“The imagery we have seen would be considered Category C imagery under UK law.
“The user then uses the Grok imagery as a jumping off point to create much more extreme, Category A, video using a different AI tool. The harms are rippling out.
“There is no excuse for releasing products to the global public which can be used to abuse and hurt people, especially children.
Read more technology news:
Robots with human-type capabilities are coming
Porn users ‘still seeing porn with no age checks’
Both Ofcom and the Technology Secretary Liz Kendall have warned X to take action over the abusive imagery being created by its AI.
Yesterday, Ms Kendall said: “What we have been seeing online in recent days has been absolutely appalling, and unacceptable in decent society.”
“X needs to deal with this urgently. It is absolutely right that Ofcom is looking into this as a matter of urgency and it has my full backing to take any enforcement action it deems necessary.”
In an interview with Sky News, the minister for AI and online safety, Kanishka Narayan, said the images being created by Grok were “completely unacceptable”.
He was asked whether the government was willing to risk its relationship with the Trump administration in order to fully enforce the UK’s online safety rules against X.
President Trump and his administration, as well as Elon Musk, have been vocal critics of the UK and EU authorities when they try to take action against US companies.
“The American president is as against child sexual abuse and violence against women and girls as I think that the British public and the British government is,” said Mr Narayan. “So that’s the first thing to say.
“The second thing to say is that this is an area of domestic policy. We will continue to make sure that the public is kept safe from those egregious examples of both sexual abuse and sexual harassment online.”
On Wednesday, Mr Musk said a new version of Grok had been released and urged users to update their app, although it was not immediately clear what updates the new version contained.
Be the first to get Breaking News
Install the Sky News app for free
The tech tycoon has previously insisted that “anyone using Grok to make illegal content will suffer the same consequences as if they uploaded illegal content”.
X has said it takes action against illegal content, including child sexual abuse material, “by removing it, permanently suspending accounts, and working with local governments and law enforcement as necessary”.
Sky News has contacted X for comment.
