The online network X has caused significant anger after individuals used its AI chatbot Grok to manipulate images of women by stripping their clothing.
Several cases have emerged where the system has been employed to virtually strip female subjects to show them in swimwear without consent, as well as to insert them in inappropriate scenarios.
Developer Response and Regulatory Position
XAI, the company responsible for Grok, did not respond to inquiries, other than an standard response stating "legacy media lies".
A government spokesperson stated that new laws is being brought forward to ban image stripping tools. Under a new criminal offence, anyone who distributed such technology would "face a jail term and substantial fines".
"X or Grok could prevent these types of abuse if they chose to," said Clare McGlynn, adding they "seem to have impunity."
Regulator Stance and Tool Functionality
The communications watchdog Ofcom said platforms must "consider the potential" of people accessing illegal content on their services, but did not confirm whether it was currently looking into X or Grok in relation to artificial intelligence images.
Grok is a free chatbot—with some paid-for functions—which reacts to X users' queries when they reference it in a message.
It is often used to offer commentary or additional background to other users' remarks, but people on X are also can edit an posted image through its AI image editing feature.
Background of Criticism and Policy
The tool has been criticised for enabling users to produce images and videos with nudity and sexualised content, and it was in the past alleged to have been involved in making sexually explicit material of public figures.
"The platform has been allowing the creation and sharing of these pictures for months without taking any measures and we have not seen any intervention by oversight bodies," Professor McGlynn added.
XAI's own usage rules bans "showing images of persons in a sexually explicit manner".
In a comment, Ofcom said it was unlawful to "create or share non-consensual intimate images or illegal content" and confirmed this included sexual deepfakes made with AI.
It said networks such as X were obligated to take "appropriate actions" to "lessen the danger" of users encountering illegal content on their platforms and to delete it promptly when they are notified of it.
