The online network X has sparked widespread outrage after individuals used its AI chatbot Grok to change pictures of women by stripping their clothing.
Numerous cases have surfaced where the technology has been used to remove clothing from female subjects to show them in bikinis without permission, as well as to insert them in sexually explicit contexts.
Company Reaction and Regulatory Stance
XAI, the firm behind Grok, did not respond to requests for comment, aside from an standard message stating "traditional news outlets lies".
A Home Office spokesperson stated that legislation is being proposed to outlaw nudification tools. Under a upcoming criminal offence, anyone who provided such tools would "be subject to a jail term and significant fines".
"The platform is capable of halting these types of misuse if they desired to," said a law professor, adding they "operate with a lack of consequences."
Regulator Position and Tool Functionality
The regulator Ofcom said technology companies must "assess the danger" of people viewing illegal content on their networks, but did not confirm whether it was currently investigating X or Grok in relation to AI-generated images.
Grok is a no-cost chatbot—with some premium features—which reacts to X users' requests when they tag it in a tweet.
It is frequently used to give commentary or further context to other posters' remarks, but people on X are also can edit an posted photo through its artificial intelligence picture modification function.
History of Controversy and Rules
The system has been faulted for allowing users to generate photos and videos with explicit content and sexually suggestive content, and it was previously accused of creating sexually explicit content of celebrities.
"X has been allowing the generation and spread of these images for a long time without taking any steps and we have not seen any action by oversight bodies," Professor McGlynn added.
XAI's own usage rules bans "portraying likenesses of persons in a pornographic manner".
In a comment, Ofcom said it was illegal to "generate or share deepfakes or child sexual abuse material" and clarified this included sexual deepfakes created with AI.
It said networks such as X were mandated to take "suitable actions" to "lessen the danger" of users encountering unlawful material on their services and to take down it quickly when they become aware of it.
