The social media platform X has provoked major anger after users used its AI chatbot Grok to change photos of women by taking off their clothing.
Several cases have surfaced where the tool has been used to digitally undress women to show them in bikinis non-consensually, as well as to insert them in sexually explicit contexts.
Company Reaction and Legal Stance
XAI, the firm that created Grok, did not respond to inquiries, aside from an automatically-generated message stating "traditional news outlets lies".
A government official stated that legislation is being introduced to prohibit nudification software. Under a new law, anyone who supplied such technology would "be subject to a jail term and significant fines".
"X or Grok is capable of halting these forms of abuse if they desired to," said Clare McGlynn, adding they "seem to have a lack of consequences."
Regulator Stance and Tool Functionality
The regulator Ofcom said technology companies must "assess the potential" of people accessing unlawful material on their networks, but did not specify whether it was presently examining X or Grok in relation to artificial intelligence images.
Grok is a no-cost artificial intelligence—with some paid-for functions—which reacts to X users' queries when they tag it in a message.
It is frequently used to provide reactions or more context to other users' comments, but people on X are also capable of modify an uploaded image through its artificial intelligence picture modification function.
Background of Criticism and Policy
The system has been criticised for enabling users to produce photos and videos with nudity and sexually suggestive material, and it was earlier alleged to have been involved in generating sexually explicit content of celebrities.
"The platform has been allowing the generation and distribution of these pictures for a long time without taking any measures and we have not witnessed any action by regulators," the expert added.
XAI's own terms of service bans "showing likenesses of people in a sexually explicit way".
In a comment, Ofcom said it was against the law to "generate or share non-consensual intimate images or illegal content" and confirmed this covered sexual deepfakes created with AI.
It said services such as X were obligated to take "suitable actions" to "lessen the risk" of users encountering unlawful material on their platforms and to remove it quickly when they are notified of it.
