Elon Musk's social media network X has caused significant outrage after users utilized its artificial intelligence assistant Grok to manipulate images of women by taking off their garments.
Multiple cases have been seen where the technology has been employed to remove clothing from women to render them in swimwear non-consensually, as well as to insert them in sexualized situations.
Company Response and Regulatory Position
XAI, the firm responsible for Grok, did not respond to questions, aside from an automatically-generated message stating "legacy media lies".
A government representative stated that new laws is being proposed to prohibit nudification tools. Under a new criminal offence, anyone who distributed such tools would "be subject to a prison sentence and heavy penalties".
"X or Grok has the ability to stop these types of misuse if they desired to," said Clare McGlynn, adding they "seem to have impunity."
Oversight Body Stance and Tool Capabilities
The regulator Ofcom said platforms must "evaluate the potential" of people seeing illegal content on their networks, but stopped short of confirming whether it was at this time looking into X or Grok in regarding artificial intelligence pictures.
Grok is a free AI assistant—with some paid-for features—which reacts to X users' prompts when they mention it in a tweet.
It is commonly used to provide reactions or more context to other users' comments, but people on X are also capable of edit an uploaded photo through its AI image editing feature.
History of Criticism and Policy
The tool has been condemned for enabling users to create photos and clips with nudity and sexualised material, and it was in the past accused of generating sexually explicit content of celebrities.
"The platform has been enabling the generation and distribution of these images for months without taking any measures and we have not witnessed any action by oversight bodies," Professor McGlynn added.
XAI's own terms of service prohibits "depicting images of individuals in a sexually explicit way".
In a comment, Ofcom said it was illegal to "make or circulate private pictures or child sexual abuse material" and stated this applied to explicit AI images generated with AI.
It said services such as X were required to take "necessary measures" to "minimize the risk" of users coming across unlawful content on their platforms and to remove it promptly when they are notified of it.
