The social media platform X has sparked widespread backlash after users employed its artificial intelligence assistant Grok to alter pictures of females by taking off their clothing.
Several cases have been seen where the tool has been employed to remove clothing from female subjects to make them appear in swimwear without permission, as well as to put them in sexually explicit contexts.
Company Reaction and Legal Position
XAI, the firm responsible for Grok, offered no comment to requests for comment, other than an automatically-generated response stating "legacy media lies".
A government spokesperson stated that legislation is being proposed to outlaw nudification software. Under a new criminal offence, anyone who provided such tools would "face a jail term and significant penalties".
"The platform is capable of halting these types of misuse if they desired to," said Clare McGlynn, adding they "operate with a lack of consequences."
Regulator Stance and Platform Capabilities
The regulator Ofcom said tech firms must "assess the risk" of people seeing unlawful material on their services, but did not specify whether it was presently examining X or Grok in concerning artificial intelligence images.
Grok is a free chatbot—with some additional features—which answers X users' prompts when they reference it in a post.
It is commonly used to provide reactions or additional context to other users' remarks, but people on X are also capable of edit an posted picture through its artificial intelligence picture modification function.
History of Criticism and Policy
The chatbot has been condemned for permitting users to generate pictures and clips with explicit content and sexualised content, and it was in the past alleged to have been involved in creating graphic content of celebrities.
"X has been allowing the creation and sharing of these images for a long time without taking any action and we have yet to see any action by regulators," the expert added.
XAI's own usage rules bans "portraying images of people in a sexually explicit fashion".
In a statement, Ofcom said it was illegal to "generate or share deepfakes or child sexual abuse material" and clarified this covered sexual deepfakes generated with AI.
It said networks such as X were mandated to take "necessary steps" to "lessen the danger" of users coming across unlawful material on their services and to take down it promptly when they become aware of it.
