Elon Musk's social media network X has provoked significant backlash after people used its AI chatbot Grok to change pictures of females by removing their garments.
Several cases have emerged where the technology has been employed to virtually strip women to make them appear in swimwear non-consensually, as well as to place them in sexualized scenarios.
Company Response and Regulatory Stance
XAI, the organization that created Grok, remained silent to questions, except for an automatically-generated message stating "traditional news outlets lies".
A government official stated that new laws is being brought forward to ban nudification software. Under a proposed criminal offence, anyone who supplied such tools would "face a jail term and substantial fines".
"The platform has the ability to stop these types of abuse if they wanted to," said a law professor, adding they "operate with impunity."
Regulator Stance and Platform Functionality
The communications watchdog Ofcom said platforms must "assess the danger" of people seeing unlawful material on their services, but did not specify whether it was presently investigating X or Grok in relation to AI-generated images.
Grok is a no-cost AI assistant—with some additional functions—which reacts to X users' prompts when they mention it in a post.
It is commonly used to offer commentary or more background to other users' comments, but people on X are also can modify an posted photo through its artificial intelligence picture modification tool.
Background of Controversy and Rules
The tool has been criticised for allowing users to create pictures and videos with nudity and sexually suggestive material, and it was earlier accused of creating graphic content of celebrities.
"X has been allowing the creation and spread of these pictures for a long time without taking any measures and we have not witnessed any intervention by regulators," Professor McGlynn added.
XAI's own acceptable use policy bans "showing likenesses of persons in a graphic fashion".
In a comment, Ofcom said it was unlawful to "generate or distribute deepfakes or child sexual abuse material" and stated this included sexual deepfakes created with AI.
It said platforms such as X were required to take "suitable measures" to "minimize the danger" of users coming across unlawful material on their services and to take down it swiftly when they become aware of it.