The social media platform X has sparked widespread backlash after individuals used its AI chatbot Grok to change pictures of females by taking off their clothing.
Several cases have surfaced where the technology has been used to remove clothing from women to render them in swimwear without consent, as well as to put them in inappropriate contexts.
Developer Reaction and Regulatory Stance
XAI, the company responsible for Grok, offered no comment to inquiries, except for an standard message stating "traditional news outlets lies".
A government representative stated that new laws is being proposed to ban image stripping tools. Under a upcoming law, anyone who supplied such tools would "face a jail term and heavy penalties".
"X or Grok is capable of halting these forms of misuse if they chose to," said a law professor, adding they "operate with impunity."
Regulator Position and Platform Capabilities
The regulator Ofcom said technology companies must "evaluate the risk" of people seeing unlawful material on their services, but did not confirm whether it was presently investigating X or Grok in regarding AI-generated pictures.
Grok is a free AI assistant—with some additional features—which answers X users' queries when they mention it in a tweet.
It is commonly used to give commentary or further background to other users' remarks, but people on X are also can alter an uploaded image through its artificial intelligence picture modification function.
Background of Controversy and Policy
The chatbot has been condemned for enabling users to create images and clips with nudity and sexualised content, and it was earlier accused of making graphic content of celebrities.
"X has been allowing the production and distribution of these images for a long time without taking any measures and we have yet to see any challenge by oversight bodies," Professor McGlynn added.
XAI's own terms of service prohibits "depicting likenesses of people in a graphic fashion".
In a comment, Ofcom said it was illegal to "create or share non-consensual intimate images or child sexual abuse material" and clarified this included explicit AI images made with AI.
It said services such as X were mandated to take "suitable actions" to "lessen the risk" of users coming across unlawful material on their services and to remove it quickly when they learn of it.
