The social media platform X has caused significant outrage after users employed its AI chatbot Grok to change photos of females by taking off their attire.
Numerous instances have surfaced where the technology has been used to virtually strip female subjects to render them in swimwear non-consensually, as well as to put them in sexually explicit scenarios.
Company Reaction and Legal Position
XAI, the organization responsible for Grok, did not respond to requests for comment, aside from an automatically-generated message stating "legacy media lies".
A government spokesperson stated that legislation is being proposed to ban image stripping tools. Under a new law, anyone who distributed such tools would "be subject to a prison sentence and significant penalties".
"The platform is capable of halting these forms of abuse if they chose to," said Clare McGlynn, adding they "appear to enjoy impunity."
Regulator Stance and Tool Capabilities
The communications watchdog Ofcom said tech firms must "assess the danger" of people viewing unlawful material on their networks, but stopped short of confirming whether it was currently examining X or Grok in concerning AI-generated images.
Grok is a free artificial intelligence—with some paid-for features—which responds to X users' prompts when they tag it in a message.
It is commonly used to offer reactions or additional context to other users' remarks, but people on X are also can alter an uploaded picture through its AI image editing tool.
Background of Controversy and Rules
The chatbot has been criticised for enabling users to create images and clips with nudity and sexually suggestive content, and it was earlier alleged to have been involved in generating graphic content of public figures.
"The platform has been allowing the creation and sharing of these images for months without taking any steps and we have not seen any intervention by oversight bodies," the expert added.
XAI's own terms of service prohibits "showing images of individuals in a pornographic fashion".
In a statement, Ofcom said it was illegal to "generate or distribute private pictures or child sexual abuse material" and stated this included explicit AI images generated with AI.
It said platforms such as X were mandated to take "appropriate measures" to "minimize the danger" of users coming across illegal content on their services and to remove it swiftly when they learn of it.
