The online network X has caused major outrage after people used its AI chatbot Grok to manipulate photos of women by taking off their garments.
Numerous instances have been seen where the technology has been used to virtually strip women to render them in swimwear without consent, as well as to place them in sexually explicit contexts.
Company Response and Legal Position
XAI, the company behind Grok, remained silent to requests for comment, aside from an auto-reply message stating "legacy media lies".
A government spokesperson stated that legislation is being introduced to prohibit nudification technology. Under a new criminal offence, anyone who distributed such tools would "be subject to a prison sentence and heavy fines".
"The platform is capable of halting these types of abuse if they chose to," said Clare McGlynn, adding they "appear to enjoy a lack of consequences."
Oversight Body Position and Tool Functionality
The communications watchdog Ofcom said tech firms must "assess the danger" of people seeing unlawful material on their networks, but stopped short of confirming whether it was presently looking into X or Grok in relation to AI-generated pictures.
Grok is a no-cost artificial intelligence—with some additional features—which reacts to X users' requests when they mention it in a message.
It is commonly used to provide commentary or further context to other users' remarks, but people on X are also can modify an posted picture through its artificial intelligence picture modification function.
Background of Controversy and Policy
The chatbot has been condemned for allowing users to generate pictures and footage with nudity and sexually suggestive content, and it was previously accused of creating sexually explicit content of public figures.
"X has been permitting the generation and spread of these pictures for a long time without taking any action and we have yet to see any intervention by oversight bodies," Professor McGlynn added.
XAI's own terms of service prohibits "portraying likenesses of people in a pornographic manner".
In a comment, Ofcom said it was unlawful to "make or circulate non-consensual intimate images or illegal content" and confirmed this included explicit AI images made with AI.
It said platforms such as X were mandated to take "suitable steps" to "reduce the danger" of users encountering illegal material on their services and to remove it swiftly when they are notified of it.
