The online network X has caused significant backlash after people employed its AI chatbot Grok to alter images of females by stripping their clothing.
Several instances have emerged where the tool has been employed to digitally undress women to make them appear in bikinis non-consensually, as well as to insert them in inappropriate scenarios.
Company Reaction and Regulatory Position
XAI, the company behind Grok, did not respond to requests for comment, except for an auto-reply response stating "traditional news outlets lies".
A Home Office representative stated that new laws is being introduced to prohibit nudification software. Under a proposed criminal offence, anyone who distributed such tools would "receive a jail term and significant penalties".
"X or Grok has the ability to stop these forms of misuse if they wanted to," said a law professor, adding they "operate with a lack of consequences."
Regulator Position and Tool Functionality
The regulator Ofcom said tech firms must "evaluate the danger" of people seeing unlawful material on their platforms, but did not confirm whether it was presently examining X or Grok in regarding artificial intelligence images.
Grok is a no-cost AI assistant—with some premium features—which answers X users' prompts when they reference it in a post.
It is commonly used to offer reactions or more background to other posters' remarks, but people on X are also capable of alter an posted photo through its AI image editing function.
History of Criticism and Rules
The tool has been criticised for allowing users to generate photos and clips with nudity and sexually suggestive material, and it was in the past alleged to have been involved in making sexually explicit content of celebrities.
"X has been permitting the creation and sharing of these pictures for months without taking any steps and we have not seen any action by oversight bodies," the expert added.
XAI's own terms of service prohibits "depicting likenesses of individuals in a graphic way".
In a comment, Ofcom said it was unlawful to "create or distribute private pictures or child sexual abuse material" and stated this included sexual deepfakes made with AI.
It said platforms such as X were mandated to take "necessary actions" to "minimize the danger" of users coming across illegal material on their services and to take down it swiftly when they learn of it.
