🔗 Share this article

The social media platform X has provoked widespread backlash after users employed its AI chatbot Grok to manipulate pictures of women by stripping their garments.

Multiple cases have been seen where the system has been used to remove clothing from women to render them in swimwear non-consensually, as well as to insert them in sexualized contexts.

Company Reaction and Regulatory Position

XAI, the company responsible for Grok, offered no comment to questions, except for an automatically-generated response stating "traditional news outlets lies".

A Home Office representative stated that legislation is being introduced to outlaw nudification tools. Under a new criminal offence, anyone who provided such technology would "receive a prison sentence and substantial penalties".

"X or Grok could prevent these types of abuse if they desired to," said Clare McGlynn, adding they "seem to have a lack of consequences."

Oversight Body Stance and Tool Capabilities

The communications watchdog Ofcom said tech firms must "consider the potential" of people seeing unlawful material on their platforms, but did not specify whether it was currently examining X or Grok in relation to artificial intelligence pictures.

Grok is a no-cost AI assistant—with some additional functions—which reacts to X users' prompts when they mention it in a message.

It is frequently used to give commentary or further context to other users' remarks, but people on X are also can edit an posted photo through its artificial intelligence picture modification function.

Background of Criticism and Rules

The tool has been faulted for enabling users to produce images and clips with nudity and sexually suggestive material, and it was in the past alleged to have been involved in creating graphic material of celebrities.

"X has been enabling the production and distribution of these pictures for a long time without taking any steps and we have not witnessed any intervention by regulators," Professor McGlynn added.

XAI's own terms of service bans "depicting likenesses of people in a graphic manner".

In a comment, Ofcom said it was unlawful to "create or distribute deepfakes or child sexual abuse material" and confirmed this applied to explicit AI images made with AI.

It said services such as X were obligated to take "appropriate actions" to "lessen the risk" of users encountering unlawful content on their services and to remove it promptly when they are notified of it.