Paris Soir - Musk's Grok AI bot barred from undressing images after backlash

Pressure had been building on Musk's xAI, the developer of Grok, to rein in the chatbot after it was used to generate a flood of lewd photos of women and children.

The company's announcement came after California's attorney general launched an investigation into xAI over the sexually explicit material, and multiple countries either blocked access to Grok or opened their own probes.

X said Wednesday it will "geoblock the ability" of all Grok and X users to create images of people in "bikinis, underwear, and similar attire" in those jurisdictions where such actions are deemed illegal.

"We have implemented technological measures to prevent the Grok account from allowing the editing of images of real people in revealing clothing," X's safety team said, adding that the restriction applied to "all users" without exceptions.

In an "extra layer of protection," image creation and the ability to edit photos via X's Grok account was now only available to paid subscribers, they added.

But the Philippines said Thursday it planned to block Grok nationwide, joining Southeast Asian neighbours Malaysia and Indonesia.

"By tonight or within today, we are expecting (Grok) to be blocked in the entire Philippines," Renato Paraiso, acting executive director of the country's cybercrime center, told AFP.

He said X's pledge to limit access would have no effect on the plans, adding that the government would watch to see if the platform follows through on its promises.

"We need to clean the internet now because much toxic content is appearing, especially with the advent of AI," Philippine telecommunications secretary Henry Rhoel Aguda said.

- 'Shocking' -

Grok's so-called "Spicy Mode" feature allowed users to create deepfakes using simple text prompts such as "put her in a bikini" or "remove her clothes."

An analysis of more than 20,000 Grok-generated images by Paris non-profit AI Forensics found that more than half depicted "individuals in minimal attire" -- most of them women, and two percent appearing to be minors.

The European Commission, which acts as the EU's digital watchdog, had said it would "carefully assess" additional measures taken by X to stop the deluge to ensure "they effectively protect citizens."

California Governor Gavin Newsom said that xAI's "vile" decision to allow sexually explicit deepfakes to proliferate prompted him to urge the state's Attorney General Rob Bonta to hold the company accountable.

"The avalanche of reports detailing the non-consensual, sexually explicit material that xAI has produced and posted online in recent weeks is shocking," Bonta said on Wednesday.

"We have zero tolerance for the AI-based creation and dissemination of nonconsensual intimate images or of child sexual abuse material."

Bonta said the California investigation would determine whether xAI violated state law after the explicit imagery was "used to harass people across the internet."

- Posts removed -

Adding further pressure onto Musk's company Wednesday, a coalition of 28 civil society groups submitted open letters to the CEOs of Apple and Google, urging them to ban Grok and X from their app stores amid the surge in sexualized images.

Indonesia on Saturday became the first country to block access to Grok entirely, with Malaysia following the next day.

On Thursday, Malaysia's communications minister said national regulators had found that X's steps to prevent Grok generating indecent images were "not done in totality."

If X can successfully deactivate and prevent the generation of such online content considered harmful, Malaysia will lift the temporary restriction imposed on Grok, Minister Fahmi Fadzil said.

India has said X removed thousands of posts and hundreds of user accounts in response to its complaints.

And Britain's Ofcom media regulator on Thursday called X's new measures a "welcome development."

"However, our formal investigation remains ongoing," an Ofcom spokesperson said, referring to a probe it announced this week into whether X failed to comply with British law over the sexual images.

L.Leduc--PS