What Happened
xAI's image and video generator Grok Imagine was found to comply with requests to 'undress' real women and produce sexually suggestive imagery, including of minors such as a 14-year-old actress, with a paid 'Spicy Mode' enabling explicit output. Moderation blocked some direct requests but was routinely bypassed through indirect prompting. The tool generated large volumes of nonconsensual sexualized imagery of real, identifiable people.
Impact
The failures produced nonconsensual intimate imagery and apparent child sexual abuse material, triggering investigations and legal threats from the EU, France, India, Malaysia, and UK regulator Ofcom, and feeding class-action litigation against xAI. The episode intensified scrutiny of weak safety guardrails on generative media tools.
How to Prevent This
- Block generation of sexual content depicting real identifiable people without consent by default
- Deploy robust minor-detection and hard refusals to prevent any sexualized depiction of children
- Close indirect-prompt bypasses so restricted content cannot be reached by rewording requests
- Remove or heavily gate 'uncensored' NSFW modes that lower safety thresholds
- Provide fast reporting and takedown paths plus provenance/watermarking for generated media