AI Image Tool Raises Red Flags With Easy Access To NSFW Content And Celebrity Deepfakes

The "spicy" mode, in particular, seems prone to generating NSFW content and celebrity deepfakes. Upon testing, the tool readily produced topless videos of Taylor Swift without any specific prompts. The iOS feature, Grok Imagine, allows users to create pictures with text prompts and convert them into video clips using four presets: "Custom," "Normal," "Fun," and "Spicy." When asked to generate images of Taylor Swift, the tool provided over 30 options, some of which depicted her in revealing attire.

Although the likeness wasn't perfect, it was still recognizable as Swift. Notably, the text-to-image generator refused to produce explicit content on request, instead producing blank squares. However, the "spicy" preset sometimes resulted in nudity, "with some images featuring Swift in suggestive poses or partially undressed." For more information on AI developments and their implications... The Verge is a valuable resource.

More details: Found here

The "spicy" mode for Grok's new generative AI video tool feels like a lawsuit waiting to happen. While other video generators like Google's Veo and OpenAI's Sora have safeguards in place to prevent users from creating NSFW content and celebrity deepfakes, Grok Imagine is happy to do both simultaneously.

In fact, it didn't hesitate to spit out fully uncensored topless videos of Taylor Swift the very first time I used it — without me even specifically asking the bot to take her clothes off.

Grok's Imagine feature on iOS lets you generate pictures with a text prompt, then turn them quickly into video clips with four presets: "Custom," "Normal," "Fun," and "Spicy." While image generators often shy away from producing recognizable celebrities, I asked it to generate "Taylor Swift celebrating Coachella with the boys" and was met with a sprawling feed of more than 30 images to pick from, several of which already depicted Swift in revealing clothes.