TikTok’s AI Video Generator Let Avatars Recite Hitler’s ‘Mein Kampf’

77 0 0

TikTok mistakenly released a version of its AI video generator that let avatars recite Adolf Hitler’s Mein Kampf and Osama bin Laden’s “Letter to America” on command.

Last week, TikTok announced its new tool “Symphony Digital Avatars” which enables brands to use AI-generated versions of real people in ads to give a “human touch” to campaigns.

Symphony Digital Avatars allowed advertisers to use AI-powered dubbing so that brands could enter a script and make the avatars say what they want within TikTok’s guidelines in product campaigns. Only users with a TikTok Ads Manager account were supposed to be able to access the tool.

However, CNN discovered that TikTok had accidentally posted a link to an internal version of its new AI avatar tool which had zero guardrails or safety measures in place.

Any TikTok user could create unmoderated and unmarked videos which let AI avatars say just about anything — including the ability to recite an excerpt from Adolf Hitler’s Mein Kampf on command.

CNN tech reporter Jon Sarlin says he was able to access the AI tool with his personal TikTok account. On X (the platform formerly known as Twitter), Sarlin revealed that he was able to get TikTok’s AI avatars to recite Osama bin Laden’s “Letter to America,” say the white supremacist slogan “The Fourteen Words,” tell people to drink bleach and vote on the wrong day, as well as read out Hitler’s Mein Kampf

Furthermore, none of these videos had a watermark to indicate they were AI-generated, meaning such videos could be taken at face value by unsuspecting TikTok users if the clips were to be posted on the platform.

‘A Technical Error’

When CNN reached out to TikTok about these videos, a company spokesperson called it a “technical error” that “allowed an extremely small number of users to create content using an internal testing version of the tool for a few days.” TikTok says it has now resolved the issue.

“If CNN had attempted to upload the harmful content it created, this content would have been rejected for violating our policies,” TikTok adds.

“TikTok is an industry leader in responsible AIGC creation, and we will continue to test and build in the safety mitigations we apply to all TikTok products before public launch.”

On X, Sarlin revealed that TikTok did not respond to follow-up queries from CNN about what the company is doing “to prevent a mistake of this magnitude from happening again.”