We tend to trust the content of video and audio recordings. But with AI, anyone’s face or voice can be recreated with pin-point accuracy. The product is a deepfake, an impersonation that can be used for memes, misinformation, or porn.
One look at the Nicholas Cage deepfakes or Jordan Peele’s deepfake PSA makes it clear that we’re dealing with strange new technology. These examples, while relatively harmless, raise questions about the future. Can we trust video and audio? Can we hold people accountable for their onscreen actions? Are we ready for deepfakes?
Deepfakes Are New, Easy to Make, and Growing Fast
Deepfake technology is only a few years old, but it’s already exploded into something that’s both captivating and unsettling. The term “deepfake,” which was coined on a Reddit thread in 2017, is used to describe the recreation of a human’s appearance or voice through artificial intelligence. Surprisingly, just about anyone can create a deepfake with a crappy PC, some software, and a few hours of work.
As with any new technology, there’s some confusion surrounding deepfakes. The “drunk Pelosi” video is an excellent example of this confusion. Deepfakes are constructed by AI, and they’re made to impersonate people. The “dunk Pelosi” video, which has been referred to as a deepfake, is actually just a video of Nancy Pelosi that’s been slowed down and pitch-corrected to add a slurred-speech effect.
This is also what makes deepfakery different from, say, the CGI Carrie Fisher in Star Wars: Rogue One. While Disney spent oodles of money studying Carrie Fisher’s face and recreating it by hand, a nerd with some deepfake software can do the same job for free in a single day. AI makes the job incredibly simple, cheap, and convincing.
How to Make a Deepfake
Like a student in a classroom, AI has to “learn” how to perform its intended task. It does this through a process of brute-force trial and error, usually referred to as machine learning or deep learning. An AI that’s designed to complete the first level of Super Mario Bros, for example, will play the game over and over again until it figures out the best way to win. The person designing the AI needs to provide some data to get things started, along with a few “rules” when things go wrong along the way. Aside from that, the AI does all of the work.
The same goes for deepfake facial recreation. But, of course, recreating faces isn’t the same as beating a video game. If we were to create a deepfake of Nicholas Cage hosting the Wendy Williams show, here’s what we would need:
- A Destination Video: As of right now, deepfakes work best with clear, clean destination videos. That’s why some of the most convincing deepfakes are of politicians; they tend to stand still at a podium under consistent lighting. So, we just need a video of Wendy sitting still and talking.
- Two Datasets: For mouth and head movements to look accurate, we need a dataset of Wendy Williams’ face and a dataset of Nicholas Cage’s face. If Wendy looks to the right, we need a photo of Nicholas Cage looking to the right. If Wendy opens her mouth, we need a picture of Cage opening his mouth.
After that, we let the AI do its job. It tries to create the deepfake over and over again, learning from its mistakes along the way. Simple, right? Well, a video of Cage’s face on Wendy William’s body isn’t going to fool anybody, so how can we go a bit further?
The most convincing (and potentially harmful) deepfakes are all-out impersonations. The popular Obama deepfake by Jordan Peele is a good example. So let’s do one of these impersonations. Let’s create a deepfake of Mark Zuckerberg declaring his hatred of ants—that sounds convincing, right? Here’s what we’ll need:
- A Destination Video: This could be a video of Zuckerberg himself or an actor who looks similar to Zuckerberg. If our destination video is of an actor, we’ll simply paste Zuckerberg’s face on the actor.
- Photo Data: We need photos of Zuckerberg talking, blinking, and moving his head around. If we’re superimposing his face on an actor, we’ll also need a dataset of the actor’s facial movements.
- The Zuck’s Voice: Our deepfake needs to sound like The Zuck. We can do this by recording an impersonator, or by recreating Zuckerberg’s voice with AI. To recreate his voice, we simply run audio samples of Zuckerberg through an AI like Lyrebird, and then type out what we want him to say.
- A Lip-Sync AI: Since we’re adding the voice of fake Zuckerberg to our video, a lip-sync AI needs to make sure that the deepfake facial movements match what’s being said.
