“Deep Fake” Video Technology Is Advancing Faster Than Our Policies Can Keep Up
Artificial avatars for hire and sophisticated video manipulation carry profound implications for society.
This article is part of the magazine, "The Future of Science In America: The Election Issue," co-published by LeapsMag, the Aspen Institute Science & Society Program, and GOOD.
Alethea.ai sports a grid of faces smiling, blinking and looking about. Some are beautiful, some are oddly familiar, but all share one thing in common—they are fake.
Alethea creates "synthetic media"— including digital faces customers can license saying anything they choose with any voice they choose. Companies can hire these photorealistic avatars to appear in explainer videos, advertisements, multimedia projects or any other applications they might dream up without running auditions or paying talent agents or actor fees. Licenses begin at a mere $99. Companies may also license digital avatars of real celebrities or hire mashups created from real celebrities including "Don Exotic" (a mashup of Donald Trump and Joe Exotic) or "Baby Obama" (a large-eared toddler that looks remarkably similar to a former U.S. President).
Naturally, in the midst of the COVID pandemic, the appeal is understandable. Rather than flying to a remote location to film a beer commercial, an actor can simply license their avatar to do the work for them. The question is—where and when this tech will cross the line between legitimately licensed and authorized synthetic media to deep fakes—synthetic videos designed to deceive the public for financial and political gain.
Deep fakes are not new. From written quotes that are manipulated and taken out of context to audio quotes that are spliced together to mean something other than originally intended, misrepresentation has been around for centuries. What is new is the technology that allows this sort of seamless and sophisticated deception to be brought to the world of video.
"At one point, video content was considered more reliable, and had a higher threshold of trust," said Alethea CEO and co-founder, Arif Khan. "We think video is harder to fake and we aren't yet as sensitive to detecting those fakes. But the technology is definitely there."
"In the future, each of us will only trust about 15 people and that's it," said Phil Lelyveld, who serves as Immersive Media Program Lead at the Entertainment Technology Center at the University of Southern California. "It's already very difficult to tell true footage from fake. In the future, I expect this will only become more difficult."
How do we know what's true in a world where original videos created with avatars of celebrities and politicians can be manipulated to say virtually anything?
As the U.S. 2020 Presidential Election nears, the potential moral and ethical implications of this technology are startling. A number of cases of truth tampering have recently been widely publicized. On August 5, President Donald Trump's campaign released an ad featuring several photos of Joe Biden that were altered to make it seem like was hiding all alone in his basement. In one photo, at least ten people who had been sitting with Biden in the original shot were cut out. In other photos, Biden's image was removed from a nature preserve and praying in church to make it appear Biden was in that same basement. Recently several videos of Speaker of the House Nancy Pelosi were slowed down by 75 percent to make her sound as if her speech was slurred.
During a campaign event in Florida on September 15 of this year, former Vice President Joe Biden was introduced by Puerto Rican singer-songwriter Luis Fonsi. After he was introduced, Biden paid tribute to the singer-songwriter—he held up his cell phone and played the hit song "Despecito". Shortly afterward, a doctored version of this video appeared on self-described parody site the United Spot replacing the Despicito with N.W.A.'s "F—- Tha Police". By September 16, Donald Trump retweeted the video, twice—first with the line "What is this all about" and second with the line "China is drooling. They can't believe this!" Twitter was quick to mark the video in these tweets as manipulated media.
Twitter had previously addressed several of Donald Trump's tweets—flagging a video shared in June as manipulated media and removing altogether a video shared by Trump in July showing a group promoting the hydroxychloroquine as an effective cure for COVID-19. Many of these manipulated videos are ultimately flagged or taken down, but not before they are seen and shared by millions of online viewers.
These faked videos were exposed rather quickly, as they could be compared with the original, publicly available source material. But what happens when there is no original source material? How do we know what's true in a world where original videos created with avatars of celebrities and politicians can be manipulated to say virtually anything?
"This type of fake media is a profound threat to our democracy," said Reid Blackman, the CEO of VIRTUE--an ethics consultancy for AI leaders. "Democracy depends on well-informed citizens. When citizens can't or won't discern between real and fake news, the implications are huge."
In light of the importance of reliable information in the political system, there's a clear and present need to verify that the images and news we consume is authentic. So how can anyone ever know that the content they are viewing is real?
"This will not be a simple technological solution," said Blackman. "There is no 'truth' button to push to verify authenticity. There's plenty of blame and condemnation to go around. Purveyors of information have a responsibility to vet the reliability of their sources. And consumers also have a responsibility to vet their sources."
Yet the process of verifying sources has never been more challenging. More and more citizens are choosing to live in a "media bubble"—gathering and sharing news only from and with people who share their political leanings and opinions. At one time, United States broadcasters were bound by the Fairness Doctrine—requiring them to present controversial issues important to the public in a way that the FCC deemed honest, equitable and balanced. The repeal of this doctrine in 1987 paved the way for new forms of cable news channels such as Fox News and MSNBC that appealed to viewers with a particular point of view. The Internet has only exacerbated these tendencies. Social media algorithms are designed to keep people clicking within their comfort zones by presenting members with only the thoughts and opinions they want to hear.
"I sometimes laugh when I hear people tell me they can back a particular opinion they hold with research," said Blackman. "Having conducted a fair bit of true scientific research, I am aware that clicking on one article on the Internet hardly qualifies. But a surprising number of people believe that finding any source online that states the fact they choose to believe is the same as proving it true."
Back to the fundamental challenge: How do we as a society root out what's false online? Lelyveld suggests that it will begin by verifying things that are known to be true rather than trying to call out everything that is fake. "The EU called me in to talk about how to deal with fake news coming out of Russia," said Lelyveld. "I told them Hollywood has spent 100 years developing special effects technology to make things that are wholly fictional indistinguishable from the truth. I told them that you'll never chase down every source of fake news. You're better off focusing on what can be proved true."
Arif Khan agrees. "There are probably 100 accounts attributed to Elon Musk on Twitter, but only one has the blue checkmark," said Khan. "That means Twitter has verified that an account of public interest is real. That's what we're trying to do with our platform. Allow celebrities to verify that specific videos were licensed and authorized directly by them."
Alethea will use another key technology called blockchain to mark all authentic authorized videos with celebrity avatars. Blockchain uses a distributed ledger technology to make sure that no undetected changes have been made to the content. Think of the difference between editing a document in a traditional word processing program and editing in a distributed online editing system like Google Docs. In a traditional word processing program, you can edit and copy a document without revealing any changes. In a shared editing system like Google Docs, every person who shares the document can see a record of every edit, addition and copy made of any portion of the document. In a similar way, blockchain helps Alethea ensure that approved videos have not been copied or altered inappropriately.
While AI companies like Alethea are moving to ensure that avatars based on real individuals aren't wrongly identified, the situation becomes a bit murkier when it comes to the question of representing groups, races, creeds, and other forms of identity. Alethea is rightly proud that the completely artificial avatars visually represent a variety of ages, races and sexes. However, companies could conceivably license an avatar to represent a marginalized group without actually hiring a person within that group to decide what the avatar will do or say.
"I don't know if I would call this tokenism, as that is difficult to identify without understanding the hiring company's intent," said Blackman. "Where this becomes deeply troubling is when avatars are used to represent a marginalized group without clearly pointing out the actor is an avatar. It's one thing for an African American woman avatar to say, 'I like ice cream.' It's entirely different thing for an African American woman avatar to say she supports a particular political candidate. In the second case, the avatar is being used as social proof that real people of a certain type back a certain political idea. And there the deception is far more problematic."
"It always comes down to unintended consequences of technology," said Lelyveld. "Technology is neutral—it's only the implementation that has the power to be good or bad. Without a thoughtful approach to the cultural, moral and political implications of technology, it often drifts towards the bad. We need to make a conscious decision as we release new technology to ensure it moves towards the good."
When presented with the idea that his avatars might be used to misrepresent marginalized groups, Khan was thoughtful. "Yes, I can see that is an unintended consequence of our technology. We would like to encourage people to license the avatars of real people, who would have final approval over what their avatars say or do. As to what people do with our completely artificial avatars, we will have to consider that moving forward."
Lelyveld frankly sees the ability for advertisers to create avatars that are our assistants or even our friends as a greater moral concern. "Once our digital assistant or avatar becomes an integral part of our life—even a friend as it were, what's to stop marketers from having those digital friends make suggestions about what drink we buy, which shirt we wear or even which candidate we elect? The possibilities for bad actors to reach us through our digital circle is mind-boggling."
Ultimately, Blackman suggests, we as a society will need to make decisions about what matters to us. "We will need to build policies and write laws—tackling the biggest problems like political deep fakes first. And then we have to figure out how to make the penalties stiff enough to matter. Fining a multibillion-dollar company a few million for a major offense isn't likely to move the needle. The punishment will need to fit the crime."
Until then, media consumers will need to do their own due diligence—to do the difficult work of uncovering the often messy and deeply uncomfortable news that's the truth.
[Editor's Note: To read other articles in this special magazine issue, visit the beautifully designed e-reader version.]
How sharing, hearing, and remembering positive stories can help shape our brains for the better
Across cultures and through millennia, human beings have always told stories. Whether it’s a group of boy scouts around a campfire sharing ghost stories or the paleolithic Cro-Magnons etching pictures of bison on cave walls, researchers believe that storytelling has been universal to human beings since the development of language.
But storytelling was more than just a way for our ancestors to pass the time. Researchers believe that storytelling served an important evolutionary purpose, helping humans learn empathy, share important information (such as where predators were or what berries were safe to eat), as well as strengthen social bonds. Quite literally, storytelling has made it possible for the human race to survive.
Today, neuroscientists are discovering that storytelling is just as important now as it was millions of years ago. Particularly in sharing positive stories, humans can more easily form relational bonds, develop a more flexible perspective, and actually grow new brain circuitry that helps us survive. Here’s how.
How sharing stories positively impacts the brain
When human beings share stories, it increases the levels of certain neurochemicals in the brain, neuroscientists have found. In a 2021 study published in Proceedings of the National Academy of Sciences (PNAS), Swedish researchers found that simply hearing a story could make hospitalized children feel better, compared to other hospitalized children who played a riddle game for the same amount of time. In their research, children in the intensive care unit who heard stories for just 30 minutes had higher levels of oxytocin, a hormone that promotes positive feelings and is linked to relaxation, trust, social connectedness, and overall psychological stability. Furthermore, the same children showed lower levels of cortisol, a hormone associated with stress. Afterward, the group of children who heard stories tended to describe their hospital experiences more positively, and even reported lower levels of pain.
Annie Brewster, MD, knows the positive effect of storytelling from personal experience. An assistant professor at Harvard Medical School and the author of The Healing Power of Storytelling: Using Personal Narrative to Navigate Illness, Trauma, and Loss, Brewster started sharing her personal experience with chronic illness after being diagnosed with multiple sclerosis in 2001. In doing so, Brewster says it has enabled her to accept her diagnosis and integrate it into her identity. Brewster believes so much in the power of hearing and sharing stories that in 2013 she founded Health Story Collaborative, a forum for others to share their mental and physical health challenges.“I wanted to hear stories of people who had found ways to move forward in positive ways, in spite of health challenges,” Brewster said. In doing so, Brewster believes people with chronic conditions can “move closer to self-acceptance and self-love.”
While hearing and sharing positive stories has been shown to increase oxytocin and other “feel good” chemicals, simply remembering a positive story has an effect on our brains as well. Mark Hoelterhoff, PhD, a lecturer in clinical psychology at the University of Edinburgh, recalling and “savoring” a positive story, thought, or feedback “begins to create new brain circuitry—a new neural network that’s geared toward looking for the positive,” he says. Over time, other research shows, savoring positive stories or thoughts can literally change the shape of your brain, hard-wiring someone to see things in a more positive light.How stories can change your behavior
In 2009, Paul Zak, PhD, a neuroscientist and professor at Claremont Graduate University, set out to measure how storytelling can actually change human behavior for the better. In his study, Zak wanted to measure the behavioral effects of oxytocin, and did this by showing test subjects two short video clips designed to elicit an emotional response.
In the first video they showed the study participants, a father spoke to the camera about his two-year-old son, Ben, who had been diagnosed with terminal brain cancer. The father told the audience that he struggled to connect with and enjoy Ben, as Ben had only a few months left to live. In the end, the father finds the strength to stay emotionally connected to his son until he dies.
The second video clip, however, was much less emotional. In that clip, the same father and son are shown spending the day at the zoo. Ben is only suggested to have cancer (he is bald from chemotherapy and referred to as a ‘miracle’, but the cancer isn’t mentioned directly). The second story lacked the dramatic narrative arc of the first video.
Zak’s team took blood before and after the participants watched one of the two videos and found that the first story increased the viewers’ cortisol and oxytocin, suggesting that they felt distress over the boy’s diagnosis and empathy toward the boy and his father. The second narrative, however, didn’t increase oxytocin or cortisol at all.
But Zak took the experiment a step further. After the movie clips, his team gave the study participants a chance to share money with a stranger in the lab. The participants who had an increase in cortisol and oxytocin were more likely to donate money generously. The participants who had increased cortisol and oxytocin were also more likely to donate money to a charity that works with children who are ill. Zak also found that the amount of oxytocin that was released was correlated with how much money people felt comfortable giving—in other words, the more oxytocin that was released, the more generous they felt, and the more money they donated.
How storytelling strengthens our bond with others
Sharing, hearing, and remembering stories can be a powerful tool for social change–not only in the way it changes our brain and our behavior, but also because it can positively affect our relationships with other people
Emotional stimulation from telling stories, writes Zak, is the foundation for empathy, and empathy strengthens our relationships with other people. “By knowing someone’s story—where they come from, what they do, and who you might know in common—relationships with strangers are formed.”
But why are these relationships important for humanity? Because human beings can use storytelling to build empathy and form relationships, it enables them to “engage in the kinds of large-scale cooperation that builds massive bridges and sends humans into space,” says Zak.
Storytelling, Zak found, and the oxytocin release that follows, also makes people more sensitive to social cues. This sensitivity not only motivates us to form relationships, but also to engage with other people and offer help, particularly if the other person seems to need help.
But as Zak found in his experiments, the type of storytelling matters when it comes to affecting relationships. Where Zak found that storytelling with a dramatic arc helps release oxytocin and cortisol, enabling people to feel more empathic and generous, other researchers have found that sharing happy stories allows for greater closeness between individuals and speakers. A group of Chinese researchers found that, compared to emotionally-neutral stories, happy stories were more “emotionally contagious.” Test subjects who heard happy stories had greater activation in certain areas of their brains, experienced more significant, positive changes in their mood, and felt a greater sense of closeness between themselves and the speaker.
“This finding suggests that when individuals are happy, they become less self-focused and then feel more intimate with others,” the authors of the study wrote. “Therefore, sharing happiness could strengthen interpersonal bonding.” The researchers went on to say that this could lead to developing better social networks, receiving more social support, and leading more successful social lives.
Since the start of the COVID pandemic, social isolation, loneliness, and resulting mental health issues have only gotten worse. In light of this, it’s safe to say that hearing, sharing, and remembering stories isn’t just something we can do for entertainment. Storytelling has always been central to the human experience, and now more than ever it’s become something crucial for our survival.
Want to know how you can reap the benefits of hearing happy stories? Keep an eye out for Upworthy’s first book, GOOD PEOPLE: Stories from the Best of Humanity, published by National Geographic/Disney, available on September 3, 2024. GOOD PEOPLE is a much-needed trove of life-affirming stories told straight from the heart. Handpicked from Upworthy’s community, these 101 stories speak to the breadth, depth, and beauty of the human experience, reminding us we have a lot more in common than we realize.
A new type of cancer therapy is shrinking deadly brain tumors with just one treatment
MRI scans after a new kind of immunotherapy for brain cancer show remarkable progress in one patient just days after the first treatment.
Few cancers are deadlier than glioblastomas—aggressive and lethal tumors that originate in the brain or spinal cord. Five years after diagnosis, less than five percent of glioblastoma patients are still alive—and more often, glioblastoma patients live just 14 months on average after receiving a diagnosis.
But an ongoing clinical trial at Mass General Cancer Center is giving new hope to glioblastoma patients and their families. The trial, called INCIPIENT, is meant to evaluate the effects of a special type of immune cell, called CAR-T cells, on patients with recurrent glioblastoma.
How CAR-T cell therapy works
CAR-T cell therapy is a type of cancer treatment called immunotherapy, where doctors modify a patient’s own immune system specifically to find and destroy cancer cells. In CAR-T cell therapy, doctors extract the patient’s T-cells, which are immune system cells that help fight off disease—particularly cancer. These T-cells are harvested from the patient and then genetically modified in a lab to produce proteins on their surface called chimeric antigen receptors (thus becoming CAR-T cells), which makes them able to bind to a specific protein on the patient’s cancer cells. Once modified, these CAR-T cells are grown in the lab for several weeks so that they can multiply into an army of millions. When enough cells have been grown, these super-charged T-cells are infused back into the patient where they can then seek out cancer cells, bind to them, and destroy them. CAR-T cell therapies have been approved by the US Food and Drug Administration (FDA) to treat certain types of lymphomas and leukemias, as well as multiple myeloma, but haven’t been approved to treat glioblastomas—yet.
CAR-T cell therapies don’t always work against solid tumors, such as glioblastomas. Because solid tumors contain different kinds of cancer cells, some cells can evade the immune system’s detection even after CAR-T cell therapy, according to a press release from Massachusetts General Hospital. For the INCIPIENT trial, researchers modified the CAR-T cells even further in hopes of making them more effective against solid tumors. These second-generation CAR-T cells (called CARv3-TEAM-E T cells) contain special antibodies that attack EFGR, a protein expressed in the majority of glioblastoma tumors. Unlike other CAR-T cell therapies, these particular CAR-T cells were designed to be directly injected into the patient’s brain.
The INCIPIENT trial results
The INCIPIENT trial involved three patients who were enrolled in the study between March and July 2023. All three patients—a 72-year-old man, a 74-year-old man, and a 57-year-old woman—were treated with chemo and radiation and enrolled in the trial with CAR-T cells after their glioblastoma tumors came back.
The results, which were published earlier this year in the New England Journal of Medicine (NEJM), were called “rapid” and “dramatic” by doctors involved in the trial. After just a single infusion of the CAR-T cells, each patient experienced a significant reduction in their tumor sizes. Just two days after receiving the infusion, the glioblastoma tumor of the 72-year-old man decreased by nearly twenty percent. Just two months later the tumor had shrunk by an astonishing 60 percent, and the change was maintained for more than six months. The most dramatic result was in the 57-year-old female patient, whose tumor shrank nearly completely after just one infusion of the CAR-T cells.
The results of the INCIPIENT trial were unexpected and astonishing—but unfortunately, they were also temporary. For all three patients, the tumors eventually began to grow back regardless of the CAR-T cell infusions. According to the press release from MGH, the medical team is now considering treating each patient with multiple infusions or prefacing each treatment with chemotherapy to prolong the response.
While there is still “more to do,” says co-author of the study neuro-oncologist Dr. Elizabeth Gerstner, the results are still promising. If nothing else, these second-generation CAR-T cell infusions may someday be able to give patients more time than traditional treatments would allow.
“These results are exciting but they are also just the beginning,” says Dr. Marcela Maus, a doctor and professor of medicine at Mass General who was involved in the clinical trial. “They tell us that we are on the right track in pursuing a therapy that has the potential to change the outlook for this intractable disease.”