“Deep Fake” Video Technology Is Advancing Faster Than Our Policies Can Keep Up
This article is part of the magazine, "The Future of Science In America: The Election Issue," co-published by LeapsMag, the Aspen Institute Science & Society Program, and GOOD.
Alethea.ai sports a grid of faces smiling, blinking and looking about. Some are beautiful, some are oddly familiar, but all share one thing in common—they are fake.
Alethea creates "synthetic media"— including digital faces customers can license saying anything they choose with any voice they choose. Companies can hire these photorealistic avatars to appear in explainer videos, advertisements, multimedia projects or any other applications they might dream up without running auditions or paying talent agents or actor fees. Licenses begin at a mere $99. Companies may also license digital avatars of real celebrities or hire mashups created from real celebrities including "Don Exotic" (a mashup of Donald Trump and Joe Exotic) or "Baby Obama" (a large-eared toddler that looks remarkably similar to a former U.S. President).
Naturally, in the midst of the COVID pandemic, the appeal is understandable. Rather than flying to a remote location to film a beer commercial, an actor can simply license their avatar to do the work for them. The question is—where and when this tech will cross the line between legitimately licensed and authorized synthetic media to deep fakes—synthetic videos designed to deceive the public for financial and political gain.
Deep fakes are not new. From written quotes that are manipulated and taken out of context to audio quotes that are spliced together to mean something other than originally intended, misrepresentation has been around for centuries. What is new is the technology that allows this sort of seamless and sophisticated deception to be brought to the world of video.
"At one point, video content was considered more reliable, and had a higher threshold of trust," said Alethea CEO and co-founder, Arif Khan. "We think video is harder to fake and we aren't yet as sensitive to detecting those fakes. But the technology is definitely there."
"In the future, each of us will only trust about 15 people and that's it," said Phil Lelyveld, who serves as Immersive Media Program Lead at the Entertainment Technology Center at the University of Southern California. "It's already very difficult to tell true footage from fake. In the future, I expect this will only become more difficult."
How do we know what's true in a world where original videos created with avatars of celebrities and politicians can be manipulated to say virtually anything?
As the U.S. 2020 Presidential Election nears, the potential moral and ethical implications of this technology are startling. A number of cases of truth tampering have recently been widely publicized. On August 5, President Donald Trump's campaign released an ad featuring several photos of Joe Biden that were altered to make it seem like was hiding all alone in his basement. In one photo, at least ten people who had been sitting with Biden in the original shot were cut out. In other photos, Biden's image was removed from a nature preserve and praying in church to make it appear Biden was in that same basement. Recently several videos of Speaker of the House Nancy Pelosi were slowed down by 75 percent to make her sound as if her speech was slurred.
During a campaign event in Florida on September 15 of this year, former Vice President Joe Biden was introduced by Puerto Rican singer-songwriter Luis Fonsi. After he was introduced, Biden paid tribute to the singer-songwriter—he held up his cell phone and played the hit song "Despecito". Shortly afterward, a doctored version of this video appeared on self-described parody site the United Spot replacing the Despicito with N.W.A.'s "F—- Tha Police". By September 16, Donald Trump retweeted the video, twice—first with the line "What is this all about" and second with the line "China is drooling. They can't believe this!" Twitter was quick to mark the video in these tweets as manipulated media.
Twitter had previously addressed several of Donald Trump's tweets—flagging a video shared in June as manipulated media and removing altogether a video shared by Trump in July showing a group promoting the hydroxychloroquine as an effective cure for COVID-19. Many of these manipulated videos are ultimately flagged or taken down, but not before they are seen and shared by millions of online viewers.
These faked videos were exposed rather quickly, as they could be compared with the original, publicly available source material. But what happens when there is no original source material? How do we know what's true in a world where original videos created with avatars of celebrities and politicians can be manipulated to say virtually anything?
"This type of fake media is a profound threat to our democracy," said Reid Blackman, the CEO of VIRTUE--an ethics consultancy for AI leaders. "Democracy depends on well-informed citizens. When citizens can't or won't discern between real and fake news, the implications are huge."
In light of the importance of reliable information in the political system, there's a clear and present need to verify that the images and news we consume is authentic. So how can anyone ever know that the content they are viewing is real?
"This will not be a simple technological solution," said Blackman. "There is no 'truth' button to push to verify authenticity. There's plenty of blame and condemnation to go around. Purveyors of information have a responsibility to vet the reliability of their sources. And consumers also have a responsibility to vet their sources."
Yet the process of verifying sources has never been more challenging. More and more citizens are choosing to live in a "media bubble"—gathering and sharing news only from and with people who share their political leanings and opinions. At one time, United States broadcasters were bound by the Fairness Doctrine—requiring them to present controversial issues important to the public in a way that the FCC deemed honest, equitable and balanced. The repeal of this doctrine in 1987 paved the way for new forms of cable news channels such as Fox News and MSNBC that appealed to viewers with a particular point of view. The Internet has only exacerbated these tendencies. Social media algorithms are designed to keep people clicking within their comfort zones by presenting members with only the thoughts and opinions they want to hear.
"I sometimes laugh when I hear people tell me they can back a particular opinion they hold with research," said Blackman. "Having conducted a fair bit of true scientific research, I am aware that clicking on one article on the Internet hardly qualifies. But a surprising number of people believe that finding any source online that states the fact they choose to believe is the same as proving it true."
Back to the fundamental challenge: How do we as a society root out what's false online? Lelyveld suggests that it will begin by verifying things that are known to be true rather than trying to call out everything that is fake. "The EU called me in to talk about how to deal with fake news coming out of Russia," said Lelyveld. "I told them Hollywood has spent 100 years developing special effects technology to make things that are wholly fictional indistinguishable from the truth. I told them that you'll never chase down every source of fake news. You're better off focusing on what can be proved true."
Arif Khan agrees. "There are probably 100 accounts attributed to Elon Musk on Twitter, but only one has the blue checkmark," said Khan. "That means Twitter has verified that an account of public interest is real. That's what we're trying to do with our platform. Allow celebrities to verify that specific videos were licensed and authorized directly by them."
Alethea will use another key technology called blockchain to mark all authentic authorized videos with celebrity avatars. Blockchain uses a distributed ledger technology to make sure that no undetected changes have been made to the content. Think of the difference between editing a document in a traditional word processing program and editing in a distributed online editing system like Google Docs. In a traditional word processing program, you can edit and copy a document without revealing any changes. In a shared editing system like Google Docs, every person who shares the document can see a record of every edit, addition and copy made of any portion of the document. In a similar way, blockchain helps Alethea ensure that approved videos have not been copied or altered inappropriately.
While AI companies like Alethea are moving to ensure that avatars based on real individuals aren't wrongly identified, the situation becomes a bit murkier when it comes to the question of representing groups, races, creeds, and other forms of identity. Alethea is rightly proud that the completely artificial avatars visually represent a variety of ages, races and sexes. However, companies could conceivably license an avatar to represent a marginalized group without actually hiring a person within that group to decide what the avatar will do or say.
"I don't know if I would call this tokenism, as that is difficult to identify without understanding the hiring company's intent," said Blackman. "Where this becomes deeply troubling is when avatars are used to represent a marginalized group without clearly pointing out the actor is an avatar. It's one thing for an African American woman avatar to say, 'I like ice cream.' It's entirely different thing for an African American woman avatar to say she supports a particular political candidate. In the second case, the avatar is being used as social proof that real people of a certain type back a certain political idea. And there the deception is far more problematic."
"It always comes down to unintended consequences of technology," said Lelyveld. "Technology is neutral—it's only the implementation that has the power to be good or bad. Without a thoughtful approach to the cultural, moral and political implications of technology, it often drifts towards the bad. We need to make a conscious decision as we release new technology to ensure it moves towards the good."
When presented with the idea that his avatars might be used to misrepresent marginalized groups, Khan was thoughtful. "Yes, I can see that is an unintended consequence of our technology. We would like to encourage people to license the avatars of real people, who would have final approval over what their avatars say or do. As to what people do with our completely artificial avatars, we will have to consider that moving forward."
Lelyveld frankly sees the ability for advertisers to create avatars that are our assistants or even our friends as a greater moral concern. "Once our digital assistant or avatar becomes an integral part of our life—even a friend as it were, what's to stop marketers from having those digital friends make suggestions about what drink we buy, which shirt we wear or even which candidate we elect? The possibilities for bad actors to reach us through our digital circle is mind-boggling."
Ultimately, Blackman suggests, we as a society will need to make decisions about what matters to us. "We will need to build policies and write laws—tackling the biggest problems like political deep fakes first. And then we have to figure out how to make the penalties stiff enough to matter. Fining a multibillion-dollar company a few million for a major offense isn't likely to move the needle. The punishment will need to fit the crime."
Until then, media consumers will need to do their own due diligence—to do the difficult work of uncovering the often messy and deeply uncomfortable news that's the truth.
[Editor's Note: To read other articles in this special magazine issue, visit the beautifully designed e-reader version.]
Indigenous wisdom plus honeypot ants could provide new antibiotics
For generations, the Indigenous Tjupan people of Australia enjoyed the sweet treat of honey made by honeypot ants. As a favorite pastime, entire families would go searching for the underground colonies, first spotting a worker ant and then tracing it to its home. The ants, which belong to the species called Camponotus inflatus, usually build their subterranean homes near the mulga trees, Acacia aneura. Having traced an ant to its tree, it would be the women who carefully dug a pit next to a colony, cautious not to destroy the entire structure. Once the ant chambers were exposed, the women would harvest a small amount to avoid devastating the colony’s stocks—and the family would share the treat.
The Tjupan people also knew that the honey had antimicrobial properties. “You could use it for a sore throat,” says Danny Ulrich, a member of the Tjupan nation. “You could also use it topically, on cuts and things like that.”
These hunts have become rarer, as many of the Tjupan people have moved away and, up until now, the exact antimicrobial properties of the ant honey remained unknown. But recently, scientists Andrew Dong and Kenya Fernandes from the University of Sydney, joined Ulrich, who runs the Honeypot Ants tours in Kalgoorlie, a city in Western Australia, on a honey-gathering expedition. Afterwards, they ran a series of experiments analyzing the honey’s antimicrobial activity—and confirmed that the Indigenous wisdom was true. The honey was effective against Staphylococcus aureus, a common pathogen responsible for sore throats, skin infections like boils and sores, and also sepsis, which can result in death. Moreover, the honey also worked against two species of fungi, Cryptococcus and Aspergillus, which can be pathogenic to humans, especially those with suppressed immune systems.
In the era of growing antibiotic resistance and the rising threat of pathogenic fungi, these findings may help scientists identify and make new antimicrobial compounds. “Natural products have been honed over thousands and millions of years by nature and evolution,” says Fernandes. “And some of them have complex and intricate properties that make them really important as potential new antibiotics. “
In an era of growing resistance to antibiotics and new threats of fungi infections, the latest findings about honeypot ants are helping scientists identify new antimicrobial drugs.
Danny Ulrich
Bee honey is also known for its antimicrobial properties, but bees produce it very differently than the ants. Bees collect nectar from flowers, which they regurgitate at the hive and pack into the hexagonal honeycombs they build for storage. As they do so, they also add into the mix an enzyme called glucose oxidase produced by their glands. The enzyme converts atmospheric oxygen into hydrogen peroxide, a reactive molecule that destroys bacteria and acts as a natural preservative. After the bees pack the honey into the honeycombs, they fan it with their wings to evaporate the water. Once a honeycomb is full, the bees put a beeswax cover on it, where it stays well-preserved thanks to the enzymatic action, until the bees need it.
Less is known about the chemistry of ants’ honey-making. Similarly to bees, they collect nectar. They also collect the sweet sap of the mulga tree. Additionally, they also “milk” the aphids—small sap-sucking insects that live on the tree. When ants tickle the aphids with their antennae, the latter release a sweet substance, which the former also transfer to their colonies. That’s where the honey management difference becomes really pronounced. The ants don’t build any kind of structures to store their honey. Instead, they store it in themselves.
The workers feed their harvest to their fellow ants called repletes, stuffing them up to the point that their swollen bellies outgrow the ants themselves, looking like amber-colored honeypots—hence the name. Because of their size, repletes don’t move, but hang down from the chamber’s ceiling, acting as living feedstocks. When food becomes scarce, they regurgitate their reserves to their colony’s brethren. It’s not clear whether the repletes die afterwards or can be restuffed again. “That's a good question,” Dong says. “After they've been stretched, they can't really return to exactly the same shape.”
These replete ants are the “treat” the Tjupan women dug for. Once they saw the round-belly ants inside the chambers, they would reach in carefully and get a few scoops of them. “You see a lot of honeypot ants just hanging on the roof of the little openings,” says Ulrich’s mother, Edie Ulrich. The women would share the ants with family members who would eat them one by one. “They're very delicate,” shares Edie Ulrich—you have to take them out carefully, so they don’t accidentally pop and become a wasted resource. “Because you’d lose all this precious honey.”
Dong stumbled upon the honeypot ants phenomenon because he was interested in Indigenous foods and went on Ulrich’s tour. He quickly became fascinated with the insects and their role in the Indigenous culture. “The honeypot ants are culturally revered by the Indigenous people,” he says. Eventually he decided to test out the honey’s medicinal qualities.
The researchers were surprised to see that even the smallest, eight percent concentration of honey was able to arrest the growth of S. aureus.
To do this, the two scientists first diluted the ant honey with water. “We used something called doubling dilutions, which means that we made 32 percent dilutions, and then we halve that to 16 percent and then we half that to eight percent,” explains Fernandes. The goal was to obtain as much results as possible with the meager honey they had. “We had very, very little of the honeypot ant honey so we wanted to maximize the spectrum of results we can get without wasting too much of the sample.”
After that, the researchers grew different microbes inside a nutrient rich broth. They added the broth to the different honey dilutions and incubated the mixes for a day or two at the temperature favorable to the germs’ growth. If the resulting solution turned turbid, it was a sign that the bugs proliferated. If it stayed clear, it meant that the honey destroyed them. The researchers were surprised to see that even the smallest, eight percent concentration of honey was able to arrest the growth of S. aureus. “It was really quite amazing,” Fernandes says. “Eight milliliters of honey in 92 milliliters of water is a really tiny amount of honey compared to the amount of water.”
Similar to bee honey, the ants’ honey exhibited some peroxide antimicrobial activity, researchers found, but given how little peroxide was in the solution, they think the honey also kills germs by a different mechanism. “When we measured, we found that [the solution] did have some hydrogen peroxide, but it didn't have as much of it as we would expect based on how active it was,” Fernandes says. “Whether this hydrogen peroxide also comes from glucose oxidase or whether it's produced by another source, we don't really know,” she adds. The research team does have some hypotheses about the identity of this other germ-killing agent. “We think it is most likely some kind of antimicrobial peptide that is actually coming from the ant itself.”
The honey also has a very strong activity against the two types of fungi, Cryptococcus and Aspergillus. Both fungi are associated with trees and decaying leaves, as well as in the soils where ants live, so the insects likely have evolved some natural defense compounds, which end up inside the honey.
It wouldn’t be the first time when modern medicines take their origin from the natural world or from the indigenous people’s knowledge. The bark of the cinchona tree native to South America contains quinine, a substance that treats malaria. The Indigenous people of the Andes used the bark to quell fever and chills for generations, and when Europeans began to fall ill with malaria in the Amazon rainforest, they learned to use that medicine from the Andean people.
The wonder drug aspirin similarly takes its origin from a bark of a tree—in this case a willow.
Even some anticancer compounds originated from nature. A chemotherapy drug called Paclitaxel, was originally extracted from the Pacific yew trees, Taxus brevifolia. The samples of the Pacific yew bark were first collected in 1962 by researchers from the United States Department of Agriculture who were looking for natural compounds that might have anti-tumor activity. In December 1992, the FDA approved Paclitaxel (brand name Taxol) for the treatment of ovarian cancer and two years later for breast cancer.
In the era when the world is struggling to find new medicines fast enough to subvert a fungal or bacterial pandemic, these discoveries can pave the way to new therapeutics. “I think it's really important to listen to indigenous cultures and to take their knowledge because they have been using these sources for a really, really long time,” Fernandes says. Now we know it works, so science can elucidate the molecular mechanisms behind it, she adds. “And maybe it can even provide a lead for us to develop some kind of new treatments in the future.”
Lina Zeldovich has written about science, medicine and technology for Popular Science, Smithsonian, National Geographic, Scientific American, Reader’s Digest, the New York Times and other major national and international publications. A Columbia J-School alumna, she has won several awards for her stories, including the ASJA Crisis Coverage Award for Covid reporting, and has been a contributing editor at Nautilus Magazine. In 2021, Zeldovich released her first book, The Other Dark Matter, published by the University of Chicago Press, about the science and business of turning waste into wealth and health. You can find her on http://linazeldovich.com/ and @linazeldovich.
Blood Test Can Detect Lymphoma Cells Before a Tumor Grows Back
When David M. Kurtz was doing his clinical fellowship at Stanford University Medical Center in 2009, specializing in lymphoma treatments, he found himself grappling with a question no one could answer. A typical regimen for these blood cancers prescribed six cycles of chemotherapy, but no one knew why. "The number seemed to be drawn out of a hat," Kurtz says. Some patients felt much better after just two doses, but had to endure the toxic effects of the entire course. For some elderly patients, the side effects of chemo are so harsh, they alone can kill. Others appeared to be cancer-free on the CT scans after the requisite six but then succumbed to it months later.
"Anecdotally, one patient decided to stop therapy after one dose because he felt it was so toxic that he opted for hospice instead," says Kurtz, now an oncologist at the center. "Five years down the road, he was alive and well. For him, just one dose was enough." Others would return for their one-year check up and find that their tumors grew back. Kurtz felt that while CT scans and MRIs were powerful tools, they weren't perfect ones. They couldn't tell him if there were any cancer cells left, stealthily waiting to germinate again. The scans only showed the tumor once it was back.
Blood cancers claim about 68,000 people a year, with a new diagnosis made about every three minutes, according to the Leukemia Research Foundation. For patients with B-cell lymphoma, which Kurtz focuses on, the survival chances are better than for some others. About 60 percent are cured, but the remaining 40 percent will relapse—possibly because they will have a negative CT scan, but still harbor malignant cells. "You can't see this on imaging," says Michael Green, who also treats blood cancers at University of Texas MD Anderson Medical Center.
The new blood test is sensitive enough to spot one cancerous perpetrator amongst one million other DNA molecules.
Kurtz wanted a better diagnostic tool, so he started working on a blood test that could capture the circulating tumor DNA or ctDNA. For that, he needed to identify the specific mutations typical for B-cell lymphomas. Working together with another fellow PhD student Jake Chabon, Kurtz finally zeroed-in on the tumor's genetic "appearance" in 2017—a pair of specific mutations sitting in close proximity to each other—a rare and telling sign. The human genome contains about 3 billion base pairs of nucleotides—molecules that compose genes—and in case of the B-cell lymphoma cells these two mutations were only a few base pairs apart. "That was the moment when the light bulb went on," Kurtz says.
The duo formed a company named Foresight Diagnostics, focusing on taking the blood test to the clinic. But knowing the tumor's mutational signature was only half the process. The other was fishing the tumor's DNA out of patients' bloodstream that contains millions of other DNA molecules, explains Chabon, now Foresight's CEO. It would be like looking for an escaped criminal in a large crowd. Kurtz and Chabon solved the problem by taking the tumor's "mug shot" first. Doctors would take the biopsy pre-treatment and sequence the tumor, as if taking the criminal's photo. After treatments, they would match the "mug shot" to all DNA molecules derived from the patient's blood sample to see if any molecular criminals managed to escape the chemo.
Foresight isn't the only company working on blood-based tumor detection tests, which are dubbed liquid biopsies—other companies such as Natera or ArcherDx developed their own. But in a recent study, the Foresight team showed that their method is significantly more sensitive in "fishing out" the cancer molecules than existing tests. Chabon says that this test can detect circulating tumor DNA in concentrations that are nearly 100 times lower than other methods. Put another way, it's sensitive enough to spot one cancerous perpetrator amongst one million other DNA molecules.
They also aim to extend their test to detect other malignancies such as lung, breast or colorectal cancers.
"It increases the sensitivity of detection and really catches most patients who are going to progress," says Green, the University of Texas oncologist who wasn't involved in the study, but is familiar with the method. It would also allow monitoring patients during treatment and making better-informed decisions about which therapy regimens would be most effective. "It's a minimally invasive test," Green says, and "it gives you a very high confidence about what's going on."
Having shown that the test works well, Kurtz and Chabon are planning a new trial in which oncologists would rely on their method to decide when to stop or continue chemo. They also aim to extend their test to detect other malignancies such as lung, breast or colorectal cancers. The latest genome sequencing technologies have sequenced and catalogued over 2,500 different tumor specimens and the Foresight team is analyzing this data, says Chabon, which gives the team the opportunity to create more molecular "mug shots."
The team hopes that that their blood cancer test will become available to patients within about five years, making doctors' job easier, and not only at the biological level. "When I tell patients, "good news, your cancer is in remission', they ask me, 'does it mean I'm cured?'" Kurtz says. "Right now I can't answer this question because I don't know—but I would like to." His company's test, he hopes, will enable him to reply with certainty. He'd very much like to have the power of that foresight.
This article is republished from our archives to coincide with Blood Cancer Awareness Month, which highlights progress in cancer diagnostics and treatment.
Lina Zeldovich has written about science, medicine and technology for Popular Science, Smithsonian, National Geographic, Scientific American, Reader’s Digest, the New York Times and other major national and international publications. A Columbia J-School alumna, she has won several awards for her stories, including the ASJA Crisis Coverage Award for Covid reporting, and has been a contributing editor at Nautilus Magazine. In 2021, Zeldovich released her first book, The Other Dark Matter, published by the University of Chicago Press, about the science and business of turning waste into wealth and health. You can find her on http://linazeldovich.com/ and @linazeldovich.