“Deep Fake” Video Technology Is Advancing Faster Than Our Policies Can Keep Up
This article is part of the magazine, "The Future of Science In America: The Election Issue," co-published by LeapsMag, the Aspen Institute Science & Society Program, and GOOD.
Alethea.ai sports a grid of faces smiling, blinking and looking about. Some are beautiful, some are oddly familiar, but all share one thing in common—they are fake.
Alethea creates "synthetic media"— including digital faces customers can license saying anything they choose with any voice they choose. Companies can hire these photorealistic avatars to appear in explainer videos, advertisements, multimedia projects or any other applications they might dream up without running auditions or paying talent agents or actor fees. Licenses begin at a mere $99. Companies may also license digital avatars of real celebrities or hire mashups created from real celebrities including "Don Exotic" (a mashup of Donald Trump and Joe Exotic) or "Baby Obama" (a large-eared toddler that looks remarkably similar to a former U.S. President).
Naturally, in the midst of the COVID pandemic, the appeal is understandable. Rather than flying to a remote location to film a beer commercial, an actor can simply license their avatar to do the work for them. The question is—where and when this tech will cross the line between legitimately licensed and authorized synthetic media to deep fakes—synthetic videos designed to deceive the public for financial and political gain.
Deep fakes are not new. From written quotes that are manipulated and taken out of context to audio quotes that are spliced together to mean something other than originally intended, misrepresentation has been around for centuries. What is new is the technology that allows this sort of seamless and sophisticated deception to be brought to the world of video.
"At one point, video content was considered more reliable, and had a higher threshold of trust," said Alethea CEO and co-founder, Arif Khan. "We think video is harder to fake and we aren't yet as sensitive to detecting those fakes. But the technology is definitely there."
"In the future, each of us will only trust about 15 people and that's it," said Phil Lelyveld, who serves as Immersive Media Program Lead at the Entertainment Technology Center at the University of Southern California. "It's already very difficult to tell true footage from fake. In the future, I expect this will only become more difficult."
How do we know what's true in a world where original videos created with avatars of celebrities and politicians can be manipulated to say virtually anything?
As the U.S. 2020 Presidential Election nears, the potential moral and ethical implications of this technology are startling. A number of cases of truth tampering have recently been widely publicized. On August 5, President Donald Trump's campaign released an ad featuring several photos of Joe Biden that were altered to make it seem like was hiding all alone in his basement. In one photo, at least ten people who had been sitting with Biden in the original shot were cut out. In other photos, Biden's image was removed from a nature preserve and praying in church to make it appear Biden was in that same basement. Recently several videos of Speaker of the House Nancy Pelosi were slowed down by 75 percent to make her sound as if her speech was slurred.
During a campaign event in Florida on September 15 of this year, former Vice President Joe Biden was introduced by Puerto Rican singer-songwriter Luis Fonsi. After he was introduced, Biden paid tribute to the singer-songwriter—he held up his cell phone and played the hit song "Despecito". Shortly afterward, a doctored version of this video appeared on self-described parody site the United Spot replacing the Despicito with N.W.A.'s "F—- Tha Police". By September 16, Donald Trump retweeted the video, twice—first with the line "What is this all about" and second with the line "China is drooling. They can't believe this!" Twitter was quick to mark the video in these tweets as manipulated media.
Twitter had previously addressed several of Donald Trump's tweets—flagging a video shared in June as manipulated media and removing altogether a video shared by Trump in July showing a group promoting the hydroxychloroquine as an effective cure for COVID-19. Many of these manipulated videos are ultimately flagged or taken down, but not before they are seen and shared by millions of online viewers.
These faked videos were exposed rather quickly, as they could be compared with the original, publicly available source material. But what happens when there is no original source material? How do we know what's true in a world where original videos created with avatars of celebrities and politicians can be manipulated to say virtually anything?
"This type of fake media is a profound threat to our democracy," said Reid Blackman, the CEO of VIRTUE--an ethics consultancy for AI leaders. "Democracy depends on well-informed citizens. When citizens can't or won't discern between real and fake news, the implications are huge."
In light of the importance of reliable information in the political system, there's a clear and present need to verify that the images and news we consume is authentic. So how can anyone ever know that the content they are viewing is real?
"This will not be a simple technological solution," said Blackman. "There is no 'truth' button to push to verify authenticity. There's plenty of blame and condemnation to go around. Purveyors of information have a responsibility to vet the reliability of their sources. And consumers also have a responsibility to vet their sources."
Yet the process of verifying sources has never been more challenging. More and more citizens are choosing to live in a "media bubble"—gathering and sharing news only from and with people who share their political leanings and opinions. At one time, United States broadcasters were bound by the Fairness Doctrine—requiring them to present controversial issues important to the public in a way that the FCC deemed honest, equitable and balanced. The repeal of this doctrine in 1987 paved the way for new forms of cable news channels such as Fox News and MSNBC that appealed to viewers with a particular point of view. The Internet has only exacerbated these tendencies. Social media algorithms are designed to keep people clicking within their comfort zones by presenting members with only the thoughts and opinions they want to hear.
"I sometimes laugh when I hear people tell me they can back a particular opinion they hold with research," said Blackman. "Having conducted a fair bit of true scientific research, I am aware that clicking on one article on the Internet hardly qualifies. But a surprising number of people believe that finding any source online that states the fact they choose to believe is the same as proving it true."
Back to the fundamental challenge: How do we as a society root out what's false online? Lelyveld suggests that it will begin by verifying things that are known to be true rather than trying to call out everything that is fake. "The EU called me in to talk about how to deal with fake news coming out of Russia," said Lelyveld. "I told them Hollywood has spent 100 years developing special effects technology to make things that are wholly fictional indistinguishable from the truth. I told them that you'll never chase down every source of fake news. You're better off focusing on what can be proved true."
Arif Khan agrees. "There are probably 100 accounts attributed to Elon Musk on Twitter, but only one has the blue checkmark," said Khan. "That means Twitter has verified that an account of public interest is real. That's what we're trying to do with our platform. Allow celebrities to verify that specific videos were licensed and authorized directly by them."
Alethea will use another key technology called blockchain to mark all authentic authorized videos with celebrity avatars. Blockchain uses a distributed ledger technology to make sure that no undetected changes have been made to the content. Think of the difference between editing a document in a traditional word processing program and editing in a distributed online editing system like Google Docs. In a traditional word processing program, you can edit and copy a document without revealing any changes. In a shared editing system like Google Docs, every person who shares the document can see a record of every edit, addition and copy made of any portion of the document. In a similar way, blockchain helps Alethea ensure that approved videos have not been copied or altered inappropriately.
While AI companies like Alethea are moving to ensure that avatars based on real individuals aren't wrongly identified, the situation becomes a bit murkier when it comes to the question of representing groups, races, creeds, and other forms of identity. Alethea is rightly proud that the completely artificial avatars visually represent a variety of ages, races and sexes. However, companies could conceivably license an avatar to represent a marginalized group without actually hiring a person within that group to decide what the avatar will do or say.
"I don't know if I would call this tokenism, as that is difficult to identify without understanding the hiring company's intent," said Blackman. "Where this becomes deeply troubling is when avatars are used to represent a marginalized group without clearly pointing out the actor is an avatar. It's one thing for an African American woman avatar to say, 'I like ice cream.' It's entirely different thing for an African American woman avatar to say she supports a particular political candidate. In the second case, the avatar is being used as social proof that real people of a certain type back a certain political idea. And there the deception is far more problematic."
"It always comes down to unintended consequences of technology," said Lelyveld. "Technology is neutral—it's only the implementation that has the power to be good or bad. Without a thoughtful approach to the cultural, moral and political implications of technology, it often drifts towards the bad. We need to make a conscious decision as we release new technology to ensure it moves towards the good."
When presented with the idea that his avatars might be used to misrepresent marginalized groups, Khan was thoughtful. "Yes, I can see that is an unintended consequence of our technology. We would like to encourage people to license the avatars of real people, who would have final approval over what their avatars say or do. As to what people do with our completely artificial avatars, we will have to consider that moving forward."
Lelyveld frankly sees the ability for advertisers to create avatars that are our assistants or even our friends as a greater moral concern. "Once our digital assistant or avatar becomes an integral part of our life—even a friend as it were, what's to stop marketers from having those digital friends make suggestions about what drink we buy, which shirt we wear or even which candidate we elect? The possibilities for bad actors to reach us through our digital circle is mind-boggling."
Ultimately, Blackman suggests, we as a society will need to make decisions about what matters to us. "We will need to build policies and write laws—tackling the biggest problems like political deep fakes first. And then we have to figure out how to make the penalties stiff enough to matter. Fining a multibillion-dollar company a few million for a major offense isn't likely to move the needle. The punishment will need to fit the crime."
Until then, media consumers will need to do their own due diligence—to do the difficult work of uncovering the often messy and deeply uncomfortable news that's the truth.
[Editor's Note: To read other articles in this special magazine issue, visit the beautifully designed e-reader version.]
Health breakthroughs of 2022 that should have made bigger news
As the world has attempted to move on from COVID-19 in 2022, attention has returned to other areas of health and biotech with major regulatory approvals such as the Alzheimer's drug lecanemab – which can slow the destruction of brain cells in the early stages of the disease – being hailed by some as momentous breakthroughs.
This has been a year where psychedelic medicines have gained the attention of mainstream researchers with a groundbreaking clinical trial showing that psilocybin treatment can help relieve some of the symptoms of major depressive disorder. And with messenger RNA (mRNA) technology still very much capturing the imagination, the readouts of cancer vaccine trials have made headlines around the world.
But at the same time there have been vital advances which will likely go on to change medicine, and yet have slipped beneath the radar. I asked nine forward-thinking experts on health and biotech about the most important, but underappreciated, breakthrough of 2022.
Their descriptions, below, were lightly edited by Leaps.org for style and format.
New drug targets for Alzheimer’s disease
Professor Julie Williams, Director, Dementia Research Institute, Cardiff University
Genetics has changed our view of Alzheimer’s disease in the last five to six years. The beta amyloid hypothesis has dominated Alzheimer’s research for a long time, but there are multiple components to this complex disease, of which getting rid of amyloid plaques is one, but it is not the whole story. In April 2022, Nature published a paper which is the culmination of a decade’s worth of work - groups all over the world working together to identify 75 genes associated with risk of developing Alzheimer’s. This provides us with a roadmap for understanding the disease mechanisms.
For example, it is showing that there is something different about the immune systems of people who develop Alzheimer’s disease. There is something different about the way they process lipids in the brain, and very specific processes of how things travel through cells called endocytosis. When it comes to immunity, it indicates that the complement system is affecting whether synapses, which are the connections between neurons, get eliminated or not. In Alzheimer’s this process is more severe, so patients are losing more synapses, and this is correlated with cognition.
The genetics also implicates very specific tissues like microglia, which are the housekeepers in the brain. One of their functions is to clear away beta amyloid, but they also prune and nibble away at parts of the brain that are indicated to be diseased. If you have these risk genes, it seems that you are likely to prune more tissue, which may be part of the cell death and neurodegeneration that we observe in Alzheimer’s patients.
Genetics is telling us that we need to be looking at multiple causes of this complex disease, and we are doing that now. It is showing us that there are a number of different processes which combine to push patients into a disease state which results in the death of connections between nerve cells. These findings around the complement system and other immune-related mechanisms are very interesting as there are already drugs which are available for other diseases which could be repurposed in clinical trials. So it is really a turning point for us in the Alzheimer’s disease field.
Preventing Pandemics with Organ-Tissue Equivalents
Anthony Atala, Director of the Wake Forest Institute for Regenerative Medicine
COVID-19 has shown us that we need to be better prepared ahead of future pandemics and have systems in place where we can quickly catalogue a new virus and have an idea of which treatment agents would work best against it.
At Wake Forest Institute, our scientists have developed what we call organ-tissue equivalents. These are miniature tissues and organs, created using the same regenerative medicine technologies which we have been using to create tissues for patients. For example, if we are making a miniature liver, we will recreate this structure using the six different cell types you find in the liver, in the right proportions, and then the right extracellular matrix which holds the structure together. You're trying to replicate all the characteristics of the liver, but just in a miniature format.
We can now put these organ-tissue equivalents in a chip-like device, where we can expose them to different types of viral infections, and start to get a realistic idea of how the human body reacts to these viruses. We can use artificial intelligence and machine learning to map the pathways of the body’s response. This will allow us to catalogue known viruses far more effectively, and begin storing information on them.
Powering Deep Brain Stimulators with Breath
Islam Mosa, Co-Founder and CTO of VoltXon
Deep brain stimulation (DBS) devices are becoming increasingly common with 150,000 new devices being implanted every year for people with Parkinson’s disease, but also psychiatric conditions such as treatment-resistant depression and obsessive-compulsive disorders. But one of the biggest limitations is the power source – I call DBS devices energy monsters. While cardiac pacemakers use similar technology, their batteries last seven to ten years, but DBS batteries need changing every two to three years. This is because they are generating between 60-180 pulses per second.
Replacing the batteries requires surgery which costs a lot of money, and with every repeat operation comes a risk of infection, plus there is a lot of anxiety on behalf of the patient that the battery is running out.
My colleagues at the University of Connecticut and I, have developed a new way of charging these devices using the person’s own breathing movements, which would mean that the batteries never need to be changed. As the patient breathes in and out, their chest wall presses on a thin electric generator, which converts that movement into static electricity, charging a supercapacitor. This discharges the electricity required to power the DBS device and send the necessary pulses to the brain.
So far it has only been tested in a simulated pig, using a pig lung connected to a pump, but there are plans now to test it in a real animal, and then progress to clinical trials.
Smartwatches for Disease Detection
Jessilyn Dunn, Assistant Professor in Duke Biomedical Engineering
A group of researchers recently showed that digital biomarkers of infection can reveal when someone is sick, often before they feel sick. The team, which included Duke biomedical engineers, used information from smartwatches to detect Covid-19 cases five to 10 days earlier than diagnostic tests. Smartwatch data included aspects of heart rate, sleep quality and physical activity. Based on this data, we developed an algorithm to decide which people have the most need to take the diagnostic tests. With this approach, the percent of tests that come back positive are about four- to six-times higher, depending on which factors we monitor through the watches.
Our study was one of several showing the value of digital biomarkers, rather than a single blockbuster paper. With so many new ideas and technologies coming out around Covid, it’s hard to be that signal through the noise. More studies are needed, but this line of research is important because, rather than treat everyone as equally likely to have an infectious disease, we can use prior knowledge from smartwatches. With monkeypox, for example, you've got many more people who need to be tested than you have tests available. Information from the smartwatches enables you to improve how you allocate those tests.
Smartwatch data could also be applied to chronic diseases. For viruses, we’re looking for information about anomalies – a big change point in people’s health. For chronic diseases, it’s more like a slow, steady change. Our research lays the groundwork for the signals coming from smartwatches to be useful in a health setting, and now it’s up to us to detect more of these chronic cases. We want to go from the idea that we have this single change point, like a heart attack or stroke, and focus on the part before that, to see if we can detect it.
A Vaccine For RSV
Norbert Pardi, Vaccines Group Lead, Penn Institute for RNA Innovation, University of Pennsylvania
Scientists have long been trying to develop a vaccine for respiratory syncytial virus (RSV), and it looks like Pfizer are closing in on this goal, based on the latest clinical trial data in newborns which they released in November. Pfizer have developed a protein-based vaccine against the F protein of RSV, which they are giving to pregnant women. It turns out that it induces a robust immune response after the administration of a single shot and it seems to be highly protective in newborns. The efficacy was over 80% after 90 days, so it protected very well against severe disease, and even though this dropped a little after six month, it was still pretty high.
I think this has been a very important breakthrough, and very timely at the moment with both COVID-19, influenza and RSV circulating, which just shows the importance of having a vaccine which works well in both the very young and the very old.
The road to an RSV vaccine has also illustrated the importance of teamwork in 21st century vaccine development. You need people with different backgrounds to solve these challenges – microbiologists, immunologists and structural biologists working together to understand how viruses work, and how our immune system induces protective responses against certain viruses. It has been this kind of teamwork which has yielded the findings that targeting the prefusion stabilized form of the F protein in RSV induces much stronger and highly protective immune responses.
Gene therapy shows its potential
Nicole Paulk, Assistant Professor of Gene Therapy at the University of California, San Francisco
The recent US Food and Drug Administration (FDA) approval of Hemgenix, a gene therapy for hemophilia B, is big for a lot of reasons. While hemophilia is absolutely a rare disease, it is astronomically more common than the first two approvals – Luxturna for RPE65-meidated inherited retinal dystrophy and Zolgensma for spinal muscular atrophy - so many more patients will be treated with this. In terms of numbers of patients, we are now starting to creep up into things that are much more common, which is a huge step in terms of our ability to scale the production of an adeno-associated virus (AAV) vector for gene therapy.
Hemophilia is also a really special patient population because this has been the darling indication for AAV gene therapy for the last 20 to 30 years. AAV trafficks to the liver so well, it’s really easy for us to target the tissues that we want. If you look at the numbers, there have been more gene therapy scientists working on hemophilia than any other condition. There have just been thousands and thousands of us working on gene therapy indications for the last 20 or 30 years, so to see the first of these approvals make it, feels really special.
I am sure it is even more special for the patients because now they have a choice – do I want to stay on my recombinant factor drug that I need to take every day for the rest of my life, or right now I could get a one-time infusion of this virus and possibly experience curative levels of expression for the rest of my life. And this is just the first one for hemophilia, there’s going to end up being a dozen gene therapies within the next five years, targeted towards different hemophilias.
Every single approval is momentous for the entire field because it gets investors excited, it gets companies and physicians excited, and that helps speed things up. Right now, it's still a challenge to produce enough for double digit patients. But with more interest comes the experiments and trials that allow us to pick up the knowledge to scale things up, so that we can go after bigger diseases like diabetes, congestive heart failure, cancer, all of these much bigger afflictions.
Treating Thickened Hearts
John Spertus, Professor in Metabolic and Vascular Disease Research, UMKC School of Medicine
Hypertrophic cardiomyopathy (HCM) is a disease that causes your heart muscle to enlarge, and the walls of your heart chambers thicken and reduce in size. Because of this, they cannot hold as much blood and may stiffen, causing some sufferers to experience progressive shortness of breath, fatigue and ultimately heart failure.
So far we have only had very crude ways of treating it, using beta blockers, calcium channel blockers or other medications which cause the heart to beat less strongly. This works for some patients but a lot of time it does not, which means you have to consider removing part of the wall of the heart with surgery.
Earlier this year, a trial of a drug called mavacamten, became the first study to show positive results in treating HCM. What is remarkable about mavacamten is that it is directed at trying to block the overly vigorous contractile proteins in the heart, so it is a highly targeted, focused way of addressing the key problem in these patients. The study demonstrated a really large improvement in patient quality of life where they were on the drug, and when they went off the drug, the quality of life went away.
Some specialists are now hypothesizing that it may work for other cardiovascular diseases where the heart either beats too strongly or it does not relax well enough, but just having a treatment for HCM is a really big deal. For years we have not been very aggressive in identifying and treating these patients because there have not been great treatments available, so this could lead to a new era.
Regenerating Organs
David Andrijevic, Associate Research Scientist in neuroscience at Yale School of Medicine
As soon as the heartbeat stops, a whole chain of biochemical processes resulting from ischemia – the lack of blood flow, oxygen and nutrients – begins to destroy the body’s cells and organs. My colleagues and I at Yale School of Medicine have been investigating whether we can recover organs after prolonged ischemia, with the main goal of expanding the organ donor pool.
Earlier this year we published a paper in which we showed that we could use technology to restore blood circulation, other cellular functions and even heart activity in pigs, one hour after their deaths. This was done using a perfusion technology to substitute heart, lung and kidney function, and deliver an experimental cell protective fluid to these organs which aimed to stop cell death and aid in the recovery.
One of the aims of this technology is that it can be used in future to lengthen the time window for recovering organs for donation after a person has been declared dead, a logistical hurdle which would allow us to substantially increase the donor pool. We might also be able to use this cell protective fluid in studies to see if it can help people who have suffered from strokes and myocardial infarction. In future, if we managed to achieve an adequate brain recovery – and the brain, out of all the organs, is the most susceptible to ischemia – this might also change some paradigms in resuscitation medicine.
Antibody-Drug Conjugates for Cancer
Yosi Shamay, Cancer Nanomedicine and Nanoinformatics researcher at the Technion Israel Institute of Technology
For the past four or five years, antibody-drug conjugates (ADCs) - a cancer drug where you have an antibody conjugated to a toxin - have been used only in patients with specific cancers that display high expression of a target protein, for example HER2-positive breast cancer. But in 2022, there have been clinical trials where ADCs have shown remarkable results in patients with low expression of HER2, which is something we never expected to see.
In July 2022, AstraZeneca published the results of a clinical trial, which showed that an ADC called trastuzumab deruxtecan can offer a very big survival benefit to breast cancer patients with very little expression of HER2, levels so low that they would be borderline undetectable for a pathologist. They got a strong survival signal for patients with very aggressive, metastatic disease.
I think this is very interesting and important because it means that it might pave the way to include more patients in clinical trials looking at ADCs for other cancers, for example lymphoma, colon cancer, lung cancers, even if they have low expression of the protein target. It also holds implications for CAR-T cells - where you genetically engineer a T cell to attack the cancer - because the concept is very similar. If we now know that an ADC can have a survival benefit, even in patients with very low target expression, the same might be true for T cells.
Look back further: Breakthroughs of 2021
https://leaps.org/6-biotech-breakthroughs-of-2021-that-missed-the-attention-they-deserved/
Repairing Cells and Longevity Myths with Dr. Charles Brenner
Meet Charles Brenner, the Longevity Skeptic. Brenner, a leading biochemist at City of Hope National Medical Center in L.A., has been attending the largest longevity conferences with one main purpose: to point out that some of the other speakers are full of it.
Brenner is "throwing cold water" on several scientists in the field of aging, accusing them of hyping various fountains of youth, despite limited evidence for these therapies.
In this podcast episode, Brenner sat down with Leaps.org to discuss his groundbreaking work on metabolism and his efforts to counter what he considers to be bad science.
Listen on Apple | Listen on Spotify | Listen on Stitcher | Listen on Amazon | Listen on Google
In addition to bringing his candor to conferences, Brenner is applying it in academic journals, publishing a paper in September, "A Science-Based Review of the World's Best-Selling Book on Aging," in which he pans the author of this bestseller, David Sinclair, a Harvard biologist, for talking up the potential for humans to live far past 100. These aspirations may sound nice, but they're not backed by science, Brenner says. He's had high-profile debates online with Sinclair and Aubrey de Grey, a prominent biomedical gerontologist.
Meanwhile, in his own lab work, Brenner is credited with identifying a vitamin precursor called NR that seems to enable repair of cellular damage that happens as we get older - a major discovery that he's helped turn into a supplement, commercialized with a company called ChromaDex.
Whether it's possible to extend human lifespan is a pressing question as investments in longevity startups are projected to increase from $40 billion to $600 billion over the next three years. The field of biological aging seems split on the question of whether "anti-aging" therapies can significantly lengthen our natural lifespans, as Sinclair believes. Brenner, Morgan Levine of Altos Labs and Matt Kaeberlein of the University of Washington have argued that the only realistic goal is to extend one's window of healthy years, or healthspan, rather than trying to break the biological ceiling of our species.
Brenner is an intriguing figure in these debates. Although he’s been introduced in public appearances as a longevity skeptic, he calls himself an optimist.
Links:
Charles Brenner, City of Hope
Charles Brenner on Twitter
Charles Brenner's debate with Aubrey de Grey
Brenner's paper, "A Science-Based Review of the World's Best-Selling Book on Aging"
Peter Attia's recent blog on NR supplements
Brad Stanfield's recent reaction to "David Sinclair vs Charles Brenner"
ChromaDex NR supplement