Who Qualifies as an “Expert” And How Can We Decide Who Is Trustworthy?
Discerning a real expert from a charlatan is crucial during the COVID-19 pandemic and beyond.
This article is part of the magazine, "The Future of Science In America: The Election Issue," co-published by LeapsMag, the Aspen Institute Science & Society Program, and GOOD.
Expertise is a slippery concept. Who has it, who claims it, and who attributes or yields it to whom is a culturally specific, sociological process. During the COVID-19 pandemic, we have witnessed a remarkable emergence of legitimate and not-so-legitimate scientists publicly claiming or being attributed to have academic expertise in precisely my field: infectious disease epidemiology. From any vantage point, it is clear that charlatans abound out there, garnering TV coverage and hundreds of thousands of Twitter followers based on loud opinions despite flimsy credentials. What is more interesting as an insider is the gradient of expertise beyond these obvious fakers.
A person's expertise is not a fixed attribute; it is a hierarchical trait defined relative to others. Despite my protestations, I am the go-to expert on every aspect of the pandemic to my family. To a reporter, I might do my best to answer a question about the immune response to SARS-CoV-2, noting that I'm not an immunologist. Among other academic scientists, my expertise is more well-defined as a subfield of epidemiology, and within that as a particular area within infectious disease epidemiology. There's a fractal quality to it; as you zoom in on a particular subject, a differentiation of expertise emerges among scientists who, from farther out, appear to be interchangeable.
We all have our scientific domain and are less knowledgeable outside it, of course, and we are often asked to comment on a broad range of topics. But many scientists without a track record in the field have become favorites among university administrators, senior faculty in unrelated fields, policymakers, and science journalists, using institutional prestige or social connections to promote themselves. This phenomenon leads to a distorted representation of science—and of academic scientists—in the public realm.
Trustworthy experts will direct you to others in their field who know more about particular topics, and will tend to be honest about what is and what isn't "in their lane."
Predictably, white male voices have been disproportionately amplified, and men are certainly over-represented in the category of those who use their connections to inappropriately claim expertise. Generally speaking, we are missing women, racial minorities, and global perspectives. This is not only important because it misrepresents who scientists are and reinforces outdated stereotypes that place white men in the Global North at the top of a credibility hierarchy. It also matters because it can promote bad science, and it passes over scientists who can lend nuance to the scientific discourse and give global perspectives on this quintessentially global crisis.
Also at work, in my opinion, are two biases within academia: the conflation of institutional prestige with individual expertise, and the bizarre hierarchy among scientists that attributes greater credibility to those in quantitative fields like physics. Regardless of mathematical expertise or institutional affiliation, lack of experience working with epidemiological data can lead to over-confidence in the deceptively simple mathematical models that we use to understand epidemics, as well as the inappropriate use of uncertain data to inform them. Prominent and vocal scientists from different quantitative fields have misapplied the methods of infectious disease epidemiology during the COVID-19 pandemic so far, creating enormous confusion among policymakers and the public. Early forecasts that predicted the epidemic would be over by now, for example, led to a sense that epidemiological models were all unreliable.
Meanwhile, legitimate scientific uncertainties and differences of opinion, as well as fundamentally different epidemic dynamics arising in diverse global contexts and in different demographic groups, appear in the press as an indistinguishable part of this general chaos. This leads many people to question whether the field has anything worthwhile to contribute, and muddies the facts about COVID-19 policies for reducing transmission that most experts agree on, like wearing masks and avoiding large indoor gatherings.
So how do we distinguish an expert from a charlatan? I believe a willingness to say "I don't know" and to openly describe uncertainties, nuances, and limitations of science are all good signs. Thoughtful engagement with questions and new ideas is also an indication of expertise, as opposed to arrogant bluster or a bullish insistence on a particular policy strategy regardless of context (which is almost always an attempt to hide a lack of depth of understanding). Trustworthy experts will direct you to others in their field who know more about particular topics, and will tend to be honest about what is and what isn't "in their lane." For example, some expertise is quite specific to a given subfield: epidemiologists who study non-infectious conditions or nutrition, for example, use different methods from those of infectious disease experts, because they generally don't need to account for the exponential growth that is inherent to a contagion process.
Academic scientists have a specific, technical contribution to make in containing the COVID-19 pandemic and in communicating research findings as they emerge. But the liminal space between scientists and the public is subject to the same undercurrents of sexism, racism, and opportunism that society and the academy have always suffered from. Although none of the proxies for expertise described above are fool-proof, they are at least indicative of integrity and humility—two traits the world is in dire need of at this moment in history.
[Editor's Note: To read other articles in this special magazine issue, visit the beautifully designed e-reader version.]
A vaccine from Moderna could be available as early as January, Fauci recently said.
Earlier this year, biotech company Moderna broke world records for speed in vaccine development. Their researchers translated the genetic code of the coronavirus into a vaccine candidate in just 42 days.
We're about to expand our safety data in Phase II.
Phase I of the clinical trial started in Seattle on March 16th, with the already-iconic image of volunteer Jennifer Haller calmly receiving the very first dose.
Instead of traditional methods, this vaccine uses a new -- and so far unproven -- technology based on synthetic biology: It hijacks the software of life – messenger RNA – to deliver a copy of the virus's genetic sequence into cells, which, in theory, triggers the body to produce antibodies to fight off a coronavirus infection.
U.S. National Institute of Allergy and Infectious Diseases Director Anthony Fauci called the vaccine's preclinical data "impressive" and told National Geographic this week that a vaccine could be ready for general use as early as January.
The Phase I trial has dosed 45 healthy adults. Phase II trials are about to start, enrolling around 600 adults. Pivotal efficacy trials would follow soon thereafter, bankrolled in collaboration with the government office BARDA (Biomedical Advanced Research and Development Authority).
Today, the chief medical officer of Moderna, Tal Zaks, answered burning questions from the public in a webinar hosted by STAT. Here's an edited and condensed summary of his answers.
1) When will a vaccine become available?
We expect to have data in early summer about the antibody levels from our mRNA vaccine. At the same time, we can measure the antibody levels of people who have had the disease, and we should be able to measure the ability of those antibodies to prevent disease.
We will not yet know if the mRNA vaccine works to prevent disease, but we could soon talk about a potential for benefit. We don't yet know about risk. We're about to expand our safety data in Phase II.
In the summer, there is an expectation that we will be launching pivotal trials, in collaboration with government agencies that are helping fund the research. The trials would be launched with the vaccine vs. a placebo with the goal of establishing: How many cases can we show we prevented with the vaccine?
This is determined by two factors: How big is the trial? And what's the attack rate in the population we vaccinate? The challenge will be to vaccinate in the areas where the risk of infection is still high in the coming months, and we're able to vaccinate and demonstrate fewer infections compared to a placebo. If the disease is happening faster in a given area, you will be able to see an outcome faster. Potentially by the end of the year, we will have the data to say if the vaccine works.
Will that be enough for regulatory approval? The main question is: When will we cross the threshold for the anticipated benefit of a presumed vaccine to be worth the risk?
There is a distinction between approval for those who need it most, like the elderly. Their unmet need and risk/benefit is not the same as it is for younger adults.
My private opinion: I don't think it's a one-size-fits-all. It will be a more measured stance.
2) Can you speed up the testing process with challenge studies, where volunteers willingly get infected?
It's a great question and I applaud the people who ask it and I applaud those signing up to do it. I'm not sure I am a huge fan, for both practical and ethical reasons. The devil is in the details. A challenge study has to show us a vaccine can prevent not just infection but prevent disease. Otherwise, how do I know the dose in the challenge study is the right dose? If you take 100 young people, 90 of them will get mild or no disease. Ten may end up in hospital and one in the ICU.
Also, the timeline. Can it let you skip Phase II of large efficacy trial? The reality for us is that we are about to start Phase II anyway. It would be months before a challenge trial could be designed. And ethically: everybody agrees there is a risk that is not zero of having very serious disease. To justify the risk, we have to be sure the benefit is worth it - that it actually shrunk the timeline. To just give us another data point, I find it hard to accept.
This technology allows us to scale up manufacturing and production.
3) What was seen preclinically in the animal models with Moderna's mRNA vaccines?
We have taken vaccines using our technology against eight different viruses, including two flu strains. In every case, in the preclinical model, we showed we could prevent disease, and when we got to antibody levels, we got the data we wanted to see. In doses of 25-100 micrograms, that usually ends up being a sweet spot where we see an effect. It's a good place as to the expectation of what we will see in Phase I trials.
4) Why is Moderna pursuing an mRNA virus instead of a traditional inactivated virus or recombinant one? This is an untried technology.
First, speed matters in a pandemic. If you have tech that can move much quicker, that makes a difference. The reason we have broken world records is that we have invested time and effort to be ready. We're starting from a platform where it's all based on synthetic biology.
Second, it's fundamental biology - we do not need to make an elaborate vaccine or stick a new virus in an old virus, or try to make a neutralizing but not binding virus. Our technology is basically mimicking the virus. All life works on making proteins through RNA. We have a biological advantage by teaching the immune system to do the right thing.
Third, this technology allows us to scale up manufacturing and production. We as a company have always seen this ahead of us. We invested in our own manufacturing facility two years ago. We have already envisioned scale up on two dimensions. Lot size and vaccines. Vaccines is the easier piece of it. If everybody gets 100 micrograms, it's not a heck of a lot. Prior to COVID, our lead program was a CMV (Cytomegalovirus) vaccine. We had envisioned launching Phase III next year. We had been already well on the path to scale up when COVID-19 caught us by surprise. This would be millions and millions of doses, but the train tracks have been laid.
5) People tend to think of vaccines as an on-off switch -- you get a vaccine and you're protected. But efficacy can be low or high (like the flu vs. measles vaccines). How good is good enough here for protection, and could we need several doses?
Probably around 50-60 percent efficacy is good enough for preventing a significant amount of disease and decreasing the R0. We will aim higher, but it's hard to estimate what degree of efficacy to prepare for until we do the trial. (For comparison, the average flu vaccine efficacy is around 50 percent.)
We anticipate a prime boost. If our immune system has never seen a virus, you can show you're getting to a certain antibody level and then remind the immune system (with another dose). A prime boost is optimal.
My only two competitors are the virus and the clock.
6) How would mutations affect a vaccine?
Coronaviruses tend to mutate the least compared to other viruses but it's entirely possible that it mutates. The report this week about those projected mutations on the spike protein have not been predicted to alter the critical antibodies.
As we scale up manufacturing, the ability to plug in a new genetic sequence and get a new vaccine out there will be very rapid.
For flu vaccine, we don't prove efficacy every year. If we get to the same place with an mRNA vaccine, we will just change the sequence and come out with a new vaccine. The path to approval would be much faster if we leverage the totality of efficacy data like we do for flu.
7) Will there be more than one vaccine and how will they be made available?
I hope so, I don't know. The path to making these available will go through a public-private partnership. It's not your typical commercial way of deploying a vaccine. But my only two competitors are the virus and the clock. We need everybody to be successful.
Kira Peikoff was the editor-in-chief of Leaps.org from 2017 to 2021. As a journalist, her work has appeared in The New York Times, Newsweek, Nautilus, Popular Mechanics, The New York Academy of Sciences, and other outlets. She is also the author of four suspense novels that explore controversial issues arising from scientific innovation: Living Proof, No Time to Die, Die Again Tomorrow, and Mother Knows Best. Peikoff holds a B.A. in Journalism from New York University and an M.S. in Bioethics from Columbia University. She lives in New Jersey with her husband and two young sons. Follow her on Twitter @KiraPeikoff.
Meet the Scientists on the Frontlines of Protecting Humanity from a Man-Made Pathogen
From left: Jean Peccoud, Randall Murch, and Neeraj Rao.
Jean Peccoud wasn't expecting an email from the FBI. He definitely wasn't expecting the agency to invite him to a meeting. "My reaction was, 'What did I do wrong to be on the FBI watch list?'" he recalls.
You use those blueprints for white-hat research—which is, indeed, why the open blueprints exist—or you can do the same for a black-hat attack.
He didn't know what the feds could possibly want from him. "I was mostly scared at this point," he says. "I was deeply disturbed by the whole thing."
But he decided to go anyway, and when he traveled to San Francisco for the 2008 gathering, the reason for the e-vite became clear: The FBI was reaching out to researchers like him—scientists interested in synthetic biology—in anticipation of the potential nefarious uses of this technology. "The whole purpose of the meeting was, 'Let's start talking to each other before we actually need to talk to each other,'" says Peccoud, now a professor of chemical and biological engineering at Colorado State University. "'And let's make sure next time you get an email from the FBI, you don't freak out."
Synthetic biology—which Peccoud defines as "the application of engineering methods to biological systems"—holds great power, and with that (as always) comes great responsibility. When you can synthesize genetic material in a lab, you can create new ways of diagnosing and treating people, and even new food ingredients. But you can also "print" the genetic sequence of a virus or virulent bacterium.
And while it's not easy, it's also not as hard as it could be, in part because dangerous sequences have publicly available blueprints. You use those blueprints for white-hat research—which is, indeed, why the open blueprints exist—or you can do the same for a black-hat attack. You could synthesize a dangerous pathogen's code on purpose, or you could unwittingly do so because someone tampered with your digital instructions. Ordering synthetic genes for viral sequences, says Peccoud, would likely be more difficult today than it was a decade ago.
"There is more awareness of the industry, and they are taking this more seriously," he says. "There is no specific regulation, though."
Trying to lock down the interconnected machines that enable synthetic biology, secure its lab processes, and keep dangerous pathogens out of the hands of bad actors is part of a relatively new field: cyberbiosecurity, whose name Peccoud and colleagues introduced in a 2018 paper.
Biological threats feel especially acute right now, during the ongoing pandemic. COVID-19 is a natural pathogen -- not one engineered in a lab. But future outbreaks could start from a bug nature didn't build, if the wrong people get ahold of the right genetic sequences, and put them in the right sequence. Securing the equipment and processes that make synthetic biology possible -- so that doesn't happen -- is part of why the field of cyberbiosecurity was born.
The Origin Story
It is perhaps no coincidence that the FBI pinged Peccoud when it did: soon after a journalist ordered a sequence of smallpox DNA and wrote, for The Guardian, about how easy it was. "That was not good press for anybody," says Peccoud. Previously, in 2002, the Pentagon had funded SUNY Stonybrook researchers to try something similar: They ordered bits of polio DNA piecemeal and, over the course of three years, strung them together.
Although many years have passed since those early gotchas, the current patchwork of regulations still wouldn't necessarily prevent someone from pulling similar tricks now, and the technological systems that synthetic biology runs on are more intertwined — and so perhaps more hackable — than ever. Researchers like Peccoud are working to bring awareness to those potential problems, to promote accountability, and to provide early-detection tools that would catch the whiff of a rotten act before it became one.
Peccoud notes that if someone wants to get access to a specific pathogen, it is probably easier to collect it from the environment or take it from a biodefense lab than to whip it up synthetically. "However, people could use genetic databases to design a system that combines different genes in a way that would make them dangerous together without each of the components being dangerous on its own," he says. "This would be much more difficult to detect."
After his meeting with the FBI, Peccoud grew more interested in these sorts of security questions. So he was paying attention when, in 2010, the Department of Health and Human Services — now helping manage the response to COVID-19 — created guidance for how to screen synthetic biology orders, to make sure suppliers didn't accidentally send bad actors the sequences that make up bad genomes.
Guidance is nice, Peccoud thought, but it's just words. He wanted to turn those words into action: into a computer program. "I didn't know if it was something you can run on a desktop or if you need a supercomputer to run it," he says. So, one summer, he tasked a team of student researchers with poring over the sentences and turning them into scripts. "I let the FBI know," he says, having both learned his lesson and wanting to get in on the game.
Peccoud later joined forces with Randall Murch, a former FBI agent and current Virginia Tech professor, and a team of colleagues from both Virginia Tech and the University of Nebraska-Lincoln, on a prototype project for the Department of Defense. They went into a lab at the University of Nebraska at Lincoln and assessed all its cyberbio-vulnerabilities. The lab develops and produces prototype vaccines, therapeutics, and prophylactic components — exactly the kind of place that you always, and especially right now, want to keep secure.
"We were creating wiki of all these nasty things."
The team found dozens of Achilles' heels, and put them in a private report. Not long after that project, the two and their colleagues wrote the paper that first used the term "cyberbiosecurity." A second paper, led by Murch, came out five months later and provided a proposed definition and more comprehensive perspective on cyberbiosecurity. But although it's now a buzzword, it's the definition, not the jargon, that matters. "Frankly, I don't really care if they call it cyberbiosecurity," says Murch. Call it what you want: Just pay attention to its tenets.
A Database of Scary Sequences
Peccoud and Murch, of course, aren't the only ones working to screen sequences and secure devices. At the nonprofit Battelle Memorial Institute in Columbus, Ohio, for instance, scientists are working on solutions that balance the openness inherent to science and the closure that can stop bad stuff. "There's a challenge there that you want to enable research but you want to make sure that what people are ordering is safe," says the organization's Neeraj Rao.
Rao can't talk about the work Battelle does for the spy agency IARPA, the Intelligence Advanced Research Projects Activity, on a project called Fun GCAT, which aims to use computational tools to deep-screen gene-sequence orders to see if they pose a threat. It can, though, talk about a twin-type internal project: ThreatSEQ (pronounced, of course, "threat seek").
The project started when "a government customer" (as usual, no one will say which) asked Battelle to curate a list of dangerous toxins and pathogens, and their genetic sequences. The researchers even started tagging sequences according to their function — like whether a particular sequence is involved in a germ's virulence or toxicity. That helps if someone is trying to use synthetic biology not to gin up a yawn-inducing old bug but to engineer a totally new one. "How do you essentially predict what the function of a novel sequence is?" says Rao. You look at what other, similar bits of code do.
"We were creating wiki of all these nasty things," says Rao. As they were working, they realized that DNA manufacturers could potentially scan in sequences that people ordered, run them against the database, and see if anything scary matched up. Kind of like that plagiarism software your college professors used.
Battelle began offering their screening capability, as ThreatSEQ. When customers -- like, currently, Twist Bioscience -- throw their sequences in, and get a report back, the manufacturers make the final decision about whether to fulfill a flagged order — whether, in the analogy, to give an F for plagiarism. After all, legitimate researchers do legitimately need to have DNA from legitimately bad organisms.
"Maybe it's the CDC," says Rao. "If things check out, oftentimes [the manufacturers] will fulfill the order." If it's your aggrieved uncle seeking the virulent pathogen, maybe not. But ultimately, no one is stopping the manufacturers from doing so.
Beyond that kind of tampering, though, cyberbiosecurity also includes keeping a lockdown on the machines that make the genetic sequences. "Somebody now doesn't need physical access to infrastructure to tamper with it," says Rao. So it needs the same cyber protections as other internet-connected devices.
Scientists are also now using DNA to store data — encoding information in its bases, rather than into a hard drive. To download the data, you sequence the DNA and read it back into a computer. But if you think like a bad guy, you'd realize that a bad guy could then, for instance, insert a computer virus into the genetic code, and when the researcher went to nab her data, her desktop would crash or infect the others on the network.
Something like that actually happened in 2017 at the USENIX security symposium, an annual programming conference: Researchers from the University of Washington encoded malware into DNA, and when the gene sequencer assembled the DNA, it corrupted the sequencer's software, then the computer that controlled it.
"This vulnerability could be just the opening an adversary needs to compromise an organization's systems," Inspirion Biosciences' J. Craig Reed and Nicolas Dunaway wrote in a paper for Frontiers in Bioengineering and Biotechnology, included in an e-book that Murch edited called Mapping the Cyberbiosecurity Enterprise.
Where We Go From Here
So what to do about all this? That's hard to say, in part because we don't know how big a current problem any of it poses. As noted in Mapping the Cyberbiosecurity Enterprise, "Information about private sector infrastructure vulnerabilities or data breaches is protected from public release by the Protected Critical Infrastructure Information (PCII) Program," if the privateers share the information with the government. "Government sector vulnerabilities or data breaches," meanwhile, "are rarely shared with the public."
"What I think is encouraging right now is the fact that we're even having this discussion."
The regulations that could rein in problems aren't as robust as many would like them to be, and much good behavior is technically voluntary — although guidelines and best practices do exist from organizations like the International Gene Synthesis Consortium and the National Institute of Standards and Technology.
Rao thinks it would be smart if grant-giving agencies like the National Institutes of Health and the National Science Foundation required any scientists who took their money to work with manufacturing companies that screen sequences. But he also still thinks we're on our way to being ahead of the curve, in terms of preventing print-your-own bioproblems: "What I think is encouraging right now is the fact that we're even having this discussion," says Rao.
Peccoud, for his part, has worked to keep such conversations going, including by doing training for the FBI and planning a workshop for students in which they imagine and work to guard against the malicious use of their research. But actually, Peccoud believes that human error, flawed lab processes, and mislabeled samples might be bigger threats than the outside ones. "Way too often, I think that people think of security as, 'Oh, there is a bad guy going after me,' and the main thing you should be worried about is yourself and errors," he says.
Murch thinks we're only at the beginning of understanding where our weak points are, and how many times they've been bruised. Decreasing those contusions, though, won't just take more secure systems. "The answer won't be technical only," he says. It'll be social, political, policy-related, and economic — a cultural revolution all its own.