Who Qualifies as an “Expert” And How Can We Decide Who Is Trustworthy?
Discerning a real expert from a charlatan is crucial during the COVID-19 pandemic and beyond.
This article is part of the magazine, "The Future of Science In America: The Election Issue," co-published by LeapsMag, the Aspen Institute Science & Society Program, and GOOD.
Expertise is a slippery concept. Who has it, who claims it, and who attributes or yields it to whom is a culturally specific, sociological process. During the COVID-19 pandemic, we have witnessed a remarkable emergence of legitimate and not-so-legitimate scientists publicly claiming or being attributed to have academic expertise in precisely my field: infectious disease epidemiology. From any vantage point, it is clear that charlatans abound out there, garnering TV coverage and hundreds of thousands of Twitter followers based on loud opinions despite flimsy credentials. What is more interesting as an insider is the gradient of expertise beyond these obvious fakers.
A person's expertise is not a fixed attribute; it is a hierarchical trait defined relative to others. Despite my protestations, I am the go-to expert on every aspect of the pandemic to my family. To a reporter, I might do my best to answer a question about the immune response to SARS-CoV-2, noting that I'm not an immunologist. Among other academic scientists, my expertise is more well-defined as a subfield of epidemiology, and within that as a particular area within infectious disease epidemiology. There's a fractal quality to it; as you zoom in on a particular subject, a differentiation of expertise emerges among scientists who, from farther out, appear to be interchangeable.
We all have our scientific domain and are less knowledgeable outside it, of course, and we are often asked to comment on a broad range of topics. But many scientists without a track record in the field have become favorites among university administrators, senior faculty in unrelated fields, policymakers, and science journalists, using institutional prestige or social connections to promote themselves. This phenomenon leads to a distorted representation of science—and of academic scientists—in the public realm.
Trustworthy experts will direct you to others in their field who know more about particular topics, and will tend to be honest about what is and what isn't "in their lane."
Predictably, white male voices have been disproportionately amplified, and men are certainly over-represented in the category of those who use their connections to inappropriately claim expertise. Generally speaking, we are missing women, racial minorities, and global perspectives. This is not only important because it misrepresents who scientists are and reinforces outdated stereotypes that place white men in the Global North at the top of a credibility hierarchy. It also matters because it can promote bad science, and it passes over scientists who can lend nuance to the scientific discourse and give global perspectives on this quintessentially global crisis.
Also at work, in my opinion, are two biases within academia: the conflation of institutional prestige with individual expertise, and the bizarre hierarchy among scientists that attributes greater credibility to those in quantitative fields like physics. Regardless of mathematical expertise or institutional affiliation, lack of experience working with epidemiological data can lead to over-confidence in the deceptively simple mathematical models that we use to understand epidemics, as well as the inappropriate use of uncertain data to inform them. Prominent and vocal scientists from different quantitative fields have misapplied the methods of infectious disease epidemiology during the COVID-19 pandemic so far, creating enormous confusion among policymakers and the public. Early forecasts that predicted the epidemic would be over by now, for example, led to a sense that epidemiological models were all unreliable.
Meanwhile, legitimate scientific uncertainties and differences of opinion, as well as fundamentally different epidemic dynamics arising in diverse global contexts and in different demographic groups, appear in the press as an indistinguishable part of this general chaos. This leads many people to question whether the field has anything worthwhile to contribute, and muddies the facts about COVID-19 policies for reducing transmission that most experts agree on, like wearing masks and avoiding large indoor gatherings.
So how do we distinguish an expert from a charlatan? I believe a willingness to say "I don't know" and to openly describe uncertainties, nuances, and limitations of science are all good signs. Thoughtful engagement with questions and new ideas is also an indication of expertise, as opposed to arrogant bluster or a bullish insistence on a particular policy strategy regardless of context (which is almost always an attempt to hide a lack of depth of understanding). Trustworthy experts will direct you to others in their field who know more about particular topics, and will tend to be honest about what is and what isn't "in their lane." For example, some expertise is quite specific to a given subfield: epidemiologists who study non-infectious conditions or nutrition, for example, use different methods from those of infectious disease experts, because they generally don't need to account for the exponential growth that is inherent to a contagion process.
Academic scientists have a specific, technical contribution to make in containing the COVID-19 pandemic and in communicating research findings as they emerge. But the liminal space between scientists and the public is subject to the same undercurrents of sexism, racism, and opportunism that society and the academy have always suffered from. Although none of the proxies for expertise described above are fool-proof, they are at least indicative of integrity and humility—two traits the world is in dire need of at this moment in history.
[Editor's Note: To read other articles in this special magazine issue, visit the beautifully designed e-reader version.]
Embrace the mess: how to choose which scientists to trust
A dozen bioethicists and researchers shared their advice on how to spot the scientists searching for the truth more than money, ego or fame.
It’s no easy task these days for people to pick the scientists they should follow. According to a recent poll by NORC at the University of Chicago, only 39 percent of Americans have a "great deal" of confidence in the scientific community. The finding is similar to Pew research last year showing that 29 percent of Americans have this level of confidence in medical scientists.
Not helping: All the money in science. Just 20 percent of Pew’s survey respondents think scientists are transparent about conflicts of interest with industry. While this issue is common to many fields, the recent gold rush to foot the bill for research on therapies for healthy aging may be contributing to the overall sense of distrust. “There’s a feeling that at some point, the FDA may actually designate aging as a disease,” said Pam Maher, a neuroscientist who studies aging at Salk Institute. “That may be another impetus for a lot of these companies to start up.”
But partnering with companies is an important incentive for researchers across biomedical fields. Many scientists – with and without financial ties and incentives – are honest, transparent and doing important, inspiring work. I asked more than a dozen bioethicists and researchers in aging how to spot the scientists who are searching for the truth more than money, ego or fame.
Avoid Scientists Who Sound Overly Confident in messaging to the public. Some multi-talented scientists are adept at publishing in both top journals and media outlets. They’re great at dropping science without the confusing jargon, in ways the public can enjoy and learn from.
But do they talk in simple soundbites, painting scientific debates in pastels or black and white when colleagues use shades of gray? Maybe they crave your attention more than knowledge seeking. “When scientists speak in a very unnuanced way, that can be irresponsible,” said Josephine Johnston, a bioethicist at the Hastings Center.
Scientists should avoid exaggerations like “without a doubt” and even “we know” – unless they absolutely do. “I feel like there’s more and more hyperbole and attention seeking…[In aging research,] the loudest voices in the room are the fringe people,” said the biogenerontologist Matt Kaeberlein.
Separate Hype from Passion. Scientists should be, need to be passionate, Johnston explained. In the realm of aging, for example, Leonard Guarente, an MIT biologist and pioneer in the field of aging, told me about his belief that longer lifespans would make for a better world.
Instead of expecting scientists to be lab-dwelling robots, we should welcome their passion. It fuels scientific dedication and creativity. Fields like aging, AI and gene editing inspire the imaginations of the public and scientists alike. That’s not a bad thing.
But it does lay fertile ground for overstatements, such as claims by some that the first 1,000-year-old has already been born. If it sounds like sci-fi, it’s probably sci-fi.
Watch Out for Cult Behavior, some experts told me. Follow scientists who mix it up and engage in debates, said NYU bioethicist Arthur Caplan, not those who hang out only with researchers in the same ideological camp.
Look for whether they’re open to working with colleagues who don’t share their views. Through collaboration, they can resolve conflicting study results and data, said Danica Chen, a biologist at UC Berkeley. We should trust science as long as it doesn’t trust itself.
Messiness is Good. You want to find and follow scientists who’ve published research over the years that does not tell a clean story. “Our goal is to disprove our models,” Kaeberlein said. Scientific findings and views should zig and zag as their careers – and science – progress.
Follow scientists who write and talk publicly about new evidence that’s convinced them to reevaluate their own positions. Who embrace the inherent messiness of science – that’s the hallmark of an honest researcher.
The flipside is a very linear publishing history. Some scientists have a pet theory they’ve managed to support with more and more evidence over time, like a bricklayer gradually, flawlessly building the prettiest house in the neighborhood. Too pretty.
There’s a dark side to this charming simplicity: scientists sometimes try and succeed at engineering the very findings they’re hoping to get, said Charles Brenner, a biochemist at City of Hope National Medical Center.
These scientists “try to prove their model and ignore data that doesn’t fit their model because everybody likes a clean story,” Kaeberlein said. “People want to become famous,” said Samuel Klein, a biologist at Washington University. “So there’s always that bias to try to get positive results.”
Don’t Overvalue Credentials. Just because a scientist works at a top university doesn’t mean they’re completely trustworthy. “The institution means almost nothing,” Kaeberlein said.
Same goes for publishing in top journals, Kaeberlein added. “There’s an incentive structure that favors poor quality science and irreproducible results in high profile journals.”
Traditional proxies for credibility aren’t quite as reliable these days. Shortcuts don’t cut it anymore; you’ve got to scrutinize the actual research the scientist is producing. “You have to look at the literature and try to interpret it for yourself,” said Rafael de Cabo, a scientist at the National Institute on Aging, run by the U.S. National Institutes of Health. Or find journalists you trust to distill this information for you, Klein suggested.
Consider Company Ties. Companies can help scientists bring their research to the public more directly and efficiently than the slower grind of academia, where “the opportunities and challenges weren’t big enough for me,” said Kaeberlein, who left the University of Washington earlier this year.
"It’s generally not universities that can take technology through what we call the valley of death,” Brenner said. “There are rewards associated with taking risks.”
Many scientists are upfront about their financial conflicts of interest – sometimes out of necessity. “At a place like Duke, our conflicts of interest are very closely managed, said Matthew Hirschey, who researchers metabolism at Duke’s Molecular Physiology Institute. “We have to be incredibly explicit about our partnerships.”
But the willingness to disclose conflicts doesn’t necessarily mean the scientist is any less biased. Those conflicts can still affect their views and outcomes of their research, said Johnston, the Hastings bioethicist.
“The proof is in the pudding, and it’s got to be done by people who are not vested in making money off the results,” Klein said. Worth noting: even if scientists eschew companies, they’re almost always financially motivated to get grants for their research.
Bottom line: lots of scientists work for and with companies, and many are highly trustworthy leaders in their fields. But if a scientist is in thick with companies and checks some of the other boxes on this list, their views and research may be compromised.
A company has slashed the cost of assessing a person's genome to just $100. With lower costs - and as other genetic tools mature and evolve - a wave of new therapies could be coming in the near future.
In May 2022, Californian biotech Ultima Genomics announced that its UG 100 platform was capable of sequencing an entire human genome for just $100, a landmark moment in the history of the field. The announcement was particularly remarkable because few had previously heard of the company, a relative unknown in an industry long dominated by global giant Illumina which controls about 80 percent of the world’s sequencing market.
Ultima’s secret was to completely revamp many technical aspects of the way Illumina have traditionally deciphered DNA. The process usually involves first splitting the double helix DNA structure into single strands, then breaking these strands into short fragments which are laid out on a glass surface called a flow cell. When this flow cell is loaded into the sequencing machine, color-coded tags are attached to each individual base letter. A laser scans the bases individually while a camera simultaneously records the color associated with them, a process which is repeated until every single fragment has been sequenced.
Instead, Ultima has found a series of shortcuts to slash the cost and boost efficiency. “Ultima Genomics has developed a fundamentally new sequencing architecture designed to scale beyond conventional approaches,” says Josh Lauer, Ultima’s chief commercial officer.
This ‘new architecture’ is a series of subtle but highly impactful tweaks to the sequencing process ranging from replacing the costly flow cell with a silicon wafer which is both cheaper and allows more DNA to be read at once, to utilizing machine learning to convert optical data into usable information.
To put $100 genome in perspective, back in 2012 the cost of sequencing a single genome was around $10,000, a price tag which dropped to $1,000 a few years later. Before Ultima’s announcement, the cost of sequencing an individual genome was around $600.
Several studies have found that nearly 12 percent of healthy people who have their genome sequenced, then discover they have a variant pointing to a heightened risk of developing a disease that can be monitored, treated or prevented.
While Ultima’s new machine is not widely available yet, Illumina’s response has been rapid. In September 2022, the company unveiled the NovaSeq X series, which it describes as its fastest most cost-efficient sequencing platform yet, capable of sequencing genomes at $200, with further price cuts likely to follow.
But what will the rapidly tumbling cost of sequencing actually mean for medicine? “Well to start with, obviously it’s going to mean more people getting their genome sequenced,” says Michael Snyder, professor of genetics at Stanford University. “It'll be a lot more accessible to people.”
At the moment sequencing is mainly limited to certain cancer patients where it is used to inform treatment options, and individuals with undiagnosed illnesses. In the past, initiatives such as SeqFirst have attempted further widen access to genome sequencing based on growing amounts of research illustrating the potential benefits of the technology in healthcare. Several studies have found that nearly 12 percent of healthy people who have their genome sequenced, then discover they have a variant pointing to a heightened risk of developing a disease that can be monitored, treated or prevented.
“While whole genome sequencing is not yet widely used in the U.S., it has started to come into pediatric critical care settings such as newborn intensive care units,” says Professor Michael Bamshad, who heads the genetic medicine division in the University of Washington’s pediatrics department. “It is also being used more often in outpatient clinical genetics services, particularly when conventional testing fails to identify explanatory variants.”
But the cost of sequencing itself is only one part of the price tag. The subsequent clinical interpretation and genetic counselling services often come to several thousand dollars, a cost which insurers are not always willing to pay.
As a result, while Bamshad and others hope that the arrival of the $100 genome will create new opportunities to use genetic testing in innovative ways, the most immediate benefits are likely to come in the realm of research.
Bigger Data
There are numerous ways in which cheaper sequencing is likely to advance scientific research, for example the ability to collect data on much larger patient groups. This will be a major boon to scientists working on complex heterogeneous diseases such as schizophrenia or depression where there are many genes involved which all exert subtle effects, as well as substantial variance across the patient population. Bigger studies could help scientists identify subgroups of patients where the disease appears to be driven by similar gene variants, who can then be more precisely targeted with specific drugs.
If insurers can figure out the economics, Snyder even foresees a future where at a certain age, all of us can qualify for annual sequencing of our blood cells to search for early signs of cancer or the potential onset of other diseases like type 2 diabetes.
David Curtis, a genetics professor at University College London, says that scientists studying these illnesses have previously been forced to rely on genome-wide association studies which are limited because they only identify common gene variants. “We might see a significant increase in the number of large association studies using sequence data,” he says. “It would be far preferable to use this because it provides information about rare, potentially functional variants.”
Cheaper sequencing will also aid researchers working on diseases which have traditionally been underfunded. Bamshad cites cystic fibrosis, a condition which affects around 40,000 children and adults in the U.S., as one particularly pertinent example.
“Funds for gene discovery for rare diseases are very limited,” he says. “We’re one of three sites that did whole genome sequencing on 5,500 people with cystic fibrosis, but our statistical power is limited. A $100 genome would make it much more feasible to sequence everyone in the U.S. with cystic fibrosis and make it more likely that we discover novel risk factors and pathways influencing clinical outcomes.”
For progressive diseases that are more common like cancer and type 2 diabetes, as well as neurodegenerative conditions like multiple sclerosis and ALS, geneticists will be able to go even further and afford to sequence individual tumor cells or neurons at different time points. This will enable them to analyze how individual DNA modifications like methylation, change as the disease develops.
In the case of cancer, this could help scientists understand how tumors evolve to evade treatments. Within in a clinical setting, the ability to sequence not just one, but many different cells across a patient’s tumor could point to the combination of treatments which offer the best chance of eradicating the entire cancer.
“What happens at the moment with a solid tumor is you treat with one drug, and maybe 80 percent of that tumor is susceptible to that drug,” says Neil Ward, vice president and general manager in the EMEA region for genomics company PacBio. “But the other 20 percent of the tumor has already got mutations that make it resistant, which is probably why a lot of modern therapies extend life for sadly only a matter of months rather than curing, because they treat a big percentage of the tumor, but not the whole thing. So going forwards, I think that we will see genomics play a huge role in cancer treatments, through using multiple modalities to treat someone's cancer.”
If insurers can figure out the economics, Snyder even foresees a future where at a certain age, all of us can qualify for annual sequencing of our blood cells to search for early signs of cancer or the potential onset of other diseases like type 2 diabetes.
“There are companies already working on looking for cancer signatures in methylated DNA,” he says. “If it was determined that you had early stage cancer, pre-symptomatically, that could then be validated with targeted MRI, followed by surgery or chemotherapy. It makes a big difference catching cancer early. If there were signs of type 2 diabetes, you could start taking steps to mitigate your glucose rise, and possibly prevent it or at least delay the onset.”
This would already revolutionize the way we seek to prevent a whole range of illnesses, but others feel that the $100 genome could also usher in even more powerful and controversial preventative medicine schemes.
Newborn screening
In the eyes of Kári Stefánsson, the Icelandic neurologist who been a visionary for so many advances in the field of human genetics over the last 25 years, the falling cost of sequencing means it will be feasible to sequence the genomes of every baby born.
“We have recently done an analysis of genomes in Iceland and the UK Biobank, and in 4 percent of people you find mutations that lead to serious disease, that can be prevented or dealt with,” says Stefansson, CEO of deCODE genetics, a subsidiary of the pharmaceutical company Amgen. “This could transform our healthcare systems.”
As well as identifying newborns with rare diseases, this kind of genomic information could be used to compute a person’s risk score for developing chronic illnesses later in life. If for example, they have a higher than average risk of colon or breast cancer, they could be pre-emptively scheduled for annual colonoscopies or mammograms as soon as they hit adulthood.
To a limited extent, this is already happening. In the UK, Genomics England has launched the Newborn Genomes Programme, which plans to undertake whole-genome sequencing of up to 200,000 newborn babies, with the aim of enabling the early identification of rare genetic diseases.
"I have not had my own genome sequenced and I would not have wanted my parents to have agreed to this," Curtis says. "I don’t see that sequencing children for the sake of some vague, ill-defined benefits could ever be justifiable.”
However, some scientists feel that it is tricky to justify sequencing the genomes of apparently healthy babies, given the data privacy issues involved. They point out that we still know too little about the links which can be drawn between genetic information at birth, and risk of chronic illness later in life.
“I think there are very difficult ethical issues involved in sequencing children if there are no clear and immediate clinical benefits,” says Curtis. “They cannot consent to this process. I have not had my own genome sequenced and I would not have wanted my parents to have agreed to this. I don’t see that sequencing children for the sake of some vague, ill-defined benefits could ever be justifiable.”
Curtis points out that there are many inherent risks about this data being available. It may fall into the hands of insurance companies, and it could even be used by governments for surveillance purposes.
“Genetic sequence data is very useful indeed for forensic purposes. Its full potential has yet to be realized but identifying rare variants could provide a quick and easy way to find relatives of a perpetrator,” he says. “If large numbers of people had been sequenced in a healthcare system then it could be difficult for a future government to resist the temptation to use this as a resource to investigate serious crimes.”
While sequencing becoming more widely available will present difficult ethical and moral challenges, it will offer many benefits for society as a whole. Cheaper sequencing will help boost the diversity of genomic datasets which have traditionally been skewed towards individuals of white, European descent, meaning that much of the actionable medical information which has come out of these studies is not relevant to people of other ethnicities.
Ward predicts that in the coming years, the growing amount of genetic information will ultimately change the outcomes for many with rare, previously incurable illnesses.
“If you're the parent of a child that has a susceptible or a suspected rare genetic disease, their genome will get sequenced, and while sadly that doesn’t always lead to treatments, it’s building up a knowledge base so companies can spring up and target that niche of a disease,” he says. “As a result there’s a whole tidal wave of new therapies that are going to come to market over the next five years, as the genetic tools we have, mature and evolve.”
This article was first published by Leaps.org in October 2022.