Who Qualifies as an “Expert” And How Can We Decide Who Is Trustworthy?
Discerning a real expert from a charlatan is crucial during the COVID-19 pandemic and beyond.
This article is part of the magazine, "The Future of Science In America: The Election Issue," co-published by LeapsMag, the Aspen Institute Science & Society Program, and GOOD.
Expertise is a slippery concept. Who has it, who claims it, and who attributes or yields it to whom is a culturally specific, sociological process. During the COVID-19 pandemic, we have witnessed a remarkable emergence of legitimate and not-so-legitimate scientists publicly claiming or being attributed to have academic expertise in precisely my field: infectious disease epidemiology. From any vantage point, it is clear that charlatans abound out there, garnering TV coverage and hundreds of thousands of Twitter followers based on loud opinions despite flimsy credentials. What is more interesting as an insider is the gradient of expertise beyond these obvious fakers.
A person's expertise is not a fixed attribute; it is a hierarchical trait defined relative to others. Despite my protestations, I am the go-to expert on every aspect of the pandemic to my family. To a reporter, I might do my best to answer a question about the immune response to SARS-CoV-2, noting that I'm not an immunologist. Among other academic scientists, my expertise is more well-defined as a subfield of epidemiology, and within that as a particular area within infectious disease epidemiology. There's a fractal quality to it; as you zoom in on a particular subject, a differentiation of expertise emerges among scientists who, from farther out, appear to be interchangeable.
We all have our scientific domain and are less knowledgeable outside it, of course, and we are often asked to comment on a broad range of topics. But many scientists without a track record in the field have become favorites among university administrators, senior faculty in unrelated fields, policymakers, and science journalists, using institutional prestige or social connections to promote themselves. This phenomenon leads to a distorted representation of science—and of academic scientists—in the public realm.
Trustworthy experts will direct you to others in their field who know more about particular topics, and will tend to be honest about what is and what isn't "in their lane."
Predictably, white male voices have been disproportionately amplified, and men are certainly over-represented in the category of those who use their connections to inappropriately claim expertise. Generally speaking, we are missing women, racial minorities, and global perspectives. This is not only important because it misrepresents who scientists are and reinforces outdated stereotypes that place white men in the Global North at the top of a credibility hierarchy. It also matters because it can promote bad science, and it passes over scientists who can lend nuance to the scientific discourse and give global perspectives on this quintessentially global crisis.
Also at work, in my opinion, are two biases within academia: the conflation of institutional prestige with individual expertise, and the bizarre hierarchy among scientists that attributes greater credibility to those in quantitative fields like physics. Regardless of mathematical expertise or institutional affiliation, lack of experience working with epidemiological data can lead to over-confidence in the deceptively simple mathematical models that we use to understand epidemics, as well as the inappropriate use of uncertain data to inform them. Prominent and vocal scientists from different quantitative fields have misapplied the methods of infectious disease epidemiology during the COVID-19 pandemic so far, creating enormous confusion among policymakers and the public. Early forecasts that predicted the epidemic would be over by now, for example, led to a sense that epidemiological models were all unreliable.
Meanwhile, legitimate scientific uncertainties and differences of opinion, as well as fundamentally different epidemic dynamics arising in diverse global contexts and in different demographic groups, appear in the press as an indistinguishable part of this general chaos. This leads many people to question whether the field has anything worthwhile to contribute, and muddies the facts about COVID-19 policies for reducing transmission that most experts agree on, like wearing masks and avoiding large indoor gatherings.
So how do we distinguish an expert from a charlatan? I believe a willingness to say "I don't know" and to openly describe uncertainties, nuances, and limitations of science are all good signs. Thoughtful engagement with questions and new ideas is also an indication of expertise, as opposed to arrogant bluster or a bullish insistence on a particular policy strategy regardless of context (which is almost always an attempt to hide a lack of depth of understanding). Trustworthy experts will direct you to others in their field who know more about particular topics, and will tend to be honest about what is and what isn't "in their lane." For example, some expertise is quite specific to a given subfield: epidemiologists who study non-infectious conditions or nutrition, for example, use different methods from those of infectious disease experts, because they generally don't need to account for the exponential growth that is inherent to a contagion process.
Academic scientists have a specific, technical contribution to make in containing the COVID-19 pandemic and in communicating research findings as they emerge. But the liminal space between scientists and the public is subject to the same undercurrents of sexism, racism, and opportunism that society and the academy have always suffered from. Although none of the proxies for expertise described above are fool-proof, they are at least indicative of integrity and humility—two traits the world is in dire need of at this moment in history.
[Editor's Note: To read other articles in this special magazine issue, visit the beautifully designed e-reader version.]
Scientists Used Fruit Flies to Quickly Develop a Personalized Cancer Treatment for a Dying Man
Genetically engineered fruit flies are being used in early research to screen novel combinations of cancer drugs for individual patients.
Imagine a man with colorectal cancer that has spread throughout his body. His tumor is not responding to traditional chemotherapy. He needs a radically effective treatment as soon as possible and there's no time to wait for a new drug or a new clinical trial.
A plethora of novel combinations of treatments can be screened quickly on as many as 400,000 flies at once.
This was the very real, and terrifying, situation of a recent patient at Mount Sinai Medical Center in New York City. So his doctors turned to a new tactic to speed up the search for a treatment that would save him: Fruit flies.
Yes, fruit flies. Those annoying little buggers that descend on opened food containers are actually leading scientists to fully personalized cancer treatments. Oncology advances often are more about about utilizing old drugs in new combinations than about adding new drugs. But classically, the development of each new chemotherapy drug combination has required studies involving numerous patients spread over many years or decades.
With the fruit fly method, however, a novel treatment -- in the sense that a particular combination of drugs and the timing of their administration has never been used before -- is developed for each patient, almost like on Star Trek, when, faced suddenly with an unknown disease, a futuristic physician researches it and develops a cure quickly enough to save the patient's life.
How It Works
Using genetic engineering techniques, researchers produce a population of fruit fly embryos, each of which is programmed to develop a replica of the patient's cancer.
Since a lot of genetically identical fly embryos can be created, and since they hatch from eggs within 30 hours and then mature within days, a plethora of novel combinations of treatments can be screened quickly on as many as 400,000 flies at once. Then, only the regimens that are effective are administered to the patient.
Biotech entrepreneur Laura Towart, CEO of the UK- and Ireland-based company, My Personal Therapeutics, is partnering with Mount Sinai to develop and test the fruit fly tactic. The researchers recently published a paper demonstrating that the tumor of the man with metastatic colorectal cancer had shrunk considerably following the treatment, and remained stable for 11 months, although he eventually succumbed to his illness.
Open Questions
Cancer is in fact many different diseases, even if it strikes two people in the same place, and both cancers look the same under a microscope. At the level of DNA, RNA, proteins, and other molecular factors, each cancer is unique – and may require a unique treatment approach.
Determining the true impact on cancer mortality will require clinical trials involving many more patients.
"Anatomy of a cancer still plays a major role, if you're a surgeon or radiation oncologist, but the medical approach to cancer therapy is moving toward treatments that are personalized based on other factors," notes Dr. Howard McLeod, an internationally recognized expert on cancer genetics at the Moffitt Cancer Center, in Tampa, Florida. "We are also headed into an era when even the methods for monitoring patients are individualized."
One big unresolved question about the fruit fly screening approach is how effective it will be in terms of actually extending life. Determining the true impact on cancer mortality will require clinical trials involving many more patients.
Next Up
Using machine learning and artificial intelligence, Towart is now working to build a service called TuMatch that will offer rapid and affordable personalized treatment recommendations for all genetically driven cancers. "We hope to have TuMatch available to patients with colorectal/GI cancers by January 2020," she says. "We are also offering [the fruit fly approach] for patients with rare genetic diseases and for patients who are diabetic."
Are Towart's fruit flies the answer to why the man's tumor shrunk? To be sure, the definitive answer will come from further research that is expected soon, but it's also clear that, prior to the treatment, there was nothing left to do for that particular patient. Thus, although it's early in the game, there's a pretty good rationale for optimism.
A Million Patients Have Innovated Their Own Medical Solutions, And Doctors Are Terrified
Diabetes patient advocate Renza Scibilia with her continuous glucose monitor, used in innovative DIY health technology.
In the fall of 2017, patient advocate Renza Scibilia told a conference of endocrinologists in Australia about new, patient-developed artificial pancreas technology that helped her manage her Type 1 diabetes.
"Because it's not a regulated product, some [doctors] were worried and said 'What if it goes wrong?'"
"They were in equal measure really interested and really scared," recalled Scibilia. "Because it's not a regulated product, some were worried and said 'What if it goes wrong? What is my liability going to be?'"
That was two years ago. Asked if physicians have been more receptive to the same "looping" technology now that its benefits have been supported by considerable data (as Leapsmag pointed out in May), Scibilia said, "No. Clinicians are still really insecure. They're always going to be reluctant to accept consumer-driven technology."
This exemplifies a major challenge to the growing Do-It-Yourself (DIY) biohealth movement: physicians are unnerved and worried about innovations developed by patients and other consumers that haven't been tested in elaborate clinical trials or sanctioned by regulatory authorities.
"It's difficult for patients who develop new health technology to demonstrate the advantage in a way that physicians would accept." said Howard DeMonaco, visiting scientist at MIT's Sloan School of Management. "New approaches to the treatment of diseases are by definition suspect to clinicians. Most are risk averse unless there is a substantial advantage to the new approach and the risks in doing so appear to be minimized."
Nevertheless, the DIY biohealth movement is booming. About a million people reported that they created medical innovations to address their own medical needs in surveys conducted from 2010-2015 in the U.S., U.K., Finland, Canada and South Korea.
Add in other DIY health innovations created in homes, community biolabs and "Maker" health fairs, and it's clear that health care providers are increasingly confronted with medical devices, information technology, and even medications that were developed in unconventional settings and lack the blessing of regulatory authorities.
Researchers in Portugal have tried to spread the word about many of these solutions on the Patent Innovations website, which has more than 500 examples, ranging from a 3-D printed arm and hand to a sensor device that warns someone when an osteomy bag is full.
When Reddit asked medical professionals, "What is the craziest DIY health treatment you've seen a patient attempt?" thousands shared horror stories.
But even in this era of patient empowerment, more widespread use of DIY health solutions still depends upon the approval and cooperation of physicians, nurses and other caregivers. And health care providers still lack awareness of promising patient-developed innovations, according to Dr. Joyce Lee, a pediatric endocrinologist at the University of Michigan who advocates involving patients in the design of healthcare technology. "Most physicians are scared of what they don't know," she said.
They're also understandably worried about patients who don't know what they're doing and make irresponsible decisions. When Reddit asked medical professionals, "What is the craziest DIY health treatment you've seen a patient attempt?" thousands shared horror stories, including a man who poked a hole in his belly button with a knitting needle to relieve gas.
Yet DeMonaco and Lee think it's possible to start bridging the gaps between responsible patient innovators and skeptical doctors as well as unprepared regulatory systems.
One obstacle to consumer-driven health innovations is that clinical trials to prove their safety and effectiveness are expensive and time-consuming, as De Monaco points out in a recent article. He and his colleagues suggested that low-cost clinical trials by and for patients could help address this challenge. They urged patients to publish their own research and detail the impact of innovations on their own health, and create databases that incorporate the findings of other patients.
For example, Adam Brown, who has Type 1 diabetes, compared the effects of low and high carbohydrate diets on his blood sugar management, and conveyed the results in an online journal. "Sharing the information allowed others to copy the experiment," the article noted, suggesting that this could be a model to create multi-patient trials that could be "analyzed by expert patients and/or by professionals."
Asked how to convince health care providers to consider such research, DeMonaco cited the example of doctors prescribing "off label" drugs for purposes that aren't approved by the FDA. "The secret to off label use, like any other user innovation, is dissemination," he said. Sharing case reports and other low-cost research serves to disseminate the information "in a way that is comfortable for physicians," he said, and urged patient innovators to take the same approach.
The FDA regulates commercial products and has no authority if consumers want to use medical devices, medications, or information systems that they find on their own.
Physicians should also be encouraged to engage in patient-driven research, said Dr. Lee. She suggests forming "maker spaces in which patients and physicians are involved in designing personalized technology for chronic diseases. In my vision, patient peers would build, iterate, and learn from each other and the doctor would be part of the team, constantly assessing and evaluating the technology and facilitating the process."
Some kind of regulatory oversight of DIY health technology is also necessary, said Todd Kuiken, senior research scholar at NC State and former principal investigator at the Woodrow Wilson Center's Synthetic Biology Project.
The FDA regulates commercial products and has no authority if consumers want to use medical devices, medications, or information systems that they find on their own. But that doesn't stop regulators from worrying about patients who use them. For example, the FDA issued a warning about diabetes looping technology earlier this year after one diabetic was hospitalized with hypoglycemia.
Kuiken, for one, believes that citizen-driven innovation requires oversight "to move forward." He suggested that Internal Review Boards, with experts on medical technology, safety and ethics, could play a helpful role in validating the work of patient innovators and others engaged in DIY health research. "As people are developing health products, there would be experts available to take a look and check in," he said.
Kuiken pointed out that in native American territories, tribally based IRBs working with the national Indian Health Services help to oversee new health science research. The model could be applied more broadly.
He also offered hope to those who want to integrate the current health regulatory structure into the ecosystem of DIY health innovations. "I didn't expect people from the FDA or NIH to show up" he said about a workshop on citizen-driven biomedical research that he helped organize at the Wilson Center last year. But senior officials from both agencies attended.
He indicated they "were open to new ideas." While he wouldn't disclose contributions made by individual participants in the workshop, he said the government staffers were "very interested in figuring out how to engage with citizen health innovators, to build bridges with the DIY community."
"Why should we wait for regulatory bodies? Why wait for trials that take too long?"
Time will tell whether those bridges will be built quickly enough to increase the comfort of physicians with health innovations developed by patients and other consumers. In the meantime, DIY health innovators like patient advocate Scibilia are undeterred.
"Why should we wait for regulatory bodies?" she asked. "Why wait for trials that take too long? There are plenty of data out there indicating the [diabetes looping] technology works. So we're just going to do it. We're not waiting."