Sloppy Science Happens More Than You Think
The media loves to tout scientific breakthroughs, and few are as toutable – and in turn, have been as touted – as CRISPR. This method of targeted DNA excision was discovered in bacteria, which use it as an adaptive immune system to combat reinfection with a previously encountered virus.
Shouldn't the editors at a Nature journal know better than to have published an incorrect paper in the first place?
It is cool on so many levels: not only is the basic function fascinating, reminding us that we still have more to discover about even simple organisms that we thought we knew so well, but the ability it grants us to remove and replace any DNA of interest has almost limitless applications in both the lab and the clinic. As if that didn't make it sexy enough, add in a bicoastal, male-female, very public and relatively ugly patent battle, and the CRISPR story is irresistible.
And then last summer, a bombshell dropped. The prestigious journal Nature Methods published a paper in which the authors claimed that CRISPR could cause many unintended mutations, rendering it unfit for clinical use. Havoc duly ensued; stocks in CRISPR-based companies plummeted. Thankfully, the authors of the offending paper were responsible, good scientists; they reassessed, then recanted. Their attention- and headline- grabbing results were wrong, and they admitted as much, leading Nature Methods to formally retract the paper this spring.
How did this happen? Shouldn't the editors at a Nature journal know better than to have published this in the first place?
Alas, high-profile scientific journals publish misleading and downright false results fairly regularly. Some errors are unavoidable – that's how the scientific method works. Hypotheses and conclusions will invariably be overturned as new data becomes available and new technologies are developed that allow for deeper and deeper studies. That's supposed to happen. But that's not what we're talking about here. Nor are we talking about obvious offenses like outright plagiarism. We're talking about mistakes that are avoidable, and that still have serious ramifications.
The cultures of both industry and academia promote research that is poorly designed and even more poorly analyzed.
Two parties are responsible for a scientific publication, and thus two parties bear the blame when things go awry: the scientists who perform and submit the work, and the journals who publish it. Unfortunately, both are incentivized for speedy and flashy publications, and not necessarily for correct publications. It is hardly a surprise, then, that we end up with papers that are speedy and flashy – and not necessarily correct.
"Scientists don't lie and submit falsified data," said Andy Koff, a professor of Molecular Biology at Sloan Kettering Institute, the basic research arm of Memorial Sloan Kettering Cancer Center. Richard Harris, who wrote the book on scientific misconduct running the gamut from unconscious bias and ignorance to more malicious fraudulence, largely concurs (full disclosure: I reviewed the book here). "Scientists want to do good science and want to be recognized as such," he said. But even so, the cultures of both industry and academia promote research that is poorly designed and even more poorly analyzed. In Rigor Mortis: How Sloppy Science Creates Worthless Cures, Crushes Hope, and Wastes Millions, Harris describes how scientists must constantly publish in order to maintain their reputations and positions, to get grants and tenure and students. "They are disincentivized from doing that last extra experiment to prove their results," he said; it could prove too risky if it could cost them a publication.
Ivan Oransky and Adam Marcus founded Retraction Watch, a blog that tracks the retraction of scientific papers, in 2010. Oransky pointed out that blinded peer review – the pride and joy of the scientific publishing enterprise – is a large part of the problem. "Pre-publication peer review is still important, but we can't treat it like the only check on the system. Papers are being reviewed by non-experts, and reviewers are asked to review papers only tangentially related to their field. Moreover, most peer reviewers don't look at the underlying or raw data, even when it is available. How then can they tell if the analysis is flawed or the data is accurate?" he wondered.
Mistaken publications also erode the public's opinion of legitimate science, which is problematic since that opinion isn't especially high to begin with.
Koff agreed that anonymous peer review is valuable, but severely flawed. "Blinded review forces a collective view of importance," he said. "If an article disagrees with the reviewer's worldview, the article gets rejected or forced to adhere to that worldview – even if that means pushing the data someplace it shouldn't necessarily go." We have lost the scientific principle behind review, he thinks, which was to critically analyze a paper. But instead of challenging fundamental assumptions within a paper, reviewers now tend to just ask for more and more supplementary data. And don't get him started on editors. "Editors are supposed to arbitrate between reviewers and writers and they have completely abdicated this responsibility, at every journal. They do not judge, and that's a real failing."
Harris laments the wasted time, effort, and resources that result when erroneous ideas take hold in a field, not to mention lives lost when drug discovery is predicated on basic science findings that end up being wrong. "When no one takes the time, care, and money to reproduce things, science isn't stopping – but it is slowing down," he noted. Mistaken publications also erode the public's opinion of legitimate science, which is problematic since that opinion isn't especially high to begin with.
Scientists and publishers don't only cause the problem, though – they may also provide the solution. Both camps are increasingly recognizing and dealing with the crisis. The self-proclaimed "data thugs" Nick Brown and James Heathers use pretty basic arithmetic to reveal statistical errors in papers. The microbiologist Elisabeth Bik scans the scientific literature for problematic images "in her free time." The psychologist Brian Nosek founded the Center for Open Science, a non-profit organization dedicated to promoting openness, integrity, and reproducibility in scientific research. The Nature family of journals – yes, the one responsible for the latest CRISPR fiasco – has its authors complete a checklist to combat irreproducibility, à la Atul Gawande. And Nature Communications, among other journals, uses transparent peer review, in which authors can opt to have the reviews of their manuscript published anonymously alongside the completed paper. This practice "shows people how the paper evolved," said Koff "and keeps the reviewer and editor accountable. Did the reviewer identify the major problems with the paper? Because there are always major problems with a paper."
Time to visit your TikTok doc? The good and bad of doctors on social media
Rakhi Patel has carved a hobby out of reviewing pizza — her favorite food — on Instagram. In a nod to her preferred topping, she calls herself thepepperoniqueen. Photos and videos show her savoring slices from scores of pizzerias. In some of them, she’s wearing scrubs — her attire as an inpatient neurology physician associate at Tufts Medical Center in Boston.
“Depending on how you dress your pizza, it can be more nutritious,” said Patel, who suggests a thin crust, sugarless tomato sauce and vegetables galore as healthier alternatives. “There are no boundaries for a health care professional to enjoy pizza.”
Beyond that, “pizza fuels my mental health and makes me happy, especially when loaded with pepperoni,” she said. “If I’m going to be a pizza connoisseur, then I also need to take care of my physical health by ensuring that I get at least three days of exercise per week and eat nutritiously when I’m not eating pizza.”
She’s among an increasing number of health care professionals, including doctors and nurses, who maintain an active persona on social media, according to bioethics researchers. They share their hobbies and interests with people inside and outside the world of medicine, helping patients and the public become acquainted with the humans behind the scrubs or white coats. Other health care experts limit their posts to medical topics, while some opt for a combination of personal and professional commentaries. Depending on the posts, ethical issues may come into play.
“Health care professionals are quite prevalent on social media,” said Mercer Gary, a postdoctoral researcher at The Hastings Center, an independent bioethics research institute in Garrison, New York. “They’ve been posting on #medTwitter for many years, mainly to communicate with one another, but, of course, anyone can see the threads. Most recently, doctors and nurses have become a presence on TikTok.”
On social media, many health care providers perceive themselves to be “humanizing” their profession by coming across as more approachable — “reminding patients that providers are people and workers, as well as repositories of medical expertise,” Gary said. As a result, she noted that patients who are often intimidated by clinicians may feel comfortable enough to overcome barriers to scheduling health care appointments. The use of TikTok in particular may help doctors and nurses connect with younger followers.
When health care providers post on social media, they must bear in mind that they have legal and ethical duties to their patients, profession and society, said Elizabeth Levy, founder and director of Physicians for Justice.
While enduring three years of pandemic conditions, many health care professionals have struggled with burnout, exhaustion and moral distress. “Much health care provider content on social media seeks to expose the difficulties of the work,” Gary added. “TikTok and Instagram reels have shown health care providers crying after losing a patient or exhausted after a night shift in the emergency department.”
A study conducted in Beijing, China and published last year found that TikTok is the world’s most rapidly growing video application, amassing 1.6 billion users in 2021. “More and more patients are searching for information on genitourinary cancers via TikTok,” the study’s authors wrote in Frontiers in Oncology, referring to cancers of the urinary tracts and male reproductive organs. Among the 61 sample videos examined by the researchers, health care practitioners contributed the content in 29, or 47 percent, of them. Yet, 22 posts, 36 percent, were misinformative, mostly due to outdated information.
More than half of the videos offered good content on disease symptoms and examinations. The authors concluded that “most videos on genitourinary cancers on TikTok are of poor to medium quality and reliability. However, videos posted by media agencies enjoyed great public attention and interaction. Medical practitioners could improve the video quality by cooperating with media agencies and avoiding unexplained terminologies.”
When health care providers post on social media, they must bear in mind that they have legal and ethical duties to their patients, profession and society, said Elizabeth Levy, founder and director of Physicians for Justice in Irvine, Calif., a nonprofit network of volunteer physicians partnering with public interest lawyers to address the social determinants of health.
“Providers are also responsible for understanding the mechanics of their posts,” such as who can see these messages and how long they stay up, Levy said. As a starting point for figuring what’s acceptable, providers could look at social media guidelines put out by their professional associations. Even beyond that, though, they must exercise prudent judgment. “As social media continues to evolve, providers will also need to stay updated with the changing risks and benefits of participation.”
Patients often research their providers online, so finding them on social media can help inform about values and approaches to care, said M. Sara Rosenthal, a professor and founding director of the program for bioethics and chair of the hospital ethics committee at the University of Kentucky College of Medicine.
Health care providers’ posts on social media also could promote patient education. They can advance informed consent and help patients navigate the risks and benefits of various treatments or preventive options. However, providers could violate ethical principles if they espouse “harmful, risky or questionable therapies or medical advice that is contrary to clinical practice guidelines or accepted standards of care,” Rosenthal said.
Inappropriate self-disclosure also can affect a provider’s reputation, said Kelly Michelson, a professor of pediatrics and director of the Center for Bioethics and Medical Humanities at Northwestern University’s Feinberg School of Medicine. A clinician’s obligations to professionalism extend beyond those moments when they are directly taking care of their patients, she said. “Many experts recommend against clinicians ‘friending’ patients or the families on social media because it blurs the patient-clinician boundary.”
Meanwhile, clinicians need to adhere closely to confidentiality. In sharing a patient’s case online for educational purposes, safeguarding identity becomes paramount. Removing names and changing minor details is insufficient, Michelson said.
“The patient-clinician relationship is sacred, and it can only be effective if patients have 100 percent confidence that all that happens with their clinician is kept in the strictest of confidence,” she said, adding that health care providers also should avoid obtaining information about their patients from social media because it can lead to bias and risk jeopardizing objectivity.
Academic clinicians can use social media as a recruitment tool to expand the pool of research participants for their studies, Michelson said. Because the majority of clinical research is conducted at academic medical centers, large segments of the population are excluded. “This affects the quality of the data and knowledge we gain from research,” she said.
Don S. Dizon, a professor of medicine and surgery at the Warren Alpert Medical School of Brown University in Providence, Rhode Island, uses LinkedIn and Doximity, as well as Twitter, Instagram, TikTok, Facebook, and most recently, YouTube and Post. He’s on Twitter nearly every day, where he interacts with the oncology community and his medical colleagues.
Also, he said, “I really like Instagram. It’s where you will see a hybrid of who I am professionally and personally. I’ve become comfortable sharing both up to a limit, but where else can I combine my appreciation of clothes with my professional life?” On that site, he’s seen sporting shirts with polka dots or stripes and an occasional bow-tie. He also posts photos of his cats.
Don S. Dizon, a professor of medicine and surgery at Brown, started using TikTok several years ago, telling medical stories in short-form videos.
Don S. Dizon
Dizon started using TikTok several years ago, telling medical stories in short-form videos. He may talk about an inspirational patient, his views on end-of-life care and death, or memories of people who have passed. But he is careful not to divulge any details that would identify anyone.
Recently, some people have become his patients after viewing his content on social media or on the Internet in general, which he clearly states isn’t a forum for medical advice. “In both situations, they are so much more relaxed when we meet, because it’s as if they have a sense of who I am as a person,” Dizon said. “I think that has helped so much in talking through a cancer diagnosis and a treatment plan, and yes, even discussions about prognosis.”
He also posts about equity and diversity. “I have found myself more likely to repost or react to issues that are inherently political, including racism, homophobia, transphobia and lack-of-access issues, because medicine is not isolated from society, and I truly believe that medicine is a social justice issue,” said Dizon, who is vice chair of diversity, equity, inclusion and professional integrity at the SWOG Cancer Research Network.
Through it all, Dizon likes “to break through the notion of doctor as infallible and all-knowing, the doctor as deity,” he said. “Humanizing what I do, especially in oncology, is something that challenges me on social media, and I appreciate the opportunities to do it on TikTok.”
Could this habit related to eating slow down rates of aging?
Last Thursday, scientists at Columbia University published a new study finding that cutting down on calories could lead to longer, healthier lives. In the phase 2 trial, 220 healthy people without obesity dropped their calories significantly and, at least according to one test, their rate of biological aging slowed by 2 to 3 percent in over a couple of years. Small though that may seem, the researchers estimate that it would translate into a decline of about 10 percent in the risk of death as people get older. That's basically the same as quitting smoking.
Previous research has shown that restricting calories results in longer lives for mice, worms and flies. This research is unique because it applies those findings to people. It was published in Nature Aging.
But what did the researchers actually show? Why did two other tests indicate that the biological age of the research participants didn't budge? Does the new paper point to anything people should be doing for more years of healthy living? Spoiler alert: Maybe, but don't try anything before talking with a medical expert about it. I had the chance to chat with someone with inside knowledge of the research -- Dr. Evan Hadley, director of the National Institute of Aging's Division of Geriatrics and Clinical Gerontology, which funded the study. Dr. Hadley describes how the research participants went about reducing their calories, as well as the risks and benefits involved. He also explains the "aging clock" used to measure the benefits.
Evan Hadley, Director of the Division of Geriatrics and Clinical Gerontology at the National Institute of Aging
NIA