Sloppy Science Happens More Than You Think
Manipulating DNA through gene editing.
The media loves to tout scientific breakthroughs, and few are as toutable – and in turn, have been as touted – as CRISPR. This method of targeted DNA excision was discovered in bacteria, which use it as an adaptive immune system to combat reinfection with a previously encountered virus.
Shouldn't the editors at a Nature journal know better than to have published an incorrect paper in the first place?
It is cool on so many levels: not only is the basic function fascinating, reminding us that we still have more to discover about even simple organisms that we thought we knew so well, but the ability it grants us to remove and replace any DNA of interest has almost limitless applications in both the lab and the clinic. As if that didn't make it sexy enough, add in a bicoastal, male-female, very public and relatively ugly patent battle, and the CRISPR story is irresistible.
And then last summer, a bombshell dropped. The prestigious journal Nature Methods published a paper in which the authors claimed that CRISPR could cause many unintended mutations, rendering it unfit for clinical use. Havoc duly ensued; stocks in CRISPR-based companies plummeted. Thankfully, the authors of the offending paper were responsible, good scientists; they reassessed, then recanted. Their attention- and headline- grabbing results were wrong, and they admitted as much, leading Nature Methods to formally retract the paper this spring.
How did this happen? Shouldn't the editors at a Nature journal know better than to have published this in the first place?
Alas, high-profile scientific journals publish misleading and downright false results fairly regularly. Some errors are unavoidable – that's how the scientific method works. Hypotheses and conclusions will invariably be overturned as new data becomes available and new technologies are developed that allow for deeper and deeper studies. That's supposed to happen. But that's not what we're talking about here. Nor are we talking about obvious offenses like outright plagiarism. We're talking about mistakes that are avoidable, and that still have serious ramifications.
The cultures of both industry and academia promote research that is poorly designed and even more poorly analyzed.
Two parties are responsible for a scientific publication, and thus two parties bear the blame when things go awry: the scientists who perform and submit the work, and the journals who publish it. Unfortunately, both are incentivized for speedy and flashy publications, and not necessarily for correct publications. It is hardly a surprise, then, that we end up with papers that are speedy and flashy – and not necessarily correct.
"Scientists don't lie and submit falsified data," said Andy Koff, a professor of Molecular Biology at Sloan Kettering Institute, the basic research arm of Memorial Sloan Kettering Cancer Center. Richard Harris, who wrote the book on scientific misconduct running the gamut from unconscious bias and ignorance to more malicious fraudulence, largely concurs (full disclosure: I reviewed the book here). "Scientists want to do good science and want to be recognized as such," he said. But even so, the cultures of both industry and academia promote research that is poorly designed and even more poorly analyzed. In Rigor Mortis: How Sloppy Science Creates Worthless Cures, Crushes Hope, and Wastes Millions, Harris describes how scientists must constantly publish in order to maintain their reputations and positions, to get grants and tenure and students. "They are disincentivized from doing that last extra experiment to prove their results," he said; it could prove too risky if it could cost them a publication.
Ivan Oransky and Adam Marcus founded Retraction Watch, a blog that tracks the retraction of scientific papers, in 2010. Oransky pointed out that blinded peer review – the pride and joy of the scientific publishing enterprise – is a large part of the problem. "Pre-publication peer review is still important, but we can't treat it like the only check on the system. Papers are being reviewed by non-experts, and reviewers are asked to review papers only tangentially related to their field. Moreover, most peer reviewers don't look at the underlying or raw data, even when it is available. How then can they tell if the analysis is flawed or the data is accurate?" he wondered.
Mistaken publications also erode the public's opinion of legitimate science, which is problematic since that opinion isn't especially high to begin with.
Koff agreed that anonymous peer review is valuable, but severely flawed. "Blinded review forces a collective view of importance," he said. "If an article disagrees with the reviewer's worldview, the article gets rejected or forced to adhere to that worldview – even if that means pushing the data someplace it shouldn't necessarily go." We have lost the scientific principle behind review, he thinks, which was to critically analyze a paper. But instead of challenging fundamental assumptions within a paper, reviewers now tend to just ask for more and more supplementary data. And don't get him started on editors. "Editors are supposed to arbitrate between reviewers and writers and they have completely abdicated this responsibility, at every journal. They do not judge, and that's a real failing."
Harris laments the wasted time, effort, and resources that result when erroneous ideas take hold in a field, not to mention lives lost when drug discovery is predicated on basic science findings that end up being wrong. "When no one takes the time, care, and money to reproduce things, science isn't stopping – but it is slowing down," he noted. Mistaken publications also erode the public's opinion of legitimate science, which is problematic since that opinion isn't especially high to begin with.
Scientists and publishers don't only cause the problem, though – they may also provide the solution. Both camps are increasingly recognizing and dealing with the crisis. The self-proclaimed "data thugs" Nick Brown and James Heathers use pretty basic arithmetic to reveal statistical errors in papers. The microbiologist Elisabeth Bik scans the scientific literature for problematic images "in her free time." The psychologist Brian Nosek founded the Center for Open Science, a non-profit organization dedicated to promoting openness, integrity, and reproducibility in scientific research. The Nature family of journals – yes, the one responsible for the latest CRISPR fiasco – has its authors complete a checklist to combat irreproducibility, à la Atul Gawande. And Nature Communications, among other journals, uses transparent peer review, in which authors can opt to have the reviews of their manuscript published anonymously alongside the completed paper. This practice "shows people how the paper evolved," said Koff "and keeps the reviewer and editor accountable. Did the reviewer identify the major problems with the paper? Because there are always major problems with a paper."
In this week's Friday Five, an old diabetes drug finds an exciting new purpose. Plus, how to make the cities of the future less toxic, making old mice younger with EVs, a new reason for mysterious stillbirths - and much more.
The Friday Five covers five stories in research that you may have missed this week. There are plenty of controversies and troubling ethical issues in science – and we get into many of them in our online magazine – but this news roundup focuses on scientific creativity and progress to give you a therapeutic dose of inspiration headed into the weekend.
Here is the promising research covered in this week's Friday Five:
Listen on Apple | Listen on Spotify | Listen on Stitcher | Listen on Amazon | Listen on Google
- How to make cities of the future less noisy
- An old diabetes drug could have a new purpose: treating an irregular heartbeat
- A new reason for mysterious stillbirths
- Making old mice younger with EVs
- No pain - or mucus - no gain
And an honorable mention this week: How treatments for depression can change the structure of the brain
Researchers at Stanford have found that the virus that causes Covid-19 can infect fat cells, which could help explain why obesity is linked to worse outcomes for those who catch Covid-19.
Obesity is a risk factor for worse outcomes for a variety of medical conditions ranging from cancer to Covid-19. Most experts attribute it simply to underlying low-grade inflammation and added weight that make breathing more difficult.
Now researchers have found a more direct reason: SARS-CoV-2, the virus that causes Covid-19, can infect adipocytes, more commonly known as fat cells, and macrophages, immune cells that are part of the broader matrix of cells that support fat tissue. Stanford University researchers Catherine Blish and Tracey McLaughlin are senior authors of the study.
Most of us think of fat as the spare tire that can accumulate around the middle as we age, but fat also is present closer to most internal organs. McLaughlin's research has focused on epicardial fat, “which sits right on top of the heart with no physical barrier at all,” she says. So if that fat got infected and inflamed, it might directly affect the heart.” That could help explain cardiovascular problems associated with Covid-19 infections.
Looking at tissue taken from autopsy, there was evidence of SARS-CoV-2 virus inside the fat cells as well as surrounding inflammation. In fat cells and immune cells harvested from health humans, infection in the laboratory drove "an inflammatory response, particularly in the macrophages…They secreted proteins that are typically seen in a cytokine storm” where the immune response runs amok with potential life-threatening consequences. This suggests to McLaughlin “that there could be a regional and even a systemic inflammatory response following infection in fat.”
It is easy to see how the airborne SARS-CoV-2 virus infects the nose and lungs, but how does it get into fat tissue? That is a mystery and the source of ample speculation.
The macrophages studied by McLaughlin and Blish were spewing out inflammatory proteins, While the the virus within them was replicating, the new viral particles were not able to replicate within those cells. It was a different story in the fat cells. “When [the virus] gets into the fat cells, it not only replicates, it's a productive infection, which means the resulting viral particles can infect another cell,” including microphages, McLaughlin explains. It seems to be a symbiotic tango of the virus between the two cell types that keeps the cycle going.
It is easy to see how the airborne SARS-CoV-2 virus infects the nose and lungs, but how does it get into fat tissue? That is a mystery and the source of ample speculation.
Macrophages are mobile; they engulf and carry invading pathogens to lymphoid tissue in the lymph nodes, tonsils and elsewhere in the body to alert T cells of the immune system to the pathogen. Perhaps some of them also carry the virus through the bloodstream to more distant tissue.
ACE2 receptors are the means by which SARS-CoV-2 latches on to and enters most cells. They are not thought to be common on fat cells, so initially most researchers thought it unlikely they would become infected.
However, while some cell receptors always sit on the surface of the cell, other receptors are expressed on the surface only under certain conditions. Philipp Scherer, a professor of internal medicine and director of the Touchstone Diabetes Center at the University of Texas Southwestern Medical Center, suggests that, in people who have obesity, “There might be higher levels of dysfunctional [fat cells] that facilitate entry of the virus,” either through transiently expressed ACE2 or other receptors. Inflammatory proteins generated by macrophages might contribute to this process.
Another hypothesis is that viral RNA might be smuggled into fat cells as cargo in small bits of material called extracellular vesicles, or EVs, that can travel between cells. Other researchers have shown that when EVs express ACE2 receptors, they can act as decoys for SARS-CoV-2, where the virus binds to them rather than a cell. These scientists are working to create drugs that mimic this decoy effect as an approach to therapy.
Do fat cells play a role in Long Covid? “Fat cells are a great place to hide. You have all the energy you need and fat cells turn over very slowly; they have a half-life of ten years,” says Scherer. Observational studies suggest that acute Covid-19 can trigger the onset of diabetes especially in people who are overweight, and that patients taking medicines to regulate their diabetes “were actually quite protective” against acute Covid-19. Scherer has funding to study the risks and benefits of those drugs in animal models of Long Covid.
McLaughlin says there are two areas of potential concern with fat tissue and Long Covid. One is that this tissue might serve as a “big reservoir where the virus continues to replicate and is sent out” to other parts of the body. The second is that inflammation due to infected fat cells and macrophages can result in fibrosis or scar tissue forming around organs, inhibiting their function. Once scar tissue forms, the tissue damage becomes more difficult to repair.
Current Covid-19 treatments work by stopping the virus from entering cells through the ACE2 receptor, so they likely would have no effect on virus that uses a different mechanism. That means another approach will have to be developed to complement the treatments we already have. So the best advice McLaughlin can offer today is to keep current on vaccinations and boosters and lose weight to reduce the risk associated with obesity.