Sloppy Science Happens More Than You Think
Manipulating DNA through gene editing.
The media loves to tout scientific breakthroughs, and few are as toutable – and in turn, have been as touted – as CRISPR. This method of targeted DNA excision was discovered in bacteria, which use it as an adaptive immune system to combat reinfection with a previously encountered virus.
Shouldn't the editors at a Nature journal know better than to have published an incorrect paper in the first place?
It is cool on so many levels: not only is the basic function fascinating, reminding us that we still have more to discover about even simple organisms that we thought we knew so well, but the ability it grants us to remove and replace any DNA of interest has almost limitless applications in both the lab and the clinic. As if that didn't make it sexy enough, add in a bicoastal, male-female, very public and relatively ugly patent battle, and the CRISPR story is irresistible.
And then last summer, a bombshell dropped. The prestigious journal Nature Methods published a paper in which the authors claimed that CRISPR could cause many unintended mutations, rendering it unfit for clinical use. Havoc duly ensued; stocks in CRISPR-based companies plummeted. Thankfully, the authors of the offending paper were responsible, good scientists; they reassessed, then recanted. Their attention- and headline- grabbing results were wrong, and they admitted as much, leading Nature Methods to formally retract the paper this spring.
How did this happen? Shouldn't the editors at a Nature journal know better than to have published this in the first place?
Alas, high-profile scientific journals publish misleading and downright false results fairly regularly. Some errors are unavoidable – that's how the scientific method works. Hypotheses and conclusions will invariably be overturned as new data becomes available and new technologies are developed that allow for deeper and deeper studies. That's supposed to happen. But that's not what we're talking about here. Nor are we talking about obvious offenses like outright plagiarism. We're talking about mistakes that are avoidable, and that still have serious ramifications.
The cultures of both industry and academia promote research that is poorly designed and even more poorly analyzed.
Two parties are responsible for a scientific publication, and thus two parties bear the blame when things go awry: the scientists who perform and submit the work, and the journals who publish it. Unfortunately, both are incentivized for speedy and flashy publications, and not necessarily for correct publications. It is hardly a surprise, then, that we end up with papers that are speedy and flashy – and not necessarily correct.
"Scientists don't lie and submit falsified data," said Andy Koff, a professor of Molecular Biology at Sloan Kettering Institute, the basic research arm of Memorial Sloan Kettering Cancer Center. Richard Harris, who wrote the book on scientific misconduct running the gamut from unconscious bias and ignorance to more malicious fraudulence, largely concurs (full disclosure: I reviewed the book here). "Scientists want to do good science and want to be recognized as such," he said. But even so, the cultures of both industry and academia promote research that is poorly designed and even more poorly analyzed. In Rigor Mortis: How Sloppy Science Creates Worthless Cures, Crushes Hope, and Wastes Millions, Harris describes how scientists must constantly publish in order to maintain their reputations and positions, to get grants and tenure and students. "They are disincentivized from doing that last extra experiment to prove their results," he said; it could prove too risky if it could cost them a publication.
Ivan Oransky and Adam Marcus founded Retraction Watch, a blog that tracks the retraction of scientific papers, in 2010. Oransky pointed out that blinded peer review – the pride and joy of the scientific publishing enterprise – is a large part of the problem. "Pre-publication peer review is still important, but we can't treat it like the only check on the system. Papers are being reviewed by non-experts, and reviewers are asked to review papers only tangentially related to their field. Moreover, most peer reviewers don't look at the underlying or raw data, even when it is available. How then can they tell if the analysis is flawed or the data is accurate?" he wondered.
Mistaken publications also erode the public's opinion of legitimate science, which is problematic since that opinion isn't especially high to begin with.
Koff agreed that anonymous peer review is valuable, but severely flawed. "Blinded review forces a collective view of importance," he said. "If an article disagrees with the reviewer's worldview, the article gets rejected or forced to adhere to that worldview – even if that means pushing the data someplace it shouldn't necessarily go." We have lost the scientific principle behind review, he thinks, which was to critically analyze a paper. But instead of challenging fundamental assumptions within a paper, reviewers now tend to just ask for more and more supplementary data. And don't get him started on editors. "Editors are supposed to arbitrate between reviewers and writers and they have completely abdicated this responsibility, at every journal. They do not judge, and that's a real failing."
Harris laments the wasted time, effort, and resources that result when erroneous ideas take hold in a field, not to mention lives lost when drug discovery is predicated on basic science findings that end up being wrong. "When no one takes the time, care, and money to reproduce things, science isn't stopping – but it is slowing down," he noted. Mistaken publications also erode the public's opinion of legitimate science, which is problematic since that opinion isn't especially high to begin with.
Scientists and publishers don't only cause the problem, though – they may also provide the solution. Both camps are increasingly recognizing and dealing with the crisis. The self-proclaimed "data thugs" Nick Brown and James Heathers use pretty basic arithmetic to reveal statistical errors in papers. The microbiologist Elisabeth Bik scans the scientific literature for problematic images "in her free time." The psychologist Brian Nosek founded the Center for Open Science, a non-profit organization dedicated to promoting openness, integrity, and reproducibility in scientific research. The Nature family of journals – yes, the one responsible for the latest CRISPR fiasco – has its authors complete a checklist to combat irreproducibility, à la Atul Gawande. And Nature Communications, among other journals, uses transparent peer review, in which authors can opt to have the reviews of their manuscript published anonymously alongside the completed paper. This practice "shows people how the paper evolved," said Koff "and keeps the reviewer and editor accountable. Did the reviewer identify the major problems with the paper? Because there are always major problems with a paper."
Friday Five: These boots were made for walking, even for people who can't
New boots could have you moving like Iron Man, based on research this month by Stanford scientists.
The Friday Five covers important stories in health and science research that you may have missed - usually over the previous week but, today, we're doing a lookback on breakthrough research over the month of October. There are plenty of controversies and troubling ethical issues in science – and we get into many of them in our online magazine – but this news roundup focuses on scientific creativity and progress to give you a therapeutic dose of inspiration headed into the weekend.
Listen on Apple | Listen on Spotify | Listen on Stitcher | Listen on Amazon | Listen on Google
This Friday Five episode covers the following studies published and announced over the past month:
- New boots could have you moving like Iron Man
- The problem with bedtime munching
- The perfect recipe for tiny brains
- The best sports for kids to avoid lifelong health risks
- Can virtual reality reduce pain?
Should egg and sperm donors reveal their identities? The debate pivots on genetics and medical history.
Cassandra Adams performs on stage at the Jersey City Theater Center in March 2019 to raise awareness about traumatic experiences that she and others have had with anonymous donor conception.
Until age 35, Cassandra Adams assumed her mother and father were her biological parents. Then she took saliva tests through two genealogy databases—23andMe and AncestryDNA—and discovered a discrepancy in her heritage. In bringing up the matter with her parents, she learned that fertility issues had led the couple to use a sperm donor.
“Most people my age were not told,” said Adams, now 40 and a stay-at-home mom in Jersey City, New Jersey, who is involved with donor-conception advocacy. “Even now, there’s still a lot of secrecy in the industry. There are still many parents who aren’t truthful or planning not to be truthful with their children.”
While some of those offspring may never know a significant part of their medical history, Adams is grateful that she does. Surprisingly, the DNA test revealed Jewish ancestry.
“There are a lot more genetic conditions that run in Jewish families, so it was really important that I get my medical history, because it’s very different from my dad who raised me,” said Adams, who has met her biological father and two of three known half-siblings. As a result of this experience, she converted to Judaism. “It has been a big journey,” she said.
In an era of advancing assisted reproduction technologies, genetics and medical history have become front and center of the debate as to whether or not egg and sperm donations should be anonymous – and whether secrecy is even possible in many cases.
Obstacles to staying anonymous
People looking to become parents can choose what’s called an “identity-release donor,” meaning their child can receive information about the donor when he or she turns 18. There’s no way to ensure that the donor will consent to a relationship at that time. Instead, if a relationship between the donor and child is a priority, parents may decide to use a known donor.
The majority of donors want to remain anonymous, said reproductive endocrinologist Robert Kiltz, founder and director of CNY Fertility in Syracuse, New York. “In general, egg and sperm donation is mostly anonymous, meaning the recipient doesn’t know the donor and the donor doesn’t know the recipient.”
Even if the donor isn’t disclosed, though, the mystery may become unraveled when a donor-conceived person undergoes direct-to-consumer genetic testing through ancestry databases, which are growing in number and popularity. These services offer DNA testing and links to relatives with identifiable information.
In the future, another obstacle to anonymity could be laws that prohibit anonymous sperm and egg donations, if they catch on. In June, Colorado became the first state in the nation to ban anonymous sperm and egg donations. The law, which takes effect in 2025, will give donor-conceived adults the legal authority to obtain their donor’s identity and medical history. It also requires banks that provide sperm and egg collection to keep current medical records and contact information for all donors. Meanwhile, it prohibits donations from those who won’t consent to identity disclosures.
“The tradition of anonymous sperm or egg donation has created a vast array of problems, most significantly that the people thus created want to know who their mommy and daddy are,” said Kenneth W. Goodman, professor and director of the Institute for Bioethics and Health Policy at the University of Miami Miller School of Medicine.
“There are counter arguments on both sides. But the current situation has led to great uncertainty and, in many cases, grief,” Goodman said.
Donors should bear some moral responsibility for their role in reproduction by allowing their identity to be disclosed to donor-conceived individuals when they turn 18, Goodman added, noting that “there are counter arguments on both sides. But the current situation has led to great uncertainty and, in many cases, grief.”
Adams, the Jersey City woman who learned she was Jewish, has channeled these feelings into several works of art and performances on stage at venues such as the Jersey City Theater Center. During these performances, she describes the trauma of “not knowing where we come from [or] who we look like.”
In the last five years, Kathleen “Casey” DiPaola, a lawyer in Albany, New York, who focuses her practice on adoption, assisted reproduction and surrogacy, has observed a big shift toward would-be parents looking to use known sperm donors. On the other hand, with egg donation, “I’m not seeing a whole lot of change,” she said. Compared to sperm donation, more medical screening is involved with egg donation, so donors are primarily found through fertility clinics and egg donor agencies that prefer anonymity. This leads to fewer options for prospective parents seeking an egg donor with disclosed identity, DiPaola said.
Some donors want to keep in touch
Rachel Lemmons, 32, who lives in Denver, grew interested in becoming an egg donor when, as a graduate student in environmental sciences, she saw an online advertisement. “It seemed like a good way to help pay off my student loan debt,” said Lemmons, who is married and has a daughter who will turn 2-years-old in December. She didn’t end up donating until many years later, after she’d paid off the debt. “The primary motivation at that point wasn’t financial,” she said. “Instead, it felt like a really wonderful way to help someone else have a family in a few weeks’ time.”
Lemmons originally donated anonymously because she didn’t know open donations existed. She was content with that until she became aware of donor-conceived individuals’ struggles. “It concerned me that I could potentially be contributing to this,” she said, adding that the egg donor and surrogacy agency and fertility clinic wouldn’t allow her to disclose her identity retroactively.
Since then, she has donated as an open donor, and kept in touch with the recipients through email and video calls. Knowing that they were finally able to have children is “incredibly rewarding,” Lemmons said.
When to tell the kids
Stanton Honig, professor of urology and division chief of sexual and reproductive medicine at Yale School of Medicine, said for years his team has recommended that couples using donor sperm inform children about the role of the donor and their identity. “Honesty is always the best policy, and it is likely that when they become of age, they might or will be able to find out about their biological sperm donor,” he said. “Hiding it creates more of a complicated situation for children in the long run.”
Amy Jones, a 45-year-old resident of Syracuse, N.Y., has three children, including twins, who know they were conceived with anonymous donor eggs from the same individual, so they share the same genetics. Jones, who is a registered nurse and asked for her real name not to be published, told them around age seven.
“The thought of using a known donor brought more concerns—what if she wanted my babies after they were born, or how would I feel if she treated them as her own every time I saw her?” said Jones.
“I did a lot of reading, and all psychologists said that it is best to start the conversation early,” she recalled. “They understood very little of what I was telling them, but through the years, I have brought it up in discussion and encouraged them to ask questions. To this day, they don't seem to be all that interested, but I expect that later on in life they may have more questions.”
Jones and her husband opted to use a donor because premature ovarian failure at age 27 had rendered her infertile. “The decision to use an egg donor was hard enough,” she said. “The thought of using a known donor brought more concerns—what if she wanted my babies after they were born, or how would I feel if she treated them as her own every time I saw her?”
Susan C. Klock, a clinical psychologist in the section of fertility and reproductive medicine at Northwestern University Feinberg School of Medicine, said, “Anonymity is virtually impossible in the age of direct-to-consumer genetic testing.” In addition, “selecting an identity-release donor is typically not the first thing parents are looking at when they select a donor. First and foremost, they are looking for a donor with a healthy medical background. Then they may consider donor characteristics that resemble the parents.”
The donor’s medical history can be critical
Donor agencies rely on the self-reported medical history of egg and sperm donors, which can lead to gaps in learning important information. Knowing a donor’s medical history may have led some families to make different or more well-informed choices.
After Steven Gunner, a donor-conceived adult, suffered from schizophrenia and died of a drug overdose at age 27 in 2020, his parents, who live in New York, learned of a potential genetic link to his mental illness. A website, Donor Sibling Registry, revealed that the sperm donor the couple had used, a college student at the time of donation, had been hospitalized during childhood for schizophrenia and died of a drug overdose at age 46. Gunner’s story inspired Steven’s Law, a bill that was introduced in Congress in July. If passed, it would mandate sperm banks to collect information on donors’ medical conditions, and donors would have to disclose medical information the banks weren’t able to find.
With limited exceptions, the U.S. Food and Drug Administration requires donors to be screened and tested for relevant communicable disease agents and diseases such as HIV, hepatitis viruses B and C, the Zika virus and several STDs. With current technology, it is also impossible to screen for thousands of rare genetic diseases. “If a couple is using IVF (in vitro fertilization) to conceive with the donor gamete, some may opt for pre-implantation genetic testing to assess for chromosomal abnormalities,” Klock said.
Even these precautions wouldn't cover every disease, and some would-be parents don't get the genetic screening. In a situation where one donor has a large number of offspring, it is concerning that he or she can spread a rare disease to multiple people, said Nick Isel, 37, of Yorkville, Illinois, who was conceived with donor sperm due to his parents’ fertility issues. They told him the truth when he was a teenager, and he found his biological father with a journalist’s help.
Since 2016, Isel, who owns a roofing company, has been petitioning the FDA to extend the retention of medical records, requiring the fertility establishment to maintain information on sperm and egg donors for 50 years instead of the current 10-year mandate.
“The lack of family health information,” he said, “is an ongoing, slow-motion public health crisis since donor conception began being regulated by the FDA as a practice.”