Researchers Behaving Badly: Known Frauds Are "the Tip of the Iceberg"
Last week, the whistleblowers in the Paolo Macchiarini affair at Sweden's Karolinska Institutet went on the record here to detail the retaliation they suffered for trying to expose a star surgeon's appalling research misconduct.
Scientific fraud of the type committed by Macchiarini is rare, but studies suggest that it's on the rise.
The whistleblowers had discovered that in six published papers, Macchiarini falsified data, lied about the condition of patients and circumvented ethical approvals. As a result, multiple patients suffered and died. But Karolinska turned a blind eye for years.
Scientific fraud of the type committed by Macchiarini is rare, but studies suggest that it's on the rise. Just this week, for example, Retraction Watch and STAT together broke the news that a Harvard Medical School cardiologist and stem cell researcher, Piero Anversa, falsified data in a whopping 31 papers, which now have to be retracted. Anversa had claimed that he could regenerate heart muscle by injecting bone marrow cells into damaged hearts, a result that no one has been able to duplicate.
A 2009 study published in the Public Library of Science (PLOS) found that about two percent of scientists admitted to committing fabrication, falsification or plagiarism in their work. That's a small number, but up to one third of scientists admit to committing "questionable research practices" that fall into a gray area between rigorous accuracy and outright fraud.
These dubious practices may include misrepresentations, research bias, and inaccurate interpretations of data. One common questionable research practice entails formulating a hypothesis after the research is done in order to claim a successful premise. Another highly questionable practice that can shape research is ghost-authoring by representatives of the pharmaceutical industry and other for-profit fields. Still another is gifting co-authorship to unqualified but powerful individuals who can advance one's career. Such practices can unfairly bolster a scientist's reputation and increase the likelihood of getting the work published.
The above percentages represent what scientists admit to doing themselves; when they evaluate the practices of their colleagues, the numbers jump dramatically. In a 2012 study published in the Journal of Research in Medical Sciences, researchers estimated that 14 percent of other scientists commit serious misconduct, while up to 72 percent engage in questionable practices. While these are only estimates, the problem is clearly not one of just a few bad apples.
In the PLOS study, Daniele Fanelli says that increasing evidence suggests the known frauds are "just the 'tip of the iceberg,' and that many cases are never discovered" because fraud is extremely hard to detect.
Essentially everyone wants to be associated with big breakthroughs, and they may overlook scientifically shaky foundations when a major advance is claimed.
In addition, it's likely that most cases of scientific misconduct go unreported because of the high price of whistleblowing. Those in the Macchiarini case showed extraordinary persistence in their multi-year campaign to stop his deadly trachea implants, while suffering serious damage to their careers. Such heroic efforts to unmask fraud are probably rare.
To make matters worse, there are numerous players in the scientific world who may be complicit in either committing misconduct or covering it up. These include not only primary researchers but co-authors, institutional executives, journal editors, and industry leaders. Essentially everyone wants to be associated with big breakthroughs, and they may overlook scientifically shaky foundations when a major advance is claimed.
Another part of the problem is that it's rare for students in science and medicine to receive an education in ethics. And studies have shown that older, more experienced and possibly jaded researchers are more likely to fudge results than their younger, more idealistic colleagues.
So, given the steep price that individuals and institutions pay for scientific misconduct, what compels them to go down that road in the first place? According to the JRMS study, individuals face intense pressures to publish and to attract grant money in order to secure teaching positions at universities. Once they have acquired positions, the pressure is on to keep the grants and publishing credits coming in order to obtain tenure, be appointed to positions on boards, and recruit flocks of graduate students to assist in research. And not to be underestimated is the human ego.
Paolo Macchiarini is an especially vivid example of a scientist seeking not only fortune, but fame. He liberally (and falsely) claimed powerful politicians and celebrities, even the Pope, as patients or admirers. He may be an extreme example, but we live in an age of celebrity scientists who bring huge amounts of grant money and high prestige to the institutions that employ them.
The media plays a significant role in both glorifying stars and unmasking frauds. In the Macchiarini scandal, the media first lifted him up, as in NBC's laudatory documentary, "A Leap of Faith," which painted him as a kind of miracle-worker, and then brought him down, as in the January 2016 documentary, "The Experiments," which chronicled the agonizing death of one of his patients.
Institutions can also play a crucial role in scientific fraud by putting more emphasis on the number and frequency of papers published than on their quality. The whole course of a scientist's career is profoundly affected by something called the h-index. This is a number based on both the frequency of papers published and how many times the papers are cited by other researchers. Raising one's ranking on the h-index becomes an overriding goal, sometimes eclipsing the kind of patient, time-consuming research that leads to true breakthroughs based on reliable results.
Universities also create a high-pressured environment that encourages scientists to cut corners. They, too, place a heavy emphasis on attracting large monetary grants and accruing fame and prestige. This can lead them, just as it led Karolinska, to protect a star scientist's sloppy or questionable research. According to Dr. Andrew Rosenberg, who is director of the Center for Science and Democracy at the U.S.-based Union of Concerned Scientists, "Karolinska defended its investment in an individual as opposed to the long-term health of the institution. People were dying, and they should have outsourced the investigation from the very beginning."
Having institutions investigate their own practices is a conflict of interest from the get-go, says Rosenberg.
Scientists, universities, and research institutions are also not immune to fads. "Hot" subjects attract grant money and confer prestige, incentivizing scientists to shift their research priorities in a direction that garners more grants. This can mean neglecting the scientist's true area of expertise and interests in favor of a subject that's more likely to attract grant money. In Macchiarini's case, he was allegedly at the forefront of the currently sexy field of regenerative medicine -- a field in which Karolinska was making a huge investment.
The relative scarcity of resources intensifies the already significant pressure on scientists. They may want to publish results rapidly, since they face many competitors for limited grant money, academic positions, students, and influence. The scarcity means that a great many researchers will fail while only a few succeed. Once again, the temptation may be to rush research and to show it in the most positive light possible, even if it means fudging or exaggerating results.
Though the pressures facing scientists are very real, the problem of misconduct is not inevitable.
Intense competition can have a perverse effect on researchers, according to a 2007 study in the journal Science of Engineering and Ethics. Not only does it place undue pressure on scientists to succeed, it frequently leads to the withholding of information from colleagues, which undermines a system in which new discoveries build on the previous work of others. Researchers may feel compelled to withhold their results because of the pressure to be the first to publish. The study's authors propose that more investment in basic research from governments could alleviate some of these competitive pressures.
Scientific journals, although they play a part in publishing flawed science, can't be expected to investigate cases of suspected fraud, says the German science blogger Leonid Schneider. Schneider's writings helped to expose the Macchiarini affair.
"They just basically wait for someone to retract problematic papers," he says.
He also notes that, while American scientists can go to the Office of Research Integrity to report misconduct, whistleblowers in Europe have no external authority to whom they can appeal to investigate cases of fraud.
"They have to go to their employer, who has a vested interest in covering up cases of misconduct," he says.
Science is increasingly international. Major studies can include collaborators from several different countries, and he suggests there should be an international body accessible to all researchers that will investigate suspected fraud.
Ultimately, says Rosenberg, the scientific system must incorporate trust. "You trust co-authors when you write a paper, and peer reviewers at journals trust that scientists at research institutions like Karolinska are acting with integrity."
Without trust, the whole system falls apart. It's the trust of the public, an elusive asset once it has been betrayed, that science depends upon for its very existence. Scientific research is overwhelmingly financed by tax dollars, and the need for the goodwill of the public is more than an abstraction.
The Macchiarini affair raises a profound question of trust and responsibility: Should multiple co-authors be held responsible for a lead author's misconduct?
Karolinska apparently believes so. When the institution at last owned up to the scandal, it vindictively found Karl Henrik-Grinnemo, one of the whistleblowers, guilty of scientific misconduct as well. It also designated two other whistleblowers as "blameworthy" for their roles as co-authors of the papers on which Macchiarini was the lead author.
As a result, the whistleblowers' reputations and employment prospects have become collateral damage. Accusations of research misconduct can be a career killer. Research grants dry up, employment opportunities evaporate, publishing becomes next to impossible, and collaborators vanish into thin air.
Grinnemo contends that co-authors should only be responsible for their discrete contributions, not for the data supplied by others.
"Different aspects of a paper are highly specialized," he says, "and that's why you have multiple authors. You cannot go through every single bit of data because you don't understand all the parts of the article."
This is especially true in multidisciplinary, translational research, where there are sometimes 20 or more authors. "You have to trust co-authors, and if you find something wrong you have to notify all co-authors. But you couldn't go through everything or it would take years to publish an article," says Grinnemo.
Though the pressures facing scientists are very real, the problem of misconduct is not inevitable. Along with increased support from governments and industry, a change in academic culture that emphasizes quality over quantity of published studies could help encourage meritorious research.
But beyond that, trust will always play a role when numerous specialists unite to achieve a common goal: the accumulation of knowledge that will promote human health, wealth, and well-being.
[Correction: An earlier version of this story mistakenly credited The New York Times with breaking the news of the Anversa retractions, rather than Retraction Watch and STAT, which jointly published the exclusive on October 14th. The piece in the Times ran on October 15th. We regret the error.]
From Crap to Cure: The Story of Fecal Transplants
C. difficile had Meg Newman's number; it had struck her 18 different times beginning in 1985. The bacterial infection takes over the gut bringing explosive diarrhea, dehydration, weight loss, and at its worst depletes blood platelets. It causes nearly 30,000 deaths each year in the U.S. alone.
"I was one sick puppy as that point and literally three days after the transplant I was doing pretty well, day four even better."
Meg knew these statistics not just from personal experience but also because she was a doctor at San Francisco General Hospital. Antibiotics had sometimes helped to treat the infection, but it never quite seemed to go away. Now, "It felt like part of my colon was sort of sliding off as I had the bowel movement." On her worst day she counted 33 bowel movements. It was 2005 and she knew she was at the end of her rope.
Medical training had taught Meg to look at the data. So when antibiotics failed, she searched the literature for other options. One was a seemingly off-the-wall treatment called fecal transplants, which essentially gives poop from a healthy person to one who is sick.
Its roots stretch back to "yellow soup" used to treat dysentery in China nearly two thousand years ago, in which ancient Chinese treaters would combine stool with liquid, mash it up, and administer it. The approach also is commonly used in veterinary medicine today. However, there were only about three papers on its use in humans in the medical literature at that time, she recalls. Still, the logic of the intervention appealed to her.
The gut microbiome as a concept and even a word were not widely known fifteen years ago. But the idea that the microbial community in her gut was in disarray, and a transplant of organisms from a healthy gut might help restore a more normal ecology made sense. And besides, the failure of standard medicine left her few options.
Meg spoke with a colleague, gastroenterologist Neil Stollman, about a possible fecal microbial transplant (FMT). Only a handful of doctors in the U.S. had ever done the procedure; Stollman had tried it just once before. After conversation with Newman, he agreed to do it.
They decided on Meg's partner Sherry as the donor. "I try very hard to use intimate sexual partners as the donor," explains Stollman. The reason is to reduce disease risk: "The logic there is pretty straightforward. If you have unprotected sex with this individual, in a monogamous way for a period of time, you have assumed pretty much any infectious risk," like hepatitis, HIV, and syphilis, he says. Other donors would be screened using the same criteria used to screen blood donations, plus screening for parasites that can live in stool but not blood.
The procedure
Martini aficionados fall into two camps, fans of shaken or stirred. In the early days the options for producing of fecal transplants were a blender or hand shaken. Stollman took the hands-on approach, mixing Sherry's fecal donation with saline to create "a milkshake in essence." He filtered it several times through gauze to get out the lumps.
Then he inserted a colonoscope, a long flexible tube, through the anus into Meg's colon. Generally a camera goes through the tube to look for polyps and cancers, while other tools can snip off polyps and retrieve tissue samples. Today he used it to insert the fecal "milkshake" as high up the colon as he could go. Imodium and bed rest were the final pieces. It works about 90 percent of the time today.
Meg went home with fingers crossed. "And within about two weeks things just slowed down; the diarrhea just stopped. I felt better so my appetite was better." The tide had turned, though it would take months to slowly repair the toll taken on her body from disease and antibiotics.
Then in 2011 another serious medical challenge required heavy use of antibiotics and Meg's C. difficile came roaring back; she needed a second FMT. Sherry had a bad sinus infection and had been on antibiotics, so that ruled her out as a donor. Red, Meg's godson, volunteered. He was twenty-one and had little or no exposure to antibiotics, which can harm friendly bacteria living in the gut.
"I was one sick puppy as that point," Meg recalls, "and literally three days after the transplant [from Red] I was doing pretty well, day four even better. It was unbelievable." It illustrated that donors are not the same, and that while an intimate partner may be the safest option, it also may not be the most efficacious donation in terms of providing missing parts of the microbial ecosystem.
Going mainstream
By then, FMTs were starting to come out of the shadows as more than just a medical oddity. One gigantic milestone in changing perceptions was a Dutch study on using the procedure to treat C. difficile that was published in January 2013 in the New England Journal of Medicine. "That was the first trial where people said, look this isn't voodoo. This wasn't made up; it really worked," says Colleen Kelly, another pioneer in using FMTs to treat C. difficile and a researcher at Brown University. A single dose was successful more than 80 percent of the time in resolving disease in patients who had failed multiple regimens of antibiotics.
Charlatans pounced on the growing interest in the microbiome, promoting FMT as a cure for all sorts of ailments for which there was no scientific evidence. The FDA stepped in, announcing it would regulate the procedure as a drug, and essentially banned use of FMTs until it wrote regulations. But the outcry from physicians and patients was so great it was forced to retreat and has allowed continued use in treating C. difficile albeit on an interim regulatory basis that has continued since 2013.
Another major change was the emergence of stool banks, modeled on blood banks. The most widely know is OpenBiome, established in 2012 as a nonprofit by young researchers at Harvard and MIT. It aimed to standardize donation of stool and screening for disease, and package them in frozen form for colonoscopic delivery, or pill form. It greatly simplified the process and more doctors became willing to use FMTs to treat C. difficile. Today, some gastroenterologists specialize in administering the transplants as a feature of their practice.
To be sure, there have been some setbacks, including a transplant between family members where the recipient became obese, likely in part because of bacteria in the material she received. The same thing has occurred in studies in mice. And last year, an elderly man died from a toxic strain of E. coli that was in material provided by a stool bank. That caused the FDA to add that pathogen to the list of those one must screen for in products designed for use as fecal transplants.
Cost remains an issue. OpenBiome charges $1500-$2000 per transplant dose, depending on whether a frozen or pill form of the product is used. And that is likely to go up as the FDA increases the number of diseases that must be screened for, such as the virus that causes COVID-19, which is present in feces and likely can be transmitted through feces. Most insurance companies do not cover FMTs because no product has been formally approved for use by the FDA.
One of the greatest treatment challenges is making the correct diagnosis, says physician Robin Patel, who initially treated patients at the Mayo Clinic in Rochester, Minnesota but now spends most of her time there developing new diagnostics. Many things can cause diarrhea and the simple presence of the organism does not mean that C. difficile is causing it. In addition, many people are colonized with the bug but never develop symptoms of the disease.
Patel used the expensive tool of whole genome sequencing to look in great detail at C. difficile in patients who were treated with antibiotics for the infection and had recurrent diarrhea. "Some of them, as you might predict, were getting their symptoms with the same exact strain [of C. difficile] but others were not, it was a different strain," suggesting that they had been reinfected.
If it is a different strain, you might want to try antibiotics, she says, but if the same strain is present, then you might want to try a different approach such as FMT. Whole genome sequencing is still too slow and expensive to use in regularly treating patients today, but Patel hopes to develop a rapid, cost-effective test to help doctors make those types of decisions.
Biotech companies are trying to develop alternatives to poop as a source for transplant to treat C. difficile. They are picking and choosing different bacteria that they can grow and then combine into a product, to varying degrees of success, but none have yet crossed the finish line of FDA approval.
"I think [the future of FMTs] is going to be targeted, even custom-made."
The FDA would like such a product because it is used to dealing with small molecule drugs that are standardized and produced in batches. Companies are pursing it because, as Kelly says, they are like sharks "smelling money in the water." Approval of such a product might cause the FDA to shut down existing stool banks that now exist in a regulatory limbo, leaving the company with a monopoly of exclusive rights to the treatment.
Back when Meg received her first fecal transplant, the procedure was so obscure that the guidelines for treating C. difficile put out by the American College of Gastroenterology didn't even mention FMT. The procedure crept into the 2013 revision of those guidelines as a treatment of last resort. Guidance under review for release later this year or early next year will ease use further but stop short of making it a first option.
Stollman imagines a future holy grail in treating C. difficile: "You give me a stool specimen and I run it through a scanner that tells me you have too much of this and too little of that. I have a sense of what normal [microbial composition of the gut] should be and add some of this and subtract some of that. Maybe I even give you some antibiotics to get rid of some of the bad guys, give you some probiotics. I think it is going to be targeted, even custom-made."
His complete vision for treating C. difficile won't arrive tomorrow, but given how rapidly FMTs have become part of medicine, it is likely to arrive in pieces and more quickly than one might think.
About five years ago Meg discovered she had an antibody deficiency that contributed to her health problems, including vulnerability to C. difficile. She began supplementation with immunoglobulin and "that has made a huge difference in my health. It is just unbelievable how much better I am." She is grateful that fecal transplants gave her the time to figure that out. She believes "there's every reason to consider it [FMT] as a first-line treatment and do the studies, ASAP."
“You First”: Who Will Be Front in Line to Get a COVID Vaccine?
There is a huge amount riding on the discovery of a vaccine effective against the Covid-19 virus.
Making 660 million of anything without a glitch is—to put it mildly—a tall order in a nation that remains short on masks, gowns, and diagnostic tests despite months of trying to meet demand.
The world is waiting for a vaccine that can liberate everyone from the constraints on liberty required by existing efforts to fight the virus with public health measures such as masks, isolation, and quarantining. President Trump, for the most part, has rejected tough public health measures. Instead he has staked his political future and those of the governors and Congressional Republicans who have followed his lead on delivering a vaccine before Election Day as the solution to the COVID-19 pandemic in the USA. Many scientific experts have been sounding encouraging notes about having a vaccine by the end of this year or early next, as have many CEOs among the more than 160 companies chasing various strategies to identify a safe and effective vaccine.
But the reality is that no matter how fast a vaccine appears, those who might benefit will face a significant period of time before they could receive one. This is due to a variety of realities. Any vaccine faces various regulatory hurdles to insure safety and efficacy. This means completing large-scale studies in tens of thousands of subjects hoping for enough cases of blunted natural infection versus a large placebo control group to determine that a vaccine works. And that takes time--plus adding in delays in manufacturing and delivery, which will create logjams for most prospective recipients.
Shipping is not going to be easy with cold chain storage requirements from -20 to -70 degrees Celsius, from factory to a doctor's office, depending on the vaccine. In addition, many of the vaccines under development require two doses--that is 660 million shots to cover just those in the United States. Making 660 million of anything without a glitch is—to put it mildly—a tall order in a nation that remains short on masks, gowns, and diagnostic tests, despite months of trying to meet demand.
There are three scenarios under which a vaccine can appear but without being in any way available to all Americans.
The first is a vaccine under development in the USA or with some USA financing begins to show promise before a full clinical trial is completed. Current vaccine trials are supervised by Data Safety and Monitoring Boards and those committees could tell a CEO eager to be first to market that their vaccine is looking good at the study's half-way point.
The CEO and vaccine manufacturing company's board then let the White House know that a magic bullet which can ensure the President's reelection is in hand. The President, as he has done many times with other COVID treatments, most recently convalescent plasma, intervenes with the FDA and demands approval using an Emergency Use Authorization, or invoking the Federal Right to Try law he and Mike Pence are constantly touting. FDA Commissioner Steve Hahn folds and an extremely limited supply of vaccine, maybe only 100,000 doses, is available just before Election Day.
The second scenario is that another nation discovers a vaccine that looks safe and effective and the USA is able to buy some supply of it. But again, we are likely, initially, to get an extremely limited amount.
Lastly, the vaccine is approved in a standard manner. A full randomized trial is done, the endpoints are met, and no serious adverse events are identified. It is a USA-funded vaccine so most of it is coming here first. Still the vials and needles and plugs need to be quality-controlled and shipped and stored at the right temperatures. Information sheets and consent forms need to be readied, offered, and signed. Odds are you won't see any of this vaccine until late next year. So, who is going to get the first shots?
Some people under all of these scenarios are going to say, "Count me out." They don't trust vaccines or they don't trust the government to provide a safe one. Others may say, "The first one out of the box may be OK, but I am going to wait for the 'best' one before I take one." Even if those numbers are large, it is still certain that there will be more takers than can be vaccinated.
If you look at the discussion of vaccine rationing, almost everybody — including government officials, FDA officials, advisory panelists and ethicists — says the first group that should get vaccinated are at-risk healthcare workers. They say it, although they're not always clear about why.
One reason is that you need to give it to health care workers first because they will keep the healthcare system going. Another is that you need to give it to them first because they face more risk and they should get rewarded for having done and continuing to do that -- their bravery ought to be rewarded and their risk reduced.
A subset of hospitals and institutions in high risk areas will [go first] and that will be it for a significant period of time.
Both of these arguments for health care worker priority are not completely convincing. Food and power and vaccine manufacturing are arguably as important as health care, but workers in those areas don't get priority attention in most guidelines. And many Americans face risks from COVID comparable to health care workers, especially those who are not on the front lines in ERs and ICUs. Prisoners, military personnel who work on warships, the elderly, nursing home residents, and poor minorities are disproportionately affected by COVID. However, none of them are going first, nor is it clear how to weigh their claims in competing against one another for a scarce vaccine.
But, there's something else that's interesting in deciding who goes first. When people all agree, as they almost always do, that it's health care workers who must go first, a huge problem remains. What is the definition of who's a healthcare worker? You could easily get millions and millions of people designated as healthcare workers who would have a claim to go first.
We normally think that health care worker means doctors and nurses. But, if we go beyond those who work in ERs and ICUs, the number is big. And we must, because no ER or ICU can run without huge numbers of supporting individuals.
If you don't vaccinate lab technicians, people who clean the rooms, make food, transport patients, provide security, do the laundry, run the IT, students, volunteers and so on, you're not going to have a functioning hospital. If you don't include those working in nursing homes, home care and hospices along with those making and supplying vital equipment and bringing in patients via ambulances, police cars, and fire trucks, you don't have a functioning ICU, much less a health care system.
The total number involved could easily exceed tens of millions depending on how broadly the definition is set.
So, what is likely to happen is that health care workers will not go first. A subset of hospitals and institutions in high risk areas will and that will be it for a significant period of time. Health care institutions in hot spots, plus the supporting services they need will go first and then vaccine availability will slowly expand to other health care institutions and the essential workers needed to keep them functioning. Then consideration will also be given to how best to control the spread of the virus in selecting hot spots versus saving prisoners or the poor. And you can be sure, whatever the guidelines are, that the military and security folks will demand their share.
For many, many months if not a year or more, most people will not have to face a choice about vaccinating. The supply just won't be there for the general public. It is a small sample of high-risk health care workers including vaccine manufacturing employees and shippers, plus essential workers to keep hospitals and nursing homes going, who will be first in line. Odds are you and your family will still be wearing masks and social distancing well into next year.