Researchers Behaving Badly: Known Frauds Are "the Tip of the Iceberg"
Last week, the whistleblowers in the Paolo Macchiarini affair at Sweden's Karolinska Institutet went on the record here to detail the retaliation they suffered for trying to expose a star surgeon's appalling research misconduct.
Scientific fraud of the type committed by Macchiarini is rare, but studies suggest that it's on the rise.
The whistleblowers had discovered that in six published papers, Macchiarini falsified data, lied about the condition of patients and circumvented ethical approvals. As a result, multiple patients suffered and died. But Karolinska turned a blind eye for years.
Scientific fraud of the type committed by Macchiarini is rare, but studies suggest that it's on the rise. Just this week, for example, Retraction Watch and STAT together broke the news that a Harvard Medical School cardiologist and stem cell researcher, Piero Anversa, falsified data in a whopping 31 papers, which now have to be retracted. Anversa had claimed that he could regenerate heart muscle by injecting bone marrow cells into damaged hearts, a result that no one has been able to duplicate.
A 2009 study published in the Public Library of Science (PLOS) found that about two percent of scientists admitted to committing fabrication, falsification or plagiarism in their work. That's a small number, but up to one third of scientists admit to committing "questionable research practices" that fall into a gray area between rigorous accuracy and outright fraud.
These dubious practices may include misrepresentations, research bias, and inaccurate interpretations of data. One common questionable research practice entails formulating a hypothesis after the research is done in order to claim a successful premise. Another highly questionable practice that can shape research is ghost-authoring by representatives of the pharmaceutical industry and other for-profit fields. Still another is gifting co-authorship to unqualified but powerful individuals who can advance one's career. Such practices can unfairly bolster a scientist's reputation and increase the likelihood of getting the work published.
The above percentages represent what scientists admit to doing themselves; when they evaluate the practices of their colleagues, the numbers jump dramatically. In a 2012 study published in the Journal of Research in Medical Sciences, researchers estimated that 14 percent of other scientists commit serious misconduct, while up to 72 percent engage in questionable practices. While these are only estimates, the problem is clearly not one of just a few bad apples.
In the PLOS study, Daniele Fanelli says that increasing evidence suggests the known frauds are "just the 'tip of the iceberg,' and that many cases are never discovered" because fraud is extremely hard to detect.
Essentially everyone wants to be associated with big breakthroughs, and they may overlook scientifically shaky foundations when a major advance is claimed.
In addition, it's likely that most cases of scientific misconduct go unreported because of the high price of whistleblowing. Those in the Macchiarini case showed extraordinary persistence in their multi-year campaign to stop his deadly trachea implants, while suffering serious damage to their careers. Such heroic efforts to unmask fraud are probably rare.
To make matters worse, there are numerous players in the scientific world who may be complicit in either committing misconduct or covering it up. These include not only primary researchers but co-authors, institutional executives, journal editors, and industry leaders. Essentially everyone wants to be associated with big breakthroughs, and they may overlook scientifically shaky foundations when a major advance is claimed.
Another part of the problem is that it's rare for students in science and medicine to receive an education in ethics. And studies have shown that older, more experienced and possibly jaded researchers are more likely to fudge results than their younger, more idealistic colleagues.
So, given the steep price that individuals and institutions pay for scientific misconduct, what compels them to go down that road in the first place? According to the JRMS study, individuals face intense pressures to publish and to attract grant money in order to secure teaching positions at universities. Once they have acquired positions, the pressure is on to keep the grants and publishing credits coming in order to obtain tenure, be appointed to positions on boards, and recruit flocks of graduate students to assist in research. And not to be underestimated is the human ego.
Paolo Macchiarini is an especially vivid example of a scientist seeking not only fortune, but fame. He liberally (and falsely) claimed powerful politicians and celebrities, even the Pope, as patients or admirers. He may be an extreme example, but we live in an age of celebrity scientists who bring huge amounts of grant money and high prestige to the institutions that employ them.
The media plays a significant role in both glorifying stars and unmasking frauds. In the Macchiarini scandal, the media first lifted him up, as in NBC's laudatory documentary, "A Leap of Faith," which painted him as a kind of miracle-worker, and then brought him down, as in the January 2016 documentary, "The Experiments," which chronicled the agonizing death of one of his patients.
Institutions can also play a crucial role in scientific fraud by putting more emphasis on the number and frequency of papers published than on their quality. The whole course of a scientist's career is profoundly affected by something called the h-index. This is a number based on both the frequency of papers published and how many times the papers are cited by other researchers. Raising one's ranking on the h-index becomes an overriding goal, sometimes eclipsing the kind of patient, time-consuming research that leads to true breakthroughs based on reliable results.
Universities also create a high-pressured environment that encourages scientists to cut corners. They, too, place a heavy emphasis on attracting large monetary grants and accruing fame and prestige. This can lead them, just as it led Karolinska, to protect a star scientist's sloppy or questionable research. According to Dr. Andrew Rosenberg, who is director of the Center for Science and Democracy at the U.S.-based Union of Concerned Scientists, "Karolinska defended its investment in an individual as opposed to the long-term health of the institution. People were dying, and they should have outsourced the investigation from the very beginning."
Having institutions investigate their own practices is a conflict of interest from the get-go, says Rosenberg.
Scientists, universities, and research institutions are also not immune to fads. "Hot" subjects attract grant money and confer prestige, incentivizing scientists to shift their research priorities in a direction that garners more grants. This can mean neglecting the scientist's true area of expertise and interests in favor of a subject that's more likely to attract grant money. In Macchiarini's case, he was allegedly at the forefront of the currently sexy field of regenerative medicine -- a field in which Karolinska was making a huge investment.
The relative scarcity of resources intensifies the already significant pressure on scientists. They may want to publish results rapidly, since they face many competitors for limited grant money, academic positions, students, and influence. The scarcity means that a great many researchers will fail while only a few succeed. Once again, the temptation may be to rush research and to show it in the most positive light possible, even if it means fudging or exaggerating results.
Though the pressures facing scientists are very real, the problem of misconduct is not inevitable.
Intense competition can have a perverse effect on researchers, according to a 2007 study in the journal Science of Engineering and Ethics. Not only does it place undue pressure on scientists to succeed, it frequently leads to the withholding of information from colleagues, which undermines a system in which new discoveries build on the previous work of others. Researchers may feel compelled to withhold their results because of the pressure to be the first to publish. The study's authors propose that more investment in basic research from governments could alleviate some of these competitive pressures.
Scientific journals, although they play a part in publishing flawed science, can't be expected to investigate cases of suspected fraud, says the German science blogger Leonid Schneider. Schneider's writings helped to expose the Macchiarini affair.
"They just basically wait for someone to retract problematic papers," he says.
He also notes that, while American scientists can go to the Office of Research Integrity to report misconduct, whistleblowers in Europe have no external authority to whom they can appeal to investigate cases of fraud.
"They have to go to their employer, who has a vested interest in covering up cases of misconduct," he says.
Science is increasingly international. Major studies can include collaborators from several different countries, and he suggests there should be an international body accessible to all researchers that will investigate suspected fraud.
Ultimately, says Rosenberg, the scientific system must incorporate trust. "You trust co-authors when you write a paper, and peer reviewers at journals trust that scientists at research institutions like Karolinska are acting with integrity."
Without trust, the whole system falls apart. It's the trust of the public, an elusive asset once it has been betrayed, that science depends upon for its very existence. Scientific research is overwhelmingly financed by tax dollars, and the need for the goodwill of the public is more than an abstraction.
The Macchiarini affair raises a profound question of trust and responsibility: Should multiple co-authors be held responsible for a lead author's misconduct?
Karolinska apparently believes so. When the institution at last owned up to the scandal, it vindictively found Karl Henrik-Grinnemo, one of the whistleblowers, guilty of scientific misconduct as well. It also designated two other whistleblowers as "blameworthy" for their roles as co-authors of the papers on which Macchiarini was the lead author.
As a result, the whistleblowers' reputations and employment prospects have become collateral damage. Accusations of research misconduct can be a career killer. Research grants dry up, employment opportunities evaporate, publishing becomes next to impossible, and collaborators vanish into thin air.
Grinnemo contends that co-authors should only be responsible for their discrete contributions, not for the data supplied by others.
"Different aspects of a paper are highly specialized," he says, "and that's why you have multiple authors. You cannot go through every single bit of data because you don't understand all the parts of the article."
This is especially true in multidisciplinary, translational research, where there are sometimes 20 or more authors. "You have to trust co-authors, and if you find something wrong you have to notify all co-authors. But you couldn't go through everything or it would take years to publish an article," says Grinnemo.
Though the pressures facing scientists are very real, the problem of misconduct is not inevitable. Along with increased support from governments and industry, a change in academic culture that emphasizes quality over quantity of published studies could help encourage meritorious research.
But beyond that, trust will always play a role when numerous specialists unite to achieve a common goal: the accumulation of knowledge that will promote human health, wealth, and well-being.
[Correction: An earlier version of this story mistakenly credited The New York Times with breaking the news of the Anversa retractions, rather than Retraction Watch and STAT, which jointly published the exclusive on October 14th. The piece in the Times ran on October 15th. We regret the error.]
“You First”: Who Will Be Front in Line to Get a COVID Vaccine?
There is a huge amount riding on the discovery of a vaccine effective against the Covid-19 virus.
Making 660 million of anything without a glitch is—to put it mildly—a tall order in a nation that remains short on masks, gowns, and diagnostic tests despite months of trying to meet demand.
The world is waiting for a vaccine that can liberate everyone from the constraints on liberty required by existing efforts to fight the virus with public health measures such as masks, isolation, and quarantining. President Trump, for the most part, has rejected tough public health measures. Instead he has staked his political future and those of the governors and Congressional Republicans who have followed his lead on delivering a vaccine before Election Day as the solution to the COVID-19 pandemic in the USA. Many scientific experts have been sounding encouraging notes about having a vaccine by the end of this year or early next, as have many CEOs among the more than 160 companies chasing various strategies to identify a safe and effective vaccine.
But the reality is that no matter how fast a vaccine appears, those who might benefit will face a significant period of time before they could receive one. This is due to a variety of realities. Any vaccine faces various regulatory hurdles to insure safety and efficacy. This means completing large-scale studies in tens of thousands of subjects hoping for enough cases of blunted natural infection versus a large placebo control group to determine that a vaccine works. And that takes time--plus adding in delays in manufacturing and delivery, which will create logjams for most prospective recipients.
Shipping is not going to be easy with cold chain storage requirements from -20 to -70 degrees Celsius, from factory to a doctor's office, depending on the vaccine. In addition, many of the vaccines under development require two doses--that is 660 million shots to cover just those in the United States. Making 660 million of anything without a glitch is—to put it mildly—a tall order in a nation that remains short on masks, gowns, and diagnostic tests, despite months of trying to meet demand.
There are three scenarios under which a vaccine can appear but without being in any way available to all Americans.
The first is a vaccine under development in the USA or with some USA financing begins to show promise before a full clinical trial is completed. Current vaccine trials are supervised by Data Safety and Monitoring Boards and those committees could tell a CEO eager to be first to market that their vaccine is looking good at the study's half-way point.
The CEO and vaccine manufacturing company's board then let the White House know that a magic bullet which can ensure the President's reelection is in hand. The President, as he has done many times with other COVID treatments, most recently convalescent plasma, intervenes with the FDA and demands approval using an Emergency Use Authorization, or invoking the Federal Right to Try law he and Mike Pence are constantly touting. FDA Commissioner Steve Hahn folds and an extremely limited supply of vaccine, maybe only 100,000 doses, is available just before Election Day.
The second scenario is that another nation discovers a vaccine that looks safe and effective and the USA is able to buy some supply of it. But again, we are likely, initially, to get an extremely limited amount.
Lastly, the vaccine is approved in a standard manner. A full randomized trial is done, the endpoints are met, and no serious adverse events are identified. It is a USA-funded vaccine so most of it is coming here first. Still the vials and needles and plugs need to be quality-controlled and shipped and stored at the right temperatures. Information sheets and consent forms need to be readied, offered, and signed. Odds are you won't see any of this vaccine until late next year. So, who is going to get the first shots?
Some people under all of these scenarios are going to say, "Count me out." They don't trust vaccines or they don't trust the government to provide a safe one. Others may say, "The first one out of the box may be OK, but I am going to wait for the 'best' one before I take one." Even if those numbers are large, it is still certain that there will be more takers than can be vaccinated.
If you look at the discussion of vaccine rationing, almost everybody — including government officials, FDA officials, advisory panelists and ethicists — says the first group that should get vaccinated are at-risk healthcare workers. They say it, although they're not always clear about why.
One reason is that you need to give it to health care workers first because they will keep the healthcare system going. Another is that you need to give it to them first because they face more risk and they should get rewarded for having done and continuing to do that -- their bravery ought to be rewarded and their risk reduced.
A subset of hospitals and institutions in high risk areas will [go first] and that will be it for a significant period of time.
Both of these arguments for health care worker priority are not completely convincing. Food and power and vaccine manufacturing are arguably as important as health care, but workers in those areas don't get priority attention in most guidelines. And many Americans face risks from COVID comparable to health care workers, especially those who are not on the front lines in ERs and ICUs. Prisoners, military personnel who work on warships, the elderly, nursing home residents, and poor minorities are disproportionately affected by COVID. However, none of them are going first, nor is it clear how to weigh their claims in competing against one another for a scarce vaccine.
But, there's something else that's interesting in deciding who goes first. When people all agree, as they almost always do, that it's health care workers who must go first, a huge problem remains. What is the definition of who's a healthcare worker? You could easily get millions and millions of people designated as healthcare workers who would have a claim to go first.
We normally think that health care worker means doctors and nurses. But, if we go beyond those who work in ERs and ICUs, the number is big. And we must, because no ER or ICU can run without huge numbers of supporting individuals.
If you don't vaccinate lab technicians, people who clean the rooms, make food, transport patients, provide security, do the laundry, run the IT, students, volunteers and so on, you're not going to have a functioning hospital. If you don't include those working in nursing homes, home care and hospices along with those making and supplying vital equipment and bringing in patients via ambulances, police cars, and fire trucks, you don't have a functioning ICU, much less a health care system.
The total number involved could easily exceed tens of millions depending on how broadly the definition is set.
So, what is likely to happen is that health care workers will not go first. A subset of hospitals and institutions in high risk areas will and that will be it for a significant period of time. Health care institutions in hot spots, plus the supporting services they need will go first and then vaccine availability will slowly expand to other health care institutions and the essential workers needed to keep them functioning. Then consideration will also be given to how best to control the spread of the virus in selecting hot spots versus saving prisoners or the poor. And you can be sure, whatever the guidelines are, that the military and security folks will demand their share.
For many, many months if not a year or more, most people will not have to face a choice about vaccinating. The supply just won't be there for the general public. It is a small sample of high-risk health care workers including vaccine manufacturing employees and shippers, plus essential workers to keep hospitals and nursing homes going, who will be first in line. Odds are you and your family will still be wearing masks and social distancing well into next year.
Herman Taylor, director of the cardiovascular research institute at Morehouse college, got in touch with UnitedHealth Group early in the pandemic.
The very people who most require solutions to COVID are those who are least likely to be involved in the search to find them.
A colleague he worked with at Grady Hospital in Atlanta was the guy when it came to studying sickle cell disease, a recessive genetic disorder that causes red blood cells to harden into half-moon shapes, causing cardiovascular problems. Sickle cell disease is more common in African Americans than it is in Caucasians, in part because having just one gene for the disease, called sickle cell trait, is protective against malaria, which is endemic to much of Africa. Roughly one in 12 African Americans carry sickle cell trait, and Taylor's colleague wondered if this could be one factor affecting differential outcomes for COVID-19.
UnitedHealth Group granted Taylor and his colleague the money to study sickle cell trait in COVID, and then, as they continued working together, they began to ask Taylor his opinion on other topics. As an insurance company, United had realized early in the pandemic that it was sitting on a goldmine of patient data—some 120 million patients' worth—that it could sift through to look for potential COVID treatments.
Their researchers thought they had found one: In a small subset of 14,000 people who'd contracted COVID, there was a group whose bills were paid by Medicare (which the researchers took as a proxy for older age). The people in this group who were taking ACE inhibitors, blood vessel dilators often used to treat high blood pressure, were 40 percent less likely to be hospitalized than those who were not taking the drug.
The connection between ACE inhibitors and COVID hospitalizations was a correlation, a statistical association. To determine whether the drugs had any real effect on COVID outcomes, United would have to perform another, more rigorous study. They would have to assign some people to receive small doses of ACE inhibitors, and others to receive placebos, and measure the outcomes under each condition. They planned to do this virtually, allowing study participants to sign up and be screened online, and sending drugs, thermometers, and tests through the mail. There were two reasons to do it this way: First, the U.S. Food and Drug Administration had been advising medical researchers to embrace new strategies in clinical trials as a way to protect participants during the pandemic.
The second reason was why they asked Herman Taylor to co-supervise it: Clinical trials have long had a diversity problem. And going virtual is a potential solution.
Since the beginning of the pandemic, COVID-19 has infected people of color at a rate of three times that of Caucasians (killing Black people at a rate 2.5 times as high, and Hispanic and American Indian or Alaska Native people at a rate 1.3 times as high). A number of explanations have been put forth to explain this disproportionate toll. Among them: higher levels of poverty, essential jobs that increase exposure, and lower quality or inadequate access to medical care.
Unfortunately, these same factors also affect who participates in research. People of color may be less likely to have doctors recommend studies to them. They may not have the time or the resources to hang out in a waiting room for hours. They may not live near large research institutions that conduct trials. The result is that new treatments, even for diseases that affect Latin, Native American, or African American populations in greater proportions, are studied mostly in white volunteers. The very people who most require solutions to COVID are those who are least likely to be involved in the search to find them.
Virtual trials can alleviate a number of these problems. Not only can people find and request to participate in these types of trials through their phones or computers, virtual trials also cover more costs, include a larger geographical range, and have inherently flexible hours.
"[In a traditional study] you have to go to a doctor's office to enroll and drive a couple of hours and pay $20 for parking and pay $15 for a sandwich in the hospital cafeteria and arrange for daycare for your kids and take time off of work," says Dr. Jonathan Cotliar, chief medical officer of Science37, a platform that investigators can hire to host and organize their trials virtually. "That's a lot just for one visit, much less over the course of a six to 12-month study."
Cotliar's data suggests that virtual trials' enhanced access seriously affects the racial makeup of a given study's participant pool. Sixty percent of patients enrolled in Science37 trials are non-Caucasian, which is, Cotliar says, "staggering compared to what you find in traditional site-based research."
But access is not the only barrier to including more people of color in clinical trials. There is also trust. When agreeing to sign up for research, undocumented immigrants may worry about finding themselves in legal trouble or without any medical support should something go wrong. In a country with a history of experimenting on African Americans without their consent, black people may not trust institutions not to use them as guinea pigs.
"A lot of people report being somewhat disregarded or disrespected once entering the healthcare system," Taylor says. "You take it all together, then people wonder, well, okay, this is how the system tends to regard me, yet now here come these people doing research, and they're all about getting me into their studies." Not so surprising that a lot of people may respond with a resounding "No thanks."
United's ACE inhibitor trial was notable for addressing both of these challenges. In addition to covering costs and allowing study subjects to participate from their own homes, it was being co-sponsored by a professor at Morehouse, one of the country's historic black colleges and universities (often abbreviated HBCUs). United was recruiting heavily in Atlanta, whose population is 52 percent African American. The study promised a thoughtful introduction to a more egalitarian future of medical research.
There's just one problem: It isn't going to happen.
This month, in preparation for the study, United reanalyzed their ACE inhibitor data with all the new patients who'd contracted COVID in the months since their first analysis. Their original data set had been concentrated in the Northeast, mostly New York City, where the earliest outbreak took place. In the 12 weeks it had taken them to set up the virtual followup study, epicenters had shifted. United's second, more geographically comprehensive sample had ten times the number of people in it. And in that sample, the signal simply disappeared.
"I was shocked, but that's the reality," says Deneen Vojta, executive vice president of enterprise research and development for UnitedHealth Group. "You make decisions based on the data, but when you get more data, more information, you might make a different decision. The answer is the answer."
There was no point in running a virtual ACE inhibitor study if a larger, more representative sample of people indicated the drug was unlikely to help anyone. Still, the model United had established to run the trial remains viable. Even as she scrapped the ACE inhibitor study, Vojta hoped not just to continue United's relationship with Dr. Taylor and Morehouse, but to formalize it. Virtual platforms are still an important part of their forthcoming trials.
If people don't believe a vaccine has been created with them in mind, then they won't take it, and it won't matter whether it exists or not.
United is not alone in this approach. As phase three trials for vaccines against SARS CoV-2 get underway, big pharma companies have been publicly articulating their own strategies for including more people of color in clinical trials, and many of these include virtual elements. Janelle Sabo, global head of clinical innovation, systems and clinical supply chain at Eli Lilly, told me that the company is employing home health and telemedicine, direct-to-patient shipping and delivery, and recruitment using social media and geolocation to expand access to more diverse populations.
Dr. Macaya Douoguih, Head of Clinical Development and Medical Affairs for Janssen Vaccines under Johnson & Johnson, spoke to Congress about this issue during a July hearing before the House Energy and Commerce Oversight and Investigations Subcommittee. She said that the company planned to institute a "focused digital and community outreach plan to provide resources and opportunities to encourage participation in our clinical trials," and had partnered with Johns Hopkins Bloomberg School of Public Health "to understand how the COVID-19 crisis is affecting different communities in the United States."
But while some of these plans are well thought-out, others are concerningly nebulous, featuring big pronouncements but fewer tangible strategies. In that same July hearing, Massachusetts representative Joe Kennedy III (D) sounded like a frustrated teacher when admonishing four of the five leads of the present pharma companies (AstraZeneca, Johnson & Johnson, Merck, Moderna, and Pfizer) for not explaining exactly how they'd ensure diversity both in the study of their vaccines, and in their eventual distribution.
This matters: The uptake of the flu vaccine is 10 percentage points lower in both the African American and Hispanic communities than it is in Caucasians. A Pew research study conducted early in the pandemic found that just 54 percent of Black adults said they "would definitely or probably get a coronavirus vaccine," compared to 74 percent of Whites and Hispanics.
"As a good friend of mine, Dr. [James] Hildreth, president at Meharry, another HBC medical school, likes to say: 'A vaccine is great, but it is the vaccination that saves people,'" Taylor says. If people don't believe a vaccine has been created with them in mind, then they won't take it, and it won't matter whether it exists or not.
In this respect, virtual platforms remain an important first step, at least in expanding admittance. In June, United Health opened up a trial to their entire workforce for a computer game that could treat ADHD. In less than two months, 1,743 people had signed up for it, from all different socioeconomic groups, from all over the country. It was inching closer to the kind of number you need for a phase three vaccine trial, which can require tens of thousands of people. Back when they'd been planning the ACE inhibitor study, United had wanted 9,600 people to agree to participate.
Now, with the help of virtual enrollment, they hope they can pull off similarly high numbers for the COVID vaccine trial they will be running for an as-yet-unnamed pharmaceutical partner. It stands to open in September.