Researchers Behaving Badly: Known Frauds Are "the Tip of the Iceberg"
Last week, the whistleblowers in the Paolo Macchiarini affair at Sweden's Karolinska Institutet went on the record here to detail the retaliation they suffered for trying to expose a star surgeon's appalling research misconduct.
Scientific fraud of the type committed by Macchiarini is rare, but studies suggest that it's on the rise.
The whistleblowers had discovered that in six published papers, Macchiarini falsified data, lied about the condition of patients and circumvented ethical approvals. As a result, multiple patients suffered and died. But Karolinska turned a blind eye for years.
Scientific fraud of the type committed by Macchiarini is rare, but studies suggest that it's on the rise. Just this week, for example, Retraction Watch and STAT together broke the news that a Harvard Medical School cardiologist and stem cell researcher, Piero Anversa, falsified data in a whopping 31 papers, which now have to be retracted. Anversa had claimed that he could regenerate heart muscle by injecting bone marrow cells into damaged hearts, a result that no one has been able to duplicate.
A 2009 study published in the Public Library of Science (PLOS) found that about two percent of scientists admitted to committing fabrication, falsification or plagiarism in their work. That's a small number, but up to one third of scientists admit to committing "questionable research practices" that fall into a gray area between rigorous accuracy and outright fraud.
These dubious practices may include misrepresentations, research bias, and inaccurate interpretations of data. One common questionable research practice entails formulating a hypothesis after the research is done in order to claim a successful premise. Another highly questionable practice that can shape research is ghost-authoring by representatives of the pharmaceutical industry and other for-profit fields. Still another is gifting co-authorship to unqualified but powerful individuals who can advance one's career. Such practices can unfairly bolster a scientist's reputation and increase the likelihood of getting the work published.
The above percentages represent what scientists admit to doing themselves; when they evaluate the practices of their colleagues, the numbers jump dramatically. In a 2012 study published in the Journal of Research in Medical Sciences, researchers estimated that 14 percent of other scientists commit serious misconduct, while up to 72 percent engage in questionable practices. While these are only estimates, the problem is clearly not one of just a few bad apples.
In the PLOS study, Daniele Fanelli says that increasing evidence suggests the known frauds are "just the 'tip of the iceberg,' and that many cases are never discovered" because fraud is extremely hard to detect.
Essentially everyone wants to be associated with big breakthroughs, and they may overlook scientifically shaky foundations when a major advance is claimed.
In addition, it's likely that most cases of scientific misconduct go unreported because of the high price of whistleblowing. Those in the Macchiarini case showed extraordinary persistence in their multi-year campaign to stop his deadly trachea implants, while suffering serious damage to their careers. Such heroic efforts to unmask fraud are probably rare.
To make matters worse, there are numerous players in the scientific world who may be complicit in either committing misconduct or covering it up. These include not only primary researchers but co-authors, institutional executives, journal editors, and industry leaders. Essentially everyone wants to be associated with big breakthroughs, and they may overlook scientifically shaky foundations when a major advance is claimed.
Another part of the problem is that it's rare for students in science and medicine to receive an education in ethics. And studies have shown that older, more experienced and possibly jaded researchers are more likely to fudge results than their younger, more idealistic colleagues.
So, given the steep price that individuals and institutions pay for scientific misconduct, what compels them to go down that road in the first place? According to the JRMS study, individuals face intense pressures to publish and to attract grant money in order to secure teaching positions at universities. Once they have acquired positions, the pressure is on to keep the grants and publishing credits coming in order to obtain tenure, be appointed to positions on boards, and recruit flocks of graduate students to assist in research. And not to be underestimated is the human ego.
Paolo Macchiarini is an especially vivid example of a scientist seeking not only fortune, but fame. He liberally (and falsely) claimed powerful politicians and celebrities, even the Pope, as patients or admirers. He may be an extreme example, but we live in an age of celebrity scientists who bring huge amounts of grant money and high prestige to the institutions that employ them.
The media plays a significant role in both glorifying stars and unmasking frauds. In the Macchiarini scandal, the media first lifted him up, as in NBC's laudatory documentary, "A Leap of Faith," which painted him as a kind of miracle-worker, and then brought him down, as in the January 2016 documentary, "The Experiments," which chronicled the agonizing death of one of his patients.
Institutions can also play a crucial role in scientific fraud by putting more emphasis on the number and frequency of papers published than on their quality. The whole course of a scientist's career is profoundly affected by something called the h-index. This is a number based on both the frequency of papers published and how many times the papers are cited by other researchers. Raising one's ranking on the h-index becomes an overriding goal, sometimes eclipsing the kind of patient, time-consuming research that leads to true breakthroughs based on reliable results.
Universities also create a high-pressured environment that encourages scientists to cut corners. They, too, place a heavy emphasis on attracting large monetary grants and accruing fame and prestige. This can lead them, just as it led Karolinska, to protect a star scientist's sloppy or questionable research. According to Dr. Andrew Rosenberg, who is director of the Center for Science and Democracy at the U.S.-based Union of Concerned Scientists, "Karolinska defended its investment in an individual as opposed to the long-term health of the institution. People were dying, and they should have outsourced the investigation from the very beginning."
Having institutions investigate their own practices is a conflict of interest from the get-go, says Rosenberg.
Scientists, universities, and research institutions are also not immune to fads. "Hot" subjects attract grant money and confer prestige, incentivizing scientists to shift their research priorities in a direction that garners more grants. This can mean neglecting the scientist's true area of expertise and interests in favor of a subject that's more likely to attract grant money. In Macchiarini's case, he was allegedly at the forefront of the currently sexy field of regenerative medicine -- a field in which Karolinska was making a huge investment.
The relative scarcity of resources intensifies the already significant pressure on scientists. They may want to publish results rapidly, since they face many competitors for limited grant money, academic positions, students, and influence. The scarcity means that a great many researchers will fail while only a few succeed. Once again, the temptation may be to rush research and to show it in the most positive light possible, even if it means fudging or exaggerating results.
Though the pressures facing scientists are very real, the problem of misconduct is not inevitable.
Intense competition can have a perverse effect on researchers, according to a 2007 study in the journal Science of Engineering and Ethics. Not only does it place undue pressure on scientists to succeed, it frequently leads to the withholding of information from colleagues, which undermines a system in which new discoveries build on the previous work of others. Researchers may feel compelled to withhold their results because of the pressure to be the first to publish. The study's authors propose that more investment in basic research from governments could alleviate some of these competitive pressures.
Scientific journals, although they play a part in publishing flawed science, can't be expected to investigate cases of suspected fraud, says the German science blogger Leonid Schneider. Schneider's writings helped to expose the Macchiarini affair.
"They just basically wait for someone to retract problematic papers," he says.
He also notes that, while American scientists can go to the Office of Research Integrity to report misconduct, whistleblowers in Europe have no external authority to whom they can appeal to investigate cases of fraud.
"They have to go to their employer, who has a vested interest in covering up cases of misconduct," he says.
Science is increasingly international. Major studies can include collaborators from several different countries, and he suggests there should be an international body accessible to all researchers that will investigate suspected fraud.
Ultimately, says Rosenberg, the scientific system must incorporate trust. "You trust co-authors when you write a paper, and peer reviewers at journals trust that scientists at research institutions like Karolinska are acting with integrity."
Without trust, the whole system falls apart. It's the trust of the public, an elusive asset once it has been betrayed, that science depends upon for its very existence. Scientific research is overwhelmingly financed by tax dollars, and the need for the goodwill of the public is more than an abstraction.
The Macchiarini affair raises a profound question of trust and responsibility: Should multiple co-authors be held responsible for a lead author's misconduct?
Karolinska apparently believes so. When the institution at last owned up to the scandal, it vindictively found Karl Henrik-Grinnemo, one of the whistleblowers, guilty of scientific misconduct as well. It also designated two other whistleblowers as "blameworthy" for their roles as co-authors of the papers on which Macchiarini was the lead author.
As a result, the whistleblowers' reputations and employment prospects have become collateral damage. Accusations of research misconduct can be a career killer. Research grants dry up, employment opportunities evaporate, publishing becomes next to impossible, and collaborators vanish into thin air.
Grinnemo contends that co-authors should only be responsible for their discrete contributions, not for the data supplied by others.
"Different aspects of a paper are highly specialized," he says, "and that's why you have multiple authors. You cannot go through every single bit of data because you don't understand all the parts of the article."
This is especially true in multidisciplinary, translational research, where there are sometimes 20 or more authors. "You have to trust co-authors, and if you find something wrong you have to notify all co-authors. But you couldn't go through everything or it would take years to publish an article," says Grinnemo.
Though the pressures facing scientists are very real, the problem of misconduct is not inevitable. Along with increased support from governments and industry, a change in academic culture that emphasizes quality over quantity of published studies could help encourage meritorious research.
But beyond that, trust will always play a role when numerous specialists unite to achieve a common goal: the accumulation of knowledge that will promote human health, wealth, and well-being.
[Correction: An earlier version of this story mistakenly credited The New York Times with breaking the news of the Anversa retractions, rather than Retraction Watch and STAT, which jointly published the exclusive on October 14th. The piece in the Times ran on October 15th. We regret the error.]
nudgesYou are driving along the highway and see an electronic sign that reads: “3,238 traffic deaths this year.” Do you think this reminder of roadside mortality would change how you drive? According to a recent, peer-reviewed study in Science, seeing that sign would make you more likely to crash. That’s ironic, given that the sign’s creators assumed it would make you safer.
The study, led by a pair of economists at the University of Toronto and University of Minnesota, examined seven years of traffic accident data from 880 electric highway sign locations in Texas, which experienced 4,480 fatalities in 2021. For one week of each month, the Texas Department of Transportation posts the latest fatality messages on signs along select traffic corridors as part of a safety campaign. Their logic is simple: Tell people to drive with care by reminding them of the dangers on the road.
But when the researchers looked at the data, they found that the number of crashes increased by 1.52 percent within three miles of these signs when compared with the same locations during the same month in previous years when signs did not show fatality information. That impact is similar to raising the speed limit by four miles or decreasing the number of highway troopers by 10 percent.
The scientists calculated that these messages contributed to 2,600 additional crashes and 16 deaths annually. They also found a social cost, meaning the financial expense borne by society as a whole due to these crashes, of $377 million per year, in Texas alone.
The cause, they argue, is distracted driving. Much like incoming texts or phone calls, these “in-your-face” messages grab your attention and undermine your focus on the road. The signs are particularly distracting and dangerous because, in communicating that many people died doing exactly what you are doing, they cause anxiety. Supporting this hypothesis, the scientists discovered that crashes increase when the signs report higher numbers of deaths. Thus, later in the year, as that total mortality figure goes up, so do the percentage of crashes.
Boomerang effects happen when those with authority, in government or business, fail to pay attention to the science. These leaders rely on armchair psychology and gut intuitions on what should work, rather than measuring what does work.
That change over time is not simply a function of changing weather, the study’s authors observed. They also found that the increase in car crashes is greatest in more complex road segments, which require greater focus to navigate.
The overall findings represent what behavioral scientists like myself call a “boomerang effect,” meaning an intervention that produces consequences opposite to those intended. Unfortunately, these effects are all too common. Between 1998 and 2004, Congress funded the $1 billion National Youth Anti-Drug Media Campaign, which famously boomeranged. Using professional advertising and public relations firms, the campaign bombarded kids aged 9 to 18 with anti-drug messaging, focused on marijuana, on TV, radio, magazines, and websites. A 2008 study funded by the National Institutes of Health found that children and teens saw these ads two to three times per week. However, more exposure to this advertising increased the likelihood that youth used marijuana. Why? Surveys and interviews suggested that young people who saw the ads got the impression that many of their peers used marijuana. As a result, they became more likely to use the drug themselves.
Boomerang effects happen when those with authority, in government or business, fail to pay attention to the science. These leaders rely on armchair psychology and gut intuitions on what should work, rather than measuring what does work.
To be clear, message campaigns—whether on electronic signs or through advertisements—can have a substantial effect on behavior. Extensive research reveals that people can be influenced by “nudges,” which shape the environment to influence their behavior in a predictable manner. For example, a successful campaign to reduce car accidents involved sending smartphone notifications that helped drivers evaluate their performance after each trip. These messages informed drivers of their personal average and best performance, as measured by accelerometers and gyroscopes. The campaign, which ran over 21 months, significantly reduced accident frequency.
Nudges work best when rigorously tested with small-scale experiments that evaluate their impact. Because behavioral scientists are infrequently consulted in creating these policies, some studies suggest that only 62 percent have a statistically significant effect. Other research reveals that up to 15 percent of desired interventions may backfire.
In the case of roadside mortality signage, the data are damning. The new research based on the Texas signs aligns with several past studies. For instance, research has shown that increasing people’s anxiety causes them to drive worse. Another, a Virginia Tech study in a laboratory setting, found that showing drivers fatality messages increased what psychologists call “cognitive load,” or the amount of information your brain is processing, with emotionally-salient information being especially burdensome and preoccupying, thus causing more distraction.
Nonetheless, Texas, along with at least 28 other states, has pursued mortality messaging campaigns since 2012, without testing them effectively. Behavioral science is critical here: when road signs are tested by people without expertise in how minds work, the results are often counterproductive. For example, the Virginia Tech research looked at road signs that used humor, popular culture, sports, and other nontraditional themes with the goal of provoking an emotional response. When they measured how participants responded to these signs, they noticed greater cognitive activation and attention in the brain. Thus, the researchers decided, the signs worked. But a behavioral scientist would note that increased attention likely contributes to the signs’ failure. As the just-published study in Science makes clear, distracting, emotionally-loaded signs are dangerous to drivers.
But there is good news. First, in most cases, it’s very doable to run an effective small-scale study testing an intervention. States could set up a safety campaign with a few electric signs in a diversity of settings and evaluate the impact over three months on driver crashes after seeing the signs. Policymakers could ask researchers to track the data as they run ads for a few months in a variety of nationally representative markets for a few months and assess their effectiveness. They could also ask behavioral scientists whether their proposals are well designed, whether similar policies have been tried previously in other places, and how these policies have worked in practice.
Everyday citizens can write to and call their elected officials to ask them to make this kind of research a priority before embracing an untested safety campaign. More broadly, you can encourage them to avoid relying on armchair psychology and to test their intuitions before deploying initiatives that might place the public under threat.
Why we should put insects on the menu
I walked through the Dong Makkhai forest-products market, just outside of Vientiane, the laid-back capital of the Lao Peoples Democratic Republic or Lao PDR. Piled on rough display tables were varieties of six-legged wildlife–grasshoppers, small white crickets, house crickets, mole crickets, wasps, wasp eggs and larvae, dragonflies, and dung beetles. Some were roasted or fried, but in a few cases, still alive and scrabbling at the bottom of deep plastic bowls. I crunched on some fried crickets and larvae.
One stall offered Giant Asian hornets, both babies and adults. I suppressed my inner squirm and, in the interests of world food security and equity, accepted an offer of the soft, velvety larva; they were smooth on the tongue and of a pleasantly cool, buttery-custard consistency. Because the seller had already given me a free sample, I felt obliged to buy a chunk of the nest with larvae and some dead adults, which the seller mixed with kaffir lime leaves.
The year was 2016 and I was in Lao PDR because Veterinarians without Borders/Vétérinaires sans Frontières-Canada had initiated a project on small-scale cricket farming. The intent was to organize and encourage rural women to grow crickets as a source of supplementary protein and sell them at the market for cash. As a veterinary epidemiologist, I had been trained to exterminate disease spreading insects—Lyme disease-carrying ticks, kissing bugs that carry American Sleeping Sickness and mosquitoes carrying malaria, West Nile and Zika. Now, as part of a global wave promoting insects as a sustainable food source, I was being asked to view arthropods as micro-livestock, and devise management methods to keep them alive and healthy. It was a bit of a mind-bender.
The 21st century wave of entomophagy, or insect eating, first surged in the early 2010s, promoted by a research centre in Wageningen, a university in the Netherlands, conferences organized by the Food and Agriculture Organization of the United Nations, and enthusiastic endorsements by culinary adventurers and celebrities from Europeanized cultures. Headlines announced that two billion people around the world already ate insects, and that if everyone adopted entomophagy we could reduce greenhouse gases, mitigate climate change, and reign in profligate land and water use associated with industrial livestock production.
Furthermore, eating insects was better for human health than eating beef. If we were going to feed the estimated nine billion people with whom we will share the earth in 2050, we would need to make some radical changes in our agriculture and food systems. As one author proclaimed, entomophagy presented us with a last great chance to save the planet.
In 2010, in Kunming, a friend had served me deep-fried bamboo worms. I ate them to be polite. They tasted like French fries, but with heads.
The more recent data suggests that the number of people who eat insects in various forms, though sizeable, may be closer to several hundreds of millions. I knew that from several decades of international veterinary work. Sometimes, for me, insect eating has been simply a way of acknowledging cultural diversity. In 2010, in Kunming, a friend had served me deep-fried bamboo worms. I ate them to be polite. They tasted like French fries, but with heads. My friend said he preferred them chewier. I never thought about them much after that. I certainly had not thought about them as ingredients for human health.
Is consuming insects good for human health? Researchers over the past decade have begun to tease that apart. Some think it might not be useful to use the all-encompassing term insect at all; we don’t lump cows, pigs, chickens into one culinary category. Which insects are we talking about? What are they fed? Were they farmed or foraged? Which stages of the insects are we eating? Do we eat them directly or roasted and ground up?
The overall research indicates that, in general, the usual farmed insects (crickets, locusts, mealworms, soldier fly larvae) have high levels of protein and other important nutrients. If insects are foraged by small groups in Laos, they provide excellent food supplements. Large scale foraging in response to global markets can be incredibly destructive, but soldier fly larvae fed on food waste and used as a substitute for ground up anchovies for farmed fish (as Enterra Feed in Canada does) improves ecological sustainability.
Entomophagy alone might not save the planet, but it does give us an unprecedented opportunity to rethink how we produce and harvest protein.
The author enjoys insects from the Dong Makkhai forest-products market, just outside of Vientiane, the capital of the Lao Peoples Democratic Republic.
David Waltner-Toews
Between 1961 and 2018, world chicken production increased from 4 billion to 20 billion, pork from 200 million to over 100 billion pigs, human populations doubled from 3.5 billion to more than 7 billion, and life expectancy (on average) from 52 to 72 years. These dramatic increases in food production are the result of narrowly focused scientific studies, identifying specific nutrients, antibiotics, vaccines and genetics. What has been missing is any sort of peripheral vision: what are the unintended consequences of our narrowly defined success?
If we look more broadly, we can see that this narrowly defined success led to industrial farming, which caused wealth, health and labor inequities; polluted the environment; and created grounds for disease outbreaks. Recent generations of Europeanized people inherited the ideas of eating cows, pigs and chickens, along with their products, so we were focused only on growing them as efficiently as possible. With insects, we have an exciting chance to start from scratch. Because, for Europeanized people, insect eating is so strange, we are given the chance to reimagine our whole food system in consultation with local experts in Asia and Africa (many of them villagers), and to bring together the best of both locally adapted food production and global distribution.
For this to happen, we will need to change the dietary habits of the big meat eaters. How can we get accustomed to eating bugs? There’s no one answer, but there are a few ways. In many cases, insects are ground up and added as protein supplements to foods like crackers or bars. In certain restaurants, the chefs want you to get used to seeing the bugs as you eat them. At Le Feston Nu in Paris, the Arlo Guthrie look-alike bartender poured me a beer and brought out five small plates, each featuring a different insect in a nest of figs, sun-dried tomatoes, raisins, and chopped dried tropical fruits: buffalo worms, crickets, large grasshoppers (all just crunchy and no strong flavour, maybe a little nutty), small black ants (sour bite), and fat grubs with a beak, which I later identified as palm weevil larvae, tasting a bit like dried figs.
Some entomophagy advertising has used esthetically pleasing presentations in classy restaurants. In London, at the Archipelago restaurant, I dined on Summer Nights (pan fried chermoula crickets, quinoa, spinach and dried fruit), Love-Bug Salad (baby greens with an accompanying dish of zingy, crunchy mealworms fried in olive oil, chilis, lemon grass, and garlic), Bushman’s Cavi-Err (caramel mealworms, bilinis, coconut cream and vodka jelly), and Medieaval Hive (brown butter ice cream, honey and butter caramel sauce and a baby bee drone).
The Archipelago restaurant in London serves up a Love-Bug Salad: baby greens with an accompanying dish of zingy, crunchy mealworms fried in olive oil, chilis, lemon grass, and garlic.
David Waltner-Toews
Some chefs, like Tokyo-based Shoichi Uchiyama, try to entice people with sidewalk cooking lessons. Uchiyama's menu included hornet larvae, silkworm pupae, and silkworms. The silkworm pupae were white and pink and yellow. We snipped off the ends and the larvae dropped out. My friend Zen Kawabata roasted them in a small pan over a camp stove in the street to get the "chaff" off. We made tea from the feces of worms that had fed on cherry blossoms—the tea smelled of the blossoms. One of Uchiyama-san’s assistants made noodles from buckwheat dough that included powdered whole bees.
At a book reading in a Tokyo bookstore, someone handed me a copy of the Japanese celebrity scandal magazine Friday, opened to an article celebrating the “charms of insect eating.” In a photo, scantily-clad girls were drinking vodka and nibbling giant water bugs dubbed as toe-biters, along with pickled and fried locusts and butterfly larvae. If celebrities embraced bug-eating, others might follow. When asked to prepare an article on entomophagy for the high fashion Sorbet Magazine, I started by describing a clip of Nicole Kidman delicately snacking on insects.
Taking a page from the success story of MacDonald’s, we might consider targeting children and school lunches. Kids don’t lug around the same dietary baggage as the grownups, and they can carry forward new eating habits for the long term. When I offered roasted crickets to my grandchildren, they scarfed them down. I asked my five-year-old granddaughter what she thought: she preferred the mealworms to the crickets – they didn’t have legs that caught in her teeth.
Entomo Farms in Ontario, the province where I live, was described in 2015 by Canadian Business magazine as North America’s largest supplier of edible insects for human consumption. When visiting, I popped some of their roasted crickets into my mouth. They were crunchy, a little nutty. Nothing to get squeamish over. Perhaps the human consumption is indeed growing—my wife, at least, has joined me in my entomophagy adventures. When we celebrated our wedding anniversary at the Public Bar and Restaurant in Brisbane, Australia, the “Kang Kong Worms” and “Salmon, Manuka Honey, and Black Ants” seemed almost normal. Of course, the champagne helped.