Researchers Behaving Badly: Known Frauds Are "the Tip of the Iceberg"
Last week, the whistleblowers in the Paolo Macchiarini affair at Sweden's Karolinska Institutet went on the record here to detail the retaliation they suffered for trying to expose a star surgeon's appalling research misconduct.
Scientific fraud of the type committed by Macchiarini is rare, but studies suggest that it's on the rise.
The whistleblowers had discovered that in six published papers, Macchiarini falsified data, lied about the condition of patients and circumvented ethical approvals. As a result, multiple patients suffered and died. But Karolinska turned a blind eye for years.
Scientific fraud of the type committed by Macchiarini is rare, but studies suggest that it's on the rise. Just this week, for example, Retraction Watch and STAT together broke the news that a Harvard Medical School cardiologist and stem cell researcher, Piero Anversa, falsified data in a whopping 31 papers, which now have to be retracted. Anversa had claimed that he could regenerate heart muscle by injecting bone marrow cells into damaged hearts, a result that no one has been able to duplicate.
A 2009 study published in the Public Library of Science (PLOS) found that about two percent of scientists admitted to committing fabrication, falsification or plagiarism in their work. That's a small number, but up to one third of scientists admit to committing "questionable research practices" that fall into a gray area between rigorous accuracy and outright fraud.
These dubious practices may include misrepresentations, research bias, and inaccurate interpretations of data. One common questionable research practice entails formulating a hypothesis after the research is done in order to claim a successful premise. Another highly questionable practice that can shape research is ghost-authoring by representatives of the pharmaceutical industry and other for-profit fields. Still another is gifting co-authorship to unqualified but powerful individuals who can advance one's career. Such practices can unfairly bolster a scientist's reputation and increase the likelihood of getting the work published.
The above percentages represent what scientists admit to doing themselves; when they evaluate the practices of their colleagues, the numbers jump dramatically. In a 2012 study published in the Journal of Research in Medical Sciences, researchers estimated that 14 percent of other scientists commit serious misconduct, while up to 72 percent engage in questionable practices. While these are only estimates, the problem is clearly not one of just a few bad apples.
In the PLOS study, Daniele Fanelli says that increasing evidence suggests the known frauds are "just the 'tip of the iceberg,' and that many cases are never discovered" because fraud is extremely hard to detect.
Essentially everyone wants to be associated with big breakthroughs, and they may overlook scientifically shaky foundations when a major advance is claimed.
In addition, it's likely that most cases of scientific misconduct go unreported because of the high price of whistleblowing. Those in the Macchiarini case showed extraordinary persistence in their multi-year campaign to stop his deadly trachea implants, while suffering serious damage to their careers. Such heroic efforts to unmask fraud are probably rare.
To make matters worse, there are numerous players in the scientific world who may be complicit in either committing misconduct or covering it up. These include not only primary researchers but co-authors, institutional executives, journal editors, and industry leaders. Essentially everyone wants to be associated with big breakthroughs, and they may overlook scientifically shaky foundations when a major advance is claimed.
Another part of the problem is that it's rare for students in science and medicine to receive an education in ethics. And studies have shown that older, more experienced and possibly jaded researchers are more likely to fudge results than their younger, more idealistic colleagues.
So, given the steep price that individuals and institutions pay for scientific misconduct, what compels them to go down that road in the first place? According to the JRMS study, individuals face intense pressures to publish and to attract grant money in order to secure teaching positions at universities. Once they have acquired positions, the pressure is on to keep the grants and publishing credits coming in order to obtain tenure, be appointed to positions on boards, and recruit flocks of graduate students to assist in research. And not to be underestimated is the human ego.
Paolo Macchiarini is an especially vivid example of a scientist seeking not only fortune, but fame. He liberally (and falsely) claimed powerful politicians and celebrities, even the Pope, as patients or admirers. He may be an extreme example, but we live in an age of celebrity scientists who bring huge amounts of grant money and high prestige to the institutions that employ them.
The media plays a significant role in both glorifying stars and unmasking frauds. In the Macchiarini scandal, the media first lifted him up, as in NBC's laudatory documentary, "A Leap of Faith," which painted him as a kind of miracle-worker, and then brought him down, as in the January 2016 documentary, "The Experiments," which chronicled the agonizing death of one of his patients.
Institutions can also play a crucial role in scientific fraud by putting more emphasis on the number and frequency of papers published than on their quality. The whole course of a scientist's career is profoundly affected by something called the h-index. This is a number based on both the frequency of papers published and how many times the papers are cited by other researchers. Raising one's ranking on the h-index becomes an overriding goal, sometimes eclipsing the kind of patient, time-consuming research that leads to true breakthroughs based on reliable results.
Universities also create a high-pressured environment that encourages scientists to cut corners. They, too, place a heavy emphasis on attracting large monetary grants and accruing fame and prestige. This can lead them, just as it led Karolinska, to protect a star scientist's sloppy or questionable research. According to Dr. Andrew Rosenberg, who is director of the Center for Science and Democracy at the U.S.-based Union of Concerned Scientists, "Karolinska defended its investment in an individual as opposed to the long-term health of the institution. People were dying, and they should have outsourced the investigation from the very beginning."
Having institutions investigate their own practices is a conflict of interest from the get-go, says Rosenberg.
Scientists, universities, and research institutions are also not immune to fads. "Hot" subjects attract grant money and confer prestige, incentivizing scientists to shift their research priorities in a direction that garners more grants. This can mean neglecting the scientist's true area of expertise and interests in favor of a subject that's more likely to attract grant money. In Macchiarini's case, he was allegedly at the forefront of the currently sexy field of regenerative medicine -- a field in which Karolinska was making a huge investment.
The relative scarcity of resources intensifies the already significant pressure on scientists. They may want to publish results rapidly, since they face many competitors for limited grant money, academic positions, students, and influence. The scarcity means that a great many researchers will fail while only a few succeed. Once again, the temptation may be to rush research and to show it in the most positive light possible, even if it means fudging or exaggerating results.
Though the pressures facing scientists are very real, the problem of misconduct is not inevitable.
Intense competition can have a perverse effect on researchers, according to a 2007 study in the journal Science of Engineering and Ethics. Not only does it place undue pressure on scientists to succeed, it frequently leads to the withholding of information from colleagues, which undermines a system in which new discoveries build on the previous work of others. Researchers may feel compelled to withhold their results because of the pressure to be the first to publish. The study's authors propose that more investment in basic research from governments could alleviate some of these competitive pressures.
Scientific journals, although they play a part in publishing flawed science, can't be expected to investigate cases of suspected fraud, says the German science blogger Leonid Schneider. Schneider's writings helped to expose the Macchiarini affair.
"They just basically wait for someone to retract problematic papers," he says.
He also notes that, while American scientists can go to the Office of Research Integrity to report misconduct, whistleblowers in Europe have no external authority to whom they can appeal to investigate cases of fraud.
"They have to go to their employer, who has a vested interest in covering up cases of misconduct," he says.
Science is increasingly international. Major studies can include collaborators from several different countries, and he suggests there should be an international body accessible to all researchers that will investigate suspected fraud.
Ultimately, says Rosenberg, the scientific system must incorporate trust. "You trust co-authors when you write a paper, and peer reviewers at journals trust that scientists at research institutions like Karolinska are acting with integrity."
Without trust, the whole system falls apart. It's the trust of the public, an elusive asset once it has been betrayed, that science depends upon for its very existence. Scientific research is overwhelmingly financed by tax dollars, and the need for the goodwill of the public is more than an abstraction.
The Macchiarini affair raises a profound question of trust and responsibility: Should multiple co-authors be held responsible for a lead author's misconduct?
Karolinska apparently believes so. When the institution at last owned up to the scandal, it vindictively found Karl Henrik-Grinnemo, one of the whistleblowers, guilty of scientific misconduct as well. It also designated two other whistleblowers as "blameworthy" for their roles as co-authors of the papers on which Macchiarini was the lead author.
As a result, the whistleblowers' reputations and employment prospects have become collateral damage. Accusations of research misconduct can be a career killer. Research grants dry up, employment opportunities evaporate, publishing becomes next to impossible, and collaborators vanish into thin air.
Grinnemo contends that co-authors should only be responsible for their discrete contributions, not for the data supplied by others.
"Different aspects of a paper are highly specialized," he says, "and that's why you have multiple authors. You cannot go through every single bit of data because you don't understand all the parts of the article."
This is especially true in multidisciplinary, translational research, where there are sometimes 20 or more authors. "You have to trust co-authors, and if you find something wrong you have to notify all co-authors. But you couldn't go through everything or it would take years to publish an article," says Grinnemo.
Though the pressures facing scientists are very real, the problem of misconduct is not inevitable. Along with increased support from governments and industry, a change in academic culture that emphasizes quality over quantity of published studies could help encourage meritorious research.
But beyond that, trust will always play a role when numerous specialists unite to achieve a common goal: the accumulation of knowledge that will promote human health, wealth, and well-being.
[Correction: An earlier version of this story mistakenly credited The New York Times with breaking the news of the Anversa retractions, rather than Retraction Watch and STAT, which jointly published the exclusive on October 14th. The piece in the Times ran on October 15th. We regret the error.]
Niklas Anzinger is the founder of Infinita VC based in the charter city of Prospera in Honduras. Infinita focuses on a new trend of charter cities and other forms of alternative jurisdictions. Healso hosts a podcast about how to accelerate the future by unblocking “stranded technologies”.This spring he was a part of the network city experiment Zuzalu spearheaded by Ethereum founder Vitalik Buterin where a few hundred invited guests from the spheres of longevity, biotechnology, crypto, artificial intelligence and investment came together to form a two-monthlong community. It has been described as the world’s first pop-up city. Every morning Vitalians would descend on a long breakfast—the menu had been carefully designed by famed radical longevity self-experimenter Bryan Johnson—and there is where I first met Anzinger who told me about Prospera. Intrigued to say the least, I caught up with him later the same week and the following is a record of our conversation.
Q. We are sitting here in the so-called pop-up network state Zuzalu temporarily realized in the village of Lusticia Bay by the beautiful Mediterranean Sea. To me this is an entirely new concept: What is a network state?
A. A network state is a highly aligned online community that has a level of in-person civility; it crowd-funds territory, and it eventually seeks diplomatic recognition. In a way it's about starting a new country. The term was coined by the crypto influencer and former CTO of Coinbase Balaji Srinivasan in a book by the same title last year [2022]. What many people don't know is that it is a more recent addition or innovation in a space called competitive governance. The idea is that you have multiple jurisdictions competing to provide you services as a customer. When you have competition among governments or government service providers, these entities are forced to provide you with a better service instead of the often worse service at higher prices or higher taxes that we're currently getting. The idea went from seasteading, which was hardly feasible because of costs, to charter cities getting public/private partnerships with existing governments and a level of legal autonomy, to special economic zones, to now network states.
Q. How do network states compare to charter cities and similar jurisdictions?
A. Charter cities and special economic zones were legal forks from other existing states. Dubai, Shenzhen in China, to some degree Hong Kong, to some degree Singapore are some examples. There's a host of other charter cities, one of which I'm based in myself, which is Prospera located in Honduras on the island Roatán. Charter cities provide the full stack of governance; they provide new laws and regulations, business registration, tax codes and governance services, Estonia style: you log on to the government platform and you get services as a citizen.
When conceptualizing network states, Balagi Srinivasan turns the idea of a charter city a bit on its head: he doesn't want to start with this full stack because it's still very hard to get these kinds of partnerships with government. It's very expensive and requires lots of experience and lots of social capital. He is saying that network states could instead start as an online community. They could have a level of alignment where they trade with each other; they have their own economy; they meet in person in regular gatherings like we're doing here in Zuzulu for two months, and then they negotiate with existing governments or host cities to get a certain degree of legal autonomy that is centered around a moral innovation. So, his idea is: don't focus on building a completely new country or city; focus on a moral innovation.
Q. What would be an example of such a moral innovation?
A. An example would be longevity—life is good; death is bad—let's see what we can do to foster progress around that moral innovation and see how we can get legal forks from the existing system that allow us to accelerate progress in that area. There is an increasing realization in the science that there are hallmarks of aging and that aging is a cause of other diseases like cancer, ALS or Alzheimer's. But aging is not recognized as a disease by the FDA in the United States and in most countries around the world, so it's very hard to get scientific funding for biotechnology that would attack the hallmarks of aging and allow us potentially to reverse aging and extend life. This is a significant shortcoming of existing government systems that groups such as the ones that have come together here in Montenegro are now seeking alternatives too. Charter cities and now network states are such alternatives.
Q. Would it not be better to work within the current systems, and try to improve them, rather than abandon them for new experimental jurisdictions?
A. There are numerous failures of public policies. These failures are hard, if not impossible, to reverse, because as soon as you have these policies, you have entrenched interests who benefit from the regulations. The only way to disrupt incumbent industries is with start-ups, but the way the system is set up makes it excessively hard for such start-ups to become big companies. In fact, larger companies are weaponizing the legal system against small companies, because they can afford the lawyers and the fixed cost of compliance.
I don't believe that our institutions in many developed countries are beyond hope. I just think it's easier to change them if you could point at successful examples. ‘Hey, this country or this zone is already doing it very successfully’; if they can extend people’s lifespan by 10 years, if they can reduce maternal mortality, and if they have a massive medical tourism where people come back healthier, then that is just very embarrassing for the FDA.
Q. Perhaps a comparison here would be the relationship between Hong Kong and China?
A. Correct, so having Hong Kong right in front of your door … ‘Hey, this capitalism thing seems to work, why don't we try it here?’ It was due to the very bold leadership by Deng Xiaoping that they experimented with it in the development zone of Shenzhen. It worked really well and then they expanded with more special economic zones that also worked.
Próspera is a private city and special economic zone on the island of Roatán in the Central American state of Honduras.
Q. Tell us about Prospera, the charter city in Honduras, that you are intimately connected with.
A. Honduras is a very poor country. It has a lot of crime, never had a single VC investment, and has a GDP per capita of 2,000 per year. Honduras has suffered tremendously. The goal of these special economic zones is to bring in economic development. That's their sole purpose. It's a homegrown innovation from Honduras that started in 2009 with a very forward-thinking statesman, Octavio Sanchez, who was the chief of staff to the president of Honduras, and then president. He had his own ideas about making Honduras a more decentralized system, where more of the power lies in the municipalities.
Inspired by the ideas of Nobel laureate economist Paul Romer, who gave a famous Ted Talk in 2009 about charter cities, Sanchez initiated a process that lasted for years and eventually led to the creation of a special economic zone legal regime that’s anchored in the Hunduran constitution that provides the highest legal autonomy in the world to these zones. There are today three special economic zones approved by the Honduran government: Prospera, Ciudad Morazan and Orchidea.
Q. How did you become interested and then involved in Prospera?
A. I read about it first in an article by Scott Alexander, a famous rationalist blogger, who wrote a very long article about Prospera, and I thought, this is amazing! Then I came to Prospera and I found it to be one of the most if not the most exciting project in the world going on right now and that it also opened my heart to the country and its people. Most of my friends there are Honduran, they have been working on this for 10 or more years. They want to remake Honduras and put it on the map as the place in the world where this legal and governance innovation started.
Q. To what extent is Prospera autonomous relative to the Honduran government?
A. What's interesting about the Honduran model is that it's anchored within the Honduran constitution, and it has a very clear framework for what's possible and what's not possible, and what's possible ensures the highest degree of legal autonomy anywhere seen in the world. Prospera has really pushed the model furthest in creating a common law-based polycentric legal system. The idea is that you don't have a legislature, instead you have common law and it's based on the best practice common law principles that a legal scholar named Tom W. Bell created.
One of the core ideas is that as a business you're not obligated to follow one regulatory monopoly like the FDA. You have regulatory flexibility so you can choose what you're regulated under. So, you can say: ‘if I do a medical clinic, I do it under Norwegian law here’. And you even have the possibility to amend it a bit. You're still required to have liability insurance, and have to agree to binding arbitration in case there's a legal dispute. And your insurance has to approve you. So, under that model the insurance becomes the regulator and they regulate through prices. The limiting factor is criminal law; Honduran criminal law fully applies. So does immigration law. And we pay taxes.
Q. Is there also an idea of creating a kind of healthy living there, and encourage medical tourism?
A. Yes, we specifically look for legal advantages in autonomy around creating new drugs, doing clinical trials, doing self-medication and experimentation. There is a stem cell clinic here and they're doing clinical trials. The island of Roatán is very easily accessible for American tourists. It's a beautiful island, and it's for regulatory reasons hard to do stem cell therapies in the United States, so they're flying in patients from the United States. Most of them are very savvy and often have PhDs in biotech and are able to assess the risk for themselves of taking drugs and doing clinical trials. We're also going to get a wellness center, and there have been ideas around establishing a peptide clinic and a compound pharmacy and things like that. We are developing a healthcare ecosystem.
Q. This kind of experimental tourism raises some ethical issues. What happens if patients are harmed? And what are the moral implications for society of these new treatments?
A. As a moral principle we believe in medical freedom: people have rights over their bodies, even at the (informed) risk of harm to themselves if no unconsenting third-parties are harmed; this is a fundamental right currently not protected effectively.
What we do differently is not changing ethical norms around safety and efficacy, we’re just changing the institutional setup. Instead of one centralized bureaucracy, like the FDA, we have regulatory pluralism that allows different providers of safety and efficacy to compete under market rules. Like under any legal system, common law in Prospera punishes malpractice, fraud, murder etc. This system will still produce safe and effective drugs, and it will still work with common sense legal notions like informed consent and liability for harm. There are regulations for medical practice, there is liability insurance and things like that. It will just do so more efficiently than the current way of doing things (unless it won’t, in which case it will change and evolve – or fail).
A direct moral benefit ´to what we do is that we increase accessibility. Typical gene therapies on the market cost $1 million dollars in the US. The gene therapy developed in Prospera costs $25,000. As to concern about whether such treatments are problematic, we do not share this perspective. We are for advancing science responsibly and we believe that both individuals and society stand to gain from improving the resiliency of the human body through advanced biotechnology.
Q. How does Prospera relate to the local Honduran population?
A. I think it's very important that our projects deliver local benefits and that they're well anchored in local communities. Because when you go to a new place, you're seen as a foreigner, and you're seen as potentially a danger or a threat. The most important thing for Prospera and Ciudad Morazan is to show we're creating jobs; we're creating employment; we're improving people's lives on the ground. Prospera is directly and indirectly employing 1,100 people. More than 2/3 of the people who are working for Prospera are Honduran. It has a lot of local service workers from the island, and it has educated Hondurans from the mainland for whom it's an alternative to going to the United States.
Q. What makes a good Prosperian citizen?
A. People in Prospera are very entrepreneurial. They're opening companies on a small scale. For example, Vehinia, who is the cook in the kitchen at Prospera, she's from the neighboring village and she started an NGO that is now funding a school where children from the local village can go to instead of a school that's 45 minutes away. There's very much a spirit of ‘let's exchange and trade with each other’. Some people might see that as a bit too commercial, but that's something about the culture that people accept and that people see as a good thing.
Q. Five years from now, if everything goes well, what do we see in Prospera?
A. I think Prospera will have at least 10,000 residents and I think Honduras hopefully will have more zones. There could be zones with a thriving industrial sector and sort of a labor-intensive economy and some that are very strong in pharmaceuticals, there could also be other zones for synthetic biology, and other zones focused on agriculture. The zones of Prospera, Ciudad Morazan and Orchidea are already showing the results we want to see, the results that we will eventually be measured by, and I'm tremendously excited about Honduras.
How to Measure Your Stress, with Dr. Rosalind Picard
Today’s podcast guest is Rosalind Picard, a researcher, inventor named on over 100 patents, entrepreneur, author, professor and engineer. When it comes to the science related to endowing computer software with emotional intelligence, she wrote the book. It’s published by MIT Press and called Affective Computing.
Dr. Picard is founder and director of the MIT Media Lab’s Affective Computing Research Group. Her research and engineering contributions have been recognized internationally. For example, she received the 2022 International Lombardy Prize for Computer Science Research, considered by many to be the Nobel prize in computer science.
Through her research and companies, Dr. Picard has developed wearable sensors, algorithms and systems for sensing, recognizing and responding to information about human emotion. Her products are focused on using fitness trackers to advance clinical quality treatments for a range of conditions.
Meanwhile, in just the past few years, numerous fitness tracking companies have released products with their own stress sensors and systems. You may have heard about Fitbit’s Stress Management Score, or Whoop’s Stress Monitor – these features and apps measure things like your heart rhythm and a certain type of invisible sweat to identify stress. They’re designed to raise awareness about forms of stress such as anxieties and anger, and suggest strategies like meditation to relax in real time when stress occurs.
But how well do these off-the-shelf gadgets work? There’s no one more knowledgeable and experienced than Rosalind Picard to explain the science behind these stress features, what they do exactly, how they might be able to help us, and their current shortcomings.
Dr. Picard is a member of the National Academy of Engineering and a Fellow of the National Academy of Inventors, and a popular speaker who’s given over a hundred invited keynote talks and a TED talk with over 2 million views. She holds a Bachelors in Electrical Engineering from Georgia Tech, and Masters and Doctorate degrees in Electrical Engineering and Computer Science from MIT. She lives in Newton, Massachusetts with her husband, where they’ve raised three sons.
In our conversation, we discuss stress scores on fitness trackers to improve well-being. She describes the difference between commercial products that might help people become more mindful of their health and products that are FDA approved and really capable of advancing the science. We also talk about several fascinating findings and concepts discovered in Dr. Picard’s lab including the multiple arousal theory, a phenomenon you’ll want to hear about. And we explore the complexity of stress, one reason it’s so tough to measure. For example, many forms of stress are actually good for us. Can fitness trackers tell the difference between stress that’s healthy and unhealthy?
Show links:
- Dr. Picard’s book, Affective Computing
- Dr. Picard’s bio
- Dr. Picard on Twitter
- Dr. Picard’s company, Empatica - https://www.empatica.com/ - The FDA-cleared Empatica Health Monitoring Platform provides accurate, continuous health insights for researchers and clinicians, collected in the real world
- Empatica Twitter
- Dr. Picard and her team have published hundreds of peer-reviewed articles across AI, Machine Learning, Affective Computing, Digital Health, and Human-computer interaction.
- Dr. Picard’s TED talk
Rosalind Picard