Who’s Responsible If a Scientist’s Work Is Used for Harm?
Are scientists morally responsible for the uses of their work? To some extent, yes. Scientists are responsible for both the uses that they intend with their work and for some of the uses they don't intend. This is because scientists bear the same moral responsibilities that we all bear, and we are all responsible for the ends we intend to help bring about and for some (but not all) of those we don't.
To not think about plausible unintended effects is to be negligent -- and to recognize, but do nothing about, such effects is to be reckless.
It should be obvious that the intended outcomes of our work are within our sphere of moral responsibility. If a scientist intends to help alleviate hunger (by, for example, breeding new drought-resistant crop strains), and they succeed in that goal, they are morally responsible for that success, and we would praise them accordingly. If a scientist intends to produce a new weapon of mass destruction (by, for example, developing a lethal strain of a virus), and they are unfortunately successful, they are morally responsible for that as well, and we would blame them accordingly. Intention matters a great deal, and we are most praised or blamed for what we intend to accomplish with our work.
But we are responsible for more than just the intended outcomes of our choices. We are also responsible for unintended but readily foreseeable uses of our work. This is in part because we are all responsible for thinking not just about what we intend, but also what else might follow from our chosen course of action. In cases where severe and egregious harms are plausible, we should act in ways that strive to prevent such outcomes. To not think about plausible unintended effects is to be negligent -- and to recognize, but do nothing about, such effects is to be reckless. To be negligent or reckless is to be morally irresponsible, and thus blameworthy. Each of us should think beyond what we intend to do, reflecting carefully on what our course of action could entail, and adjusting our choices accordingly.
It is this area, of unintended but readily foreseeable (and plausible) impacts, that often creates the most difficulty for scientists. Many scientists can become so focused on their work (which is often demanding) and so focused on achieving their intended goals, that they fail to stop and think about other possible implications.
Debates over "dual-use" research exemplify these concerns, where harmful potential uses of research might mean the work should not be pursued, or the full publication of results should be curtailed. When researchers perform gain-of-function research, pushing viruses to become more transmissible or more deadly, it is clear how dangerous such work could be in the wrong hands. In these cases, it is not enough to simply claim that such uses were not intended and that it is someone else's job to ensure that the materials remain secure. We know securing infectious materials can be error-prone (recall events at the CDC and the FDA).
In some areas of research, scientists are already worrying about the unintended possible downsides of their work.
Further, securing viral strains does nothing to secure the knowledge that could allow for reproducing the viral strain (particularly when the methodologies and/or genetic sequences are published after the fact, as was the case for H5N1 and horsepox). It is, in fact, the researcher's moral responsibility to be concerned not just about the biosafety controls in their own labs, but also which projects should be pursued (Will the gain in knowledge be worth the possible downsides?) and which results should be published (Will a result make it easier for a malicious actor to deploy a new bioweapon?).
We have not yet had (to my knowledge) a use of gain-of-function research to harm people. If that does happen, those who actually released the virus on the public will be most blameworthy–-intentions do matter. But the scientists who developed the knowledge deployed by the malicious actors may also be held blameworthy, especially if the malicious use was easy to foresee, even if it was not pleasant to think about.
In some areas of research, scientists are already worrying about the unintended possible downsides of their work. Scientists investigating gene drives have thought beyond the immediate desired benefits of their work (e.g. reducing invasive species populations) and considered the possible spread of gene drives to untargeted populations. Modeling the impacts of such possibilities has led some researchers to pull back from particular deployment possibilities. It is precisely such thinking through both the intended and unintended possible outcomes that is needed for responsible work.
The world has gotten too small, too vulnerable for scientists to act as though they are not responsible for the uses of their work, intended or not. They must seek to ensure that, as the recent AAAS Statement on Scientific Freedom and Responsibility demands, their work is done "in the interest of humanity." This requires thinking beyond one's intentions, potentially drawing on the expertise of others, sometimes from other disciplines, to help explore implications. The need for such thinking does not guarantee good outcomes, but it will ensure that we are doing the best we can, and that is what being morally responsible is all about.
Massive benefits of AI come with environmental and human costs. Can AI itself be part of the solution?
The recent explosion of generative artificial intelligence tools like ChatGPT and Dall-E enabled anyone with internet access to harness AI’s power for enhanced productivity, creativity, and problem-solving. With their ever-improving capabilities and expanding user base, these tools proved useful across disciplines, from the creative to the scientific.
But beneath the technological wonders of human-like conversation and creative expression lies a dirty secret—an alarming environmental and human cost. AI has an immense carbon footprint. Systems like ChatGPT take months to train in high-powered data centers, which demand huge amounts of electricity, much of which is still generated with fossil fuels, as well as water for cooling. “One of the reasons why Open AI needs investments [to the tune of] $10 billion from Microsoft is because they need to pay for all of that computation,” says Kentaro Toyama, a computer scientist at the University of Michigan. There’s also an ecological toll from mining rare minerals required for hardware and infrastructure. This environmental exploitation pollutes land, triggers natural disasters and causes large-scale human displacement. Finally, for data labeling needed to train and correct AI algorithms, the Big Data industry employs cheap and exploitative labor, often from the Global South.
Generative AI tools are based on large language models (LLMs), with most well-known being various versions of GPT. LLMs can perform natural language processing, including translating, summarizing and answering questions. They use artificial neural networks, called deep learning or machine learning. Inspired by the human brain, neural networks are made of millions of artificial neurons. “The basic principles of neural networks were known even in the 1950s and 1960s,” Toyama says, “but it’s only now, with the tremendous amount of compute power that we have, as well as huge amounts of data, that it’s become possible to train generative AI models.”
Though there aren’t any official figures about the power consumption or emissions from data centers, experts estimate that they use one percent of global electricity—more than entire countries.
In recent months, much attention has gone to the transformative benefits of these technologies. But it’s important to consider that these remarkable advances may come at a price.
AI’s carbon footprint
In their latest annual report, 2023 Landscape: Confronting Tech Power, the AI Now Institute, an independent policy research entity focusing on the concentration of power in the tech industry, says: “The constant push for scale in artificial intelligence has led Big Tech firms to develop hugely energy-intensive computational models that optimize for ‘accuracy’—through increasingly large datasets and computationally intensive model training—over more efficient and sustainable alternatives.”
Though there aren’t any official figures about the power consumption or emissions from data centers, experts estimate that they use one percent of global electricity—more than entire countries. In 2019, Emma Strubell, then a graduate researcher at the University of Massachusetts Amherst, estimated that training a single LLM resulted in over 280,000 kg in CO2 emissions—an equivalent of driving almost 1.2 million km in a gas-powered car. A couple of years later, David Patterson, a computer scientist from the University of California Berkeley, and colleagues, estimated GPT-3’s carbon footprint at over 550,000 kg of CO2 In 2022, the tech company Hugging Face, estimated the carbon footprint of its own language model, BLOOM, as 25,000 kg in CO2 emissions. (BLOOM’s footprint is lower because Hugging Face uses renewable energy, but it doubled when other life-cycle processes like hardware manufacturing and use were added.)
Luckily, despite the growing size and numbers of data centers, their increasing energy demands and emissions have not kept pace proportionately—thanks to renewable energy sources and energy-efficient hardware.
But emissions don’t tell the full story.
AI’s hidden human cost
“If historical colonialism annexed territories, their resources, and the bodies that worked on them, data colonialism’s power grab is both simpler and deeper: the capture and control of human life itself through appropriating the data that can be extracted from it for profit.” So write Nick Couldry and Ulises Mejias, authors of the book The Costs of Connection.
The energy requirements, hardware manufacture and the cheap human labor behind AI systems disproportionately affect marginalized communities.
Technologies we use daily inexorably gather our data. “Human experience, potentially every layer and aspect of it, is becoming the target of profitable extraction,” Couldry and Meijas say. This feeds data capitalism, the economic model built on the extraction and commodification of data. While we are being dispossessed of our data, Big Tech commodifies it for their own benefit. This results in consolidation of power structures that reinforce existing race, gender, class and other inequalities.
“The political economy around tech and tech companies, and the development in advances in AI contribute to massive displacement and pollution, and significantly changes the built environment,” says technologist and activist Yeshi Milner, who founded Data For Black Lives (D4BL) to create measurable change in Black people’s lives using data. The energy requirements, hardware manufacture and the cheap human labor behind AI systems disproportionately affect marginalized communities.
AI’s recent explosive growth spiked the demand for manual, behind-the-scenes tasks, creating an industry described by Mary Gray and Siddharth Suri as “ghost work” in their book. This invisible human workforce that lies behind the “magic” of AI, is overworked and underpaid, and very often based in the Global South. For example, workers in Kenya who made less than $2 an hour, were the behind the mechanism that trained ChatGPT to properly talk about violence, hate speech and sexual abuse. And, according to an article in Analytics India Magazine, in some cases these workers may not have been paid at all, a case for wage theft. An exposé by the Washington Post describes “digital sweatshops” in the Philippines, where thousands of workers experience low wages, delays in payment, and wage theft by Remotasks, a platform owned by Scale AI, a $7 billion dollar American startup. Rights groups and labor researchers have flagged Scale AI as one company that flouts basic labor standards for workers abroad.
It is possible to draw a parallel with chattel slavery—the most significant economic event that continues to shape the modern world—to see the business structures that allow for the massive exploitation of people, Milner says. Back then, people got chocolate, sugar, cotton; today, they get generative AI tools. “What’s invisible through distance—because [tech companies] also control what we see—is the massive exploitation,” Milner says.
“At Data for Black Lives, we are less concerned with whether AI will become human…[W]e’re more concerned with the growing power of AI to decide who’s human and who’s not,” Milner says. As a decision-making force, AI becomes a “justifying factor for policies, practices, rules that not just reinforce, but are currently turning the clock back generations years on people’s civil and human rights.”
Ironically, AI plays an important role in mitigating its own harms—by plowing through mountains of data about weather changes, extreme weather events and human displacement.
Nuria Oliver, a computer scientist, and co-founder and vice-president of the European Laboratory of Learning and Intelligent Systems (ELLIS), says that instead of focusing on the hypothetical existential risks of today’s AI, we should talk about its real, tangible risks.
“Because AI is a transverse discipline that you can apply to any field [from education, journalism, medicine, to transportation and energy], it has a transformative power…and an exponential impact,” she says.
AI's accountability
“At the core of what we were arguing about data capitalism [is] a call to action to abolish Big Data,” says Milner. “Not to abolish data itself, but the power structures that concentrate [its] power in the hands of very few actors.”
A comprehensive AI Act currently negotiated in the European Parliament aims to rein Big Tech in. It plans to introduce a rating of AI tools based on the harms caused to humans, while being as technology-neutral as possible. That sets standards for safe, transparent, traceable, non-discriminatory, and environmentally friendly AI systems, overseen by people, not automation. The regulations also ask for transparency in the content used to train generative AIs, particularly with copyrighted data, and also disclosing that the content is AI-generated. “This European regulation is setting the example for other regions and countries in the world,” Oliver says. But, she adds, such transparencies are hard to achieve.
Google, for example, recently updated its privacy policy to say that anything on the public internet will be used as training data. “Obviously, technology companies have to respond to their economic interests, so their decisions are not necessarily going to be the best for society and for the environment,” Oliver says. “And that’s why we need strong research institutions and civil society institutions to push for actions.” ELLIS also advocates for data centers to be built in locations where the energy can be produced sustainably.
Ironically, AI plays an important role in mitigating its own harms—by plowing through mountains of data about weather changes, extreme weather events and human displacement. “The only way to make sense of this data is using machine learning methods,” Oliver says.
Milner believes that the best way to expose AI-caused systemic inequalities is through people's stories. “In these last five years, so much of our work [at D4BL] has been creating new datasets, new data tools, bringing the data to life. To show the harms but also to continue to reclaim it as a tool for social change and for political change.” This change, she adds, will depend on whose hands it is in.
DNA gathered from animal poop helps protect wildlife
On the savannah near the Botswana-Zimbabwe border, elephants grazed contentedly. Nearby, postdoctoral researcher Alida de Flamingh watched and waited. As the herd moved away, she went into action, collecting samples of elephant dung that she and other wildlife conservationists would study in the months to come. She pulled on gloves, took a swab, and ran it all over the still-warm, round blob of elephant poop.
Sequencing DNA from fecal matter is a safe, non-invasive way to track and ultimately help protect over 42,000 species currently threatened by extinction. Scientists are using this DNA to gain insights into wildlife health, genetic diversity and even the broader environment. Applied to elephants, chimpanzees, toucans and other species, it helps scientists determine the genetic diversity of groups and linkages with other groups. Such analysis can show changes in rates of inbreeding. Populations with greater genetic diversity adapt better to changes and environmental stressors than those with less diversity, thus reducing their risks of extinction, explains de Flamingh, a postdoctoral researcher at the University of Illinois Urbana-Champaign.
Analyzing fecal DNA also reveals information about an animal’s diet and health, and even nearby flora that is eaten. That information gives scientists broader insights into the ecosystem, and the findings are informing conservation initiatives. Examples include restoring or maintaining genetic connections among groups, ensuring access to certain foraging areas or increasing diversity in captive breeding programs.
Approximately 27 percent of mammals and 28 percent of all assessed species are close to dying out. The IUCN Red List of threatened species, simply called the Red List, is the world’s most comprehensive record of animals’ risk of extinction status. The more information scientists gather, the better their chances of reducing those risks. In Africa, populations of vertebrates declined 69 percent between 1970 and 2022, according to the World Wildlife Fund (WWF).
“We put on sterile gloves and use a sterile swab to collect wet mucus and materials from the outside of the dung ball,” says Alida de Flamingh, a postdoctoral researcher at the University of Illinois Urbana-Champaign.
“When people talk about species, they often talk about ecosystems, but they often overlook genetic diversity,” says Christina Hvilsom, senior geneticist at the Copenhagen Zoo. “It’s easy to count (individuals) to assess whether the population size is increasing or decreasing, but diversity isn’t something we can see with our bare eyes. Yet, it’s actually the foundation for the species and populations.” DNA analysis can provide this critical information.
Assessing elephants’ health
“Africa’s elephant populations are facing unprecedented threats,” says de Flamingh, the postdoc, who has studied them since 2009. Challenges include ivory poaching, habitat destruction and smaller, more fragmented habitats that result in smaller mating pools with less genetic diversity. Additionally, de Flamingh studies the microbial communities living on and in elephants – their microbiomes – looking for parasites or dangerous microbes.
Approximately 415,000 elephants inhabit Africa today, but de Flamingh says the number would be four times higher without these challenges. The IUCN Red List reports African savannah elephants are endangered and African forest elephants are critically endangered. Elephants support ecosystem biodiversity by clearing paths that help other species travel. Their very footprints create small puddles that can host smaller organisms such as tadpoles. Elephants are often described as ecosystems’ engineers, so if they disappear, the rest of the ecosystem will suffer too.
There’s a process to collecting elephant feces. “We put on sterile gloves (which we change for each sample) and use a sterile swab to collect wet mucus and materials from the outside of the dung ball,” says de Flamingh. They rub a sample about the size of a U.S. quarter onto a paper card embedded with DNA preservation technology. Each card is air dried and stored in a packet of desiccant to prevent mold growth. This way, samples can be stored at room temperature indefinitely without the DNA degrading.
Earlier methods required collecting dung in bags, which needed either refrigeration or the addition of preservatives, or the riskier alternative of tranquilizing the animals before approaching them to draw blood samples. The ability to collect and sequence the DNA made things much easier and safer.
“Our research provides a way to assess elephant health without having to physically interact with elephants,” de Flamingh emphasizes. “We also keep track of the GPS coordinates of each sample so that we can create a map of the sampling locations,” she adds. That helps researchers correlate elephants’ health with geographic areas and their conditions.
Although de Flamingh works with elephants in the wild, the contributions of zoos in the United States and collaborations in South Africa (notably the late Professor Rudi van Aarde and the Conservation Ecology Research Unit at the University of Pretoria) were key in studying this method to ensure it worked, she points out.
Protecting chimpanzees
Genetic work with chimpanzees began about a decade ago. Hvilsom and her group at the Copenhagen Zoo analyzed DNA from nearly 1,000 fecal samples collected between 2003 and 2018 by a team of international researchers. The goal was to assess the status of the West African subspecies, which is critically endangered after rapid population declines. Of the four subspecies of chimpanzees, the West African subspecies is considered the most at-risk.
In total, the WWF estimates the numbers of chimpanzees inhabiting Africa’s forests and savannah woodlands at between 173,000 and 300,000. Poaching, disease and human-caused changes to their lands are their major risks.
By analyzing genetics obtained from fecal samples, Hvilsom estimated the chimpanzees’ population, ascertained their family relationships and mapped their migration routes.
“One of the threats is mining near the Nimba Mountains in Guinea,” a stronghold for the West African subspecies, Hvilsom says. The Nimba Mountains are a UNESCO World Heritage Site, but they are rich in iron ore, which is used to make the steel that is vital to the Asian construction boom. As she and colleagues wrote in a recent paper, “Many extractive industries are currently developing projects in chimpanzee habitat.”
Analyzing DNA allows researchers to identify individual chimpanzees more accurately than simply observing them, she says. Normally, field researchers would install cameras and manually inspect each picture to determine how many chimpanzees were in an area. But, Hvilsom says, “That’s very tricky. Chimpanzees move a lot and are fast, so it’s difficult to get clear pictures. Often, they find and destroy the cameras. Also, they live in large areas, so you need a lot of cameras.”
By analyzing genetics obtained from fecal samples, Hvilsom estimated the chimpanzees’ population, ascertained their family relationships and mapped their migration routes based upon DNA comparisons with other chimpanzee groups. The mining companies and builders are using this information to locate future roads where they won’t disrupt migration – a more effective solution than trying to build artificial corridors for wildlife.
“The current route cuts off communities of chimpanzees,” Hvilsom elaborates. That effectively prevents young adult chimps from joining other groups when the time comes, eventually reducing the currently-high levels of genetic diversity.
“The mining company helped pay for the genetics work,” Hvilsom says, “as part of its obligation to assess and monitor biodiversity and the effect of the mining in the area.”
Of 50 toucan subspecies, 11 are threatened or near-threatened with extinction because of deforestation and poaching.
Identifying toucan families
Feces aren't the only substance researchers draw DNA samples from. Jeffrey Coleman, a Ph.D. candidate at the University of Texas at Austin relies on blood tests for studying the genetic diversity of toucans---birds species native to Central America and nearby regions. They live in the jungles, where they hop among branches, snip fruit from trees, toss it in the air and catch it with their large beaks. “Toucans are beautiful, charismatic birds that are really important to the ecosystem,” says Coleman.
Of their 50 subspecies, 11 are threatened or near-threatened with extinction because of deforestation and poaching. “When people see these aesthetically pleasing birds, they’re motivated to care about conservation practices,” he points out.
Coleman works with the Dallas World Aquarium and its partner zoos to analyze DNA from blood draws, using it to identify which toucans are related and how closely. His goal is to use science to improve the genetic diversity among toucan offspring.
Specifically, he’s looking at sections of the genome of captive birds in which the nucleotides repeat multiple times, such as AGATAGATAGAT. Called microsatellites, these consecutively-repeating sections can be passed from parents to children, helping scientists identify parent-child and sibling-sibling relationships. “That allows you to make strategic decisions about how to pair (captive) individuals for mating...to avoid inbreeding,” Coleman says.
Jeffrey Coleman is studying the microsatellites inside the toucan genomes.
Courtesy Jeffrey Coleman
The alternative is to use a type of analysis that looks for a single DNA building block – a nucleotide – that differs in a given sequence. Called single nucleotide polymorphisms (SNPs, pronounced “snips”), they are very common and very accurate. Coleman says they are better than microsatellites for some uses. But scientists have already developed a large body of microsatellite data from multiple species, so microsatellites can shed more insights on relations.
Regardless of whether conservation programs use SNPs or microsatellites to guide captive breeding efforts, the goal is to help them build genetically diverse populations that eventually may supplement endangered populations in the wild. “The hope is that the ecosystem will be stable enough and that the populations (once reintroduced into the wild) will be able to survive and thrive,” says Coleman. History knows some good examples of captive breeding success.
The California condor, which had a total population of 27 in 1987, when the last wild birds were captured, is one of them. A captive breeding program boosted their numbers to 561 by the end of 2022. Of those, 347 of those are in the wild, according to the National Park Service.
Conservationists hope that their work on animals’ genetic diversity will help preserve and restore endangered species in captivity and the wild. DNA analysis is crucial to both types of efforts. The ability to apply genome sequencing to wildlife conservation brings a new level of accuracy that helps protect species and gives fresh insights that observation alone can’t provide.
“A lot of species are threatened,” Coleman says. “I hope this research will be a resource people can use to get more information on longer-term genealogies and different populations.”