In an initial study, Canary analyzed speech recordings with AI and identified early stage Alzheimer’s with 96 percent accuracy.
Dave Arnold retired in his 60s and began spending time volunteering in local schools. But then he started misplacing items, forgetting appointments and losing his sense of direction. Eventually he was diagnosed with early stage Alzheimer’s.
“Hearing the diagnosis made me very emotional and tearful,” he said. “I immediately thought of all my mom had experienced.” His mother suffered with the condition for years before passing away. Over the last year, Arnold has worked for the Alzheimer’s Association as one of its early stage advisors, sharing his insights to help others in the initial stages of the disease.
Arnold was diagnosed sooner than many others. It's important to find out early, when interventions can make the most difference. One promising avenue is looking at how people talk. Research has shown that Alzheimer’s affects a part of the brain that controls speech, resulting in small changes before people show other signs of the disease.
Now, Canary Speech, a company based in Utah, is using AI to examine elements like the pitch of a person’s voice and their pauses. In an initial study, Canary analyzed speech recordings with AI and identified early stage Alzheimer’s with 96 percent accuracy.
Developing the AI model
Canary Speech’s CEO, Henry O’Connell, met cofounder Jeff Adams about 40 years before they started the company. Back when they first crossed paths, they were both living in Bethesda, Maryland; O’Connell was a research fellow at the National Institutes of Health studying rare neurological diseases, while Adams was working to decode spy messages. Later on, Adams would specialize in building mathematical models to analyze speech and sound as a team leader in developing Amazon's Alexa.
It wasn't until 2015 that they decided to make use of the fit between their backgrounds. ““We established Canary Speech in 2017 to build a product that could be used in multiple languages in clinical environments,” O'Connell says.
The need is growing. About 55 million people worldwide currently live with Alzheimer’s, a number that is expected to double by 2050. Some scientists think the disease results from a buildup of plaque in the brain. It causes mild memory loss at first and, over time, this issue get worse while other symptoms, such as disorientation and hallucinations, can develop. Treatment to manage the disease is more effective in the earlier stages, but detection is difficult since mild symptoms are often attributed to the normal aging process.
O’Connell and Adams specialize in the complex ways that Alzheimer’s effects how people speak. Using AI, their mathematical model analyzes 15 million data points every minute, focusing on certain features of speech such as pitch, pauses and elongation of words. It also pays attention to how the vibrations of vocal cords change in different stages of the disease.
To create their model, the team used a type of machine learning called deep neural nets, which looks at multiple layers of data - in this case, the multiple features of a person’s speech patterns.
“Deep neural nets allow us to look at much, much larger data sets built out of millions of elements,” O’Connell explained. “Through machine learning and AI, we’ve identified features that are very sensitive to an Alzheimer’s patient versus [people without the disease] and also very sensitive to mild cognitive impairment, early stage and moderate Alzheimer's.” Based on their learnings, Canary is able to classify the disease stage very quickly, O’Connell said.
“When we’re listening to sublanguage elements, we’re really analyzing the direct result of changes in the brain in the physical body,” O’Connell said. “The brain controls your vocal cords: how fast they vibrate, the expansion of them, the contraction.” These factors, along with where people put their tongues when talking, function subconsciously and result in subtle changes in the sounds of speech.
Further testing is needed
In an initial trial, Canary analyzed speech recordings from phone calls to a large U.S. health insurer. They looked at the audio recordings of 651 policyholders who had early stage Alzheimer’s and 1018 who did not have the condition, aiming for a representative sample of age, gender and race. They used this data to create their first diagnostic model and found that it was 96 percent accurate in identifying Alzheimer’s.
Christian Herff, an assistant professor of neuroscience at Maastricht University in the Netherlands, praised this approach while adding that further testing is needed to assess its effectiveness.
“I think the general idea of identifying increased risk for cognitive impairment based on speech characteristics is very feasible, particularly when change in a user’s voice is monitored, for example, by recording speech every year,” Herff said. He noted that this can only be a first indication, not a full diagnosis. The accuracy still needs to be validated in studies that follows individuals over a period of time, he said.
Toby Walsh, a professor of artificial intelligence at the University of New South Wales, also thinks Canary’s tool has potential but highlights that Canary could diagnose some people who don’t really have the disease. “This is an interesting and promising application of AI,” he said, “but these tools need to be used carefully. Imagine the anxiety of being misdiagnosed with Alzheimer’s.”
As with many other AI tools, privacy and bias are additional issues to monitor closely, Walsh said.
Other languages
A related issue is that not everyone is fluent in English. Mahnaz Arvaneh, a senior lecturer in automatic control and systems engineering at the University of Sheffield, said this could be a blind spot.
“The system may not be very accurate for those who have English as their second language as their speaking patterns would be different, and any issue might be because of language deficiency rather than cognitive issues,” Arvaneh said.
The team is expanding to multiple languages starting with Japanese and Spanish. The elements of the model that make up the algorithm are very similar, but they need to be validated and retrained in a different language, which will require access to more data.
Recently, Canary analyzed the phone calls of 233 Japanese patients who had mild cognitive impairment and 704 healthy people. Using an English model they were able to identify the Japanese patients who had mild cognitive impairment with 78 percent accuracy. They also developed a model in Japanese that was 45 percent accurate, and they’re continuing to train it with more data.
The future
Canary is using their model to look at other diseases like Huntington’s and Parkinson’s. They’re also collaborating with pharmaceuticals to validate potential therapies for Alzheimer’s. By looking at speech patterns over time, Canary can get an indication of how well these drugs are working.
Dave Arnold and his wife dance at his nephew’s wedding in Rochester, New York, ten years ago, before his Alzheimer's diagnosis.
Dave Arnold
Ultimately, they want to integrate their tool into everyday life. “We want it to be used in a smartphone, or a teleconference call so that individuals could be examined in their home,” O’Connell said. “We could follow them over time and work with clinical teams and hospitals to improve the evaluation of patients and contribute towards an accurate diagnosis.”
Arnold, the patient with early stage Alzheimer’s, sees great promise. “The process of getting a diagnosis is already filled with so much anxiety,” he said. “Anything that can be done to make it easier and less stressful would be a good thing, as long as it’s proven accurate.”
A new book by Pulitzer-winning physician-scientist Siddhartha Mukherjee will be released from Simon & Schuster on October 25, 2022.
The Song of the Cell begins with the discovery of cells and of germ theory, featuring characters such as Louis Pasteur and Robert Koch, who brought the cell “into intimate contact with pathology and medicine.” This intercourse would transform biomedicine, leading to the insight that we can treat disease by thinking at the cellular level. The slightest rearrangement of sick cells might be the path toward alleviating suffering for the organism: eroding the cell walls of a bacterium while sparing our human cells; inventing a medium that coaxes sperm and egg to dance into cellular union for in vitro fertilization (IVF); designing molecular missiles that home to the receptors decorating the exterior of cancer cells; teaching adult skin cells to remember their embryonic state for regenerative medicines.
Mukherjee uses the bulk of the book to elucidate key cell types in the human body, along with their “connective relationships” that enable key organs and organ systems to function. This includes the immune system, the heart, the brain, and so on. Mukherjee’s distinctive style features compelling anecdotes and human stories that animate the scientific (and unscientific) processes that have led to our current state of understanding. In his chapter on neurons and the brain, for example, he integrates Santiago Ramon y Cajal’s meticulous black ink sketches of neurons into Mukherjee’s own personal encounter with clinical depression. In one lucid section, he interviews Dr. Helen Mayberg, a pioneering neurologist who takes seriously the descriptive power of her patients’ metaphors, as they suffer from “caves,” “holes,” “voids,” and “force fields” that render their lives gray. Dr. Mayberg aims to stimulate patients’ neuronal cells in a manner that brings back the color.
Beyond exposing the insight and inventiveness that has arisen out of cell-based thinking, it seems that Mukherjee’s bigger project is an epistemological one. The early chapters of The Song of the Cell continually hint at the potential for redefining the basic unit of biology as the cell rather than the gene. The choice to center biomedicine around cells is, above all, a conspicuous choice not to center it around genes (the subject of Mukherjee’s previous book, The Gene), because genes dominate popular science communication.
This choice of cells over genes is most welcome. Cells are alive. Genes are not. Letters—such as the As, Cs, Gs, and Ts that represent the nucleotides of DNA, which make up our genes—must be synthesized into a word or poem or song that offers a glimpse into deeper truths. A key idea embedded in this thinking is that of emergence. Whether in ancient myth or modern art, creation tends to be an emergent process, not a linearly coded script. The cell is our current best guess for the basic unit of life’s emergence, turning a finite set of chemical building blocks—nucleic acids, proteins, sugars, fats—into a replicative, evolving system for fighting stasis and entropy. The cell’s song is one for our times, for it is the song of biology’s emergence out of chemistry and physics, into the “frenetically active process” of homeostasis.
Re-centering our view of biology has practical consequences, too, for how we think about diagnosing and treating disease, and for inventing new medicines. Centering cells presents a challenge: which type of cell to place at the center? Rather than default to the apparent simplicity of DNA as a symbol because it represents the one master code for life, the tension in defining the diversity of cells—a mapping process still far from complete in cutting-edge biology laboratories—can help to create a more thoughtful library of cellular metaphors to shape both the practice and communication of biology.
Further, effective problem solving is often about operating at the right level, or the right scale. The cell feels like appropriate level at which to interrogate many of the diseases that ail us, because the senses that guide our own perceptions of sickness and health—the smoldering pain of inflammation, the tunnel vision of a migraine, the dizziness of a fluttering heart—are emergent.
This, unfortunately, is sort of where Mukherjee leaves the reader, under-exploring the consequences of a biology of emergence. Many practical and profound questions have to do with the ways that each scale of life feeds back on the others. In a tome on Cells and “the future human” I wished that Mukherjee had created more space for seeking the ways that cells will shape and be shaped by the future, of humanity and otherwise.
We are entering a phase of real-world bioengineering that features the modularization of cellular parts within cells, of cells within organs, of organs within bodies, and of bodies within ecosystems. In this reality, we would be unwise to assume that any whole is the mere sum of its parts.
For example, when discussing the regenerative power of pluripotent stem cells, Mukherjee raises the philosophical thought experiment of the Delphic boat, also known as the Ship of Theseus. The boat is made of many pieces of wood, each of which is replaced for repairs over the years, with the boat’s structure unchanged. Eventually none of the boat’s original wood remains: Is it the same boat?
Mukherjee raises the Delphic boat in one paragraph at the end of the chapter on stem cells, as a metaphor related to the possibility of stem cell-enabled regeneration in perpetuity. He does not follow any of the threads of potential answers. Given the current state of cellular engineering, about which Mukherjee is a world expert from his work as a physician-scientist, this book could have used an entire section dedicated to probing this question and, importantly, the ways this thought experiment falls apart.
We are entering a phase of real-world bioengineering that features the modularization of cellular parts within cells, of cells within organs, of organs within bodies, and of bodies within ecosystems. In this reality, we would be unwise to assume that any whole is the mere sum of its parts. Wholeness at any one of these scales of life—organelle, cell, organ, body, ecosystem—is what is at stake if we allow biological reductionism to assume away the relation between those scales.
In other words, Mukherjee succeeds in providing a masterful and compelling narrative of the lives of many of the cells that emerge to enliven us. Like his previous books, it is a worthwhile read for anyone curious about the role of cells in disease and in health. And yet, he fails to offer the broader context of The Song of the Cell.
As leading agronomist and essayist Wes Jackson has written, “The sequence of amino acids that is at home in the human cell, when produced inside the bacterial cell, does not fold quite right. Something about the E. coli internal environment affects the tertiary structure of the protein and makes it inactive. The whole in this case, the E. coli cell, affects the part—the newly made protein. Where is the priority of part now?” [1]
Beyond the ways that different kingdoms of life translate the same genetic code, the practical situation for humanity today relates to the ways that the different disciplines of modern life use values and culture to influence our genes, cells, bodies, and environment. It may be that humans will soon become a bit like the Delphic boat, infused with the buzz of fresh cells to repopulate different niches within our bodies, for healthier, longer lives. But in biology, as in writing, a mixed metaphor can cause something of a cacophony. For we are not boats with parts to be replaced piecemeal. And nor are whales, nor alpine forests, nor topsoil. Life isn’t a sum of parts, and neither is a song that rings true.
[1] Wes Jackson, "Visions and Assumptions," in Nature as Measure (p. 52-53).
