In an initial study, Canary analyzed speech recordings with AI and identified early stage Alzheimer’s with 96 percent accuracy.
Adobe Stock
Dave Arnold retired in his 60s and began spending time volunteering in local schools. But then he started misplacing items, forgetting appointments and losing his sense of direction. Eventually he was diagnosed with early stage Alzheimer’s.
“Hearing the diagnosis made me very emotional and tearful,” he said. “I immediately thought of all my mom had experienced.” His mother suffered with the condition for years before passing away. Over the last year, Arnold has worked for the Alzheimer’s Association as one of its early stage advisors, sharing his insights to help others in the initial stages of the disease.
Arnold was diagnosed sooner than many others. It's important to find out early, when interventions can make the most difference. One promising avenue is looking at how people talk. Research has shown that Alzheimer’s affects a part of the brain that controls speech, resulting in small changes before people show other signs of the disease.
Now, Canary Speech, a company based in Utah, is using AI to examine elements like the pitch of a person’s voice and their pauses. In an initial study, Canary analyzed speech recordings with AI and identified early stage Alzheimer’s with 96 percent accuracy.
Developing the AI model
Canary Speech’s CEO, Henry O’Connell, met cofounder Jeff Adams about 40 years before they started the company. Back when they first crossed paths, they were both living in Bethesda, Maryland; O’Connell was a research fellow at the National Institutes of Health studying rare neurological diseases, while Adams was working to decode spy messages. Later on, Adams would specialize in building mathematical models to analyze speech and sound as a team leader in developing Amazon's Alexa.
It wasn't until 2015 that they decided to make use of the fit between their backgrounds. ““We established Canary Speech in 2017 to build a product that could be used in multiple languages in clinical environments,” O'Connell says.
The need is growing. About 55 million people worldwide currently live with Alzheimer’s, a number that is expected to double by 2050. Some scientists think the disease results from a buildup of plaque in the brain. It causes mild memory loss at first and, over time, this issue get worse while other symptoms, such as disorientation and hallucinations, can develop. Treatment to manage the disease is more effective in the earlier stages, but detection is difficult since mild symptoms are often attributed to the normal aging process.
O’Connell and Adams specialize in the complex ways that Alzheimer’s effects how people speak. Using AI, their mathematical model analyzes 15 million data points every minute, focusing on certain features of speech such as pitch, pauses and elongation of words. It also pays attention to how the vibrations of vocal cords change in different stages of the disease.
To create their model, the team used a type of machine learning called deep neural nets, which looks at multiple layers of data - in this case, the multiple features of a person’s speech patterns.
“Deep neural nets allow us to look at much, much larger data sets built out of millions of elements,” O’Connell explained. “Through machine learning and AI, we’ve identified features that are very sensitive to an Alzheimer’s patient versus [people without the disease] and also very sensitive to mild cognitive impairment, early stage and moderate Alzheimer's.” Based on their learnings, Canary is able to classify the disease stage very quickly, O’Connell said.
“When we’re listening to sublanguage elements, we’re really analyzing the direct result of changes in the brain in the physical body,” O’Connell said. “The brain controls your vocal cords: how fast they vibrate, the expansion of them, the contraction.” These factors, along with where people put their tongues when talking, function subconsciously and result in subtle changes in the sounds of speech.
Further testing is needed
In an initial trial, Canary analyzed speech recordings from phone calls to a large U.S. health insurer. They looked at the audio recordings of 651 policyholders who had early stage Alzheimer’s and 1018 who did not have the condition, aiming for a representative sample of age, gender and race. They used this data to create their first diagnostic model and found that it was 96 percent accurate in identifying Alzheimer’s.
Christian Herff, an assistant professor of neuroscience at Maastricht University in the Netherlands, praised this approach while adding that further testing is needed to assess its effectiveness.
“I think the general idea of identifying increased risk for cognitive impairment based on speech characteristics is very feasible, particularly when change in a user’s voice is monitored, for example, by recording speech every year,” Herff said. He noted that this can only be a first indication, not a full diagnosis. The accuracy still needs to be validated in studies that follows individuals over a period of time, he said.
Toby Walsh, a professor of artificial intelligence at the University of New South Wales, also thinks Canary’s tool has potential but highlights that Canary could diagnose some people who don’t really have the disease. “This is an interesting and promising application of AI,” he said, “but these tools need to be used carefully. Imagine the anxiety of being misdiagnosed with Alzheimer’s.”
As with many other AI tools, privacy and bias are additional issues to monitor closely, Walsh said.
Other languages
A related issue is that not everyone is fluent in English. Mahnaz Arvaneh, a senior lecturer in automatic control and systems engineering at the University of Sheffield, said this could be a blind spot.
“The system may not be very accurate for those who have English as their second language as their speaking patterns would be different, and any issue might be because of language deficiency rather than cognitive issues,” Arvaneh said.
The team is expanding to multiple languages starting with Japanese and Spanish. The elements of the model that make up the algorithm are very similar, but they need to be validated and retrained in a different language, which will require access to more data.
Recently, Canary analyzed the phone calls of 233 Japanese patients who had mild cognitive impairment and 704 healthy people. Using an English model they were able to identify the Japanese patients who had mild cognitive impairment with 78 percent accuracy. They also developed a model in Japanese that was 45 percent accurate, and they’re continuing to train it with more data.
The future
Canary is using their model to look at other diseases like Huntington’s and Parkinson’s. They’re also collaborating with pharmaceuticals to validate potential therapies for Alzheimer’s. By looking at speech patterns over time, Canary can get an indication of how well these drugs are working.
Dave Arnold and his wife dance at his nephew’s wedding in Rochester, New York, ten years ago, before his Alzheimer's diagnosis.
Dave Arnold
Ultimately, they want to integrate their tool into everyday life. “We want it to be used in a smartphone, or a teleconference call so that individuals could be examined in their home,” O’Connell said. “We could follow them over time and work with clinical teams and hospitals to improve the evaluation of patients and contribute towards an accurate diagnosis.”
Arnold, the patient with early stage Alzheimer’s, sees great promise. “The process of getting a diagnosis is already filled with so much anxiety,” he said. “Anything that can be done to make it easier and less stressful would be a good thing, as long as it’s proven accurate.”
This fall, Robert Reinhart of Boston University published a study finding that electrical stimulation can boost memory - and Reinhart was surprised to discover the effects lasted a full month.
Cydney Scott
Scientists believe brain circuits tend to uncouple as we age. Since brain neurons communicate by exchanging electrical impulses with each other, the breakdown of these links and associations could be what causes the “senior moment”—when you can’t remember the name of the movie you just watched.
In 2019, Boston University researchers led by Robert Reinhart, director of the Cognitive and Clinical Neuroscience Laboratory, showed that memory loss in healthy older adults is likely caused by these disconnected brain networks. When Reinhart and his team stimulated two key areas of the brain with mild electrical current, they were able to bring the brains of older adult subjects back into sync — enough so that their ability to remember small differences between two images matched that of much younger subjects for at least 50 minutes after the testing stopped.
Reinhart wowed the neuroscience community once again this fall. His newer study in Nature Neuroscience presented 150 healthy participants, ages 65 to 88, who were able to recall more words on a given list after 20 minutes of low-intensity electrical stimulation sessions over four consecutive days. This amounted to a 50 to 65 percent boost in their recall.
Even Reinhart was surprised to discover the enhanced performance of his subjects lasted a full month when they were tested again later. Those who benefited most were the participants who were the most forgetful at the start.
An older person participates in Robert Reinhart's research on brain stimulation.
Robert Reinhart
Reinhart’s subjects only suffered normal age-related memory deficits, but NIBS has great potential to help people with cognitive impairment and dementia, too, says Krista Lanctôt, the Bernick Chair of Geriatric Psychopharmacology at Sunnybrook Health Sciences Center in Toronto. Plus, “it is remarkably safe,” she says.
Lanctôt was the senior author on a meta-analysis of brain stimulation studies published last year on people with mild cognitive impairment or later stages of Alzheimer’s disease. The review concluded that magnetic stimulation to the brain significantly improved the research participants’ neuropsychiatric symptoms, such as apathy and depression. The stimulation also enhanced global cognition, which includes memory, attention, executive function and more.
This is the frontier of neuroscience.
The two main forms of NIBS – and many questions surrounding them
There are two types of NIBS. They differ based on whether electrical or magnetic stimulation is used to create the electric field, the type of device that delivers the electrical current and the strength of the current.
Transcranial Current Brain Stimulation (tES) is an umbrella term for a group of techniques using low-wattage electrical currents to manipulate activity in the brain. The current is delivered to the scalp or forehead via electrodes attached to a nylon elastic cap or rubber headband.
Variations include how the current is delivered—in an alternating pattern or in a constant, direct mode, for instance. Tweaking frequency, potency or target brain area can produce different effects as well. Reinhart’s 2022 study demonstrated that low or high frequencies and alternating currents were uniquely tied to either short-term or long-term memory improvements.
Sessions may be 20 minutes per day over the course of several days or two weeks. “[The subject] may feel a tingling, warming, poking or itching sensation,” says Reinhart, which typically goes away within a minute.
The other main approach to NIBS is Transcranial Magnetic Simulation (TMS). It involves the use of an electromagnetic coil that is held or placed against the forehead or scalp to activate nerve cells in the brain through short pulses. The stimulation is stronger than tES but similar to a magnetic resonance imaging (MRI) scan.
The subject may feel a slight knocking or tapping on the head during a 20-to-60-minute session. Scalp discomfort and headaches are reported by some; in very rare cases, a seizure can occur.
No head-to-head trials have been conducted yet to evaluate the differences and effectiveness between electrical and magnetic current stimulation, notes Lanctôt, who is also a professor of psychiatry and pharmacology at the University of Toronto. Although TMS was approved by the FDA in 2008 to treat major depression, both techniques are considered experimental for the purpose of cognitive enhancement.
“One attractive feature of tES is that it’s inexpensive—one-fifth the price of magnetic stimulation,” Reinhart notes.
Don’t confuse either of these procedures with the horrors of electroconvulsive therapy (ECT) in the 1950s and ‘60s. ECT is a more powerful, riskier procedure used only as a last resort in treating severe mental illness today.
Clinical studies on NIBS remain scarce. Standardized parameters and measures for testing have not been developed. The high heterogeneity among the many existing small NIBS studies makes it difficult to draw general conclusions. Few of the studies have been replicated and inconsistencies abound.
Scientists are still lacking so much fundamental knowledge about the brain and how it works, says Reinhart. “We don’t know how information is represented in the brain or how it’s carried forward in time. It’s more complex than physics.”
Lanctôt’s meta-analysis showed improvements in global cognition from delivering the magnetic form of the stimulation to people with Alzheimer’s, and this finding was replicated inan analysis in the Journal of Prevention of Alzheimer’s Disease this fall. Neither meta-analysis found clear evidence that applying the electrical currents, was helpful for Alzheimer’s subjects, but Lanctôt suggests this might be merely because the sample size for tES was smaller compared to the groups that received TMS.
At the same time, London neuroscientist Marco Sandrini, senior lecturer in psychology at the University of Roehampton, critically reviewed a series of studies on the effects of tES on episodic memory. Often declining with age, episodic memory relates to recalling a person’s own experiences from the past. Sandrini’s review concluded that delivering tES to the prefrontal or temporoparietal cortices of the brain might enhance episodic memory in older adults with Alzheimer’s disease and amnesiac mild cognitive impairment (the predementia phase of Alzheimer’s when people start to have symptoms).
Researchers readily tick off studies needed to explore, clarify and validate existing NIBS data. What is the optimal stimulus session frequency, spacing and duration? How intense should the stimulus be and where should it be targeted for what effect? How might genetics or degree of brain impairment affect responsiveness? Would adjunct medication or cognitive training boost positive results? Could administering the stimulus while someone sleeps expedite memory consolidation?
Using MRI or another brain scan along with computational modeling of the current flow, a clinician could create a treatment that is customized to each person’s brain.
While Sandrini’s review reported improvements induced by tES in the recall or recognition of words and images, there is no evidence it will translate into improvements in daily activities. This is another question that will require more research and testing, Sandrini notes.
Scientists are still lacking so much fundamental knowledge about the brain and how it works, says Reinhart. “We don’t know how information is represented in the brain or how it’s carried forward in time. It’s more complex than physics.”
Where the science is headed
Learning how to apply precision medicine to NIBS is the next focus in advancing this technology, says Shankar Tumati, a post-doctoral fellow working with Lanctôt.
There is great variability in each person’s brain anatomy—the thickness of the skull, the brain’s unique folds, the amount of cerebrospinal fluid. All of these structural differences impact how electrical or magnetic stimulation is distributed in the brain and ultimately the effects.
Using MRI or another brain scan along with computational modeling of the current flow, a clinician could create a treatment that is customized to each person’s brain, from where to put the electrodes to determining the exact dose and duration of stimulation needed to achieve lasting results, Sandrini says.
Above all, most neuroscientists say that largescale research studies over long periods of time are necessary to confirm the safety and durability of this therapy for the purpose of boosting memory. Short of that, there can be no FDA approval or medical regulation for this clinical use.
Lanctôt urges people to seek out clinical NIBS trials in their area if they want to see the science advance. “That is how we’ll find the answers,” she says, predicting it will be 5 to 10 years to develop each additional clinical application of NIBS. Ultimately, she predicts that reigning in Alzheimer’s disease and mild cognitive impairment will require a multi-pronged approach that includes lifestyle and medications, too.
Sandrini believes that scientific efforts should focus on preventing or delaying Alzheimer’s. “We need to start intervention earlier—as soon as people start to complain about forgetting things,” he says. “Changes in the brain start 10 years before [there is a problem]. Once Alzheimer’s develops, it is too late.”
