In an initial study, Canary analyzed speech recordings with AI and identified early stage Alzheimer’s with 96 percent accuracy.
Dave Arnold retired in his 60s and began spending time volunteering in local schools. But then he started misplacing items, forgetting appointments and losing his sense of direction. Eventually he was diagnosed with early stage Alzheimer’s.
“Hearing the diagnosis made me very emotional and tearful,” he said. “I immediately thought of all my mom had experienced.” His mother suffered with the condition for years before passing away. Over the last year, Arnold has worked for the Alzheimer’s Association as one of its early stage advisors, sharing his insights to help others in the initial stages of the disease.
Arnold was diagnosed sooner than many others. It's important to find out early, when interventions can make the most difference. One promising avenue is looking at how people talk. Research has shown that Alzheimer’s affects a part of the brain that controls speech, resulting in small changes before people show other signs of the disease.
Now, Canary Speech, a company based in Utah, is using AI to examine elements like the pitch of a person’s voice and their pauses. In an initial study, Canary analyzed speech recordings with AI and identified early stage Alzheimer’s with 96 percent accuracy.
Developing the AI model
Canary Speech’s CEO, Henry O’Connell, met cofounder Jeff Adams about 40 years before they started the company. Back when they first crossed paths, they were both living in Bethesda, Maryland; O’Connell was a research fellow at the National Institutes of Health studying rare neurological diseases, while Adams was working to decode spy messages. Later on, Adams would specialize in building mathematical models to analyze speech and sound as a team leader in developing Amazon's Alexa.
It wasn't until 2015 that they decided to make use of the fit between their backgrounds. ““We established Canary Speech in 2017 to build a product that could be used in multiple languages in clinical environments,” O'Connell says.
The need is growing. About 55 million people worldwide currently live with Alzheimer’s, a number that is expected to double by 2050. Some scientists think the disease results from a buildup of plaque in the brain. It causes mild memory loss at first and, over time, this issue get worse while other symptoms, such as disorientation and hallucinations, can develop. Treatment to manage the disease is more effective in the earlier stages, but detection is difficult since mild symptoms are often attributed to the normal aging process.
O’Connell and Adams specialize in the complex ways that Alzheimer’s effects how people speak. Using AI, their mathematical model analyzes 15 million data points every minute, focusing on certain features of speech such as pitch, pauses and elongation of words. It also pays attention to how the vibrations of vocal cords change in different stages of the disease.
To create their model, the team used a type of machine learning called deep neural nets, which looks at multiple layers of data - in this case, the multiple features of a person’s speech patterns.
“Deep neural nets allow us to look at much, much larger data sets built out of millions of elements,” O’Connell explained. “Through machine learning and AI, we’ve identified features that are very sensitive to an Alzheimer’s patient versus [people without the disease] and also very sensitive to mild cognitive impairment, early stage and moderate Alzheimer's.” Based on their learnings, Canary is able to classify the disease stage very quickly, O’Connell said.
“When we’re listening to sublanguage elements, we’re really analyzing the direct result of changes in the brain in the physical body,” O’Connell said. “The brain controls your vocal cords: how fast they vibrate, the expansion of them, the contraction.” These factors, along with where people put their tongues when talking, function subconsciously and result in subtle changes in the sounds of speech.
Further testing is needed
In an initial trial, Canary analyzed speech recordings from phone calls to a large U.S. health insurer. They looked at the audio recordings of 651 policyholders who had early stage Alzheimer’s and 1018 who did not have the condition, aiming for a representative sample of age, gender and race. They used this data to create their first diagnostic model and found that it was 96 percent accurate in identifying Alzheimer’s.
Christian Herff, an assistant professor of neuroscience at Maastricht University in the Netherlands, praised this approach while adding that further testing is needed to assess its effectiveness.
“I think the general idea of identifying increased risk for cognitive impairment based on speech characteristics is very feasible, particularly when change in a user’s voice is monitored, for example, by recording speech every year,” Herff said. He noted that this can only be a first indication, not a full diagnosis. The accuracy still needs to be validated in studies that follows individuals over a period of time, he said.
Toby Walsh, a professor of artificial intelligence at the University of New South Wales, also thinks Canary’s tool has potential but highlights that Canary could diagnose some people who don’t really have the disease. “This is an interesting and promising application of AI,” he said, “but these tools need to be used carefully. Imagine the anxiety of being misdiagnosed with Alzheimer’s.”
As with many other AI tools, privacy and bias are additional issues to monitor closely, Walsh said.
Other languages
A related issue is that not everyone is fluent in English. Mahnaz Arvaneh, a senior lecturer in automatic control and systems engineering at the University of Sheffield, said this could be a blind spot.
“The system may not be very accurate for those who have English as their second language as their speaking patterns would be different, and any issue might be because of language deficiency rather than cognitive issues,” Arvaneh said.
The team is expanding to multiple languages starting with Japanese and Spanish. The elements of the model that make up the algorithm are very similar, but they need to be validated and retrained in a different language, which will require access to more data.
Recently, Canary analyzed the phone calls of 233 Japanese patients who had mild cognitive impairment and 704 healthy people. Using an English model they were able to identify the Japanese patients who had mild cognitive impairment with 78 percent accuracy. They also developed a model in Japanese that was 45 percent accurate, and they’re continuing to train it with more data.
The future
Canary is using their model to look at other diseases like Huntington’s and Parkinson’s. They’re also collaborating with pharmaceuticals to validate potential therapies for Alzheimer’s. By looking at speech patterns over time, Canary can get an indication of how well these drugs are working.
Dave Arnold and his wife dance at his nephew’s wedding in Rochester, New York, ten years ago, before his Alzheimer's diagnosis.
Dave Arnold
Ultimately, they want to integrate their tool into everyday life. “We want it to be used in a smartphone, or a teleconference call so that individuals could be examined in their home,” O’Connell said. “We could follow them over time and work with clinical teams and hospitals to improve the evaluation of patients and contribute towards an accurate diagnosis.”
Arnold, the patient with early stage Alzheimer’s, sees great promise. “The process of getting a diagnosis is already filled with so much anxiety,” he said. “Anything that can be done to make it easier and less stressful would be a good thing, as long as it’s proven accurate.”
She figured the pain would go away. But instead, it intensified that night. Kamil's husband drove her to the Cedars-Sinai hospital, where she was admitted to the coronary care unit. It turned out she wasn't having a heart attack after all. Instead, she was diagnosed with a much less common but nonetheless dangerous heart condition called takotsubo syndrome, or broken heart syndrome.
A heart attack happens when blood flow to the heart is obstructed—such as when an artery is blocked—causing heart muscle tissue to die. In takotsubo syndrome, the blood flow isn't blocked, but the heart doesn't pump it properly. The heart changes its shape and starts to resemble a Japanese fishing device called tako-tsubo, a clay pot with a wider body and narrower mouth, used to catch octopus.
"The heart muscle is stunned and doesn't function properly anywhere from three days to three weeks," explains Noel Bairey Merz, the cardiologist at Cedar Sinai who Kamil went to see after she was discharged.
"The heart muscle is stunned and doesn't function properly anywhere from three days to three weeks."
But even though the heart isn't permanently damaged, mortality rates due to takotsubo syndrome are comparable to those of a heart attack, Merz notes—about 4-5 percent of patients die from the attack, and 20 percent within the next five years. "It's as bad as a heart attack," Merz says—only it's much less known, even to doctors. The condition affects only about 1 percent of people, and there are around 15,000 new cases annually. It's diagnosed using a cardiac ventriculogram, an imaging test that allows doctors to see how the heart pumps blood.
Scientists don't fully understand what causes Takotsubo syndrome, but it usually occurs after extreme emotional or physical stress. Doctors think it's triggered by a so-called catecholamine storm, a phenomenon in which the body releases too much catecholamines—hormones involved in the fight-or-flight response. Evolutionarily, when early humans lived in savannas or forests and had to either fight off predators or flee from them, these hormones gave our ancestors the needed strength and stamina to take either action. Released by nerve endings and by the adrenal glands that sit on top of the kidneys, these hormones still flood our bodies in moments of stress, but an overabundance of them could sometimes be damaging.
Elaine Kamil
A study by scientists at Harvard Medical School linked increased risk of takotsubo to higher activity in the amygdala, a brain region responsible for emotions that's involved in responses to stress. The scientists believe that chronic stress makes people more susceptible to the syndrome. Notably, one small study suggested that the number of Takotsubo cases increased during the COVID-19 pandemic.
There are no specific drugs to treat takotsubo, so doctors rely on supportive therapies, which include medications typically used for high blood pressure and heart failure. In most cases, the heart returns to its normal shape within a few weeks. "It's a spontaneous recovery—the catecholamine storm is resolved, the injury trigger is removed and the heart heals itself because our bodies have an amazing healing capacity," Merz says. It also helps that tissues remain intact. 'The heart cells don't die, they just aren't functioning properly for some time."
That's the good news. The bad news is that takotsubo is likely to strike again—in 5-20 percent of patients the condition comes back, sometimes more severe than before.
That's exactly what happened to Kamil. After getting her diagnosis in 2013, she realized that she actually had a previous takotsubo episode. In 2010, she experienced similar symptoms after her son died. "The night after he died, I was having severe chest pain at night, but I was too overwhelmed with grief to do anything about it," she recalls. After a while, the pain subsided and didn't return until three years later.
For weeks after her second attack, she felt exhausted, listless and anxious. "You lose confidence in your body," she says. "You have these little twinges on your chest, or if you start having arrhythmia, and you wonder if this is another episode coming up. It's really unnerving because you don't know how to read these cues." And that's very typical, Merz says. Even when the heart muscle appears to recover, patients don't return to normal right away. They have shortens of breath, they can't exercise, and they stay anxious and worried for a while.
Women over the age of 50 are diagnosed with takotsubo more often than other demographics. However, it happens in men too, although it typically strikes after physical stress, such as a triathlon or an exhausting day of cycling. Young people can also get takotsubo. Older patients are hospitalized more often, but younger people tend to have more severe complications. It could be because an older person may go for a jog while younger one may run a marathon, which would take a stronger toll on the body of a person who's predisposed to the condition.
Notably, the emotional stressors don't always have to be negative—the heart muscle can get out of shape from good emotions, too. "There have been case reports of takotsubo at weddings," Merz says. Moreover, one out of three or four takotsubo patients experience no apparent stress, she adds. "So it could be that it's not so much the catecholamine storm itself, but the body's reaction to it—the physiological reaction deeply embedded into out physiology," she explains.
Merz and her team are working to understand what makes people predisposed to takotsubo. They think a person's genetics play a role, but they haven't yet pinpointed genes that seem to be responsible. Genes code for proteins, which affect how the body metabolizes various compounds, which, in turn, affect the body's response to stress. Pinning down the protein involved in takotsubo susceptibility would allow doctors to develop screening tests and identify those prone to severe repeating attacks. It will also help develop medications that can either prevent it or treat it better than just waiting for the body to heal itself.
Researchers at the Imperial College London found that elevated levels of certain types of microRNAs—molecules involved in protein production—increase the chances of developing takotsubo.
In one study, researchers tried treating takotsubo in mice with a drug called suberanilohydroxamic acid, or SAHA, typically used for cancer treatment. The drug improved cardiac health and reversed the broken heart in rodents. It remains to be seen if the drug would have a similar effect on humans. But identifying a drug that shows promise is progress, Merz says. "I'm glad that there's research in this area."
This article was originally published by Leaps.org on July 28, 2021.