In an initial study, Canary analyzed speech recordings with AI and identified early stage Alzheimer’s with 96 percent accuracy.
Dave Arnold retired in his 60s and began spending time volunteering in local schools. But then he started misplacing items, forgetting appointments and losing his sense of direction. Eventually he was diagnosed with early stage Alzheimer’s.
“Hearing the diagnosis made me very emotional and tearful,” he said. “I immediately thought of all my mom had experienced.” His mother suffered with the condition for years before passing away. Over the last year, Arnold has worked for the Alzheimer’s Association as one of its early stage advisors, sharing his insights to help others in the initial stages of the disease.
Arnold was diagnosed sooner than many others. It's important to find out early, when interventions can make the most difference. One promising avenue is looking at how people talk. Research has shown that Alzheimer’s affects a part of the brain that controls speech, resulting in small changes before people show other signs of the disease.
Now, Canary Speech, a company based in Utah, is using AI to examine elements like the pitch of a person’s voice and their pauses. In an initial study, Canary analyzed speech recordings with AI and identified early stage Alzheimer’s with 96 percent accuracy.
Developing the AI model
Canary Speech’s CEO, Henry O’Connell, met cofounder Jeff Adams about 40 years before they started the company. Back when they first crossed paths, they were both living in Bethesda, Maryland; O’Connell was a research fellow at the National Institutes of Health studying rare neurological diseases, while Adams was working to decode spy messages. Later on, Adams would specialize in building mathematical models to analyze speech and sound as a team leader in developing Amazon's Alexa.
It wasn't until 2015 that they decided to make use of the fit between their backgrounds. ““We established Canary Speech in 2017 to build a product that could be used in multiple languages in clinical environments,” O'Connell says.
The need is growing. About 55 million people worldwide currently live with Alzheimer’s, a number that is expected to double by 2050. Some scientists think the disease results from a buildup of plaque in the brain. It causes mild memory loss at first and, over time, this issue get worse while other symptoms, such as disorientation and hallucinations, can develop. Treatment to manage the disease is more effective in the earlier stages, but detection is difficult since mild symptoms are often attributed to the normal aging process.
O’Connell and Adams specialize in the complex ways that Alzheimer’s effects how people speak. Using AI, their mathematical model analyzes 15 million data points every minute, focusing on certain features of speech such as pitch, pauses and elongation of words. It also pays attention to how the vibrations of vocal cords change in different stages of the disease.
To create their model, the team used a type of machine learning called deep neural nets, which looks at multiple layers of data - in this case, the multiple features of a person’s speech patterns.
“Deep neural nets allow us to look at much, much larger data sets built out of millions of elements,” O’Connell explained. “Through machine learning and AI, we’ve identified features that are very sensitive to an Alzheimer’s patient versus [people without the disease] and also very sensitive to mild cognitive impairment, early stage and moderate Alzheimer's.” Based on their learnings, Canary is able to classify the disease stage very quickly, O’Connell said.
“When we’re listening to sublanguage elements, we’re really analyzing the direct result of changes in the brain in the physical body,” O’Connell said. “The brain controls your vocal cords: how fast they vibrate, the expansion of them, the contraction.” These factors, along with where people put their tongues when talking, function subconsciously and result in subtle changes in the sounds of speech.
Further testing is needed
In an initial trial, Canary analyzed speech recordings from phone calls to a large U.S. health insurer. They looked at the audio recordings of 651 policyholders who had early stage Alzheimer’s and 1018 who did not have the condition, aiming for a representative sample of age, gender and race. They used this data to create their first diagnostic model and found that it was 96 percent accurate in identifying Alzheimer’s.
Christian Herff, an assistant professor of neuroscience at Maastricht University in the Netherlands, praised this approach while adding that further testing is needed to assess its effectiveness.
“I think the general idea of identifying increased risk for cognitive impairment based on speech characteristics is very feasible, particularly when change in a user’s voice is monitored, for example, by recording speech every year,” Herff said. He noted that this can only be a first indication, not a full diagnosis. The accuracy still needs to be validated in studies that follows individuals over a period of time, he said.
Toby Walsh, a professor of artificial intelligence at the University of New South Wales, also thinks Canary’s tool has potential but highlights that Canary could diagnose some people who don’t really have the disease. “This is an interesting and promising application of AI,” he said, “but these tools need to be used carefully. Imagine the anxiety of being misdiagnosed with Alzheimer’s.”
As with many other AI tools, privacy and bias are additional issues to monitor closely, Walsh said.
Other languages
A related issue is that not everyone is fluent in English. Mahnaz Arvaneh, a senior lecturer in automatic control and systems engineering at the University of Sheffield, said this could be a blind spot.
“The system may not be very accurate for those who have English as their second language as their speaking patterns would be different, and any issue might be because of language deficiency rather than cognitive issues,” Arvaneh said.
The team is expanding to multiple languages starting with Japanese and Spanish. The elements of the model that make up the algorithm are very similar, but they need to be validated and retrained in a different language, which will require access to more data.
Recently, Canary analyzed the phone calls of 233 Japanese patients who had mild cognitive impairment and 704 healthy people. Using an English model they were able to identify the Japanese patients who had mild cognitive impairment with 78 percent accuracy. They also developed a model in Japanese that was 45 percent accurate, and they’re continuing to train it with more data.
The future
Canary is using their model to look at other diseases like Huntington’s and Parkinson’s. They’re also collaborating with pharmaceuticals to validate potential therapies for Alzheimer’s. By looking at speech patterns over time, Canary can get an indication of how well these drugs are working.
Dave Arnold and his wife dance at his nephew’s wedding in Rochester, New York, ten years ago, before his Alzheimer's diagnosis.
Dave Arnold
Ultimately, they want to integrate their tool into everyday life. “We want it to be used in a smartphone, or a teleconference call so that individuals could be examined in their home,” O’Connell said. “We could follow them over time and work with clinical teams and hospitals to improve the evaluation of patients and contribute towards an accurate diagnosis.”
Arnold, the patient with early stage Alzheimer’s, sees great promise. “The process of getting a diagnosis is already filled with so much anxiety,” he said. “Anything that can be done to make it easier and less stressful would be a good thing, as long as it’s proven accurate.”
A mosquito under the microscope.
The association between climate and infectious disease is poorly understood, says Irina Tezaur, a computational scientist at Sandia National Laboratories. “Correlations have been observed but it’s not known if these correlations translate to causal relationships.”
To make accurate longer-term predictions, scientists need more empirical data, multiple datasets specific to locations and diseases, and the ability to calculate risks that depend on unpredictable nature and human behavior. Another obstacle is that climate scientists and epidemiologists are not collaborating effectively, so some researchers are calling for a multidisciplinary approach, a new field called Outbreak Science.
Climate scientists are far ahead of epidemiologists in gathering essential data.
Earth System Models—combining the interactions of atmosphere, ocean, land, ice and biosphere—have been in place for two decades to monitor the effects of global climate change. These models must be combined with epidemiological and human model research, areas that are easily skewed by unpredictable elements, from extreme weather events to public environmental policy shifts.
“There is never just one driver in tracking the impact of climate on infectious disease,” says Joacim Rocklöv, a professor at the Heidelberg Institute of Global Health & Heidelberg Interdisciplinary Centre for Scientific Computing in Germany. Rocklöv has studied how climate affects vector-borne diseases—those transmitted to humans by mosquitoes, ticks or fleas. “You need to disentangle the variables to find out how much difference climate makes to the outcome and how much is other factors.” Determinants from deforestation to population density to lack of healthcare access influence the spread of disease.
Even though climate change is not the primary driver of infectious disease today, it poses a major threat to public health in the future, says Rocklöv.
The promise of predictive modeling
“Models are simplifications of a system we’re trying to understand,” says Jeremy Hess, who directs the Center for Health and the Global Environment at University of Washington in Seattle. “They’re tools for learning that improve over time with new observations.”
Accurate predictions depend on high-quality, long-term observational data but models must start with assumptions. “It’s not possible to apply an evidence-based approach for the next 40 years,” says Rocklöv. “Using models to experiment and learn is the only way to figure out what climate means for infectious disease. We collect data and analyze what already happened. What we do today will not make a difference for several decades.”
To improve accuracy, scientists develop and draw on thousands of models to cover as many scenarios as possible. One model may capture the dynamics of disease transmission while another focuses on immunity data or ocean influences or seasonal components of a virus. Further, each model needs to be disease-specific and often location-specific to be useful.
“All models have biases so it’s important to use a suite of models,” Tezaur stresses.
The modeling scientist chooses the drivers of change and parameters based on the question explored. The drivers could be increased precipitation, poverty or mosquito prevalence, for instance. Later, the scientist may need to isolate the effect of one driver so that will require another model.
There have been some related successes, such as the latest models for mosquito-borne diseases like Dengue, Zika and malaria as well as those for flu and tick-borne diseases, says Hess.
Rocklöv was part of a research team that used test data from 2018 and 2019 to identify regions at risk for West Nile virus outbreaks. Using AI, scientists were able to forecast outbreaks of the virus for the entire transmission season in Europe. “In the end, we want data-driven models; that’s what AI can accomplish,” says Rocklöv. Other researchers are making an important headway in creating a framework to predict novel host–parasite interactions.
Modeling studies can run months, years or decades. “The scientist is working with layers of data. The challenge is how to transform and couple different models together on a planetary scale,” says Jeanne Fair, a scientist at Los Alamos National Laboratory, Biosecurity and Public Health, in New Mexico.
Disease forecasting will require a significant investment into the infrastructure needed to collect data about the environment, vectors, and hosts a tall spatial and temporal resolutions.
And it’s a constantly changing picture. A modeling study in an April 2022 issue of Nature predicted that thousands of animals will migrate to cooler locales as temperatures rise. This means that various species will come into closer contact with people and other mammals for the first time. This is likely to increase the risk of emerging infectious disease transmitted from animals to humans, especially in Africa and Asia.
Other things can happen too. Global warming could precipitate viral mutations or new infectious diseases that don’t respond to antimicrobial treatments. Insecticide-resistant mosquitoes could evolve. Weather-related food insecurity could increase malnutrition and weaken people’s immune systems. And the impact of an epidemic will be worse if it co-occurs during a heatwave, flood, or drought, says Hess.
The devil is in the climate variables
Solid predictions about the future of climate and disease are not possible with so many uncertainties. Difficult-to-measure drivers must be added to the empirical model mix, such as land and water use, ecosystem changes or the public’s willingness to accept a vaccine or practice social distancing. Nor is there any precedent for calculating the effect of climate changes that are accelerating at a faster speed than ever before.
The most critical climate variables thought to influence disease spread are temperature, precipitation, humidity, sunshine and wind, according to Tezaur’s research. And then there are variables within variables. Influenza scientists, for example, found that warm winters were predictors of the most severe flu seasons in the following year.
The human factor may be the most challenging determinant. To what degree will people curtail greenhouse gas emissions, if at all? The swift development of effective COVID-19 vaccines was a game-changer, but will scientists be able to repeat it during the next pandemic? Plus, no model could predict the amount of internet-fueled COVID-19 misinformation, Fair noted. To tackle this issue, infectious disease teams are looking to include more sociologists and political scientists in their modeling.
Addressing the gaps
Currently, researchers are focusing on the near future, predicting for next year, says Fair. “When it comes to long-term, that’s where we have the most work to do.” While scientists cannot foresee how political influences and misinformation spread will affect models, they are positioned to make headway in collecting and assessing new data streams that have never been merged.
Disease forecasting will require a significant investment into the infrastructure needed to collect data about the environment, vectors, and hosts at all spatial and temporal resolutions, Fair and her co-authors stated in their recent study. For example real-time data on mosquito prevalence and diversity in various settings and times is limited or non-existent. Fair also would like to see standards set in mosquito data collection in every country. “Standardizing across the US would be a huge accomplishment,” she says.
Understanding how climate change contributes to the spread of disease is critical for thwarting future calamities.
Jeanne Fair
Hess points to a dearth of data in local and regional datasets about how extreme weather events play out in different geographic locations. His research indicates that Africa and the Middle East experienced substantial climate shifts, for example, but are unrepresented in the evidentiary database, which limits conclusions. “A model for dengue may be good in Singapore but not necessarily in Port-au-Prince,” Hess explains. And, he adds, scientists need a way of evaluating models for how effective they are.
The hope, Rocklöv says, is that in the future we will have data-driven models rather than theoretical ones. In turn, sharper statistical analyses can inform resource allocation and intervention strategies to prevent outbreaks.
Most of all, experts emphasize that epidemiologists and climate scientists must stop working in silos. If scientists can successfully merge epidemiological data with climatic, biological, environmental, ecological and demographic data, they will make better predictions about complex disease patterns. Modeling “cross talk” and among disciplines and, in some cases, refusal to release data between countries is hindering discovery and advances.
It’s time for bold transdisciplinary action, says Hess. He points to initiatives that need funding in disease surveillance and control; developing and testing interventions; community education and social mobilization; decision-support analytics to predict when and where infections will emerge; advanced methodologies to improve modeling; training scientists in data management and integrated surveillance.
Establishing a new field of Outbreak Science to coordinate collaboration would accelerate progress. Investment in decision-support modeling tools for public health teams, policy makers, and other long-term planning stakeholders is imperative, too. We need to invest in programs that encourage people from climate modeling and epidemiology to work together in a cohesive fashion, says Tezaur. Joining forces is the only way to solve the formidable challenges ahead.
This article originally appeared in One Health/One Planet, a single-issue magazine that explores how climate change and other environmental shifts are increasing vulnerabilities to infectious diseases by land and by sea. The magazine probes how scientists are making progress with leaders in other fields toward solutions that embrace diverse perspectives and the interconnectedness of all lifeforms and the planet.
