In an initial study, Canary analyzed speech recordings with AI and identified early stage Alzheimer’s with 96 percent accuracy.
Dave Arnold retired in his 60s and began spending time volunteering in local schools. But then he started misplacing items, forgetting appointments and losing his sense of direction. Eventually he was diagnosed with early stage Alzheimer’s.
“Hearing the diagnosis made me very emotional and tearful,” he said. “I immediately thought of all my mom had experienced.” His mother suffered with the condition for years before passing away. Over the last year, Arnold has worked for the Alzheimer’s Association as one of its early stage advisors, sharing his insights to help others in the initial stages of the disease.
Arnold was diagnosed sooner than many others. It's important to find out early, when interventions can make the most difference. One promising avenue is looking at how people talk. Research has shown that Alzheimer’s affects a part of the brain that controls speech, resulting in small changes before people show other signs of the disease.
Now, Canary Speech, a company based in Utah, is using AI to examine elements like the pitch of a person’s voice and their pauses. In an initial study, Canary analyzed speech recordings with AI and identified early stage Alzheimer’s with 96 percent accuracy.
Developing the AI model
Canary Speech’s CEO, Henry O’Connell, met cofounder Jeff Adams about 40 years before they started the company. Back when they first crossed paths, they were both living in Bethesda, Maryland; O’Connell was a research fellow at the National Institutes of Health studying rare neurological diseases, while Adams was working to decode spy messages. Later on, Adams would specialize in building mathematical models to analyze speech and sound as a team leader in developing Amazon's Alexa.
It wasn't until 2015 that they decided to make use of the fit between their backgrounds. ““We established Canary Speech in 2017 to build a product that could be used in multiple languages in clinical environments,” O'Connell says.
The need is growing. About 55 million people worldwide currently live with Alzheimer’s, a number that is expected to double by 2050. Some scientists think the disease results from a buildup of plaque in the brain. It causes mild memory loss at first and, over time, this issue get worse while other symptoms, such as disorientation and hallucinations, can develop. Treatment to manage the disease is more effective in the earlier stages, but detection is difficult since mild symptoms are often attributed to the normal aging process.
O’Connell and Adams specialize in the complex ways that Alzheimer’s effects how people speak. Using AI, their mathematical model analyzes 15 million data points every minute, focusing on certain features of speech such as pitch, pauses and elongation of words. It also pays attention to how the vibrations of vocal cords change in different stages of the disease.
To create their model, the team used a type of machine learning called deep neural nets, which looks at multiple layers of data - in this case, the multiple features of a person’s speech patterns.
“Deep neural nets allow us to look at much, much larger data sets built out of millions of elements,” O’Connell explained. “Through machine learning and AI, we’ve identified features that are very sensitive to an Alzheimer’s patient versus [people without the disease] and also very sensitive to mild cognitive impairment, early stage and moderate Alzheimer's.” Based on their learnings, Canary is able to classify the disease stage very quickly, O’Connell said.
“When we’re listening to sublanguage elements, we’re really analyzing the direct result of changes in the brain in the physical body,” O’Connell said. “The brain controls your vocal cords: how fast they vibrate, the expansion of them, the contraction.” These factors, along with where people put their tongues when talking, function subconsciously and result in subtle changes in the sounds of speech.
Further testing is needed
In an initial trial, Canary analyzed speech recordings from phone calls to a large U.S. health insurer. They looked at the audio recordings of 651 policyholders who had early stage Alzheimer’s and 1018 who did not have the condition, aiming for a representative sample of age, gender and race. They used this data to create their first diagnostic model and found that it was 96 percent accurate in identifying Alzheimer’s.
Christian Herff, an assistant professor of neuroscience at Maastricht University in the Netherlands, praised this approach while adding that further testing is needed to assess its effectiveness.
“I think the general idea of identifying increased risk for cognitive impairment based on speech characteristics is very feasible, particularly when change in a user’s voice is monitored, for example, by recording speech every year,” Herff said. He noted that this can only be a first indication, not a full diagnosis. The accuracy still needs to be validated in studies that follows individuals over a period of time, he said.
Toby Walsh, a professor of artificial intelligence at the University of New South Wales, also thinks Canary’s tool has potential but highlights that Canary could diagnose some people who don’t really have the disease. “This is an interesting and promising application of AI,” he said, “but these tools need to be used carefully. Imagine the anxiety of being misdiagnosed with Alzheimer’s.”
As with many other AI tools, privacy and bias are additional issues to monitor closely, Walsh said.
Other languages
A related issue is that not everyone is fluent in English. Mahnaz Arvaneh, a senior lecturer in automatic control and systems engineering at the University of Sheffield, said this could be a blind spot.
“The system may not be very accurate for those who have English as their second language as their speaking patterns would be different, and any issue might be because of language deficiency rather than cognitive issues,” Arvaneh said.
The team is expanding to multiple languages starting with Japanese and Spanish. The elements of the model that make up the algorithm are very similar, but they need to be validated and retrained in a different language, which will require access to more data.
Recently, Canary analyzed the phone calls of 233 Japanese patients who had mild cognitive impairment and 704 healthy people. Using an English model they were able to identify the Japanese patients who had mild cognitive impairment with 78 percent accuracy. They also developed a model in Japanese that was 45 percent accurate, and they’re continuing to train it with more data.
The future
Canary is using their model to look at other diseases like Huntington’s and Parkinson’s. They’re also collaborating with pharmaceuticals to validate potential therapies for Alzheimer’s. By looking at speech patterns over time, Canary can get an indication of how well these drugs are working.
Dave Arnold and his wife dance at his nephew’s wedding in Rochester, New York, ten years ago, before his Alzheimer's diagnosis.
Dave Arnold
Ultimately, they want to integrate their tool into everyday life. “We want it to be used in a smartphone, or a teleconference call so that individuals could be examined in their home,” O’Connell said. “We could follow them over time and work with clinical teams and hospitals to improve the evaluation of patients and contribute towards an accurate diagnosis.”
Arnold, the patient with early stage Alzheimer’s, sees great promise. “The process of getting a diagnosis is already filled with so much anxiety,” he said. “Anything that can be done to make it easier and less stressful would be a good thing, as long as it’s proven accurate.”
In today's podcast episode, law professor Gaia Bernstein talks about the challenges of keeping control over our thoughts and actions, even when some powerful forces are pushing in the other direction.
Gaia’s book is called Unwired: Gaining Control Over Addictive Technologies. It’s fascinating and I had a chance to talk with her about it for today’s podcast. At its heart, our conversation is really about how and whether we can maintain control over our thoughts and actions, even when some powerful forces are pushing in the other direction.
Listen on Apple | Listen on Spotify | Listen on Stitcher | Listen on Amazon | Listen on Google
We discuss the idea that, in certain situations, maybe it's not reasonable to expect that we’ll be able to enjoy personal freedom and autonomy. We also talk about how to be a good parent when it sometimes seems like our kids prefer to be raised by their iPads; so-called educational video games that actually don’t have anything to do with education; the root causes of tech addictions for people of all ages; and what kinds of changes we should be supporting.
Gaia is Seton’s Hall’s Technology, Privacy and Policy Professor of Law, as well as Co-Director of the Institute for Privacy Protection, and Co-Director of the Gibbons Institute of Law Science and Technology. She’s the founding director of the Institute for Privacy Protection. She created and spearheaded the Institute’s nationally recognized Outreach Program, which educated parents and students about technology overuse and privacy.
Professor Bernstein's scholarship has been published in leading law reviews including the law reviews of Vanderbilt, Boston College, Boston University, and U.C. Davis. Her work has been selected to the Stanford-Yale Junior Faculty Forum and received extensive media coverage. Gaia joined Seton Hall's faculty in 2004. Before that, she was a fellow at the Engelberg Center of Innovation Law & Policy and at the Information Law Institute of the New York University School of Law. She holds a J.S.D. from the New York University School of Law, an LL.M. from Harvard Law School, and a J.D. from Boston University.
Gaia’s work on this topic is groundbreaking I hope you’ll listen to the conversation and then consider pre-ordering her new book. It comes out on March 28.
Rakhi Patel is among an increasing number of health care professionals, including doctors and nurses, who maintain an active persona on Instagram, TikTok and other social media sites.
“Health care professionals are quite prevalent on social media,” said Mercer Gary, a postdoctoral researcher at The Hastings Center, an independent bioethics research institute in Garrison, New York. “They’ve been posting on #medTwitter for many years, mainly to communicate with one another, but, of course, anyone can see the threads. Most recently, doctors and nurses have become a presence on TikTok.”
On social media, many health care providers perceive themselves to be “humanizing” their profession by coming across as more approachable — “reminding patients that providers are people and workers, as well as repositories of medical expertise,” Gary said. As a result, she noted that patients who are often intimidated by clinicians may feel comfortable enough to overcome barriers to scheduling health care appointments. The use of TikTok in particular may help doctors and nurses connect with younger followers.
When health care providers post on social media, they must bear in mind that they have legal and ethical duties to their patients, profession and society, said Elizabeth Levy, founder and director of Physicians for Justice.
While enduring three years of pandemic conditions, many health care professionals have struggled with burnout, exhaustion and moral distress. “Much health care provider content on social media seeks to expose the difficulties of the work,” Gary added. “TikTok and Instagram reels have shown health care providers crying after losing a patient or exhausted after a night shift in the emergency department.”
A study conducted in Beijing, China and published last year found that TikTok is the world’s most rapidly growing video application, amassing 1.6 billion users in 2021. “More and more patients are searching for information on genitourinary cancers via TikTok,” the study’s authors wrote in Frontiers in Oncology, referring to cancers of the urinary tracts and male reproductive organs. Among the 61 sample videos examined by the researchers, health care practitioners contributed the content in 29, or 47 percent, of them. Yet, 22 posts, 36 percent, were misinformative, mostly due to outdated information.
More than half of the videos offered good content on disease symptoms and examinations. The authors concluded that “most videos on genitourinary cancers on TikTok are of poor to medium quality and reliability. However, videos posted by media agencies enjoyed great public attention and interaction. Medical practitioners could improve the video quality by cooperating with media agencies and avoiding unexplained terminologies.”
When health care providers post on social media, they must bear in mind that they have legal and ethical duties to their patients, profession and society, said Elizabeth Levy, founder and director of Physicians for Justice in Irvine, Calif., a nonprofit network of volunteer physicians partnering with public interest lawyers to address the social determinants of health.
“Providers are also responsible for understanding the mechanics of their posts,” such as who can see these messages and how long they stay up, Levy said. As a starting point for figuring what’s acceptable, providers could look at social media guidelines put out by their professional associations. Even beyond that, though, they must exercise prudent judgment. “As social media continues to evolve, providers will also need to stay updated with the changing risks and benefits of participation.”
Patients often research their providers online, so finding them on social media can help inform about values and approaches to care, said M. Sara Rosenthal, a professor and founding director of the program for bioethics and chair of the hospital ethics committee at the University of Kentucky College of Medicine.
Health care providers’ posts on social media also could promote patient education. They can advance informed consent and help patients navigate the risks and benefits of various treatments or preventive options. However, providers could violate ethical principles if they espouse “harmful, risky or questionable therapies or medical advice that is contrary to clinical practice guidelines or accepted standards of care,” Rosenthal said.
Inappropriate self-disclosure also can affect a provider’s reputation, said Kelly Michelson, a professor of pediatrics and director of the Center for Bioethics and Medical Humanities at Northwestern University’s Feinberg School of Medicine. A clinician’s obligations to professionalism extend beyond those moments when they are directly taking care of their patients, she said. “Many experts recommend against clinicians ‘friending’ patients or the families on social media because it blurs the patient-clinician boundary.”
Meanwhile, clinicians need to adhere closely to confidentiality. In sharing a patient’s case online for educational purposes, safeguarding identity becomes paramount. Removing names and changing minor details is insufficient, Michelson said.
“The patient-clinician relationship is sacred, and it can only be effective if patients have 100 percent confidence that all that happens with their clinician is kept in the strictest of confidence,” she said, adding that health care providers also should avoid obtaining information about their patients from social media because it can lead to bias and risk jeopardizing objectivity.
Academic clinicians can use social media as a recruitment tool to expand the pool of research participants for their studies, Michelson said. Because the majority of clinical research is conducted at academic medical centers, large segments of the population are excluded. “This affects the quality of the data and knowledge we gain from research,” she said.
Don S. Dizon, a professor of medicine and surgery at the Warren Alpert Medical School of Brown University in Providence, Rhode Island, uses LinkedIn and Doximity, as well as Twitter, Instagram, TikTok, Facebook, and most recently, YouTube and Post. He’s on Twitter nearly every day, where he interacts with the oncology community and his medical colleagues.
Also, he said, “I really like Instagram. It’s where you will see a hybrid of who I am professionally and personally. I’ve become comfortable sharing both up to a limit, but where else can I combine my appreciation of clothes with my professional life?” On that site, he’s seen sporting shirts with polka dots or stripes and an occasional bow-tie. He also posts photos of his cats.
Don S. Dizon, a professor of medicine and surgery at Brown, started using TikTok several years ago, telling medical stories in short-form videos.
Don S. Dizon
Dizon started using TikTok several years ago, telling medical stories in short-form videos. He may talk about an inspirational patient, his views on end-of-life care and death, or memories of people who have passed. But he is careful not to divulge any details that would identify anyone.
Recently, some people have become his patients after viewing his content on social media or on the Internet in general, which he clearly states isn’t a forum for medical advice. “In both situations, they are so much more relaxed when we meet, because it’s as if they have a sense of who I am as a person,” Dizon said. “I think that has helped so much in talking through a cancer diagnosis and a treatment plan, and yes, even discussions about prognosis.”
He also posts about equity and diversity. “I have found myself more likely to repost or react to issues that are inherently political, including racism, homophobia, transphobia and lack-of-access issues, because medicine is not isolated from society, and I truly believe that medicine is a social justice issue,” said Dizon, who is vice chair of diversity, equity, inclusion and professional integrity at the SWOG Cancer Research Network.
Through it all, Dizon likes “to break through the notion of doctor as infallible and all-knowing, the doctor as deity,” he said. “Humanizing what I do, especially in oncology, is something that challenges me on social media, and I appreciate the opportunities to do it on TikTok.”