In an initial study, Canary analyzed speech recordings with AI and identified early stage Alzheimer’s with 96 percent accuracy.
Dave Arnold retired in his 60s and began spending time volunteering in local schools. But then he started misplacing items, forgetting appointments and losing his sense of direction. Eventually he was diagnosed with early stage Alzheimer’s.
“Hearing the diagnosis made me very emotional and tearful,” he said. “I immediately thought of all my mom had experienced.” His mother suffered with the condition for years before passing away. Over the last year, Arnold has worked for the Alzheimer’s Association as one of its early stage advisors, sharing his insights to help others in the initial stages of the disease.
Arnold was diagnosed sooner than many others. It's important to find out early, when interventions can make the most difference. One promising avenue is looking at how people talk. Research has shown that Alzheimer’s affects a part of the brain that controls speech, resulting in small changes before people show other signs of the disease.
Now, Canary Speech, a company based in Utah, is using AI to examine elements like the pitch of a person’s voice and their pauses. In an initial study, Canary analyzed speech recordings with AI and identified early stage Alzheimer’s with 96 percent accuracy.
Developing the AI model
Canary Speech’s CEO, Henry O’Connell, met cofounder Jeff Adams about 40 years before they started the company. Back when they first crossed paths, they were both living in Bethesda, Maryland; O’Connell was a research fellow at the National Institutes of Health studying rare neurological diseases, while Adams was working to decode spy messages. Later on, Adams would specialize in building mathematical models to analyze speech and sound as a team leader in developing Amazon's Alexa.
It wasn't until 2015 that they decided to make use of the fit between their backgrounds. ““We established Canary Speech in 2017 to build a product that could be used in multiple languages in clinical environments,” O'Connell says.
The need is growing. About 55 million people worldwide currently live with Alzheimer’s, a number that is expected to double by 2050. Some scientists think the disease results from a buildup of plaque in the brain. It causes mild memory loss at first and, over time, this issue get worse while other symptoms, such as disorientation and hallucinations, can develop. Treatment to manage the disease is more effective in the earlier stages, but detection is difficult since mild symptoms are often attributed to the normal aging process.
O’Connell and Adams specialize in the complex ways that Alzheimer’s effects how people speak. Using AI, their mathematical model analyzes 15 million data points every minute, focusing on certain features of speech such as pitch, pauses and elongation of words. It also pays attention to how the vibrations of vocal cords change in different stages of the disease.
To create their model, the team used a type of machine learning called deep neural nets, which looks at multiple layers of data - in this case, the multiple features of a person’s speech patterns.
“Deep neural nets allow us to look at much, much larger data sets built out of millions of elements,” O’Connell explained. “Through machine learning and AI, we’ve identified features that are very sensitive to an Alzheimer’s patient versus [people without the disease] and also very sensitive to mild cognitive impairment, early stage and moderate Alzheimer's.” Based on their learnings, Canary is able to classify the disease stage very quickly, O’Connell said.
“When we’re listening to sublanguage elements, we’re really analyzing the direct result of changes in the brain in the physical body,” O’Connell said. “The brain controls your vocal cords: how fast they vibrate, the expansion of them, the contraction.” These factors, along with where people put their tongues when talking, function subconsciously and result in subtle changes in the sounds of speech.
Further testing is needed
In an initial trial, Canary analyzed speech recordings from phone calls to a large U.S. health insurer. They looked at the audio recordings of 651 policyholders who had early stage Alzheimer’s and 1018 who did not have the condition, aiming for a representative sample of age, gender and race. They used this data to create their first diagnostic model and found that it was 96 percent accurate in identifying Alzheimer’s.
Christian Herff, an assistant professor of neuroscience at Maastricht University in the Netherlands, praised this approach while adding that further testing is needed to assess its effectiveness.
“I think the general idea of identifying increased risk for cognitive impairment based on speech characteristics is very feasible, particularly when change in a user’s voice is monitored, for example, by recording speech every year,” Herff said. He noted that this can only be a first indication, not a full diagnosis. The accuracy still needs to be validated in studies that follows individuals over a period of time, he said.
Toby Walsh, a professor of artificial intelligence at the University of New South Wales, also thinks Canary’s tool has potential but highlights that Canary could diagnose some people who don’t really have the disease. “This is an interesting and promising application of AI,” he said, “but these tools need to be used carefully. Imagine the anxiety of being misdiagnosed with Alzheimer’s.”
As with many other AI tools, privacy and bias are additional issues to monitor closely, Walsh said.
Other languages
A related issue is that not everyone is fluent in English. Mahnaz Arvaneh, a senior lecturer in automatic control and systems engineering at the University of Sheffield, said this could be a blind spot.
“The system may not be very accurate for those who have English as their second language as their speaking patterns would be different, and any issue might be because of language deficiency rather than cognitive issues,” Arvaneh said.
The team is expanding to multiple languages starting with Japanese and Spanish. The elements of the model that make up the algorithm are very similar, but they need to be validated and retrained in a different language, which will require access to more data.
Recently, Canary analyzed the phone calls of 233 Japanese patients who had mild cognitive impairment and 704 healthy people. Using an English model they were able to identify the Japanese patients who had mild cognitive impairment with 78 percent accuracy. They also developed a model in Japanese that was 45 percent accurate, and they’re continuing to train it with more data.
The future
Canary is using their model to look at other diseases like Huntington’s and Parkinson’s. They’re also collaborating with pharmaceuticals to validate potential therapies for Alzheimer’s. By looking at speech patterns over time, Canary can get an indication of how well these drugs are working.
Dave Arnold and his wife dance at his nephew’s wedding in Rochester, New York, ten years ago, before his Alzheimer's diagnosis.
Dave Arnold
Ultimately, they want to integrate their tool into everyday life. “We want it to be used in a smartphone, or a teleconference call so that individuals could be examined in their home,” O’Connell said. “We could follow them over time and work with clinical teams and hospitals to improve the evaluation of patients and contribute towards an accurate diagnosis.”
Arnold, the patient with early stage Alzheimer’s, sees great promise. “The process of getting a diagnosis is already filled with so much anxiety,” he said. “Anything that can be done to make it easier and less stressful would be a good thing, as long as it’s proven accurate.”
When graduating college this month, many North American engineering students will take a special pledge, with a history dating back to 1925.
The tradition of taking an engineering oath began over a century ago in Canada. In 1922, Herbert E.T. Haultain, professor of mining engineering at the University of Toronto, presented the idea at the annual meeting of the Engineering Institute of Canada. The seven past presidents of that body were in attendance, heard Haultain’s speech and accepted his suggestion to form a committee to create an honor oath. Later, they formed the nonprofit Corporation of the Seven Wardens, which would oversee the ritual. Next year, in 1923, with the encouragement of the Seven Wardens, Haultain wrote to poet and writer Rudyard Kipling, asking him to develop a professional oath for engineers. “We are a tribe—a very important tribe within the community,” Haultain said in the letter, “but we are lacking in tribal spirit, or perhaps I should say, in manifestation of tribal spirit. Also, we are inarticulate. Can you help us?”
While Kipling is most famous now for “The Jungle Book” and perhaps his poem “Gunga Din,” he had also written a short story about engineers, “The Bridge Builders.” His poem “The Sons of Martha” can be read as a celebration of engineers:
It is their care in all the ages to take the buffet and cushion the shock.
It is their care that the gear engages; it is their care that the switches lock.
It is their care that the wheels run truly; it is their care to embark and entrain,
Tally, transport, and deliver duly the Sons of Mary by land and main.
Kipling accepted the ask and wrote the Ritual of the Calling of an Engineer, which he sent to Haultain a month later. In his response to Haultain, he stated that he preferred the word “Obligation” to “Oath.” He wrote the Obligation using Old English lettering and the old-fashioned capitalization. Kipling’s Obligation binds engineers upon their “Honor and Cold Iron” to not “suffer or pass, or be privy to the passing of, Bad Workmanship or Faulty Material,” and pardon is asked “in the presence of my betters and my equals in my Calling” for the engineer’s “assured failures and derelictions.” The hope is that when one is tempted to shoddy work by weakness or weariness, the memory of the Obligation “and the company before whom it was entered into, may return to me to aid, comfort, and restrain.”
Using the Obligation, The Seven Wardens created an induction ceremony, which seeks to unify the profession and recognize engineering’s ethics, including responsibility to the public and the need to make the best decisions possible. The induction ceremony included recitation of Kipling’s “Obligation” and incorporated an anvil, a hammer, an iron chain, and an iron ring. The inductee engineers sat inside an area marked off by the iron chain, with their more senior colleagues outside that area. At the start of the ritual, the leader beat out S-S-T in Morse code with the hammer and anvil—the letters standing for Steel, Stone, and Time. A more experienced and previously obligated engineer placed the ring on the small finger of the inductee engineer’s working hand. As per Kipling, the ring’s rough, faceted texture symbolized “the young engineer’s mind” and the difficulties engineers face in mastering their discipline.
A persistent myth purports that the original iron rings were made from the beams or bolts of the Quebec Bridge that failed twice during construction.
The first induction ceremony took place on April 25, 1925, in Montreal to obligate two of the Seven Wardens, along with four graduates from the University of Toronto class of 1893. On May 1 of that year, 14 more engineers were obligated at the University of Toronto. From that time to today most Canadian professional engineers have gone through that same ritual in their various camps, called Kipling camps—local chapters associated with various Canadian universities.
Henry Petroski, Duke University’s professor of civil engineering and history, notes in his book, “Forgive Design: Understanding Failure,” that Kipling’s poem “Sons of Martha” is often read as part of the ritual. However, sometimes inductees read Kipling’s “Hymn of Breaking Strain,” instead, which graphically depicts disastrous outcomes of engineering mistakes. The first stanza of that poem says:
The careful text-books measure
(Let all who build beware!)
The load, the shock, the pressure
Material can bear.
So, when the buckled girder
Lets down the grinding span,
'The blame of loss, or murder,
Is laid upon the man.
Not on the Stuff—the Man!
As if to strengthen the importance of these concepts, a persistent myth purports that the original iron rings were made from the beams or bolts of the Quebec Bridge that failed twice during construction. The bridge spans the St. Lawrence River upriver from Quebec City, and at the time of its construction was the world’s longest at 1,800 feet. Due to engineering errors and poor oversight, the bridge’s own weight exceeded its carrying capacity. Moreover, engineers downplayed danger when bridge beams began to warp under stress, saying that they were probably warped before they were installed. On August 29, 1907, the bridge collapsed, killing 75 of 86 workers. A second collapse occurred in 1916 when lifting equipment failed, and thirteen more workers died.
The ring myth, however, couldn’t be true. The original iron rings couldn’t have come from the failed bridge since it was made of steel, not wrought iron. Today the rings are made from stainless steel because iron deteriorates and stains engineers’ finger black.
On August 14, 2018, Morandi Bridge over Polcevera River in Genoa, Italy, collapsed from structural failure, killing 43 people.
Adobe Stock
The Seven Wardens decided to restrict the ritual to engineers trained in Canada. They copyrighted the obligation oath in Canada and the United States in 1935. Although the ritual is not a requirement for professional licensing, just like the Hippocratic Oath is not part of medical licensing, it remains a long-standing tradition.
The American Obligation of the Engineer has its own creation story, albeit a very different one. The American Order of the Engineer (OOE) was initiated in 1970, during the era of the anti-war protests, Apollo missions and the first Earth Day. On May 4, 1970, the National Guard shot into a crowd of protesters at Kent State University, killing four people. The two authors of the American obligation—Cleveland State University’s (CSU) engineering professor John Janssen and his wife Susan—reflected these historical events in the oath they wrote. Their version of the oath binds engineers to “practice integrity and fair dealing.” It also notes that their “skill carries with it the obligation to serve humanity by making the best use of the Earth’s precious wealth.” As Petroski explains in his book, “campus antiwar protestors around the country tended to view engineers as complicit in weapons proliferation [which] prompted some [CSU] engineering student leaders to look for a means of asserting some more positive values.”
Kip A. Wedel, associate professor of history and politics at Bethel College, wrote in his book, “The Obligation: A History of the Order of the Engineer,” that the ceremony was not a direct response to the Kent State shootings—it was already scheduled when the shootings happened. Yet, engineering students found the ceremony a positive action they could take in contrast to the overall turmoil. The first American ritual took place on June 4, 1970, at CSU. In total, 170 students, faculty members, and practicing engineers took the obligation. This established CSU as the first Link of the Order, as the OOE designates its local chapters. For their first ceremony, the CSU students fabricated smooth, unfaceted rings from stainless steel pipe. Later they were replaced by factory-made rings. According to Paula Ostaff, OOE’s Executive Director, about 20,000 eligible students and alumni obligate themselves yearly.
Societies hope that every engineer is imbued with a strong ethical sense and that their pledges are never far from mind. For some, the rings they wear serve a daily reminder that every paper they sign off on is touched by a physical reminder of their commitment.
These ethical and responsible engineering practices are especially salient today, when one in three American bridges needs repair or replacement, some have already collapsed, and engineers are working on projects related to the bipartisan infrastructure bill President Biden signed into law in 2021. Canada has committed $33 billion to its Investing in Canada Infrastructure Program. At the heart of these grand projects are many thousands of professional engineers, collectively working millions of hours. The professional vows they took aim to assure that the homes, bridges and airplanes they build will work as expected.
A movie still from the 1966 film "Fantastic Voyage"
"Chemotherapy is delivered systemically," Bionaut-founder and CEO Michael Shpigelmacher says. "Often only a small percentage arrives at the location where it is actually needed."
But what if it was possible to send a tiny robot through the body to attack a tumor or deliver a drug at exactly the right location?
Several startups and academic institutes worldwide are working to develop such a solution but Bionaut Labs seems the furthest along in advancing its invention. "You can think of the Bionaut as a tiny screw that moves through the veins as if steered by an invisible screwdriver until it arrives at the tumor," Shpigelmacher explains. Via Zoom, he shares the screen of an X-ray machine in his Culver City lab to demonstrate how the half-transparent, yellowish device winds its way along the spine in the body. The nanobot contains a tiny but powerful magnet. The "invisible screwdriver" is an external magnetic field that rotates that magnet inside the device and gets it to move and change directions.
The current model has a diameter of less than a millimeter. Shpigelmacher's engineers could build the miniature vehicle even smaller but the current size has the advantage of being big enough to see with bare eyes. It can also deliver more medicine than a tinier version. In the Zoom demonstration, the micorobot is injected into the spine, not unlike an epidural, and pulled along the spine through an outside magnet until the Bionaut reaches the brainstem. Depending which organ it needs to reach, it could be inserted elsewhere, for instance through a catheter.
"The hope is that we can develop a vehicle to transport medication deep into the body," says Max Planck scientist Tian Qiu.
Imagine moving a screw through a steak with a magnet — that's essentially how the device works. But of course, the Bionaut is considerably different from an ordinary screw: "At the right location, we give a magnetic signal, and it unloads its medicine package," Shpigelmacher says.
To start, Bionaut Labs wants to use its device to treat Parkinson's disease and brain stem gliomas, a type of cancer that largely affects children and teenagers. About 300 to 400 young people a year are diagnosed with this type of tumor. Radiation and brain surgery risk damaging sensitive brain tissue, and chemotherapy often doesn't work. Most children with these tumors live less than 18 months. A nanobot delivering targeted chemotherapy could be a gamechanger. "These patients really don't have any other hope," Shpigelmacher says.
Of course, the main challenge of the developing such a device is guaranteeing that it's safe. Because tissue is so sensitive, any mistake could risk disastrous results. In recent years, Bionaut has tested its technology in dozens of healthy sheep and pigs with no major adverse effects. Sheep make a good stand-in for humans because their brains and spines are similar to ours.
The Bionaut device is about the size of a grain of rice.
Bionaut Labs
"As the Bionaut moves through brain tissue, it creates a transient track that heals within a few weeks," Shpigelmacher says. The company is hoping to be the first to test a nanobot in humans. In December 2022, it announced that a recent round of funding drew $43.2 million, for a total of 63.2 million, enabling more research and, if all goes smoothly, human clinical trials by early next year.
Once the technique has been perfected, further applications could include addressing other kinds of brain disorders that are considered incurable now, such as Alzheimer's or Huntington's disease. "Microrobots could serve as a bridgehead, opening the gateway to the brain and facilitating precise access of deep brain structure – either to deliver medication, take cell samples or stimulate specific brain regions," Shpigelmacher says.
Robot-assisted hybrid surgery with artificial intelligence is already used in state-of-the-art surgery centers, and many medical experts believe that nanorobotics will be the instrument of the future. In 2016, three scientists were awarded the Nobel Prize in Chemistry for their development of "the world's smallest machines," nano "elevators" and minuscule motors. Since then, the scientific experiments have progressed to the point where applicable devices are moving closer to actually being implemented.
Bionaut's technology was initially developed by a research team lead by Peer Fischer, head of the independent Micro Nano and Molecular Systems Lab at the Max Planck Institute for Intelligent Systems in Stuttgart, Germany. Fischer is considered a pioneer in the research of nano systems, which he began at Harvard University more than a decade ago. He and his team are advising Bionaut Labs and have licensed their technology to the company.
"The hope is that we can develop a vehicle to transport medication deep into the body," says Max Planck scientist Tian Qiu, who leads the cooperation with Bionaut Labs. He agrees with Shpigelmacher that the Bionaut's size is perfect for transporting medication loads and is researching potential applications for even smaller nanorobots, especially in the eye, where the tissue is extremely sensitive. "Nanorobots can sneak through very fine tissue without causing damage."
In "Fantastic Voyage," Raquel Welch's adventures inside the body of a dissident scientist let her swim through his veins into his brain, but her shrunken miniature submarine is attacked by antibodies; she has to flee through the nerves into the scientist's eye where she escapes into freedom on a tear drop. In reality, the exit in the lab is much more mundane. The Bionaut simply leaves the body through the same port where it entered. But apart from the dramatization, the "Fantastic Voyage" was almost prophetic, or, as Shpigelmacher says, "Science fiction becomes science reality."
This article was first published by Leaps.org on April 12, 2021.