“Siri, Read My Mind”: A New Device Lets Users Think Commands
Sometime in the near future, we won't need to type on a smartphone or computer to silently communicate our thoughts to others.
"We're moving as fast as possible to get the technology right, to get the ethics right, to get everything right."
In fact, the devices themselves will quietly understand our intentions and express them to other people. We won't even need to move our mouths.
That "sometime in the near future" is now.
At the recent TED Conference, MIT student and TED Fellow Arnav Kapur was onstage with a colleague doing the first live public demo of his new technology. He was showing how you can communicate with a computer using signals from your brain. The usually cool, erudite audience seemed a little uncomfortable.
"If you look at the history of computing, we've always treated computers as external devices that compute and act on our behalf," Kapur said. "What I want to do is I want to weave computing, AI and Internet as part of us."
His colleague started up a device called AlterEgo. Thin like a sticker, AlterEgo picks up signals in the mouth cavity. It recognizes the intended speech and processes it through the built-in AI. The device then gives feedback to the user directly through bone conduction: It vibrates your inner ear drum and gives you a response meshing with your normal hearing.
Onstage, the assistant quietly thought of a question: "What is the weather in Vancouver?" Seconds later, AlterEgo told him in his ear. "It's 50 degrees and rainy here in Vancouver," the assistant announced.
AlterEgo essentially gives you a built-in Siri.
"We don't have a deadline [to go to market], but we're moving as fast as possible to get the technology right, to get the ethics right, to get everything right," Kapur told me after the talk. "We're developing it both as a general purpose computer interface and [in specific instances] like on the clinical side or even in people's homes."
Nearly-telepathic communication actually makes sense now. About ten years ago, the Apple iPhone replaced the ubiquitous cell phone keyboard with a blank touchscreen. A few years later, Google Glass put computer screens into a simple lens. More recently, Amazon Alexa and Microsoft Cortana have dropped the screen and gone straight for voice control. Now those voices are getting closer to our minds and may even become indistinguishable in the future.
"We knew the voice market was growing, like with getting map locations, and audio is the next frontier of user interfaces," says Dr. Rupal Patel, Founder and CEO of VocalID. The startup literally gives voices to the voiceless, particularly people unable to speak because of illness or other circumstances.
"We start with [our database of] human voices, then train our deep learning technology to learn the pattern of speech… We mix voices together from our voice bank, so it's not just Damon's voice, but three or five voices. They are different enough to blend it into a voice that does not exist today – kind of like a face morph."
The VocalID customer then has a voice as unique as he or she is, mixed together like a Sauvignon blend. It is a surrogate voice for those of us who cannot speak, just as much as AlterEgo is a surrogate companion for our brains.
"I'm very skeptical keyboards or voice-based communication will be replaced any time soon."
Voice equality will become increasingly important as Siri, Alexa and voice-based interfaces become the dominant communication method.
It may feel odd to view your voice as a privilege, but as the world becomes more voice-activated, there will be a wider gap between the speakers and the voiceless. Picture going shopping without access to the Internet or trying to eat healthily when your neighborhood is a food desert. And suffering from vocal difficulties is more common than you might think. In fact, according to government statistics, around 7.5 million people in the U.S. have trouble using their voices.
While voice communication appears to be here to stay, at least for now, a more radical shift to mind-controlled communication is not necessarily inevitable. Tech futurist Wagner James Au, for one, is dubious.
"I'm very skeptical keyboards or voice-based communication will be replaced any time soon. Generation Z has grown up with smartphones and games like Fortnite, so I don't see them quickly switching to a new form factor. It's still unclear if even head-mounted AR/VR displays will see mass adoption, and mind-reading devices are a far greater physical imposition on the user."
How adopters use the newest brain impulse-reading, voice-altering technology is a much more complicated discussion. This spring, a video showed U.S. House Speaker Nancy Pelosi stammering and slurring her words at a press conference. The problem is that it didn't really happen: the video was manufactured and heavily altered from the original source material.
So-called deepfake videos use computer algorithms to capture the visual and vocal cues of an individual, and then the creator can manipulate it to say whatever it wants. Deepfakes have already created false narratives in the political and media systems – and these are only videos. Newer tech is making the barrier between tech and our brains, if not our entire identity, even thinner.
"Last year," says Patel of VocalID, "we did penetration testing with our voices on banks that use voice control – and our generation 4 system is even tricky for you and me to identify the difference (between real and fake). As a forward-thinking company, we want to prevent risk early on by watermarking voices, creating a detector of false voices, and so on." She adds, "The line will become more blurred over time."
Onstage at TED, Kapur reassured the audience about who would be in the driver's seat. "This is why we designed the system to deliberately record from the peripheral nervous system, which is why the control in all situations resides with the user."
And, like many creators, he quickly shifted back to the possibilities. "What could the implications of something like this be? Imagine perfectly memorizing things, where you perfectly record information that you silently speak, and then hear them later when you want to, internally searching for information, crunching numbers at speeds computers do, silently texting other people."
"The potential," he concluded, "could be far-reaching."
How to Measure Your Stress, with Dr. Rosalind Picard
Today’s podcast guest is Rosalind Picard, a researcher, inventor named on over 100 patents, entrepreneur, author, professor and engineer. When it comes to the science related to endowing computer software with emotional intelligence, she wrote the book. It’s published by MIT Press and called Affective Computing.
Dr. Picard is founder and director of the MIT Media Lab’s Affective Computing Research Group. Her research and engineering contributions have been recognized internationally. For example, she received the 2022 International Lombardy Prize for Computer Science Research, considered by many to be the Nobel prize in computer science.
Through her research and companies, Dr. Picard has developed wearable sensors, algorithms and systems for sensing, recognizing and responding to information about human emotion. Her products are focused on using fitness trackers to advance clinical quality treatments for a range of conditions.
Meanwhile, in just the past few years, numerous fitness tracking companies have released products with their own stress sensors and systems. You may have heard about Fitbit’s Stress Management Score, or Whoop’s Stress Monitor – these features and apps measure things like your heart rhythm and a certain type of invisible sweat to identify stress. They’re designed to raise awareness about forms of stress such as anxieties and anger, and suggest strategies like meditation to relax in real time when stress occurs.
But how well do these off-the-shelf gadgets work? There’s no one more knowledgeable and experienced than Rosalind Picard to explain the science behind these stress features, what they do exactly, how they might be able to help us, and their current shortcomings.
Dr. Picard is a member of the National Academy of Engineering and a Fellow of the National Academy of Inventors, and a popular speaker who’s given over a hundred invited keynote talks and a TED talk with over 2 million views. She holds a Bachelors in Electrical Engineering from Georgia Tech, and Masters and Doctorate degrees in Electrical Engineering and Computer Science from MIT. She lives in Newton, Massachusetts with her husband, where they’ve raised three sons.
In our conversation, we discuss stress scores on fitness trackers to improve well-being. She describes the difference between commercial products that might help people become more mindful of their health and products that are FDA approved and really capable of advancing the science. We also talk about several fascinating findings and concepts discovered in Dr. Picard’s lab including the multiple arousal theory, a phenomenon you’ll want to hear about. And we explore the complexity of stress, one reason it’s so tough to measure. For example, many forms of stress are actually good for us. Can fitness trackers tell the difference between stress that’s healthy and unhealthy?
Show links:
- Dr. Picard’s book, Affective Computing
- Dr. Picard’s bio
- Dr. Picard on Twitter
- Dr. Picard’s company, Empatica - https://www.empatica.com/ - The FDA-cleared Empatica Health Monitoring Platform provides accurate, continuous health insights for researchers and clinicians, collected in the real world
- Empatica Twitter
- Dr. Picard and her team have published hundreds of peer-reviewed articles across AI, Machine Learning, Affective Computing, Digital Health, and Human-computer interaction.
- Dr. Picard’s TED talk
Rosalind Picard
If you look back on the last century of scientific achievements, you might notice that most of the scientists we celebrate are overwhelmingly white, while scientists of color take a backseat. Since the Nobel Prize was introduced in 1901, for example, no black scientists have landed this prestigious award.
The work of black women scientists has gone unrecognized in particular. Their work uncredited and often stolen, black women have nevertheless contributed to some of the most important advancements of the last 100 years, from the polio vaccine to GPS.
Here are five black women who have changed science forever.
Dr. May Edward Chinn
Dr. May Edward Chinn practicing medicine in Harlem
George B. Davis, PhD.
Chinn was born to poor parents in New York City just before the start of the 20th century. Although she showed great promise as a pianist, playing with the legendary musician Paul Robeson throughout the 1920s, she decided to study medicine instead. Chinn, like other black doctors of the time, were barred from studying or practicing in New York hospitals. So Chinn formed a private practice and made house calls, sometimes operating in patients’ living rooms, using an ironing board as a makeshift operating table.
Chinn worked among the city’s poor, and in doing this, started to notice her patients had late-stage cancers that often had gone undetected or untreated for years. To learn more about cancer and its prevention, Chinn begged information off white doctors who were willing to share with her, and even accompanied her patients to other clinic appointments in the city, claiming to be the family physician. Chinn took this information and integrated it into her own practice, creating guidelines for early cancer detection that were revolutionary at the time—for instance, checking patient health histories, checking family histories, performing routine pap smears, and screening patients for cancer even before they showed symptoms. For years, Chinn was the only black female doctor working in Harlem, and she continued to work closely with the poor and advocate for early cancer screenings until she retired at age 81.
Alice Ball
Pictorial Press Ltd/Alamy
Alice Ball was a chemist best known for her groundbreaking work on the development of the “Ball Method,” the first successful treatment for those suffering from leprosy during the early 20th century.
In 1916, while she was an undergraduate student at the University of Hawaii, Ball studied the effects of Chaulmoogra oil in treating leprosy. This oil was a well-established therapy in Asian countries, but it had such a foul taste and led to such unpleasant side effects that many patients refused to take it.
So Ball developed a method to isolate and extract the active compounds from Chaulmoogra oil to create an injectable medicine. This marked a significant breakthrough in leprosy treatment and became the standard of care for several decades afterward.
Unfortunately, Ball died before she could publish her results, and credit for this discovery was given to another scientist. One of her colleagues, however, was able to properly credit her in a publication in 1922.
Henrietta Lacks
onathan Newton/The Washington Post/Getty
The person who arguably contributed the most to scientific research in the last century, surprisingly, wasn’t even a scientist. Henrietta Lacks was a tobacco farmer and mother of five children who lived in Maryland during the 1940s. In 1951, Lacks visited Johns Hopkins Hospital where doctors found a cancerous tumor on her cervix. Before treating the tumor, the doctor who examined Lacks clipped two small samples of tissue from Lacks’ cervix without her knowledge or consent—something unthinkable today thanks to informed consent practices, but commonplace back then.
As Lacks underwent treatment for her cancer, her tissue samples made their way to the desk of George Otto Gey, a cancer researcher at Johns Hopkins. He noticed that unlike the other cell cultures that came into his lab, Lacks’ cells grew and multiplied instead of dying out. Lacks’ cells were “immortal,” meaning that because of a genetic defect, they were able to reproduce indefinitely as long as certain conditions were kept stable inside the lab.
Gey started shipping Lacks’ cells to other researchers across the globe, and scientists were thrilled to have an unlimited amount of sturdy human cells with which to experiment. Long after Lacks died of cervical cancer in 1951, her cells continued to multiply and scientists continued to use them to develop cancer treatments, to learn more about HIV/AIDS, to pioneer fertility treatments like in vitro fertilization, and to develop the polio vaccine. To this day, Lacks’ cells have saved an estimated 10 million lives, and her family is beginning to get the compensation and recognition that Henrietta deserved.
Dr. Gladys West
Andre West
Gladys West was a mathematician who helped invent something nearly everyone uses today. West started her career in the 1950s at the Naval Surface Warfare Center Dahlgren Division in Virginia, and took data from satellites to create a mathematical model of the Earth’s shape and gravitational field. This important work would lay the groundwork for the technology that would later become the Global Positioning System, or GPS. West’s work was not widely recognized until she was honored by the US Air Force in 2018.
Dr. Kizzmekia "Kizzy" Corbett
TIME Magazine
At just 35 years old, immunologist Kizzmekia “Kizzy” Corbett has already made history. A viral immunologist by training, Corbett studied coronaviruses at the National Institutes of Health (NIH) and researched possible vaccines for coronaviruses such as SARS (Severe Acute Respiratory Syndrome) and MERS (Middle East Respiratory Syndrome).
At the start of the COVID pandemic, Corbett and her team at the NIH partnered with pharmaceutical giant Moderna to develop an mRNA-based vaccine against the virus. Corbett’s previous work with mRNA and coronaviruses was vital in developing the vaccine, which became one of the first to be authorized for emergency use in the United States. The vaccine, along with others, is responsible for saving an estimated 14 million lives.