Giving robots self-awareness as they move through space - and maybe even providing them with gene-like methods for storing rules of behavior - could be important steps toward creating more intelligent machines.
Now roboticists are going a step further, training their creations to do even better: understand their own image in space and interact with humans like humans do. Today, there are already robot-teachers like KeeKo, robot-pets like Moffin, robot-babysitters like iPal, and robotic companions for the elderly like Pepper.
But even these reasonably intelligent creations still have huge limitations, some scientists think. “There are niche applications for the current generations of robots,” says professor Anthony Zador at Cold Spring Harbor Laboratory—but they are not “generalists” who can do varied tasks all on their own, as they mostly lack the abilities to improvise, make decisions based on a multitude of facts or emotions, and adjust to rapidly changing circumstances. “We don’t have general purpose robots that can interact with the world. We’re ages away from that.”
Robotic spatial self-awareness – the achievement by the team at Columbia – is an important step toward creating more intelligent machines. Hod Lipson, professor of mechanical engineering who runs the Columbia lab, says that future robots will need this ability to assist humans better. Knowing how you look and where in space your parts are, decreases the need for human oversight. It also helps the robot to detect and compensate for damage and keep up with its own wear-and-tear. And it allows robots to realize when something is wrong with them or their parts. “We want our robots to learn and continue to grow their minds and bodies on their own,” Chen says. That’s what Zador wants too—and on a much grander level. “I want a robot who can drive my car, take my dog for a walk and have a conversation with me.”
Columbia scientists have trained a robot to become aware of its own "body," so it can map the right path to touch a ball without running into an obstacle, in this case a square.
Jane Nisselson and Yinuo Qin/ Columbia Engineering
Today’s technological advances are making some of these leaps of progress possible. One of them is the so-called Deep Learning—a method that trains artificial intelligence systems to learn and use information similar to how humans do it. Described as a machine learning method based on neural network architectures with multiple layers of processing units, Deep Learning has been used to successfully teach machines to recognize images, understand speech and even write text.
Trained by Google, one of these language machine learning geniuses, BERT, can finish sentences. Another one called GPT3, designed by San Francisco-based company OpenAI, can write little stories. Yet, both of them still make funny mistakes in their linguistic exercises that even a child wouldn’t. According to a paper published by Stanford’s Center for Research on Foundational Models, BERT seems to not understand the word “not.” When asked to fill in the word after “A robin is a __” it correctly answers “bird.” But try inserting the word “not” into that sentence (“A robin is not a __”) and BERT still completes it the same way. Similarly, in one of its stories, GPT3 wrote that if you mix a spoonful of grape juice into your cranberry juice and drink the concoction, you die. It seems that robots, and artificial intelligence systems in general, are still missing some rudimentary facts of life that humans and animals grasp naturally and effortlessly.
How does one give robots a genome? Zador has an idea. We can’t really equip machines with real biological nucleotide-based genes, but we can mimic the neuronal blueprint those genes create.
It's not exactly the robots’ fault. Compared to humans, and all other organisms that have been around for thousands or millions of years, robots are very new. They are missing out on eons of evolutionary data-building. Animals and humans are born with the ability to do certain things because they are pre-wired in them. Flies know how to fly, fish knows how to swim, cats know how to meow, and babies know how to cry. Yet, flies don’t really learn to fly, fish doesn’t learn to swim, cats don’t learn to meow, and babies don’t learn to cry—they are born able to execute such behaviors because they’re preprogrammed to do so. All that happens thanks to the millions of years of evolutions wired into their respective genomes, which give rise to the brain’s neural networks responsible for these behaviors. Robots are the newbies, missing out on that trove of information, Zador argues.
A neuroscience professor who studies how brain circuitry generates various behaviors, Zador has a different approach to developing the robotic mind. Until their creators figure out a way to imbue the bots with that information, robots will remain quite limited in their abilities. Each model will only be able to do certain things it was programmed to do, but it will never go above and beyond its original code. So Zador argues that we have to start giving robots a genome.
How does one do that? Zador has an idea. We can’t really equip machines with real biological nucleotide-based genes, but we can mimic the neuronal blueprint those genes create. Genomes lay out rules for brain development. Specifically, the genome encodes blueprints for wiring up our nervous system—the details of which neurons are connected, the strength of those connections and other specs that will later hold the information learned throughout life. “Our genomes serve as blueprints for building our nervous system and these blueprints give rise to a human brain, which contains about 100 billion neurons,” Zador says.
If you think what a genome is, he explains, it is essentially a very compact and compressed form of information storage. Conceptually, genomes are similar to CliffsNotes and other study guides. When students read these short summaries, they know about what happened in a book, without actually reading that book. And that’s how we should be designing the next generation of robots if we ever want them to act like humans, Zador says. “We should give them a set of behavioral CliffsNotes, which they can then unwrap into brain-like structures.” Robots that have such brain-like structures will acquire a set of basic rules to generate basic behaviors and use them to learn more complex ones.
Currently Zador is in the process of developing algorithms that function like simple rules that generate such behaviors. “My algorithms would write these CliffsNotes, outlining how to solve a particular problem,” he explains. “And then, the neural networks will use these CliffsNotes to figure out which ones are useful and use them in their behaviors.” That’s how all living beings operate. They use the pre-programmed info from their genetics to adapt to their changing environments and learn what’s necessary to survive and thrive in these settings.
For example, a robot’s neural network could draw from CliffsNotes with “genetic” instructions for how to be aware of its own body or learn to adjust its movements. And other, different sets of CliffsNotes may imbue it with the basics of physical safety or the fundamentals of speech.
At the moment, Zador is working on algorithms that are trying to mimic neuronal blueprints for very simple organisms—such as earthworms, which have only 302 neurons and about 7000 synapses compared to the millions we have. That’s how evolution worked, too—expanding the brains from simple creatures to more complex to the Homo Sapiens. But if it took millions of years to arrive at modern humans, how long would it take scientists to forge a robot with human intelligence? That’s a billion-dollar question. Yet, Zador is optimistic. “My hypotheses is that if you can build simple organisms that can interact with the world, then the higher level functions will not be nearly as challenging as they currently are.”
Sonja Bauman, 39, and Mariela Florez, 83, taking one of their regular walks in Plantation, Fla. The pair met through an online platform called Papa that provides "family on demand." (Photo by Sonja Bauman)
In 2020, Papa connected Bauman, the 39-year-old Floridian, with another woman in her mid-70s who lives alone and has very limited mobility. Bauman began driving her to doctor’s appointments and helping her with chores around the house. “When I’m not there, she doesn’t leave her apartment,” said Bauman. The two have gone to the gym together, and they walk slowly through the neighborhood, chatting so it feels less like exercise.
Parker was driven to start Papa by the problem of social isolation among seniors, exacerbated by the pandemic, but he believes users of all ages can benefit. “Many of our Pals feel more comfortable opening up with older members than their same-aged friends,” he said.
Other platforms aim for similar, in-person connections. Generation Tech unites teens with seniors for technology training. And Mon Ami, which provides case management software for aging and disability service providers, has an app that connects isolated older people with college-age volunteers.
Making new connections through video
Several new sites match you with strangers for real-time video chatting on various topics, such as finding common ground on political issues. Other video platforms focus on intergenerational connections.
S. Jay Olshansky, a gerontology professor at the University of Illinois-Chicago, recalls the first time he saw Hyunseung Lee, an 11-year-old from Seoul, through his computer screen. The kid was shy, but Olshansky, 67, encouraged him to ask questions. “Turns out, he was thirsting for this kind of interaction.”
They’d connected through Eldera, a platform that pairs mentors age 60 and up with mentees, using an algorithm, for video conversations. “The time and wisdom of older adults is the most important natural resource we can give future generations,” said Dana Griffin, Eldera's CEO. “Connecting through a screen is the opposite of social media.”
In weekly meetings, Olshansky noticed Lee’s unique interest in math. “There’s something special in you,” Olshansky told him. “How do we bring it to the surface?” He suggested Lee write a book on his favorite subject, and the preteen ran with it, cranking out 70 pages in two weeks. Lee has published his love letter to theorems on Amazon.
Hyunseung Lee, age 11, of Korea, and U.S. college professor Jay Olshanksy, 67, discuss math, strategy and Hyunsung's budding career as a book author during their video chats through a platform called Eldera. (Photo by Dana Palmer/Eldera)
Lee’s parents told Olshansky that their son has become more assertive — a recurring theme, Griffin said. “Confidence is the number one thing parents tell us about.” Since Eldera’s inception last year, the number of mentors has grown exponentially. Even so, Griffin said the waitlist for mentors typically numbers 200 kids.
Another site, Big and Mini, hosts video interactions between seniors and young adults; about 10,000 active users have joined since 2019, said co-founder Aditi Merchant.
Users often bring the benefits of their video interactions to their real-world relationships. Olshansky views Lee as an older version of his grandkids. “Eldera teaches me how to interact with them.” Lee, high on confidence, began instructing his classmates in math. Griffin noted that a group of Eldera mentors in Memphis, who met initially on Eldera, now take walks together in-person to trade ideas for helping each other’s Eldera kids solve problems in their schools and communities.
“We’ve evolved into a community for older adults who want to give back to the world,” said Griffin. Other new tools for connection take the form of virtual reality apps.
Connecting in virtual reality
During pandemic isolation, record numbers of people bought devices for virtual and augmented reality. Such gadgets can convince you that you’re hanging out with friends, even if they’re in another hemisphere. Lifelike simulations from miles away could be especially useful for meaningful interactions between people of different generations, since they’re often geographically segregated.
VR’s benefits require further study, but users report less social isolation and depression, according to MIT research. The immersive, 3-D experience is more compelling than FaceTime or Zoom. “It’s like the difference between a phone call and video call,” said Rick Robinson, Vice President of AARP’s Innovation Labs.
“When VR is designed right, the medium disappears,” said Jeremy Bailenson of Stanford.
Dana Pierce, a 56-year-old government employee in Indiana, got Meta's VR headset in May, 2021, thinking she’d enjoy it more than a new laptop. After many virtual group tours of exotic destinations, she has no regrets. Her adventures occur on Alcove, an app made by Robinson’s Innovation Labs. He co-created it with VR-company Rendever and sought input from people over age 50 to tailor it to their interests. “I’m an introvert,” said Pierce. “I’ve been more socially active since getting my headset than I am in real life.”
Tagging along with her to places like Paris are avatars representing real people around the world. She’s gotten to know VR users in their 70s, 80s and 90s, as well as younger people and some her own age. One is a new friend she plays chess with in relaxing nature settings. Another is her oldest son. He lives 90 minutes away but, earlier this year, Pierce welcomed him and his girlfriend to her virtual house on Alcove. They chatted in the living room decorated with family photos uploaded by Pierce. Then they took out a boat to go VR fishing — because why not — until 2 a.m.
“When VR is designed right, the medium disappears,” said Jeremy Bailenson, a communications professor who directs Stanford’s Virtual Human Interaction Lab. He’s teaching a class of 175 students entirely in VR. After months of covid isolation, the first time the class met, “there was a big catharsis. It really feels like you’re in a big crowd.” Like-minded people meet in VR for events such as comedy shows and creative writing meetups, while the Swedish pop group ABBA has performed this year as digital versions of themselves (“ABBA-tars”) during a virtual concert tour.
Karen Fingerman, a psychologist and director of the Texas Aging and Longevity Center at the University of Texas-Austin, supports the idea of VR for social connection, though she added that some people need it more than others. Hospitals and assisted-living facilities are using products such as Penumbra’s REAL I-Series and MyndVR to bring VR excursions to isolated patients and seniors. “If you’re in a bed or facility, this gives you something to talk about,” said Gita Barry, Penumbra’s executive vice president.
Pierce uses it on most days. She may see another adult son, who lives with her, less often as a result. But VR helps her manage real-world stressors, more than escaping them. After a long workday, she visits her back porch on Alcove, which overlooks a pond. “It’s my little retreat,” she said. “VR improves my mood. It’s added a lot to my life.”
Some seniors are using more than one technology. Olshansky and Lee discuss strategy while playing Internet chess. And Olshansky recently began using VR. He sees his sister, who lives far away, in a virtual beach house. “It’d be a great way to interact with Hyunseung,” he said. “I should get him a headset.”
A version of this article first appeared in The Washington Post on December 3, 2021.