Giving robots self-awareness as they move through space - and maybe even providing them with gene-like methods for storing rules of behavior - could be important steps toward creating more intelligent machines.
Now roboticists are going a step further, training their creations to do even better: understand their own image in space and interact with humans like humans do. Today, there are already robot-teachers like KeeKo, robot-pets like Moffin, robot-babysitters like iPal, and robotic companions for the elderly like Pepper.
But even these reasonably intelligent creations still have huge limitations, some scientists think. “There are niche applications for the current generations of robots,” says professor Anthony Zador at Cold Spring Harbor Laboratory—but they are not “generalists” who can do varied tasks all on their own, as they mostly lack the abilities to improvise, make decisions based on a multitude of facts or emotions, and adjust to rapidly changing circumstances. “We don’t have general purpose robots that can interact with the world. We’re ages away from that.”
Robotic spatial self-awareness – the achievement by the team at Columbia – is an important step toward creating more intelligent machines. Hod Lipson, professor of mechanical engineering who runs the Columbia lab, says that future robots will need this ability to assist humans better. Knowing how you look and where in space your parts are, decreases the need for human oversight. It also helps the robot to detect and compensate for damage and keep up with its own wear-and-tear. And it allows robots to realize when something is wrong with them or their parts. “We want our robots to learn and continue to grow their minds and bodies on their own,” Chen says. That’s what Zador wants too—and on a much grander level. “I want a robot who can drive my car, take my dog for a walk and have a conversation with me.”
Columbia scientists have trained a robot to become aware of its own "body," so it can map the right path to touch a ball without running into an obstacle, in this case a square.
Jane Nisselson and Yinuo Qin/ Columbia Engineering
Today’s technological advances are making some of these leaps of progress possible. One of them is the so-called Deep Learning—a method that trains artificial intelligence systems to learn and use information similar to how humans do it. Described as a machine learning method based on neural network architectures with multiple layers of processing units, Deep Learning has been used to successfully teach machines to recognize images, understand speech and even write text.
Trained by Google, one of these language machine learning geniuses, BERT, can finish sentences. Another one called GPT3, designed by San Francisco-based company OpenAI, can write little stories. Yet, both of them still make funny mistakes in their linguistic exercises that even a child wouldn’t. According to a paper published by Stanford’s Center for Research on Foundational Models, BERT seems to not understand the word “not.” When asked to fill in the word after “A robin is a __” it correctly answers “bird.” But try inserting the word “not” into that sentence (“A robin is not a __”) and BERT still completes it the same way. Similarly, in one of its stories, GPT3 wrote that if you mix a spoonful of grape juice into your cranberry juice and drink the concoction, you die. It seems that robots, and artificial intelligence systems in general, are still missing some rudimentary facts of life that humans and animals grasp naturally and effortlessly.
How does one give robots a genome? Zador has an idea. We can’t really equip machines with real biological nucleotide-based genes, but we can mimic the neuronal blueprint those genes create.
It's not exactly the robots’ fault. Compared to humans, and all other organisms that have been around for thousands or millions of years, robots are very new. They are missing out on eons of evolutionary data-building. Animals and humans are born with the ability to do certain things because they are pre-wired in them. Flies know how to fly, fish knows how to swim, cats know how to meow, and babies know how to cry. Yet, flies don’t really learn to fly, fish doesn’t learn to swim, cats don’t learn to meow, and babies don’t learn to cry—they are born able to execute such behaviors because they’re preprogrammed to do so. All that happens thanks to the millions of years of evolutions wired into their respective genomes, which give rise to the brain’s neural networks responsible for these behaviors. Robots are the newbies, missing out on that trove of information, Zador argues.
A neuroscience professor who studies how brain circuitry generates various behaviors, Zador has a different approach to developing the robotic mind. Until their creators figure out a way to imbue the bots with that information, robots will remain quite limited in their abilities. Each model will only be able to do certain things it was programmed to do, but it will never go above and beyond its original code. So Zador argues that we have to start giving robots a genome.
How does one do that? Zador has an idea. We can’t really equip machines with real biological nucleotide-based genes, but we can mimic the neuronal blueprint those genes create. Genomes lay out rules for brain development. Specifically, the genome encodes blueprints for wiring up our nervous system—the details of which neurons are connected, the strength of those connections and other specs that will later hold the information learned throughout life. “Our genomes serve as blueprints for building our nervous system and these blueprints give rise to a human brain, which contains about 100 billion neurons,” Zador says.
If you think what a genome is, he explains, it is essentially a very compact and compressed form of information storage. Conceptually, genomes are similar to CliffsNotes and other study guides. When students read these short summaries, they know about what happened in a book, without actually reading that book. And that’s how we should be designing the next generation of robots if we ever want them to act like humans, Zador says. “We should give them a set of behavioral CliffsNotes, which they can then unwrap into brain-like structures.” Robots that have such brain-like structures will acquire a set of basic rules to generate basic behaviors and use them to learn more complex ones.
Currently Zador is in the process of developing algorithms that function like simple rules that generate such behaviors. “My algorithms would write these CliffsNotes, outlining how to solve a particular problem,” he explains. “And then, the neural networks will use these CliffsNotes to figure out which ones are useful and use them in their behaviors.” That’s how all living beings operate. They use the pre-programmed info from their genetics to adapt to their changing environments and learn what’s necessary to survive and thrive in these settings.
For example, a robot’s neural network could draw from CliffsNotes with “genetic” instructions for how to be aware of its own body or learn to adjust its movements. And other, different sets of CliffsNotes may imbue it with the basics of physical safety or the fundamentals of speech.
At the moment, Zador is working on algorithms that are trying to mimic neuronal blueprints for very simple organisms—such as earthworms, which have only 302 neurons and about 7000 synapses compared to the millions we have. That’s how evolution worked, too—expanding the brains from simple creatures to more complex to the Homo Sapiens. But if it took millions of years to arrive at modern humans, how long would it take scientists to forge a robot with human intelligence? That’s a billion-dollar question. Yet, Zador is optimistic. “My hypotheses is that if you can build simple organisms that can interact with the world, then the higher level functions will not be nearly as challenging as they currently are.”
In the U.S. and Europe, it is illegal to reuse pacemakers and other implants. Therefore, cardiologists export them to the global South where they save the lives of people of all ages.
Often he observed there were no doctors in the E.R.s, and hte nurses could render only basic first aid. “When somebody fell into a coma, they fell into a coma,“ Israel remembers. “There weren’t even any defibrillators to restart a patient’s heart,” while defibrillators are standard equipment in most clinics in the U.S. and Europe as lifesaving devices. When Israel finally visited the largest and most modern hospital in Nairobi, he found it better equipped but he learned that its services were only available to patients who could afford them. The cardiologist there had a drawer full of petitions from patients with heart ailments who couldn’t afford lifesaving surgery. Even two decades ago, a pacemaker cost $5,000 in Kenya, which made it unaffordable for most Kenyans who earn an average of $600 per month.
Since 2003, Israel and a team of two doctors and two nurses visit Kenya and Zambia once or twice a year to implant German pacemakers for free. Notably, the pacemakers and defibrillators Israel exports to Africa would end up in the landfill in Germany. Clinics have to pay for specialized services to dispose of this medical equipment. “In Germany, I could go to jail if I used a defibrillator that is one day past its expiration date,“ Israel says, “but in Kenya, people don’t have the money for the cheapest model. What nonsense to throw this precious medical equipment away while people in poorer countries die because they desperately need it.“
Israel works at the breakpoint between the laws in a wealthy country like Germany and the reality in the global South. The U.S. and most European countries have strict laws that ban the reuse of medical implants and enforce strict expiration dates for medical equipment. “But if a pacemaker is a few days past its expiration date, it still works perfectly fine,“ Israel says. “And it also happens that we implant a pacemaker and five months later it turns out that the patient needs a different kind. Then we replace it and we’d have to trash the first one in Germany, though it could easily run another 12 years.“
“If we get this right, we have lots of devices we can implant, hips and knees, etcetera. Where this will lead is limitless," says Eva Kline Rogers, the program coordinator for My Heart, Your Heart.
Israel has been collecting donations of pacemakers and defibrillators from manufacturers but also from other doctors and from funeral homes for his nonprofit Pacemakers for East Africa since 2003. Most funeral homes in the U.S. and Europe are legally obliged to remove pacemakers from the dead before cremation. “Most pacemakers survive their owners,“ says Israel. He sterilizes the pacemakers and finds them a new life in East Africa. Studies show that reused pacemakers carry no greater risk for the patients than new ones.
In the U.S., University of Michigan professor Thomas Crawford heads up a similar initiative, My Heart, Your Heart. “Each year 1 to 2 million individuals worldwide die due to a lack of access to pacemakers and defibrillators,” the organization notes on its website. The nonprofit was founded in 2009, but it took four years for the doctors to get permission from the FDA to export pacemakers. Since receiving permission, the organization has sent dozens of devices to the Philippines, Haiti, Venezuela, Kenya, Sierra Leone and Ukraine. “We were the first doctors ever to implant a pacemaker in Sierra Leone in 2018,” says Crawford, who has traveled extensively to most of the recipient countries.
Even individuals can donate their pacemakers; the organization offers a prepaid envelope. “My mother recently passed and she donated her device,” says Tina Alexandris-Souphis, one of the doctors at University of Michigan who collaborates on My Heart, Your Heart. The organization works with World Medical Relief and the U.K. based charity Pace4Life to maintain a registry of the most urgent patients and send devices to where they are needed the most.
My Heart, Your Heart is also conducting a randomized controlled trial to provide further evidence that reused pacemakers pose no additional risk. “Our vision is that we establish this is safe and create a blueprint for organizations around the world to safely reuse these devices instead of them being thrown in the trash,” says Eva Kline Rogers, the program’s coordinator. “If we get this right, we have lots of devices we can implant, hips and knees, etc. Where this will lead is limitless.” She points out that in addition to receiving the donated devices, the doctors in the global South also benefit from the expertise of renowned cardiologists, such as Crawford, who sometimes advise them in complex cases.
And Adrian Baranchuk, a Canadian doctor at the Kingston General Hospital at the Queen’s University, regularly travels through South America with his “cardiology van” to help villagers in remote areas with heart problems.
Israel says that he’s been accused of racism, in thinking that these pacemakers are suitable for those in the global South - many of whom are people of color - even though officials in wealthier countries consider them to be trash. The cardiologist counters such criticism with stories about desperate need of his patients. At his first medical visit to Nairobi that he organized with a local cardiologist, six patients were waiting for him. “In Germany, they would all be considered emergencies,” Israel says. One eighty-year old grandmother had a heartrate of 18. “I’ve never before seen anything like this,” Israel exclaims. “At first I thought I couldn’t find her pulse before I realized that her heart was only beating once every three seconds.” After the surgery, she got up, dressed herself and hurriedly packed her bag, explaining she had a ton of work to accomplish. Her family was in disbelief, Israel says. “They told me she had been bedridden for five years because as soon as she tried to get up she would faint.”
Israel has been accused of racism, in thinking that these pacemakers are suitable for those in the global South even though they're considered to be trash by officials in wealthier countries. The cardiologist counters such criticism with stories about desperate need of his patients.
Carsten Israel
The hospital in Nairobi where Israel conducts the surgeries, charges patients $200 for the use of its facilities. If patients can’t afford that sum, Israel will pay it from the funds of his nonprofit. For some people, $200 far exceeds their resources. Once, a family from the extremely poor Northern region of Kenya told him they couldn’t afford the $3 for the bus ride to Nairobi. Israel suspected this was a pretense because they were afraid of the surgery and agreed to reimburse the $3, “but when they came, they were wearing rags and were so rail-thin, I understood that they really needed every cent they had for food.”
Israel is a renowned cardiologists in Germany. And yet, he considers his work in East Africa to be particularly meaningful. “Generally, most patients in Germany will get the treatment they need,” he says, “and I never before experienced that people have an illness that is easily curable but simply won’t be treated.” He also feels a heavy responsibility. Many patients have his personal cell phone and call him when they have problems or good news about how they’re doing.
Some of those progress reports come much faster than in Israel’s home country. Before he implanted a pacemaker in a tall Massai in Kenya, the man had been picked on by his family because he wouldn’t help much with the hard work on the family peanut farm. “When I examined him, he had a pulse of 40,” Israel says. “It’s a miracle he was even standing upright, let alone hauling heavy bags.” After the surgery, Israel advised his patient to stay the night for observation, but the patient couldn’t wait to leave. Two hours later, he returned, covered in sweat. He’d been running sprints with his brothers to test the new device. Israel shakes his head. In Germany, it would be unthinkable for a patient to engage in athletics immediately after surgery. But the patient was exuberant: “I was the fastest!”
The success stories are notable partly because the challenges remain so steep. In Zambia, for instance, there is a single cardiologist; she determined to become one after losing her younger sister to an easily curable heart disease. Often, the hospitals not only lack pacemakers but also sterile surgery equipment, antibiotics and other essential material. Therefore, Israel and his team import everything they need for the surgeries, including medication. If necessary, they improvise. “I’ve done surgery with a desk lamp hanging from the ceiling by threads,” Israel says. He already knows that he will need to return to Kenya in six months to replace the pacemaker of one of his patients and replace the batteries in others. If he doesn’t travel, lives are at risk.