Dadbot, Wifebot, Friendbot: The Future of Memorializing Avatars
In 2016, when my family found out that my father was dying from cancer, I did something that at the time felt completely obvious: I started building a chatbot replica of him.
I simply wanted to create an interactive way to share key parts of his life story.
I was not under any delusion that the Dadbot, as I soon began calling it, would be a true avatar of him. From my research about the voice computing revolution—Siri, Alexa, the Google Assistant—I knew that fully humanlike AIs, like you see in the movies, were a vast ways from technological reality. Replicating my dad in any real sense was never the goal, anyway; that notion gave me the creeps.
Instead, I simply wanted to create an interactive way to share key parts of his life story: facts about his ancestors in Greece. Memories from growing up. Stories about his hobbies, family life, and career. And I wanted the Dadbot, which sent text messages and audio clips over Facebook Messenger, to remind me of his personality—warm, erudite, and funny. So I programmed it to use his distinctive phrasings; to tell a few of his signature jokes and sing his favorite songs.
While creating the Dadbot, a laborious undertaking that sprawled into 2017, I fixated on two things. The first was getting the programming right, which I did using a conversational agent authoring platform called PullString. The second, far more wrenching concern was my father's health. Failing to improve after chemotherapy and immunotherapy, and steadily losing energy, weight, and the animating sparkle of life, he died on February 9.
John Vlahos at a family reunion in the summer of 2016, a few months after his cancer diagnosis.
(Courtesy James Vlahos)
After a magazine article that I wrote about the Dadbot came out in the summer of 2017, messages poured in from readers. While most people simply expressed sympathy, some conveyed a more urgent message: They wanted their own memorializing chatbots. One man implored me to make a bot for him; he had been diagnosed with cancer and wanted his six-month-old daughter to have a way to remember him. A technology entrepreneur needed advice on replicating what I did for her father, who had stage IV cancer. And a teacher in India asked me to engineer a conversational replica of her son, who had recently been struck and killed by a bus.
Journalists from around the world also got in touch for interviews, and they inevitably came around to the same question. Will virtual immortality, they asked, ever become a business?
The prospect of this happening had never crossed my mind. I was consumed by my father's struggle and my own grief. But the notion has since become head-slappingly obvious. I am not the only person to confront the loss of a loved one; the experience is universal. And I am not alone in craving a way to keep memories alive. Of course people like the ones who wrote me will get Dadbots, Mombots, and Childbots of their own. If a moonlighting writer like me can create a minimum viable product, then a company employing actual computer scientists could do much more.
But this prospect raises unanswered and unsettling questions. For businesses, profit, and not some deeply personal mission, will be the motivation. This shift will raise issues that I didn't have to confront. To make money, a virtual immortality company could follow the lucrative but controversial business model that has worked so well for Google and Facebook. To wit, a company could provide the memorializing chatbot for free and then find ways to monetize the attention and data of whoever communicated with it. Given the copious amount of personal information flowing back and forth in conversations with replica bots, this would be a data gold mine for the company—and a massive privacy risk for users.
Virtual immortality as commercial product will doubtless become more sophisticated.
Alternately, a company could charge for memorializing avatars, perhaps with an annual subscription fee. This would put the business in a powerful position. Imagine the fee getting hiked each year. A customer like me would find himself facing a terrible decision—grit my teeth and keep paying, or be forced to pull the plug on the best, closest reminder of a loved one that I have. The same person would effectively wind up dying twice.
Another way that a beloved digital avatar could die is if the company that creates it ceases to exist. This is no mere academic concern for me: Earlier this year, PullString was swallowed up by Apple. I'm still able to access the Dadbot on my own computer, fortunately, but the acquisition means that other friends and family members can no longer chat with him remotely.
Startups like PullString, of course, are characterized by impermanence; they tend to get snapped up by bigger companies or run out of venture capital and fold. But even if big players like, say, Facebook or Google get into the virtual immortality game, we can't count on them existing even a few decades from now, which means that the avatars enabled by their technology would die, too.
The permanence problem is the biggest hurdle faced by the fledgling enterprise of virtual immortality. So some entrepreneurs are attempting to enable avatars whose existence isn't reliant upon any one company or set of computer servers. "By leveraging the power of blockchain and decentralized software to replicate information, we help users create avatars that live on forever," says Alex Roy, the founder and CEO of the startup Everlife.ai. But until this type of solution exists, give props to conventional technology for preserving memories: printed photos and words on paper can last for centuries.
The fidelity of avatars—just how lifelike they are—also raises serious concerns. Before I started creating the Dadbot, I worried that the tech might be just good enough to remind my family of the man it emulated, but so far off from my real father that it gave us all the creeps. But because the Dadbot was a simple chatbot and not some all-knowing AI, and because the interface was a messaging app, there was no danger of him encroaching on the reality of my actual dad.
But virtual immortality as commercial product will doubtless become more sophisticated. Avatars will have brains built by teams of computer scientists employing the latest techniques in conversational AI. The replicas will not just text but also speak, using synthetic voices that emulate the ones of the people being memorialized. They may even come to life as animated clones on computer screens or in 3D with the help of virtual reality headsets.
What fascinates me is how technology can help to preserve the past—genuine facts and memories from peoples' lives.
These are all lines that I don't personally want to cross; replicating my dad was never the goal. I also never aspired to have some synthetic version of him that continued to exist in the present, capable of acquiring knowledge about the world or my life and of reacting to it in real time.
Instead, what fascinates me is how technology can help to preserve the past—genuine facts and memories from people's lives—and their actual voices so that their stories can be shared interactively after they have gone. I'm working on ideas for doing this via voice computing platforms like Alexa and Assistant, and while I don't have all of the answers yet, I'm excited to figure out what might be possible.
[Adapted from Talk to Me: How Voice Computing Will Transform the Way We Live, Work, and Think (Houghton Mifflin Harcourt, March 26, 2019).]
This episode is about a health metric you may not have heard of before: heart rate variability, or HRV. This refers to the small changes in the length of time between each of your heart beats.
Scientists have known about and studied HRV for a long time. In recent years, though, new monitors have come to market that can measure HRV accurately whenever you want.
Five months ago, I got interested in HRV as a more scientific approach to finding the lifestyle changes that work best for me as an individual. It's at the convergence of some important trends in health right now, such as health tech, precision health and the holistic approach in systems biology, which recognizes how interactions among different parts of the body are key to health.
But HRV is just one of many numbers worth paying attention to. For this episode of Making Sense of Science, I spoke with psychologist Dr. Leah Lagos; Dr. Jessilyn Dunn, assistant professor in biomedical engineering at Duke; and Jason Moore, the CEO of Spren and an app called Elite HRV. We talked about what HRV is, research on its benefits, how to measure it, whether it can be used to make improvements in health, and what researchers still need to learn about HRV.
*Talk to your doctor before trying anything discussed in this episode related to HRV and lifestyle changes to raise it.
Listen on Apple | Listen on Spotify | Listen on Stitcher | Listen on Amazon | Listen on Google
Show notes
Spren - https://www.spren.com/
Elite HRV - https://elitehrv.com/
Jason Moore's Twitter - https://twitter.com/jasonmooreme?lang=en
Dr. Jessilyn Dunn's Twitter - https://twitter.com/drjessilyn?lang=en
Dr. Dunn's study on HRV, flu and common cold - https://jamanetwork.com/journals/jamanetworkopen/f...
Dr. Leah Lagos - https://drleahlagos.com/
Dr. Lagos on Star Talk - https://www.youtube.com/watch?v=jC2Q10SonV8
Research on HRV and intermittent fasting - https://pubmed.ncbi.nlm.nih.gov/33859841/
Research on HRV and Mediterranean diet - https://medicalxpress.com/news/2010-06-twin-medite...:~:text=Using%20data%20from%20the%20Emory,eating%20a%20Western%2Dtype%20diet
Devices for HRV biofeedback - https://elitehrv.com/heart-variability-monitors-an...
Benefits of HRV biofeedback - https://pubmed.ncbi.nlm.nih.gov/32385728/
HRV and cognitive performance - https://www.frontiersin.org/articles/10.3389/fnins...
HRV and emotional regulation - https://pubmed.ncbi.nlm.nih.gov/36030986/
Fortune article on HRV - https://fortune.com/well/2022/12/26/heart-rate-var...
Ever since he was a baby, Sharon Wong’s son Brandon suffered from rashes, prolonged respiratory issues and vomiting. In 2006, as a young child, he was diagnosed with a severe peanut allergy.
"My son had a history of reacting to traces of peanuts in the air or in food,” says Wong, a food allergy advocate who runs a blog focusing on nut free recipes, cooking techniques and food allergy awareness. “Any participation in school activities, social events, or travel with his peanut allergy required a lot of preparation.”
Peanut allergies affect around a million children in the U.S. Most never outgrow the condition. The problem occurs when the immune system mistakenly views the proteins in peanuts as a threat and releases chemicals to counteract it. This can lead to digestive problems, hives and shortness of breath. For some, like Wong’s son, even exposure to trace amounts of peanuts could be life threatening. They go into anaphylactic shock and need to take a shot of adrenaline as soon as possible.
Typically, people with peanut allergies try to completely avoid them and carry an adrenaline autoinjector like an EpiPen in case of emergencies. This constant vigilance is very stressful, particularly for parents with young children.
“The search for a peanut allergy ‘cure’ has been a vigorous one,” says Claudia Gray, a pediatrician and allergist at Vincent Pallotti Hospital in Cape Town, South Africa. The closest thing to a solution so far, she says, is the process of desensitization, which exposes the patient to gradually increasing doses of peanut allergen to build up a tolerance. The most common type of desensitization is oral immunotherapy, where patients ingest small quantities of peanut powder. It has been effective but there is a risk of anaphylaxis since it involves swallowing the allergen.
"By the end of the trial, my son tolerated approximately 1.5 peanuts," Sharon Wong says.
DBV Technologies, a company based in Montrouge, France has created a skin patch to address this problem. The Viaskin Patch contains a much lower amount of peanut allergen than oral immunotherapy and delivers it through the skin to slowly increase tolerance. This decreases the risk of anaphylaxis.
Wong heard about the peanut patch and wanted her son to take part in an early phase 2 trial for 4-to-11-year-olds.
“We felt that participating in DBV’s peanut patch trial would give him the best chance at desensitization or at least increase his tolerance from a speck of peanut to a peanut,” Wong says. “The daily routine was quite simple, remove the old patch and then apply a new one. By the end of the trial, he tolerated approximately 1.5 peanuts.”
How it works
For DBV Technologies, it all began when pediatric gastroenterologist Pierre-Henri Benhamou teamed up with fellow professor of gastroenterology Christopher Dupont and his brother, engineer Bertrand Dupont. Together they created a more effective skin patch to detect when babies have allergies to cow's milk. Then they realized that the patch could actually be used to treat allergies by promoting tolerance. They decided to focus on peanut allergies first as the more dangerous.
The Viaskin patch utilizes the fact that the skin can promote tolerance to external stimuli. The skin is the body’s first defense. Controlling the extent of the immune response is crucial for the skin. So it has defense mechanisms against external stimuli and can promote tolerance.
The patch consists of an adhesive foam ring with a plastic film on top. A small amount of peanut protein is placed in the center. The adhesive ring is attached to the back of the patient's body. The peanut protein sits above the skin but does not directly touch it. As the patient sweats, water droplets on the inside of the film dissolve the peanut protein, which is then absorbed into the skin.
The peanut protein is then captured by skin cells called Langerhans cells. They play an important role in getting the immune system to tolerate certain external stimuli. Langerhans cells take the peanut protein to lymph nodes which activate T regulatory cells. T regulatory cells suppress the allergic response.
A different patch is applied to the skin every day to increase tolerance. It’s both easy to use and convenient.
“The DBV approach uses much smaller amounts than oral immunotherapy and works through the skin significantly reducing the risk of allergic reactions,” says Edwin H. Kim, the division chief of Pediatric Allergy and Immunology at the University of North Carolina, U.S., and one of the principal investigators of Viaskin’s clinical trials. “By not going through the mouth, the patch also avoids the taste and texture issues. Finally, the ability to apply a patch and immediately go about your day may be very attractive to very busy patients and families.”
Brandon Wong displaying origami figures he folded at an Origami Convention in 2022
Sharon Wong
Clinical trials
Results from DBV's phase 3 trial in children ages 1 to 3 show its potential. For a positive result, patients who could not tolerate 10 milligrams or less of peanut protein had to be able to manage 300 mg or more after 12 months. Toddlers who could already tolerate more than 10 mg needed to be able to manage 1000 mg or more. In the end, 67 percent of subjects using the Viaskin patch met the target as compared to 33 percent of patients taking the placebo dose.
“The Viaskin peanut patch has been studied in several clinical trials to date with promising results,” says Suzanne M. Barshow, assistant professor of medicine in allergy and asthma research at Stanford University School of Medicine in the U.S. “The data shows that it is safe and well-tolerated. Compared to oral immunotherapy, treatment with the patch results in fewer side effects but appears to be less effective in achieving desensitization.”
The primary reason the patch is less potent is that oral immunotherapy uses a larger amount of the allergen. Additionally, absorption of the peanut protein into the skin could be erratic.
Gray also highlights that there is some tradeoff between risk and efficacy.
“The peanut patch is an exciting advance but not as effective as the oral route,” Gray says. “For those patients who are very sensitive to orally ingested peanut in oral immunotherapy or have an aversion to oral peanut, it has a use. So, essentially, the form of immunotherapy will have to be tailored to each patient.” Having different forms such as the Viaskin patch which is applied to the skin or pills that patients can swallow or dissolve under the tongue is helpful.
The hope is that the patch’s efficacy will increase over time. The team is currently running a follow-up trial, where the same patients continue using the patch.
“It is a very important study to show whether the benefit achieved after 12 months on the patch stays stable or hopefully continues to grow with longer duration,” says Kim, who is an investigator in this follow-up trial.
"My son now attends university in Massachusetts, lives on-campus, and eats dorm food. He has so much more freedom," Wong says.
The team is further ahead in the phase 3 follow-up trial for 4-to-11-year-olds. The initial phase 3 trial was not as successful as the trial for kids between one and three. The patch enabled patients to tolerate more peanuts but there was not a significant enough difference compared to the placebo group to be definitive. The follow-up trial showed greater potency. It suggests that the longer patients are on the patch, the stronger its effects.
They’re also testing if making the patch bigger, changing the shape and extending the minimum time it’s worn can improve its benefits in a trial for a new group of 4-to-11 year-olds.
The future
DBV Technologies is using the skin patch to treat cow’s milk allergies in children ages 1 to 17. They’re currently in phase 2 trials.
As for the peanut allergy trials in toddlers, the hope is to see more efficacy soon.
For Wong’s son who took part in the earlier phase 2 trial for 4-to-11-year-olds, the patch has transformed his life.
“My son continues to maintain his peanut tolerance and is not affected by peanut dust in the air or cross-contact,” Wong says. ”He attends university in Massachusetts, lives on-campus, and eats dorm food. He still carries an EpiPen but has so much more freedom than before his clinical trial. We will always be grateful.”