World’s First “Augmented Reality” Contact Lens Aims to Revolutionize Much More Than Medicine
Imagine a world without screens. Instead of endlessly staring at your computer or craning your neck down to scroll through social media feeds and emails, information simply appears in front of your eyes when you need it and disappears when you don't.
"The vision is super clear...I was reading the poem with my eyes closed."
No more rude interruptions during dinner, no more bumping into people on the street while trying to follow GPS directions — just the information you want, when you need it, projected directly onto your visual field.
While this screenless future sounds like science fiction, it may soon be a reality thanks to the new Silicon Valley startup Mojo Vision, creator of the world's first smart contact lens. With a 14,000 pixel-per-inch display with eye-tracking, image stabilization, and a custom wireless radio, the Mojo smart lens bills itself the "smallest and densest dynamic display ever made." Unlike current augmented reality wearables such as Google Glass or ThirdEye, which project images onto a glass screen, the Mojo smart lens can project images directly onto the retina.
A current prototype displayed at the Consumer Electronics Show earlier this year in Las Vegas includes a tiny screen positioned right above the most sensitive area of the pupil. "[The Mojo lens] is a contact lens that essentially has wireless power and data transmission for a small micro LED projector that is placed over the center of the eye," explains David Hobbs, Director of Product Management at Mojo Vision. "[It] displays critical heads-up information when you need it and fades into the background when you're ready to continue on with your day."
Eventually, Mojo Visions' technology could replace our beloved smart devices but the first generation of the Mojo smart lens will be used to help the 2.2 billion people globally who suffer from vision impairment.
"If you think of the eye as a camera [for the visually impaired], the sensors are not working properly," explains Dr. Ashley Tuan, Vice President of Medical Devices at Mojo Vision and fellow of the American Academy of Optometry. "For this population, our lens can process the image so the contrast can be enhanced, we can make the image larger, magnify it so that low-vision people can see it or we can make it smaller so they can check their environment." In January of this year, the FDA granted Breakthrough Device Designation to Mojo, allowing them to have early and frequent discussions with the FDA about technical, safety and efficacy topics before clinical trials can be done and certification granted.
For now, Dr. Tuan is one of the few people who has actually worn the Mojo lens. "I put the contact lens on my eye. It was very comfortable like any contact lenses I've worn before," she describes. "The vision is super clear and then when I put on the accessories, suddenly I see Yoda in front of me and I see my vital signs. And then I have my colleague that prepared a beautiful poem that I loved when I was young [and] I was reading the poem with my eyes closed."
At the moment, there are several electronic glasses on the market like Acesight and Nueyes Pro that provide similar solutions for those suffering from visual impairment, but they are large, cumbersome, and highly visible. Mojo lens would be a discreet, more comfortable alternative that offers users more freedom of movement and independence.
"In the case of augmented-reality contact lenses, there could be an opportunity to improve the lives of people with low vision," says Dr. Thomas Steinemann, spokesperson for the American Academy of Ophthalmology and professor of ophthalmology at MetroHealth Medical Center in Cleveland. "There are existing tools for people currently living with low vision—such as digital apps, magnifiers, etc.— but something wearable could provide more flexibility and significantly more aid in day-to-day tasks."
As one of the first examples of "invisible computing," the potential applications of Mojo lens in the medical field are endless.
According to Dr. Tuan, the visually impaired often suffer from depression due to their lack of mobility and 70 percent of them are underemployed. "We hope that they can use this device to gain their mobility so they can get that social aspect back in their lives and then, eventually, employment," she explains. "That is our first and most important goal."
But helping those with low visual capabilities is only Mojo lens' first possible medical application; augmented reality is already being used in medicine and is poised to revolutionize the field in the coming decades. For example, Accuvein, a device that uses lasers to provide real-time images of veins, is widely used by nurses and doctors to help with the insertion of needles for IVs and blood tests.
According to the National Center for Biotechnology Information, augmentation of reality has been used in surgery for many years with surgeons using devices such as Google Glass to overlay critical information about their patients into their visual field. Using software like the Holographic Navigation Platform by Scopsis, surgeons can see a mixed-reality overlay that can "show you complicated tumor boundaries, assist with implant placements and guide you along anatomical pathways," its developers say.
However, according to Dr. Tuan, augmented reality headsets have drawbacks in the surgical setting. "The advantage of [Mojo lens] is you don't need to worry about sweating or that the headset or glasses will slide down to your nose," she explains "Also, our lens is designed so that it will understand your intent, so when you don't want the image overlay it will disappear, it will not block your visual field, and when you need it, it will come back at the right time."
As one of the first examples of "invisible computing," the potential applications of Mojo lens in the medical field are endless. Possibilities include live translation of sign language for deaf people; helping those with autism to read emotions; and improving doctors' bedside manner by allowing them to fully engage with patients without relying on a computer.
"[By] monitoring those blood vessels we can [track] chronic disease progression: high blood pressure, diabetes, and Alzheimer's."
Furthermore, the lens could be used to monitor health issues. "We have image sensors in the lens right now that point to the world but we can have a camera pointing inside of your eye to your retina," says Dr. Tuan, "[By] monitoring those blood vessels we can [track] chronic disease progression: high blood pressure, diabetes, and Alzheimer's."
For the moment, the future medical applications of the Mojo lens are still theoretical, but the team is confident they can eventually become a reality after going through the proper regulatory review. The company is still in the process of design, prototype and testing of the lens, so they don't know exactly when it will be available for use, but they anticipate shipping the first available products in the next couple of years. Once it does go to market, it will be available by prescription only for those with visual impairments, but the team's goal is to bring it to broader consumer markets pending regulatory clearance.
"We see that right now there's a unique opportunity here for Mojo lens and invisible computing to help to shape what the next decade of technology development looks like," explains David Hobbs. "We can use [the Mojo lens] to better serve us as opposed to us serving technology better."
A sleek, four-foot tall white robot glides across a cafe storefront in Tokyo’s Nihonbashi district, holding a two-tiered serving tray full of tea sandwiches and pastries. The cafe’s patrons smile and say thanks as they take the tray—but it’s not the robot they’re thanking. Instead, the patrons are talking to the person controlling the robot—a restaurant employee who operates the avatar from the comfort of their home.
It’s a typical scene at DAWN, short for Diverse Avatar Working Network—a cafe that launched in Tokyo six years ago as an experimental pop-up and quickly became an overnight success. Today, the cafe is a permanent fixture in Nihonbashi, staffing roughly 60 remote workers who control the robots remotely and communicate to customers via a built-in microphone.
More than just a creative idea, however, DAWN is being hailed as a life-changing opportunity. The workers who control the robots remotely (known as “pilots”) all have disabilities that limit their ability to move around freely and travel outside their homes. Worldwide, an estimated 16 percent of the global population lives with a significant disability—and according to the World Health Organization, these disabilities give rise to other problems, such as exclusion from education, unemployment, and poverty.
These are all problems that Kentaro Yoshifuji, founder and CEO of Ory Laboratory, which supplies the robot servers at DAWN, is looking to correct. Yoshifuji, who was bedridden for several years in high school due to an undisclosed health problem, launched the company to help enable people who are house-bound or bedridden to more fully participate in society, as well as end the loneliness, isolation, and feelings of worthlessness that can sometimes go hand-in-hand with being disabled.
“It’s heartbreaking to think that [people with disabilities] feel they are a burden to society, or that they fear their families suffer by caring for them,” said Yoshifuji in an interview in 2020. “We are dedicating ourselves to providing workable, technology-based solutions. That is our purpose.”
Shota Kuwahara, a DAWN employee with muscular dystrophy. Ory Labs, Inc.
Wanting to connect with others and feel useful is a common sentiment that’s shared by the workers at DAWN. Marianne, a mother of two who lives near Mt. Fuji, Japan, is functionally disabled due to chronic pain and fatigue. Working at DAWN has allowed Marianne to provide for her family as well as help alleviate her loneliness and grief.Shota, Kuwahara, a DAWN employee with muscular dystrophy, agrees. "There are many difficulties in my daily life, but I believe my life has a purpose and is not being wasted," he says. "Being useful, able to help other people, even feeling needed by others, is so motivational."
When a patient is diagnosed with early-stage breast cancer, having surgery to remove the tumor is considered the standard of care. But what happens when a patient can’t have surgery?
Whether it’s due to high blood pressure, advanced age, heart issues, or other reasons, some breast cancer patients don’t qualify for a lumpectomy—one of the most common treatment options for early-stage breast cancer. A lumpectomy surgically removes the tumor while keeping the patient’s breast intact, while a mastectomy removes the entire breast and nearby lymph nodes.
Fortunately, a new technique called cryoablation is now available for breast cancer patients who either aren’t candidates for surgery or don’t feel comfortable undergoing a surgical procedure. With cryoablation, doctors use an ultrasound or CT scan to locate any tumors inside the patient’s breast. They then insert small, needle-like probes into the patient's breast which create an “ice ball” that surrounds the tumor and kills the cancer cells.
Cryoablation has been used for decades to treat cancers of the kidneys and liver—but only in the past few years have doctors been able to use the procedure to treat breast cancer patients. And while clinical trials have shown that cryoablation works for tumors smaller than 1.5 centimeters, a recent clinical trial at Memorial Sloan Kettering Cancer Center in New York has shown that it can work for larger tumors, too.
In this study, doctors performed cryoablation on patients whose tumors were, on average, 2.5 centimeters. The cryoablation procedure lasted for about 30 minutes, and patients were able to go home on the same day following treatment. Doctors then followed up with the patients after 16 months. In the follow-up, doctors found the recurrence rate for tumors after using cryoablation was only 10 percent.
For patients who don’t qualify for surgery, radiation and hormonal therapy is typically used to treat tumors. However, said Yolanda Brice, M.D., an interventional radiologist at Memorial Sloan Kettering Cancer Center, “when treated with only radiation and hormonal therapy, the tumors will eventually return.” Cryotherapy, Brice said, could be a more effective way to treat cancer for patients who can’t have surgery.
“The fact that we only saw a 10 percent recurrence rate in our study is incredibly promising,” she said.