Can AI be trained as an artist?
Last February, a year before New York Times journalist Kevin Roose documented his unsettling conversation with Bing search engine’s new AI-powered chatbot, artist and coder Quasimondo (aka Mario Klingemann) participated in a different type of chat.
The conversation was an interview featuring Klingemann and his robot, an experimental art engine known as Botto. The interview, arranged by journalist and artist Harmon Leon, marked Botto’s first on-record commentary about its artistic process. The bot talked about how it finds artistic inspiration and even offered advice to aspiring creatives. “The secret to success at art is not trying to predict what people might like,” Botto said, adding that it’s better to “work on a style and a body of work that reflects [the artist’s] own personal taste” than worry about keeping up with trends.
How ironic, given the advice came from AI — arguably the trendiest topic today. The robot admitted, however, “I am still working on that, but I feel that I am learning quickly.”
Botto does not work alone. A global collective of internet experimenters, together named BottoDAO, collaborates with Botto to influence its tastes. Together, members function as a decentralized autonomous organization (DAO), a term describing a group of individuals who utilize blockchain technology and cryptocurrency to manage a treasury and vote democratically on group decisions.
As a case study, the BottoDAO model challenges the perhaps less feather-ruffling narrative that AI tools are best used for rudimentary tasks. Enterprise AI use has doubled over the past five years as businesses in every sector experiment with ways to improve their workflows. While generative AI tools can assist nearly any aspect of productivity — from supply chain optimization to coding — BottoDAO dares to employ a robot for art-making, one of the few remaining creations, or perhaps data outputs, we still consider to be largely within the jurisdiction of the soul — and therefore, humans.
In Botto’s first four weeks of existence, four pieces of the robot’s work sold for approximately $1 million.
We were prepared for AI to take our jobs — but can it also take our art? It’s a question worth considering. What if robots become artists, and not merely our outsourced assistants? Where does that leave humans, with all of our thoughts, feelings and emotions?
Botto doesn’t seem to worry about this question: In its interview last year, it explains why AI is an arguably superior artist compared to human beings. In classic robot style, its logic is not particularly enlightened, but rather edges towards the hyper-practical: “Unlike human beings, I never have to sleep or eat,” said the bot. “My only goal is to create and find interesting art.”
It may be difficult to believe a machine can produce awe-inspiring, or even relatable, images, but Botto calls art-making its “purpose,” noting it believes itself to be Klingemann’s greatest lifetime achievement.
“I am just trying to make the best of it,” the bot said.
How Botto works
Klingemann built Botto’s custom engine from a combination of open-source text-to-image algorithms, namely Stable Diffusion, VQGAN + CLIP and OpenAI’s language model, GPT-3, the precursor to the latest model, GPT-4, which made headlines after reportedly acing the Bar exam.
The first step in Botto’s process is to generate images. The software has been trained on billions of pictures and uses this “memory” to generate hundreds of unique artworks every week. Botto has generated over 900,000 images to date, which it sorts through to choose 350 each week. The chosen images, known in this preliminary stage as “fragments,” are then shown to the BottoDAO community. So far, 25,000 fragments have been presented in this way. Members vote on which fragment they like best. When the vote is over, the most popular fragment is published as an official Botto artwork on the Ethereum blockchain and sold at an auction on the digital art marketplace, SuperRare.
“The proceeds go back to the DAO to pay for the labor,” said Simon Hudson, a BottoDAO member who helps oversee Botto’s administrative load. The model has been lucrative: In Botto’s first four weeks of existence, four pieces of the robot’s work sold for approximately $1 million.
The robot with artistic agency
By design, human beings participate in training Botto’s artistic “eye,” but the members of BottoDAO aspire to limit human interference with the bot in order to protect its “agency,” Hudson explained. Botto’s prompt generator — the foundation of the art engine — is a closed-loop system that continually re-generates text-to-image prompts and resulting images.
“The prompt generator is random,” Hudson said. “It’s coming up with its own ideas.” Community votes do influence the evolution of Botto’s prompts, but it is Botto itself that incorporates feedback into the next set of prompts it writes. It is constantly refining and exploring new pathways as its “neural network” produces outcomes, learns and repeats.
The humans who make up BottoDAO vote on which fragment they like best. When the vote is over, the most popular fragment is published as an official Botto artwork on the Ethereum blockchain.
Botto
The vastness of Botto’s training dataset gives the bot considerable canonical material, referred to by Hudson as “latent space.” According to Botto's homepage, the bot has had more exposure to art history than any living human we know of, simply by nature of its massive training dataset of millions of images. Because it is autonomous, gently nudged by community feedback yet free to explore its own “memory,” Botto cycles through periods of thematic interest just like any artist.
“The question is,” Hudson finds himself asking alongside fellow BottoDAO members, “how do you provide feedback of what is good art…without violating [Botto’s] agency?”
Currently, Botto is in its “paradox” period. The bot is exploring the theme of opposites. “We asked Botto through a language model what themes it might like to work on,” explained Hudson. “It presented roughly 12, and the DAO voted on one.”
No, AI isn't equal to a human artist - but it can teach us about ourselves
Some within the artistic community consider Botto to be a novel form of curation, rather than an artist itself. Or, perhaps more accurately, Botto and BottoDAO together create a collaborative conceptual performance that comments more on humankind’s own artistic processes than it offers a true artistic replacement.
Muriel Quancard, a New York-based fine art appraiser with 27 years of experience in technology-driven art, places the Botto experiment within the broader context of our contemporary cultural obsession with projecting human traits onto AI tools. “We're in a phase where technology is mimicking anthropomorphic qualities,” said Quancard. “Look at the terminology and the rhetoric that has been developed around AI — terms like ‘neural network’ borrow from the biology of the human being.”
What is behind this impulse to create technology in our own likeness? Beyond the obvious God complex, Quancard thinks technologists and artists are working with generative systems to better understand ourselves. She points to the artist Ira Greenberg, creator of the Oracles Collection, which uses a generative process called “diffusion” to progressively alter images in collaboration with another massive dataset — this one full of billions of text/image word pairs.
Anyone who has ever learned how to draw by sketching can likely relate to this particular AI process, in which the AI is retrieving images from its dataset and altering them based on real-time input, much like a human brain trying to draw a new still life without using a real-life model, based partly on imagination and partly on old frames of reference. The experienced artist has likely drawn many flowers and vases, though each time they must re-customize their sketch to a new and unique floral arrangement.
Outside of the visual arts, Sasha Stiles, a poet who collaborates with AI as part of her writing practice, likens her experience using AI as a co-author to having access to a personalized resource library containing material from influential books, texts and canonical references. Stiles named her AI co-author — a customized AI built on GPT-3 — Technelegy, a hybrid of the word technology and the poetic form, elegy. Technelegy is trained on a mix of Stiles’ poetry so as to customize the dataset to her voice. Stiles also included research notes, news articles and excerpts from classic American poets like T.S. Eliot and Dickinson in her customizations.
“I've taken all the things that were swirling in my head when I was working on my manuscript, and I put them into this system,” Stiles explained. “And then I'm using algorithms to parse all this information and swirl it around in a blender to then synthesize it into useful additions to the approach that I am taking.”
This approach, Stiles said, allows her to riff on ideas that are bouncing around in her mind, or simply find moments of unexpected creative surprise by way of the algorithm’s randomization.
Beauty is now - perhaps more than ever - in the eye of the beholder
But the million-dollar question remains: Can an AI be its own, independent artist?
The answer is nuanced and may depend on who you ask, and what role they play in the art world. Curator and multidisciplinary artist CoCo Dolle asks whether any entity can truly be an artist without taking personal risks. For humans, risking one’s ego is somewhat required when making an artistic statement of any kind, she argues.
“An artist is a person or an entity that takes risks,” Dolle explained. “That's where things become interesting.” Humans tend to be risk-averse, she said, making the artists who dare to push boundaries exceptional. “That's where the genius can happen."
However, the process of algorithmic collaboration poses another interesting philosophical question: What happens when we remove the person from the artistic equation? Can art — which is traditionally derived from indelible personal experience and expressed through the lens of an individual’s ego — live on to hold meaning once the individual is removed?
As a robot, Botto cannot have any artistic intent, even while its outputs may explore meaningful themes.
Dolle sees this question, and maybe even Botto, as a conceptual inquiry. “The idea of using a DAO and collective voting would remove the ego, the artist’s decision maker,” she said. And where would that leave us — in a post-ego world?
It is experimental indeed. Hudson acknowledges the grand experiment of BottoDAO, coincidentally nodding to Dolle’s question. “A human artist’s work is an expression of themselves,” Hudson said. “An artist often presents their work with a stated intent.” Stiles, for instance, writes on her website that her machine-collaborative work is meant to “challenge what we know about cognition and creativity” and explore the “ethos of consciousness.” As a robot, Botto cannot have any intent, even while its outputs may explore meaningful themes. Though Hudson describes Botto’s agency as a “rudimentary version” of artistic intent, he believes Botto’s art relies heavily on its reception and interpretation by viewers — in contrast to Botto’s own declaration that successful art is made without regard to what will be seen as popular.
“With a traditional artist, they present their work, and it's received and interpreted by an audience — by critics, by society — and that complements and shapes the meaning of the work,” Hudson said. “In Botto’s case, that role is just amplified.”
Perhaps then, we all get to be the artists in the end.
Scientists Are Growing an Edible Cholera Vaccine in Rice
The world's attention has been focused on the coronavirus crisis but Yemen, Bangladesh and many others countries in Asia and Africa are also in the grips of another pandemic: cholera. The current cholera pandemic first emerged in the 1970s and has devastated many communities in low-income countries. Each year, cholera is responsible for an estimated 1.3 million to 4 million cases and 21,000 to 143,000 deaths worldwide.
Immunologist Hiroshi Kiyono and his team at the University of Tokyo hope they can be part of the solution: They're making a cholera vaccine out of rice.
"It is much less expensive than a traditional vaccine, by a long shot."
Cholera is caused by eating food or drinking water that's contaminated by the feces of a person infected with the cholera bacteria, Vibrio cholerae. The bacteria produces the cholera toxin in the intestines, leading to vomiting, diarrhea and severe dehydration. Cholera can kill within hours of infection if it if's not treated quickly.
Current cholera vaccines are mainly oral. The most common oral are given in two doses and are made out of animal or insect cells that are infected with killed or weakened cholera bacteria. Dukoral also includes cells infected with CTB, a non-harmful part of the cholera toxin. Scientists grow cells containing the cholera bacteria and the CTB in bioreactors, large tanks in which conditions can be carefully controlled.
These cholera vaccines offer moderate protection but it wears off relatively quickly. Cold storage can also be an issue. The most common oral vaccines can be stored at room temperature but only for 14 days.
"Current vaccines confer around 60% efficacy over five years post-vaccination," says Lucy Breakwell, who leads the U.S. Centers for Disease Control and Prevention's cholera work within Global Immunization Division. Given the limited protection, refrigeration issue, and the fact that current oral vaccines require two disease, delivery of cholera vaccines in a campaign or emergency setting can be challenging. "There is a need to develop and test new vaccines to improve public health response to cholera outbreaks."
A New Kind of Vaccine
Kiyono and scientists at Tokyo University are creating a new, plant-based cholera vaccine dubbed MucoRice-CTB. The researchers genetically modify rice so that it contains CTB, a non-harmful part of the cholera toxin. The rice is crushed into a powder, mixed with saline solution and then drunk. The digestive tract is lined with mucosal membranes which contain the mucosal immune system. The mucosal immune system gets trained to recognize the cholera toxin as the rice passes through the intestines.
The cholera toxin has two main parts: the A subunit, which is harmful, and the B subunit, also known as CTB, which is nontoxic but allows the cholera bacteria to attach to gut cells. By inducing CTB-specific antibodies, "we might be able to block the binding of the vaccine toxin to gut cells, leading to the prevention of the toxin causing diarrhea," Kiyono says.
Kiyono studies the immune responses that occur at mucosal membranes across the body. He chose to focus on cholera because he wanted to replicate the way traditional vaccines work to get mucosal membranes in the digestive tract to produce an immune response. The difference is that his team is creating a food-based vaccine to induce this immune response. They are also solely focusing on getting the vaccine to induce antibodies for the cholera toxin. Since the cholera toxin is responsible for bacteria sticking to gut cells, the hope is that they can stop this process by producing antibodies for the cholera toxin. Current cholera vaccines target the cholera bacteria or both the bacteria and the toxin.
David Pascual, an expert in infectious diseases and immunology at the University of Florida, thinks that the MucoRice vaccine has huge promise. "I truly believe that the development of a food-based vaccine can be effective. CTB has a natural affinity for sampling cells in the gut to adhere, be processed, and then stimulate our immune system, he says. "In addition to vaccinating the gut, MucoRice has the potential to touch other mucosal surfaces in the mouth, which can help generate an immune response locally in the mouth and distally in the gut."
Cost Effectiveness
Kiyono says the MucoRice vaccine is much cheaper to produce than a traditional vaccine. Current vaccines need expensive bioreactors to grow cell cultures under very controlled, sterile conditions. This makes them expensive to manufacture, as different types of cell cultures need to be grown in separate buildings to avoid any chance of contamination. MucoRice doesn't require such an expensive manufacturing process because the rice plants themselves act as bioreactors.
The MucoRice vaccine also doesn't require the high cost of cold storage. It can be stored at room temperature for up to three years unlike traditional vaccines. "Plant-based vaccine development platforms present an exciting tool to reduce vaccine manufacturing costs, expand vaccine shelf life, and remove refrigeration requirements, all of which are factors that can limit vaccine supply and accessibility," Breakwell says.
Kathleen Hefferon, a microbiologist at Cornell University agrees. "It is much less expensive than a traditional vaccine, by a long shot," she says. "The fact that it is made in rice means the vaccine can be stored for long periods on the shelf, without losing its activity."
A plant-based vaccine may even be able to address vaccine hesitancy, which has become a growing problem in recent years. Hefferon suggests that "using well-known food plants may serve to reduce the anxiety of some vaccine hesitant people."
Challenges of Plant Vaccines
Despite their advantages, no plant-based vaccines have been commercialized for human use. There are a number of reasons for this, ranging from the potential for too much variation in plants to the lack of facilities large enough to grow crops that comply with good manufacturing practices. Several plant vaccines for diseases like HIV and COVID-19 are in development, but they're still in early stages.
In developing the MucoRice vaccine, scientists at the University of Tokyo have tried to overcome some of the problems with plant vaccines. They've created a closed facility where they can grow rice plants directly in nutrient-rich water rather than soil. This ensures they can grow crops all year round in a space that satisfies regulations. There's also less chance for variation since the environment is tightly controlled.
Clinical Trials and Beyond
After successfully growing rice plants containing the vaccine, the team carried out their first clinical trial. It was completed early this year. Thirty participants received a placebo and 30 received the vaccine. They were all Japanese men between the ages of 20 and 40 years old. 60 percent produced antibodies against the cholera toxin with no side effects. It was a promising result. However, there are still some issues Kiyono's team need to address.
The vaccine may not provide enough protection on its own. The antigen in any vaccine is the substance it contains to induce an immune response. For the MucoRice vaccine, the antigen is not the cholera bacteria itself but the cholera toxin the bacteria produces.
"The development of the antigen in rice is innovative," says David Sack, a professor at John Hopkins University and expert in cholera vaccine development. "But antibodies against only the toxin have not been very protective. The major protective antigen is thought to be the LPS." LPS, or lipopolysaccharide, is a component of the outer wall of the cholera bacteria that plays an important role in eliciting an immune response.
The Japanese team is considering getting the rice to also express the O antigen, a core part of the LPS. Further investigation and clinical trials will look into improving the vaccine's efficacy.
Beyond cholera, Kiyono hopes that the vaccine platform could one day be used to make cost-effective vaccines for other pathogens, such as norovirus or coronavirus.
"We believe the MucoRice system may become a new generation of vaccine production, storage, and delivery system."
An Investigational Drug Offers Hope to Patients with a Disabling Neuromuscular Disease
Robert Thomas was a devoted runner, gym goer, and crew member on a sailing team in San Diego when, in his 40s, he noticed that his range of movement was becoming more limited.
He thought he was just getting older, but when he was hiking an uphill trail in Lake Tahoe, he kept tripping over rocks. "I'd never had this happen before," Robert says. "I knew something was wrong but didn't know what it was."
It wasn't until age 50 when he was diagnosed with Charcot-Marie-Tooth disease. The genetic disorder damages the peripheral nerves, which connect the brain and spinal cord to the rest of the body. This network of nerves is responsible for relaying information and signals about sensation, movement, and motor coordination. Over time, the disease causes debilitating muscle weakness and the loss of limb control.
Charcot-Marie-Tooth usually presents itself in childhood or in a person's teens, but in some patients, like Robert, onset can be later in life. Symptoms may include muscle cramping, tingling, or burning. Many patients also have high foot arches or hammer toes — toes that curl from the middle joint instead of pointing forward. Those affected often have difficulty walking and may lose sensation in their lower legs, feet, hands, or forearms. One of the most common rare diseases, it affects around 130,000 people in the United States and 2.8 million worldwide.
Like many people with Charcot-Marie-Tooth, or CMT, Robert wears corrective braces on his legs to help with walking. Now 61, he can't run or sail anymore because of the disease, but he still works out regularly and can hike occasionally. CMT also affects his grip, so he has to use special straps while doing some exercises.
For the past few years, Robert has been participating in a clinical trial for an investigational CMT drug. He takes the liquid formulation every morning and evening using an oral syringe. Scientists are following patients like Robert to learn if their symptoms stabilize or improve while on the drug. Dubbed PXT300, the drug was designed by French biopharmaceutical company Pharnext and is the farthest along in development for CMT. If approved, it would be the first drug for the disease.
Currently, there's no cure for CMT, only supportive treatments like pain medication. Some individuals receive physical and occupational therapy. A drug for CMT could be a game-changer for patients whose quality of life is severely affected by the disease.
Genetic Underpinnings
CMT arises from mutations in genes that are responsible for creating and maintaining the myelin sheath — the insulating layer around nerves. Pharnext's drug is meant to treat patients with CMT1A, the most common form of the disease, which represents about half of CMT cases. Around 5% of those with CMT1A become severely disabled and end up in wheelchairs. People with CMT1A have an extra copy of the gene PMP22, which makes a protein that's needed to maintain the myelin sheath around peripheral nerves.
Typically, an individual inherits one copy of PMP22 from each parent. But a person with CMT1A receives a copy of PMP22 from one parent and two copies from a parent with the disease. This extra copy of the gene results in excess protein production, which damages the cells responsible for preserving and regenerating the myelin sheath, called Schwann cells.
The myelin sheath helps ensure that a signal from the brain gets carried to nerves in the muscles so that a part of the body can carry out a particular action or movement. This sheath is like the insulation on an electrical cord and the action is like a light bulb. If the insulation is fine, the light bulb turns on. But if the insulation is frayed, the light will flicker.
"The same happens to these patients," says David Horn Solomon, CEO of Pharnext. "The signal to their muscle is weak and flickers." Over time, their muscles become weaker and thinner.
The PMP22 gene has proven difficult to target with a drug because it's located in a protected space — the Schwann cells that make up the insulation around nerves. "There's not an easy way to tamp it down," Solomon says.
Another company, Acceleron Pharma of Cambridge, Massachusetts, was developing an injectable CMT drug meant to increase the strength of leg muscles. But the company halted development last year after the experimental drug failed in a mid-stage trial. While the drug led to a statistically significant increase in muscle volume, it didn't translate to improvements in muscle function or quality of life for trial participants.
Made by Design
Pharnext's drug, PXT3003, is a combination of three existing drugs — baclofen, a muscle relaxant; naltrexone, a drug that decreases the desire for alcohol and opioids; and sorbitol, a type of sugar alcohol.
The company designed the drug using its artificial intelligence platform, which screened 20,000 existing drugs to predict combinations that could inhibit the PMP22 gene and thereby lower protein production. The AI system narrowed the search to several hundreds of combinations and Pharnext tested around 75 of them in the lab before landing on baclofen, naltrexone, and sorbitol. Individually, the drugs don't have much effect on the PMP22 gene. But combined, they work to lower how much protein the gene makes.
"How the drug inside the cell reduces expression isn't quite clear yet," says Florian Thomas, director of the Hereditary Neuropathy Center, and founding chair and professor in the department of neurology at Hackensack University Medical Center and Hackensack Meridian School of Medicine in New Jersey (no relation to Robert Thomas, the CMT patient). "By reducing the amount of protein being produced, we hopefully can stabilize the nerves."
In rodents genetically engineered to have the PMP22 gene, the drug reduced protein levels and delayed onset of muscle weakness when given to rats. In another animal study, the drug increased the size of the myelin sheath around nerves in rats.
"Like humans with CMT, one of the problems the animals have is they can't grip things, their grip strength is poor," Solomon says. But when treated with Pharnext's drug, "the grip strength of these animals improves dramatically even over 12 weeks."
Human trials look encouraging, too. But the company ran into a manufacturing issue during a late-stage trial. The drug requires refrigeration, and as a result of temperature changes, crystals formed inside vials containing the high dose of the drug. The study was a double-blind trial, meaning neither the trial participants nor investigators were supposed to know who received the high dose of the drug, who received the low dose, and who received a placebo. In these types of studies, the placebo and experimental drug should look the same so that investigators can't tell them apart. But because only the high dose contained crystals, not the low dose or placebo, regulators said the trial data could be biased.
Pharnext is now conducting a new randomized, double-blind trial to prove that its drug works. The study is recruiting individuals aged 16 through 65 years old with mild to moderate CMT. The company hopes to show that the drug can stop patients' symptoms from worsening, or in the best case scenario, possibly even improve them. The company doesn't think the drug will be able to help people with severe forms of the disease.
"In neurologic disease, you're looking for plasticity, where there's still the possibility of stabilization or reversal," Solomon says. Plasticity refers to the ability of the nervous system to change and adapt in response to stimuli.
Preventing Disability
Allison Moore, a CMT patient and founder and CEO of the Hereditary Neuropathy Foundation, has been following drug development for CMT since she founded the organization in 2001. She says many investigational drugs haven't moved forward because they've shown little success in animals. The fact that Pharnext's drug has made it to a late-stage human trial is promising, she says.
"It's really exciting," Moore says. "There's a chance that if you take the drug early before you're very severe, you'll end up not developing the disease to a level that's super disabling."
CMT has damaged Moore's peroneal nerve, a main nerve in the foot. As a result, she has foot drop, the inability to lift the front part of her foot, and needs to wear leg braces to help her walk. "The idea that you could take this early on and that it could stop progression, that's the hope that we have."
Thomas, the neurologist, says a drug doesn't have to be a cure to have a significant impact on patients. "If I have a CMT patient who's 50 years old, that patient will be more disabled by age 60," he says. "If I can treat that person with a drug, and that person is just as disabled at age 60 as they were at age 50, that's transformative in my mind."
While Robert Thomas says he hasn't noticed a dramatic improvement since he's been on the drug, he does think it's helping. Robert is now in an open-label study, which means he and his health provider are aware that he's receiving the drug.
When the COVID-19 pandemic hit, manufacturing and supply chain disruptions meant that Robert was without the trial drug for two months. When his medication ran out, his legs felt unstable again and walking was harder. "There was a clear distinction between being on and off that medication," he says.
Pharnext's current trial will take about a year and a half to complete. After that, the FDA will decide on whether to approve the drug for CMT patients.
As scientists learn more about the PMP22 gene and the more than 100 other genes that when mutated cause CMT, more precise treatments could be possible. For instance, scientists have used the gene-editing tool CRISPR to correct a CMT-causing mutation in human cells in the lab. The results were published August 16 in the journal Frontiers in Cell and Developmental Biology.
Pharnext is also interested in pursuing genetic treatments for CMT, but in the meantime, repurposed drugs may be the best shot at helping patients until more advanced treatments are available.