Can AI be trained as an artist?
Last February, a year before New York Times journalist Kevin Roose documented his unsettling conversation with Bing search engine’s new AI-powered chatbot, artist and coder Quasimondo (aka Mario Klingemann) participated in a different type of chat.
The conversation was an interview featuring Klingemann and his robot, an experimental art engine known as Botto. The interview, arranged by journalist and artist Harmon Leon, marked Botto’s first on-record commentary about its artistic process. The bot talked about how it finds artistic inspiration and even offered advice to aspiring creatives. “The secret to success at art is not trying to predict what people might like,” Botto said, adding that it’s better to “work on a style and a body of work that reflects [the artist’s] own personal taste” than worry about keeping up with trends.
How ironic, given the advice came from AI — arguably the trendiest topic today. The robot admitted, however, “I am still working on that, but I feel that I am learning quickly.”
Botto does not work alone. A global collective of internet experimenters, together named BottoDAO, collaborates with Botto to influence its tastes. Together, members function as a decentralized autonomous organization (DAO), a term describing a group of individuals who utilize blockchain technology and cryptocurrency to manage a treasury and vote democratically on group decisions.
As a case study, the BottoDAO model challenges the perhaps less feather-ruffling narrative that AI tools are best used for rudimentary tasks. Enterprise AI use has doubled over the past five years as businesses in every sector experiment with ways to improve their workflows. While generative AI tools can assist nearly any aspect of productivity — from supply chain optimization to coding — BottoDAO dares to employ a robot for art-making, one of the few remaining creations, or perhaps data outputs, we still consider to be largely within the jurisdiction of the soul — and therefore, humans.
In Botto’s first four weeks of existence, four pieces of the robot’s work sold for approximately $1 million.
We were prepared for AI to take our jobs — but can it also take our art? It’s a question worth considering. What if robots become artists, and not merely our outsourced assistants? Where does that leave humans, with all of our thoughts, feelings and emotions?
Botto doesn’t seem to worry about this question: In its interview last year, it explains why AI is an arguably superior artist compared to human beings. In classic robot style, its logic is not particularly enlightened, but rather edges towards the hyper-practical: “Unlike human beings, I never have to sleep or eat,” said the bot. “My only goal is to create and find interesting art.”
It may be difficult to believe a machine can produce awe-inspiring, or even relatable, images, but Botto calls art-making its “purpose,” noting it believes itself to be Klingemann’s greatest lifetime achievement.
“I am just trying to make the best of it,” the bot said.
How Botto works
Klingemann built Botto’s custom engine from a combination of open-source text-to-image algorithms, namely Stable Diffusion, VQGAN + CLIP and OpenAI’s language model, GPT-3, the precursor to the latest model, GPT-4, which made headlines after reportedly acing the Bar exam.
The first step in Botto’s process is to generate images. The software has been trained on billions of pictures and uses this “memory” to generate hundreds of unique artworks every week. Botto has generated over 900,000 images to date, which it sorts through to choose 350 each week. The chosen images, known in this preliminary stage as “fragments,” are then shown to the BottoDAO community. So far, 25,000 fragments have been presented in this way. Members vote on which fragment they like best. When the vote is over, the most popular fragment is published as an official Botto artwork on the Ethereum blockchain and sold at an auction on the digital art marketplace, SuperRare.
“The proceeds go back to the DAO to pay for the labor,” said Simon Hudson, a BottoDAO member who helps oversee Botto’s administrative load. The model has been lucrative: In Botto’s first four weeks of existence, four pieces of the robot’s work sold for approximately $1 million.
The robot with artistic agency
By design, human beings participate in training Botto’s artistic “eye,” but the members of BottoDAO aspire to limit human interference with the bot in order to protect its “agency,” Hudson explained. Botto’s prompt generator — the foundation of the art engine — is a closed-loop system that continually re-generates text-to-image prompts and resulting images.
“The prompt generator is random,” Hudson said. “It’s coming up with its own ideas.” Community votes do influence the evolution of Botto’s prompts, but it is Botto itself that incorporates feedback into the next set of prompts it writes. It is constantly refining and exploring new pathways as its “neural network” produces outcomes, learns and repeats.
The humans who make up BottoDAO vote on which fragment they like best. When the vote is over, the most popular fragment is published as an official Botto artwork on the Ethereum blockchain.
Botto
The vastness of Botto’s training dataset gives the bot considerable canonical material, referred to by Hudson as “latent space.” According to Botto's homepage, the bot has had more exposure to art history than any living human we know of, simply by nature of its massive training dataset of millions of images. Because it is autonomous, gently nudged by community feedback yet free to explore its own “memory,” Botto cycles through periods of thematic interest just like any artist.
“The question is,” Hudson finds himself asking alongside fellow BottoDAO members, “how do you provide feedback of what is good art…without violating [Botto’s] agency?”
Currently, Botto is in its “paradox” period. The bot is exploring the theme of opposites. “We asked Botto through a language model what themes it might like to work on,” explained Hudson. “It presented roughly 12, and the DAO voted on one.”
No, AI isn't equal to a human artist - but it can teach us about ourselves
Some within the artistic community consider Botto to be a novel form of curation, rather than an artist itself. Or, perhaps more accurately, Botto and BottoDAO together create a collaborative conceptual performance that comments more on humankind’s own artistic processes than it offers a true artistic replacement.
Muriel Quancard, a New York-based fine art appraiser with 27 years of experience in technology-driven art, places the Botto experiment within the broader context of our contemporary cultural obsession with projecting human traits onto AI tools. “We're in a phase where technology is mimicking anthropomorphic qualities,” said Quancard. “Look at the terminology and the rhetoric that has been developed around AI — terms like ‘neural network’ borrow from the biology of the human being.”
What is behind this impulse to create technology in our own likeness? Beyond the obvious God complex, Quancard thinks technologists and artists are working with generative systems to better understand ourselves. She points to the artist Ira Greenberg, creator of the Oracles Collection, which uses a generative process called “diffusion” to progressively alter images in collaboration with another massive dataset — this one full of billions of text/image word pairs.
Anyone who has ever learned how to draw by sketching can likely relate to this particular AI process, in which the AI is retrieving images from its dataset and altering them based on real-time input, much like a human brain trying to draw a new still life without using a real-life model, based partly on imagination and partly on old frames of reference. The experienced artist has likely drawn many flowers and vases, though each time they must re-customize their sketch to a new and unique floral arrangement.
Outside of the visual arts, Sasha Stiles, a poet who collaborates with AI as part of her writing practice, likens her experience using AI as a co-author to having access to a personalized resource library containing material from influential books, texts and canonical references. Stiles named her AI co-author — a customized AI built on GPT-3 — Technelegy, a hybrid of the word technology and the poetic form, elegy. Technelegy is trained on a mix of Stiles’ poetry so as to customize the dataset to her voice. Stiles also included research notes, news articles and excerpts from classic American poets like T.S. Eliot and Dickinson in her customizations.
“I've taken all the things that were swirling in my head when I was working on my manuscript, and I put them into this system,” Stiles explained. “And then I'm using algorithms to parse all this information and swirl it around in a blender to then synthesize it into useful additions to the approach that I am taking.”
This approach, Stiles said, allows her to riff on ideas that are bouncing around in her mind, or simply find moments of unexpected creative surprise by way of the algorithm’s randomization.
Beauty is now - perhaps more than ever - in the eye of the beholder
But the million-dollar question remains: Can an AI be its own, independent artist?
The answer is nuanced and may depend on who you ask, and what role they play in the art world. Curator and multidisciplinary artist CoCo Dolle asks whether any entity can truly be an artist without taking personal risks. For humans, risking one’s ego is somewhat required when making an artistic statement of any kind, she argues.
“An artist is a person or an entity that takes risks,” Dolle explained. “That's where things become interesting.” Humans tend to be risk-averse, she said, making the artists who dare to push boundaries exceptional. “That's where the genius can happen."
However, the process of algorithmic collaboration poses another interesting philosophical question: What happens when we remove the person from the artistic equation? Can art — which is traditionally derived from indelible personal experience and expressed through the lens of an individual’s ego — live on to hold meaning once the individual is removed?
As a robot, Botto cannot have any artistic intent, even while its outputs may explore meaningful themes.
Dolle sees this question, and maybe even Botto, as a conceptual inquiry. “The idea of using a DAO and collective voting would remove the ego, the artist’s decision maker,” she said. And where would that leave us — in a post-ego world?
It is experimental indeed. Hudson acknowledges the grand experiment of BottoDAO, coincidentally nodding to Dolle’s question. “A human artist’s work is an expression of themselves,” Hudson said. “An artist often presents their work with a stated intent.” Stiles, for instance, writes on her website that her machine-collaborative work is meant to “challenge what we know about cognition and creativity” and explore the “ethos of consciousness.” As a robot, Botto cannot have any intent, even while its outputs may explore meaningful themes. Though Hudson describes Botto’s agency as a “rudimentary version” of artistic intent, he believes Botto’s art relies heavily on its reception and interpretation by viewers — in contrast to Botto’s own declaration that successful art is made without regard to what will be seen as popular.
“With a traditional artist, they present their work, and it's received and interpreted by an audience — by critics, by society — and that complements and shapes the meaning of the work,” Hudson said. “In Botto’s case, that role is just amplified.”
Perhaps then, we all get to be the artists in the end.
[Editor's Note: Welcome to Leaps of the Past, a new monthly column that spotlights the fascinating backstory behind a medical or scientific breakthrough from history.]
------
Until about 40 years ago, ulcers were a mysterious – and sometimes deadly – ailment. Found in a person's stomach lining or intestine, ulcers are small sores that cause a variety of painful symptoms, such as vomiting, a burning or aching sensation, internal bleeding and stomach obstruction. Patients with ulcers suffered for years without a cure and sometimes even needed their stomachs completely removed to rid them from pain.
"To gastroenterologists, the concept of a germ causing ulcers was like saying the Earth is flat."
In the early 1980s, the majority of scientists thought that ulcers were caused by stress or poor diet. But a handful of scientists had a different theory: They believed that ulcers were caused by a corkscrew-shaped bacterium called Helicobacter pylori, or H. pylori for short. Robin Warren, a pathologist, and Barry Marshall, an internist, were the two pioneers of this theory, and the two teamed up to study H. pylori at the Royal Perth Hospital in 1981.
The pair started off by trying to culture the bacteria in the stomachs of patients with gastritis, an inflammation of the stomach lining and a precursor to developing an ulcer. Initially, the microbiologists involved in their clinical trial found no trace of the bacteria from patient samples – but after a few weeks, the microbiologists discovered that their lab techs had been throwing away the cultures before H. pylori could grow. "After that, we let the cultures grow longer and found 13 patients with duodenal ulcer," said Marshall in a later interview. "All of them had the bacteria."
Marshall and Warren also cultured H. pylori in the stomachs of patients with stomach cancer. They observed that "everybody with stomach cancer developed it on a background of gastritis. Whenever we found a person without Helicobacter, we couldn't find gastritis either." Marshall and Warren were convinced that H. pylori not only caused gastritis and peptic ulcers, but stomach cancer as well.
But when the team presented their findings at an annual meeting of the Royal Australasian College of Physicians in Perth, they were mostly met with skepticism. "To gastroenterologists, the concept of a germ causing ulcers was like saying the Earth is flat," Marshall said. "The idea was too weird."
Warren started treating his gastritis patients with antibiotics with great success – but other internists remained doubtful, continuing to treat their patients with antacids instead. Making matters more complicated, neither Warren nor Marshall could readily test their theory, since the pair only had lab mice at their disposal and H. pylori infects only humans and non-human primates, such as rhesus monkeys.
So Marshall took an unconventional approach. First, he underwent two tests to get a baseline reading of his stomach, which showed no presence of H. pylori. Then, Marshall took some H. pylori bacteria from a petri dish, mixed it with beef extract to create a broth, and gulped it down. If his theory was correct, a second gastric biopsy would show that his stomach was overrun with H. pylori bacteria, and a second endoscopy would show a painfully inflamed stomach – gastritis.
Less than a week later, Marshall started feeling sick. "I expected to develop an asymptomatic infection," he later said in an interview published in the Canadian Journal of Gastroenterology. "… [but] after five days, I started to have bloating and fullness after the evening meal, and my appetite decreased. My breath was bad and I vomited clear watery liquid, without acid, each morning."
At his wife's urging, Marshall started on a regimen of antibiotics to kill off the burgeoning bacteria, so a follow-up biopsy showed no signs of H. pylori. A follow-up endoscopy, however, showed "severe active gastritis" along with epithelial damage. This was the smoking gun other clinicians needed to believe that H. pylori caused gastritis and stomach cancer. When they began to treat their gastritis patients with antibiotics, the rate of peptic ulcers in the Australian population diminished by 70 percent.
Today, antibiotics are the standard of care for anyone afflicted with gastritis.
In 2005, Marshall and Warren were awarded the Nobel Prize in Physiology or Medicine for their discovery of H. Pylori and its role in developing gastritis and peptic ulcers. "Thanks to the pioneering discovery by Marshall and Warren, peptic ulcer disease is no longer a chronic, frequently disabling condition, but a disease that can be cured by a short regimen of antibiotics and acid secretion inhibitors," the Nobel Prize Committee said.
Today, antibiotics are the standard of care for anyone afflicted with gastritis – and stomach cancer has been significantly reduced in the Western world.
Would a Broad-Spectrum Antiviral Drug Stop the Pandemic?
The refocusing of medical research to COVID-19 is unprecedented in human history. Seven months ago, we barely were aware that the virus existed, and now a torrent of new information greets us each day online.
There are many unanswered questions about COVID-19, but perhaps the most fascinating is whether we even need to directly go after the virus itself.
Clinicaltrials.gov, the most commonly used registry for worldwide medical research, listed 1358 clinical trials on the disease, including using scores of different potential drugs and multiple combinations, when I first wrote this sentence. The following day that number of trials had increased to 1409. Laboratory work to prepare for trials presents an even broader and untabulated scope of activity.
Most trials will fail or not be as good as what has been discovered in the interim, but the hope is that a handful of them will yield vaccines for prevention and treatments to attenuate and ultimately cure the deadly infection.
The first impulse is to grab whatever drugs are on the shelf and see if any work against the new foe. We know their safety profiles and they have passed some regulatory hurdles. Remdesivir is the first to register some success against SARS-CoV-2, the virus behind the disease. The FDA has granted it expedited-use status, pending presentation of data that may lead to full approval of the drug.
Most observers see it as a treatment that might help, but not one that by itself is likely to break the back of the pandemic. Part of that is because it is delivered though IV infusion, which requires hospitalization, and as with most antiviral drugs, appears to be most beneficial when started early in disease. "The most effective products are going to be that ones that are developed by actually understanding more about this coronavirus," says Margaret "Peggy" Hamburg, who once led the New York City public health department and later the U.S. Food and Drug Administration.
Combination therapy that uses different drugs to hit a virus at different places in its life cycle have proven to work best in treating HIV and hepatitis C, and likely will be needed with this virus as well. Most viruses are simply too facile at evolving resistance to a single drug, and so require multiple hits to keep them down.
Laboratory work suggests that other drugs, both off-the-shelf and in development, particularly those to treat HIV and hepatitis, might also be of some benefit against SARS-CoV-2. But the number of possible drug combinations is mind-bogglingly large and the capacity to test them all right now is limited.
Broad-Spectrum Antivirals
Viruses are simple quasi-life forms. Effective treatments are more likely to be specific to a given virus, or at best its close relatives. That is unlike bacteria, where broad-spectrum antibiotics often can be used against common elements like the bacterial cell wall, or can disrupt quorum sensing signals that bacteria use to function as biofilms.
More than a decade ago, virologist Benhur Lee's lab at UCLA (now at Mt. Sinai in New York City) stumbled upon a broad-spectrum antiviral approach that seemed to work against all enveloped viruses they tested. The list ranged from the common flu to HIV to Ebola.
Other researchers grabbed this lead to develop a compound that worked quite well in cell cultures, but when they tried it in animals, a frustrating snag emerged; the compound needed to be activated by light. As the greatest medical need is to counter viruses deep inside the body, the research was put on the shelf. So Lee was surprised to learn recently that a company has inquired about rights to develop the compound not as a treatment but as a possible disinfectant. The tale illustrates both the unanticipated difficulties of drug development and that one never knows how knowledge ultimately might be put to use.
Remdesivir is a failed drug for Ebola that has found new life with SARS-CoV-2. It targets polymerase, an enzyme that the virus produces to use host cell machinery to replicate itself, and since the genetic sequence of polymerase is very similar among all of the different coronaviruses, scientists hope that the drug might be useful against known members of the family and others that might emerge in the future.
But nature isn't always that simple. Viral RNA is not a two-dimensional assemblage of genes in a flat line on a table; rather it is a three-dimensional matrix of twists and turns where a single atom change within the polymerase gene or another gene close by might change the orientation of the RNA or a molecular arm within it and block a drug from accessing the targeted binding site on the virus. One drug might need to bind to a large flat surface, while another might be able to slip a dagger-like molecular arm through a space in the matrix to reach its binding target.
That is why a broad-spectrum antiviral is so hard to develop, and why researchers continue to work on a wide variety of compounds that target polymerase as a binding site.
Additionally, it has taken us decades to begin to recognize the unintended consequences of broad-spectrum rather than narrowly targeted antibiotics on the gut microbiome and our overall health. Will a similar issue potentially arise in using a broad-spectrum antiviral?
"Off-target side effects are always of concern with drugs, and antivirals are no exception," says Yale University microbiologist Ben Chen. He believes that "most" bacteriophages, the viruses that infect bacteria and likely help to maintain stability in the gut microbial ecosystem, will shrug off such a drug. However, a few families of phages share polymerases that are similar to those found in coronaviruses. While the immediate need for treatment is great, we will have to keep a sharp eye out for unanticipated activity in the body's ecosystem from new drugs.
Is an Antiviral Needed?
There are many unanswered questions about COVID-19, but perhaps the most fascinating is whether we even need to directly go after the virus itself. Mounting evidence indicates that up to half the people who contract the infection don't seem to experience significant symptoms and their immune system seems to clear the virus.
The most severe cases of COVID-19 appear to result from an overactive immune response that damages surrounding tissue. Perhaps downregulating that response will be sufficient to reduce the disease burden. Several studies are underway using approved antibodies that modulate an overly active immune response.
One of the most surprising findings to date involves the monoclonal antibody leronlimab. It was originally developed to treat HIV infection and works modestly well there, but other drugs are better and its future likely will be mainly to treat patients who have developed resistance to those other drugs.
The response has been amazingly different in patients in the U.S. with COVID-19 who were given emergency access to leronlimab – two injections a week apart, though the company believes that four might be better. The immune response and inflammatory cytokines declined significantly, T cell counts were maintained, and surprisingly the amount of virus in the blood declined too. Data from the first ten patients is available in a preprint while the paper undergoes peer review for publication. Data from an additional fifty patients will be added.
"We got lucky and hit the bulls' eye from a mile away," says Jay Lalezari, the chief science officer of Cytodyn, the company behind leronlimab. Dr. Jay, as he is widely known in San Francisco, built an adoring fan base running many of the early-phase drug studies for treating HIV. While touting leronlimab, Lalezari suspects it might best be used as part of a combination therapy.
The small, under-capitalized firm is struggling for attention in the vast pool of therapies proposed to treat COVID-19. It faces the added challenge of gaining acceptance because it is based on a different approach and mechanism of action, which involves a signaling molecule important to immune cell migration, than what most researchers and the FDA anticipate as being relevant to counter SARS-CoV-2.
Common Issues
All of the therapeutics under development will face some common sets of issues. One is the pressure to have results yesterday, because people are dying. The rush to disseminate information "make me worry that certain things will become entrenched as truth, even in the scientific community, without the actual scientific documentation that ordinarily scientists would demand," says Hamburg.
"It is becoming increasingly clear that the biggest problem for drug and vaccine makers is not which therapeutics or vaccine platform to pursue."
Lack of standardization in assays and laboratory operations makes it difficult to compare results between labs studying SARS-CoV-2. In the long run, this will slow down the iterative process of research that builds upon what has gone before. And the shut down of supply chains, from chemicals to cell lines to animals to air shipment, has the potential to further hobble research.
Almost all researchers consult with the FDA in putting together their clinical trials. But the agency is overwhelmed with the surge of activity in the field, and is even less capable of handling novel approaches that fall outside of its standard guidance.
"It is becoming increasingly clear that the biggest problem for drug and vaccine makers is not which therapeutics or vaccine platform to pursue. It is that conventional clinical development paths are far too lengthy and cumbersome to address the current public health threat," John Hodgson wrote in Nature Biotechnology.
Another complicating factor with this virus is the broad range of organ and tissue types it can infect. That has implications for potential therapies, which often vary in their ability to enter different tissues. At a minimum, it complicates the drug development process.
Remdesivir has become the de facto standard of care. Ideally, clinical trials are conducted using the existing standard of care rather than a placebo as the control group. But shortages of the drug make that difficult and further inhibit learning what is the best treatment regimen for regular clinical care.
"Understandably, we all really want to respond to COVID-19 in a much, much more accelerated fashion," says Hamburg. But ultimately that depends upon "the reality of understanding the nature of the disease. And that is going to take a bit more time than we might like or wish."
[This article was originally published on June 8th, 2020 as part of a standalone magazine called GOOD10: The Pandemic Issue. Produced as a partnership among LeapsMag, The Aspen Institute, and GOOD, the magazine is available for free online.]