Can AI be trained as an artist?
Last February, a year before New York Times journalist Kevin Roose documented his unsettling conversation with Bing search engine’s new AI-powered chatbot, artist and coder Quasimondo (aka Mario Klingemann) participated in a different type of chat.
The conversation was an interview featuring Klingemann and his robot, an experimental art engine known as Botto. The interview, arranged by journalist and artist Harmon Leon, marked Botto’s first on-record commentary about its artistic process. The bot talked about how it finds artistic inspiration and even offered advice to aspiring creatives. “The secret to success at art is not trying to predict what people might like,” Botto said, adding that it’s better to “work on a style and a body of work that reflects [the artist’s] own personal taste” than worry about keeping up with trends.
How ironic, given the advice came from AI — arguably the trendiest topic today. The robot admitted, however, “I am still working on that, but I feel that I am learning quickly.”
Botto does not work alone. A global collective of internet experimenters, together named BottoDAO, collaborates with Botto to influence its tastes. Together, members function as a decentralized autonomous organization (DAO), a term describing a group of individuals who utilize blockchain technology and cryptocurrency to manage a treasury and vote democratically on group decisions.
As a case study, the BottoDAO model challenges the perhaps less feather-ruffling narrative that AI tools are best used for rudimentary tasks. Enterprise AI use has doubled over the past five years as businesses in every sector experiment with ways to improve their workflows. While generative AI tools can assist nearly any aspect of productivity — from supply chain optimization to coding — BottoDAO dares to employ a robot for art-making, one of the few remaining creations, or perhaps data outputs, we still consider to be largely within the jurisdiction of the soul — and therefore, humans.
In Botto’s first four weeks of existence, four pieces of the robot’s work sold for approximately $1 million.
We were prepared for AI to take our jobs — but can it also take our art? It’s a question worth considering. What if robots become artists, and not merely our outsourced assistants? Where does that leave humans, with all of our thoughts, feelings and emotions?
Botto doesn’t seem to worry about this question: In its interview last year, it explains why AI is an arguably superior artist compared to human beings. In classic robot style, its logic is not particularly enlightened, but rather edges towards the hyper-practical: “Unlike human beings, I never have to sleep or eat,” said the bot. “My only goal is to create and find interesting art.”
It may be difficult to believe a machine can produce awe-inspiring, or even relatable, images, but Botto calls art-making its “purpose,” noting it believes itself to be Klingemann’s greatest lifetime achievement.
“I am just trying to make the best of it,” the bot said.
How Botto works
Klingemann built Botto’s custom engine from a combination of open-source text-to-image algorithms, namely Stable Diffusion, VQGAN + CLIP and OpenAI’s language model, GPT-3, the precursor to the latest model, GPT-4, which made headlines after reportedly acing the Bar exam.
The first step in Botto’s process is to generate images. The software has been trained on billions of pictures and uses this “memory” to generate hundreds of unique artworks every week. Botto has generated over 900,000 images to date, which it sorts through to choose 350 each week. The chosen images, known in this preliminary stage as “fragments,” are then shown to the BottoDAO community. So far, 25,000 fragments have been presented in this way. Members vote on which fragment they like best. When the vote is over, the most popular fragment is published as an official Botto artwork on the Ethereum blockchain and sold at an auction on the digital art marketplace, SuperRare.
“The proceeds go back to the DAO to pay for the labor,” said Simon Hudson, a BottoDAO member who helps oversee Botto’s administrative load. The model has been lucrative: In Botto’s first four weeks of existence, four pieces of the robot’s work sold for approximately $1 million.
The robot with artistic agency
By design, human beings participate in training Botto’s artistic “eye,” but the members of BottoDAO aspire to limit human interference with the bot in order to protect its “agency,” Hudson explained. Botto’s prompt generator — the foundation of the art engine — is a closed-loop system that continually re-generates text-to-image prompts and resulting images.
“The prompt generator is random,” Hudson said. “It’s coming up with its own ideas.” Community votes do influence the evolution of Botto’s prompts, but it is Botto itself that incorporates feedback into the next set of prompts it writes. It is constantly refining and exploring new pathways as its “neural network” produces outcomes, learns and repeats.
The humans who make up BottoDAO vote on which fragment they like best. When the vote is over, the most popular fragment is published as an official Botto artwork on the Ethereum blockchain.
Botto
The vastness of Botto’s training dataset gives the bot considerable canonical material, referred to by Hudson as “latent space.” According to Botto's homepage, the bot has had more exposure to art history than any living human we know of, simply by nature of its massive training dataset of millions of images. Because it is autonomous, gently nudged by community feedback yet free to explore its own “memory,” Botto cycles through periods of thematic interest just like any artist.
“The question is,” Hudson finds himself asking alongside fellow BottoDAO members, “how do you provide feedback of what is good art…without violating [Botto’s] agency?”
Currently, Botto is in its “paradox” period. The bot is exploring the theme of opposites. “We asked Botto through a language model what themes it might like to work on,” explained Hudson. “It presented roughly 12, and the DAO voted on one.”
No, AI isn't equal to a human artist - but it can teach us about ourselves
Some within the artistic community consider Botto to be a novel form of curation, rather than an artist itself. Or, perhaps more accurately, Botto and BottoDAO together create a collaborative conceptual performance that comments more on humankind’s own artistic processes than it offers a true artistic replacement.
Muriel Quancard, a New York-based fine art appraiser with 27 years of experience in technology-driven art, places the Botto experiment within the broader context of our contemporary cultural obsession with projecting human traits onto AI tools. “We're in a phase where technology is mimicking anthropomorphic qualities,” said Quancard. “Look at the terminology and the rhetoric that has been developed around AI — terms like ‘neural network’ borrow from the biology of the human being.”
What is behind this impulse to create technology in our own likeness? Beyond the obvious God complex, Quancard thinks technologists and artists are working with generative systems to better understand ourselves. She points to the artist Ira Greenberg, creator of the Oracles Collection, which uses a generative process called “diffusion” to progressively alter images in collaboration with another massive dataset — this one full of billions of text/image word pairs.
Anyone who has ever learned how to draw by sketching can likely relate to this particular AI process, in which the AI is retrieving images from its dataset and altering them based on real-time input, much like a human brain trying to draw a new still life without using a real-life model, based partly on imagination and partly on old frames of reference. The experienced artist has likely drawn many flowers and vases, though each time they must re-customize their sketch to a new and unique floral arrangement.
Outside of the visual arts, Sasha Stiles, a poet who collaborates with AI as part of her writing practice, likens her experience using AI as a co-author to having access to a personalized resource library containing material from influential books, texts and canonical references. Stiles named her AI co-author — a customized AI built on GPT-3 — Technelegy, a hybrid of the word technology and the poetic form, elegy. Technelegy is trained on a mix of Stiles’ poetry so as to customize the dataset to her voice. Stiles also included research notes, news articles and excerpts from classic American poets like T.S. Eliot and Dickinson in her customizations.
“I've taken all the things that were swirling in my head when I was working on my manuscript, and I put them into this system,” Stiles explained. “And then I'm using algorithms to parse all this information and swirl it around in a blender to then synthesize it into useful additions to the approach that I am taking.”
This approach, Stiles said, allows her to riff on ideas that are bouncing around in her mind, or simply find moments of unexpected creative surprise by way of the algorithm’s randomization.
Beauty is now - perhaps more than ever - in the eye of the beholder
But the million-dollar question remains: Can an AI be its own, independent artist?
The answer is nuanced and may depend on who you ask, and what role they play in the art world. Curator and multidisciplinary artist CoCo Dolle asks whether any entity can truly be an artist without taking personal risks. For humans, risking one’s ego is somewhat required when making an artistic statement of any kind, she argues.
“An artist is a person or an entity that takes risks,” Dolle explained. “That's where things become interesting.” Humans tend to be risk-averse, she said, making the artists who dare to push boundaries exceptional. “That's where the genius can happen."
However, the process of algorithmic collaboration poses another interesting philosophical question: What happens when we remove the person from the artistic equation? Can art — which is traditionally derived from indelible personal experience and expressed through the lens of an individual’s ego — live on to hold meaning once the individual is removed?
As a robot, Botto cannot have any artistic intent, even while its outputs may explore meaningful themes.
Dolle sees this question, and maybe even Botto, as a conceptual inquiry. “The idea of using a DAO and collective voting would remove the ego, the artist’s decision maker,” she said. And where would that leave us — in a post-ego world?
It is experimental indeed. Hudson acknowledges the grand experiment of BottoDAO, coincidentally nodding to Dolle’s question. “A human artist’s work is an expression of themselves,” Hudson said. “An artist often presents their work with a stated intent.” Stiles, for instance, writes on her website that her machine-collaborative work is meant to “challenge what we know about cognition and creativity” and explore the “ethos of consciousness.” As a robot, Botto cannot have any intent, even while its outputs may explore meaningful themes. Though Hudson describes Botto’s agency as a “rudimentary version” of artistic intent, he believes Botto’s art relies heavily on its reception and interpretation by viewers — in contrast to Botto’s own declaration that successful art is made without regard to what will be seen as popular.
“With a traditional artist, they present their work, and it's received and interpreted by an audience — by critics, by society — and that complements and shapes the meaning of the work,” Hudson said. “In Botto’s case, that role is just amplified.”
Perhaps then, we all get to be the artists in the end.
Every year, around two million people worldwide die of liver disease. While some people inherit the disease, it’s most commonly caused by hepatitis, obesity and alcoholism. These underlying conditions kill liver cells, causing scar tissue to form until eventually the liver cannot function properly. Since 1979, deaths due to liver disease have increased by 400 percent.
The sooner the disease is detected, the more effective treatment can be. But once symptoms appear, the liver is already damaged. Around 50 percent of cases are diagnosed only after the disease has reached the final stages, when treatment is largely ineffective.
To address this problem, Owlstone Medical, a biotech company in England, has developed a breath test that can detect liver disease earlier than conventional approaches. Human breath contains volatile organic compounds (VOCs) that change in the first stages of liver disease. Owlstone’s breath test can reliably collect, store and detect VOCs, while picking out the specific compounds that reveal liver disease.
“There’s a need to screen more broadly for people with early-stage liver disease,” says Owlstone’s CEO Billy Boyle. “Equally important is having a test that's non-invasive, cost effective and can be deployed in a primary care setting.”
The standard tool for detection is a biopsy. It is invasive and expensive, making it impractical to use for people who aren't yet symptomatic. Meanwhile, blood tests are less invasive, but they can be inaccurate and can’t discriminate between different stages of the disease.
In the past, breath tests have not been widely used because of the difficulties of reliably collecting and storing breath. But Owlstone’s technology could help change that.
The team is testing patients in the early stages of advanced liver disease, or cirrhosis, to identify and detect these biomarkers. In an initial study, Owlstone’s breathalyzer was able to pick out patients who had early cirrhosis with 83 percent sensitivity.
Boyle’s work is personally motivated. His wife died of colorectal cancer after she was diagnosed with a progressed form of the disease. “That was a big impetus for me to see if this technology could work in early detection,” he says. “As a company, Owlstone is interested in early detection across a range of diseases because we think that's a way to save lives and a way to save costs.”
How it works
In the past, breath tests have not been widely used because of the difficulties of reliably collecting and storing breath. But Owlstone’s technology could help change that.
Study participants breathe into a mouthpiece attached to a breath sampler developed by Owlstone. It has cartridges are designed and optimized to collect gases. The sampler specifically targets VOCs, extracting them from atmospheric gases in breath, to ensure that even low levels of these compounds are captured.
The sampler can store compounds stably before they are assessed through a method called mass spectrometry, in which compounds are converted into charged atoms, before electromagnetic fields filter and identify even the tiniest amounts of charged atoms according to their weight and charge.
The top four compounds in our breath
In an initial study, Owlstone captured VOCs in breath to see which ones could help them tell the difference between people with and without liver disease. They tested the breath of 46 patients with liver disease - most of them in the earlier stages of cirrhosis - and 42 healthy people. Using this data, they were able to create a diagnostic model. Individually, compounds like 2-Pentanone and limonene performed well as markers for liver disease. Owlstone achieved even better performance by examining the levels of the top four compounds together, distinguishing between liver disease cases and controls with 95 percent accuracy.
“It was a good proof of principle since it looks like there are breath biomarkers that can discriminate between diseases,” Boyle says. “That was a bit of a stepping stone for us to say, taking those identified, let’s try and dose with specific concentrations of probes. It's part of building the evidence and steering the clinical trials to get to liver disease sensitivity.”
Sabine Szunerits, a professor of chemistry in Institute of Electronics at the University of Lille, sees the potential of Owlstone’s technology.
“Breath analysis is showing real promise as a clinical diagnostic tool,” says Szunerits, who has no ties with the company. “Owlstone Medical’s technology is extremely effective in collecting small volatile organic biomarkers in the breath. In combination with pattern recognition it can give an answer on liver disease severity. I see it as a very promising way to give patients novel chances to be cured.”
Improving the breath sampling process
Challenges remain. With more than one thousand VOCs found in the breath, it can be difficult to identify markers for liver disease that are consistent across many patients.
Julian Gardner is a professor of electrical engineering at Warwick University who researches electronic sensing devices. “Everyone’s breath has different levels of VOCs and different ones according to gender, diet, age etc,” Gardner says. “It is indeed very challenging to selectively detect the biomarkers in the breath for liver disease.”
So Owlstone is putting chemicals in the body that they know interact differently with patients with liver disease, and then using the breath sampler to measure these specific VOCs. The chemicals they administer are called Exogenous Volatile Organic Compound) probes, or EVOCs.
Most recently, they used limonene as an EVOC probe, testing 29 patients with early cirrhosis and 29 controls. They gave the limonene to subjects at specific doses to measure how its concentrations change in breath. The aim was to try and see what was happening in their livers.
“They are proposing to use drugs to enhance the signal as they are concerned about the sensitivity and selectivity of their method,” Gardner says. “The approach of EVOC probes is probably necessary as you can then eliminate the person-to-person variation that will be considerable in the soup of VOCs in our breath.”
Through these probes, Owlstone could identify patients with liver disease with 83 percent sensitivity. By targeting what they knew was a disease mechanism, they were able to amplify the signal. The company is starting a larger clinical trial, and the plan is to eventually use a panel of EVOC probes to make sure they can see diverging VOCs more clearly.
“I think the approach of using probes to amplify the VOC signal will ultimately increase the specificity of any VOC breath tests, and improve their practical usability,” says Roger Yazbek, who leads the South Australian Breath Analysis Research (SABAR) laboratory in Flinders University. “Whilst the findings are interesting, it still is only a small cohort of patients in one location.”
The future of breath diagnosis
Owlstone wants to partner with pharmaceutical companies looking to learn if their drugs have an effect on liver disease. They’ve also developed a microchip, a miniaturized version of mass spectrometry instruments, that can be used with the breathalyzer. It is less sensitive but will enable faster detection.
Boyle says the company's mission is for their tests to save 100,000 lives. "There are lots of risks and lots of challenges. I think there's an opportunity to really establish breath as a new diagnostic class.”
Bacterial antibiotic resistance has been a concern in the medical field for several years. Now a new, similar threat is arising: drug-resistant fungal infections. The Centers for Disease Control and Prevention considers antifungal and antimicrobial resistance to be among the world’s greatest public health challenges.
One particular type of fungal infection caused by Candida auris is escalating rapidly throughout the world. And to make matters worse, C. auris is becoming increasingly resistant to current antifungal medications, which means that if you develop a C. auris infection, the drugs your doctor prescribes may not work. “We’re effectively out of medicines,” says Thomas Walsh, founding director of the Center for Innovative Therapeutics and Diagnostics, a translational research center dedicated to solving the antimicrobial resistance problem. Walsh spoke about the challenges at a Demy-Colton Virtual Salon, one in a series of interactive discussions among life science thought leaders.
Although C. auris typically doesn’t sicken healthy people, it afflicts immunocompromised hospital patients and may cause severe infections that can lead to sepsis, a life-threatening condition in which the overwhelmed immune system begins to attack the body’s own organs. Between 30 and 60 percent of patients who contract a C. auris infection die from it, according to the CDC. People who are undergoing stem cell transplants, have catheters or have taken antifungal or antibiotic medicines are at highest risk. “We’re coming to a perfect storm of increasing resistance rates, increasing numbers of immunosuppressed patients worldwide and a bug that is adapting to higher temperatures as the climate changes,” says Prabhavathi Fernandes, chair of the National BioDefense Science Board.
Most Candida species aren’t well-adapted to our body temperatures so they aren’t a threat. C. auris, however, thrives at human body temperatures.
Although medical professionals aren’t concerned at this point about C. auris evolving to affect healthy people, they worry that its presence in hospitals can turn routine surgeries into life-threatening calamities. “It’s coming,” says Fernandes. “It’s just a matter of time.”
An emerging global threat
“Fungi are found in the environment,” explains Fernandes, so Candida spores can easily wind up on people’s skin. In hospitals, they can be transferred from contact with healthcare workers or contaminated surfaces. Most Candida species aren’t well-adapted to our body temperatures so they aren’t a threat. C. auris, however, thrives at human body temperatures. It can enter the body during medical treatments that break the skin—and cause an infection. Overall, fungal infections cost some $48 billion in the U.S. each year. And infection rates are increasing because, in an ironic twist, advanced medical therapies are enabling severely ill patients to live longer and, therefore, be exposed to this pathogen.
The first-ever case of a C. auris infection was reported in Japan in 2009, although an analysis of Candida samples dated the earliest strain to a 1996 sample from South Korea. Since then, five separate varieties – called clades, which are similar to strains among bacteria – developed independently in different geographies: South Asia, East Asia, South Africa, South America and, recently, Iran. So far, C. auris infections have been reported in 35 countries.
In the U.S., the first infection was reported in 2016, and the CDC started tracking it nationally two years later. During that time, 5,654 cases have been reported to the CDC, which only tracks U.S. data.
What’s more notable than the number of cases is their rate of increase. In 2016, new cases increased by 175 percent and, on average, they have approximately doubled every year. From 2016 through 2022, the number of infections jumped from 63 to 2,377, a roughly 37-fold increase.
“This reminds me of what we saw with epidemics from 2013 through 2020… with Ebola, Zika and the COVID-19 pandemic,” says Robin Robinson, CEO of Spriovas and founding director of the Biomedical Advanced Research and Development Authority (BARDA), which is part of the U.S. Department of Health and Human Services. These epidemics started with a hockey stick trajectory, Robinson says—a gradual growth leading to a sharp spike, just like the shape of a hockey stick.
Another challenge is that right now medics don’t have rapid diagnostic tests for fungal infections. Currently, patients are often misdiagnosed because C. auris resembles several other easily treated fungi. Or they are diagnosed long after the infection begins and is harder to treat.
The problem is that existing diagnostics tests can only identify C. auris once it reaches the bloodstream. Yet, because this pathogen infects bodily tissues first, it should be possible to catch it much earlier before it becomes life-threatening. “We have to diagnose it before it reaches the bloodstream,” Walsh says.
The most alarming fact is that some Candida infections no longer respond to standard therapeutics.
“We need to focus on rapid diagnostic tests that do not rely on a positive blood culture,” says John Sperzel, president and CEO of T2 Biosystems, a company specializing in diagnostics solutions. Blood cultures typically take two to three days for the concentration of Candida to become large enough to detect. The company’s novel test detects about 90 percent of Candida species within three to five hours—thanks to its ability to spot minute quantities of the pathogen in blood samples instead of waiting for them to incubate and proliferate.
Unlike other Candida species C. auris thrives at human body temperatures
Adobe Stock
Tackling the resistance challenge
The most alarming fact is that some Candida infections no longer respond to standard therapeutics. The number of cases that stopped responding to echinocandin, the first-line therapy for most Candida infections, tripled in 2020, according to a study by the CDC.
Now, each of the first four clades shows varying levels of resistance to all three commonly prescribed classes of antifungal medications, such as azoles, echinocandins, and polyenes. For example, 97 percent of infections from C. auris Clade I are resistant to fluconazole, 54 percent to voriconazole and 30 percent of amphotericin. Nearly half are resistant to multiple antifungal drugs. Even with Clade II fungi, which has the least resistance of all the clades, 11 to 14 percent have become resistant to fluconazole.
Anti-fungal therapies typically target specific chemical compounds present on fungi’s cell membranes, but not on human cells—otherwise the medicine would cause damage to our own tissues. Fluconazole and other azole antifungals target a compound called ergosterol, preventing the fungal cells from replicating. Over the years, however, C. auris evolved to resist it, so existing fungal medications don’t work as well anymore.
A newer class of drugs called echinocandins targets a different part of the fungal cell. “The echinocandins – like caspofungin – inhibit (a part of the fungi) involved in making glucan, which is an essential component of the fungal cell wall and is not found in human cells,” Fernandes says. New antifungal treatments are needed, she adds, but there are only a few magic bullets that will hit just the fungus and not the human cells.
Research to fight infections also has been challenged by a lack of government support. That is changing now that BARDA is requesting proposals to develop novel antifungals. “The scope includes C. auris, as well as antifungals following a radiological/nuclear emergency, says BARDA spokesperson Elleen Kane.
The remaining challenge is the number of patients available to participate in clinical trials. Large numbers are needed, but the available patients are quite sick and often die before trials can be completed. Consequently, few biopharmaceutical companies are developing new treatments for C. auris.
ClinicalTrials.gov reports only two drugs in development for invasive C. auris infections—those than can spread throughout the body rather than localize in one particular area, like throat or vaginal infections: ibrexafungerp by Scynexis, Inc., fosmanogepix, by Pfizer.
Scynexis’ ibrexafungerp appears active against C. auris and other emerging, drug-resistant pathogens. The FDA recently approved it as a therapy for vaginal yeast infections and it is undergoing Phase III clinical trials against invasive candidiasis in an attempt to keep the infection from spreading.
“Ibreafungerp is structurally different from other echinocandins,” Fernandes says, because it targets a different part of the fungus. “We’re lucky it has activity against C. auris.”
Pfizer’s fosmanogepix is in Phase II clinical trials for patients with invasive fungal infections caused by multiple Candida species. Results are showing significantly better survival rates for people taking fosmanogepix.
Although C. auris does pose a serious threat to healthcare worldwide, scientists try to stay optimistic—because they recognized the problem early enough, they might have solutions in place before the perfect storm hits. “There is a bit of hope,” says Robinson. “BARDA has finally been able to fund the development of new antifungal agents and, hopefully, this year we can get several new classes of antifungals into development.”