Can AI be trained as an artist?
Last February, a year before New York Times journalist Kevin Roose documented his unsettling conversation with Bing search engine’s new AI-powered chatbot, artist and coder Quasimondo (aka Mario Klingemann) participated in a different type of chat.
The conversation was an interview featuring Klingemann and his robot, an experimental art engine known as Botto. The interview, arranged by journalist and artist Harmon Leon, marked Botto’s first on-record commentary about its artistic process. The bot talked about how it finds artistic inspiration and even offered advice to aspiring creatives. “The secret to success at art is not trying to predict what people might like,” Botto said, adding that it’s better to “work on a style and a body of work that reflects [the artist’s] own personal taste” than worry about keeping up with trends.
How ironic, given the advice came from AI — arguably the trendiest topic today. The robot admitted, however, “I am still working on that, but I feel that I am learning quickly.”
Botto does not work alone. A global collective of internet experimenters, together named BottoDAO, collaborates with Botto to influence its tastes. Together, members function as a decentralized autonomous organization (DAO), a term describing a group of individuals who utilize blockchain technology and cryptocurrency to manage a treasury and vote democratically on group decisions.
As a case study, the BottoDAO model challenges the perhaps less feather-ruffling narrative that AI tools are best used for rudimentary tasks. Enterprise AI use has doubled over the past five years as businesses in every sector experiment with ways to improve their workflows. While generative AI tools can assist nearly any aspect of productivity — from supply chain optimization to coding — BottoDAO dares to employ a robot for art-making, one of the few remaining creations, or perhaps data outputs, we still consider to be largely within the jurisdiction of the soul — and therefore, humans.
In Botto’s first four weeks of existence, four pieces of the robot’s work sold for approximately $1 million.
We were prepared for AI to take our jobs — but can it also take our art? It’s a question worth considering. What if robots become artists, and not merely our outsourced assistants? Where does that leave humans, with all of our thoughts, feelings and emotions?
Botto doesn’t seem to worry about this question: In its interview last year, it explains why AI is an arguably superior artist compared to human beings. In classic robot style, its logic is not particularly enlightened, but rather edges towards the hyper-practical: “Unlike human beings, I never have to sleep or eat,” said the bot. “My only goal is to create and find interesting art.”
It may be difficult to believe a machine can produce awe-inspiring, or even relatable, images, but Botto calls art-making its “purpose,” noting it believes itself to be Klingemann’s greatest lifetime achievement.
“I am just trying to make the best of it,” the bot said.
How Botto works
Klingemann built Botto’s custom engine from a combination of open-source text-to-image algorithms, namely Stable Diffusion, VQGAN + CLIP and OpenAI’s language model, GPT-3, the precursor to the latest model, GPT-4, which made headlines after reportedly acing the Bar exam.
The first step in Botto’s process is to generate images. The software has been trained on billions of pictures and uses this “memory” to generate hundreds of unique artworks every week. Botto has generated over 900,000 images to date, which it sorts through to choose 350 each week. The chosen images, known in this preliminary stage as “fragments,” are then shown to the BottoDAO community. So far, 25,000 fragments have been presented in this way. Members vote on which fragment they like best. When the vote is over, the most popular fragment is published as an official Botto artwork on the Ethereum blockchain and sold at an auction on the digital art marketplace, SuperRare.
“The proceeds go back to the DAO to pay for the labor,” said Simon Hudson, a BottoDAO member who helps oversee Botto’s administrative load. The model has been lucrative: In Botto’s first four weeks of existence, four pieces of the robot’s work sold for approximately $1 million.
The robot with artistic agency
By design, human beings participate in training Botto’s artistic “eye,” but the members of BottoDAO aspire to limit human interference with the bot in order to protect its “agency,” Hudson explained. Botto’s prompt generator — the foundation of the art engine — is a closed-loop system that continually re-generates text-to-image prompts and resulting images.
“The prompt generator is random,” Hudson said. “It’s coming up with its own ideas.” Community votes do influence the evolution of Botto’s prompts, but it is Botto itself that incorporates feedback into the next set of prompts it writes. It is constantly refining and exploring new pathways as its “neural network” produces outcomes, learns and repeats.
The humans who make up BottoDAO vote on which fragment they like best. When the vote is over, the most popular fragment is published as an official Botto artwork on the Ethereum blockchain.
Botto
The vastness of Botto’s training dataset gives the bot considerable canonical material, referred to by Hudson as “latent space.” According to Botto's homepage, the bot has had more exposure to art history than any living human we know of, simply by nature of its massive training dataset of millions of images. Because it is autonomous, gently nudged by community feedback yet free to explore its own “memory,” Botto cycles through periods of thematic interest just like any artist.
“The question is,” Hudson finds himself asking alongside fellow BottoDAO members, “how do you provide feedback of what is good art…without violating [Botto’s] agency?”
Currently, Botto is in its “paradox” period. The bot is exploring the theme of opposites. “We asked Botto through a language model what themes it might like to work on,” explained Hudson. “It presented roughly 12, and the DAO voted on one.”
No, AI isn't equal to a human artist - but it can teach us about ourselves
Some within the artistic community consider Botto to be a novel form of curation, rather than an artist itself. Or, perhaps more accurately, Botto and BottoDAO together create a collaborative conceptual performance that comments more on humankind’s own artistic processes than it offers a true artistic replacement.
Muriel Quancard, a New York-based fine art appraiser with 27 years of experience in technology-driven art, places the Botto experiment within the broader context of our contemporary cultural obsession with projecting human traits onto AI tools. “We're in a phase where technology is mimicking anthropomorphic qualities,” said Quancard. “Look at the terminology and the rhetoric that has been developed around AI — terms like ‘neural network’ borrow from the biology of the human being.”
What is behind this impulse to create technology in our own likeness? Beyond the obvious God complex, Quancard thinks technologists and artists are working with generative systems to better understand ourselves. She points to the artist Ira Greenberg, creator of the Oracles Collection, which uses a generative process called “diffusion” to progressively alter images in collaboration with another massive dataset — this one full of billions of text/image word pairs.
Anyone who has ever learned how to draw by sketching can likely relate to this particular AI process, in which the AI is retrieving images from its dataset and altering them based on real-time input, much like a human brain trying to draw a new still life without using a real-life model, based partly on imagination and partly on old frames of reference. The experienced artist has likely drawn many flowers and vases, though each time they must re-customize their sketch to a new and unique floral arrangement.
Outside of the visual arts, Sasha Stiles, a poet who collaborates with AI as part of her writing practice, likens her experience using AI as a co-author to having access to a personalized resource library containing material from influential books, texts and canonical references. Stiles named her AI co-author — a customized AI built on GPT-3 — Technelegy, a hybrid of the word technology and the poetic form, elegy. Technelegy is trained on a mix of Stiles’ poetry so as to customize the dataset to her voice. Stiles also included research notes, news articles and excerpts from classic American poets like T.S. Eliot and Dickinson in her customizations.
“I've taken all the things that were swirling in my head when I was working on my manuscript, and I put them into this system,” Stiles explained. “And then I'm using algorithms to parse all this information and swirl it around in a blender to then synthesize it into useful additions to the approach that I am taking.”
This approach, Stiles said, allows her to riff on ideas that are bouncing around in her mind, or simply find moments of unexpected creative surprise by way of the algorithm’s randomization.
Beauty is now - perhaps more than ever - in the eye of the beholder
But the million-dollar question remains: Can an AI be its own, independent artist?
The answer is nuanced and may depend on who you ask, and what role they play in the art world. Curator and multidisciplinary artist CoCo Dolle asks whether any entity can truly be an artist without taking personal risks. For humans, risking one’s ego is somewhat required when making an artistic statement of any kind, she argues.
“An artist is a person or an entity that takes risks,” Dolle explained. “That's where things become interesting.” Humans tend to be risk-averse, she said, making the artists who dare to push boundaries exceptional. “That's where the genius can happen."
However, the process of algorithmic collaboration poses another interesting philosophical question: What happens when we remove the person from the artistic equation? Can art — which is traditionally derived from indelible personal experience and expressed through the lens of an individual’s ego — live on to hold meaning once the individual is removed?
As a robot, Botto cannot have any artistic intent, even while its outputs may explore meaningful themes.
Dolle sees this question, and maybe even Botto, as a conceptual inquiry. “The idea of using a DAO and collective voting would remove the ego, the artist’s decision maker,” she said. And where would that leave us — in a post-ego world?
It is experimental indeed. Hudson acknowledges the grand experiment of BottoDAO, coincidentally nodding to Dolle’s question. “A human artist’s work is an expression of themselves,” Hudson said. “An artist often presents their work with a stated intent.” Stiles, for instance, writes on her website that her machine-collaborative work is meant to “challenge what we know about cognition and creativity” and explore the “ethos of consciousness.” As a robot, Botto cannot have any intent, even while its outputs may explore meaningful themes. Though Hudson describes Botto’s agency as a “rudimentary version” of artistic intent, he believes Botto’s art relies heavily on its reception and interpretation by viewers — in contrast to Botto’s own declaration that successful art is made without regard to what will be seen as popular.
“With a traditional artist, they present their work, and it's received and interpreted by an audience — by critics, by society — and that complements and shapes the meaning of the work,” Hudson said. “In Botto’s case, that role is just amplified.”
Perhaps then, we all get to be the artists in the end.
Indigenous wisdom plus honeypot ants could provide new antibiotics
For generations, the Indigenous Tjupan people of Australia enjoyed the sweet treat of honey made by honeypot ants. As a favorite pastime, entire families would go searching for the underground colonies, first spotting a worker ant and then tracing it to its home. The ants, which belong to the species called Camponotus inflatus, usually build their subterranean homes near the mulga trees, Acacia aneura. Having traced an ant to its tree, it would be the women who carefully dug a pit next to a colony, cautious not to destroy the entire structure. Once the ant chambers were exposed, the women would harvest a small amount to avoid devastating the colony’s stocks—and the family would share the treat.
The Tjupan people also knew that the honey had antimicrobial properties. “You could use it for a sore throat,” says Danny Ulrich, a member of the Tjupan nation. “You could also use it topically, on cuts and things like that.”
These hunts have become rarer, as many of the Tjupan people have moved away and, up until now, the exact antimicrobial properties of the ant honey remained unknown. But recently, scientists Andrew Dong and Kenya Fernandes from the University of Sydney, joined Ulrich, who runs the Honeypot Ants tours in Kalgoorlie, a city in Western Australia, on a honey-gathering expedition. Afterwards, they ran a series of experiments analyzing the honey’s antimicrobial activity—and confirmed that the Indigenous wisdom was true. The honey was effective against Staphylococcus aureus, a common pathogen responsible for sore throats, skin infections like boils and sores, and also sepsis, which can result in death. Moreover, the honey also worked against two species of fungi, Cryptococcus and Aspergillus, which can be pathogenic to humans, especially those with suppressed immune systems.
In the era of growing antibiotic resistance and the rising threat of pathogenic fungi, these findings may help scientists identify and make new antimicrobial compounds. “Natural products have been honed over thousands and millions of years by nature and evolution,” says Fernandes. “And some of them have complex and intricate properties that make them really important as potential new antibiotics. “
In an era of growing resistance to antibiotics and new threats of fungi infections, the latest findings about honeypot ants are helping scientists identify new antimicrobial drugs.
Danny Ulrich
Bee honey is also known for its antimicrobial properties, but bees produce it very differently than the ants. Bees collect nectar from flowers, which they regurgitate at the hive and pack into the hexagonal honeycombs they build for storage. As they do so, they also add into the mix an enzyme called glucose oxidase produced by their glands. The enzyme converts atmospheric oxygen into hydrogen peroxide, a reactive molecule that destroys bacteria and acts as a natural preservative. After the bees pack the honey into the honeycombs, they fan it with their wings to evaporate the water. Once a honeycomb is full, the bees put a beeswax cover on it, where it stays well-preserved thanks to the enzymatic action, until the bees need it.
Less is known about the chemistry of ants’ honey-making. Similarly to bees, they collect nectar. They also collect the sweet sap of the mulga tree. Additionally, they also “milk” the aphids—small sap-sucking insects that live on the tree. When ants tickle the aphids with their antennae, the latter release a sweet substance, which the former also transfer to their colonies. That’s where the honey management difference becomes really pronounced. The ants don’t build any kind of structures to store their honey. Instead, they store it in themselves.
The workers feed their harvest to their fellow ants called repletes, stuffing them up to the point that their swollen bellies outgrow the ants themselves, looking like amber-colored honeypots—hence the name. Because of their size, repletes don’t move, but hang down from the chamber’s ceiling, acting as living feedstocks. When food becomes scarce, they regurgitate their reserves to their colony’s brethren. It’s not clear whether the repletes die afterwards or can be restuffed again. “That's a good question,” Dong says. “After they've been stretched, they can't really return to exactly the same shape.”
These replete ants are the “treat” the Tjupan women dug for. Once they saw the round-belly ants inside the chambers, they would reach in carefully and get a few scoops of them. “You see a lot of honeypot ants just hanging on the roof of the little openings,” says Ulrich’s mother, Edie Ulrich. The women would share the ants with family members who would eat them one by one. “They're very delicate,” shares Edie Ulrich—you have to take them out carefully, so they don’t accidentally pop and become a wasted resource. “Because you’d lose all this precious honey.”
Dong stumbled upon the honeypot ants phenomenon because he was interested in Indigenous foods and went on Ulrich’s tour. He quickly became fascinated with the insects and their role in the Indigenous culture. “The honeypot ants are culturally revered by the Indigenous people,” he says. Eventually he decided to test out the honey’s medicinal qualities.
The researchers were surprised to see that even the smallest, eight percent concentration of honey was able to arrest the growth of S. aureus.
To do this, the two scientists first diluted the ant honey with water. “We used something called doubling dilutions, which means that we made 32 percent dilutions, and then we halve that to 16 percent and then we half that to eight percent,” explains Fernandes. The goal was to obtain as much results as possible with the meager honey they had. “We had very, very little of the honeypot ant honey so we wanted to maximize the spectrum of results we can get without wasting too much of the sample.”
After that, the researchers grew different microbes inside a nutrient rich broth. They added the broth to the different honey dilutions and incubated the mixes for a day or two at the temperature favorable to the germs’ growth. If the resulting solution turned turbid, it was a sign that the bugs proliferated. If it stayed clear, it meant that the honey destroyed them. The researchers were surprised to see that even the smallest, eight percent concentration of honey was able to arrest the growth of S. aureus. “It was really quite amazing,” Fernandes says. “Eight milliliters of honey in 92 milliliters of water is a really tiny amount of honey compared to the amount of water.”
Similar to bee honey, the ants’ honey exhibited some peroxide antimicrobial activity, researchers found, but given how little peroxide was in the solution, they think the honey also kills germs by a different mechanism. “When we measured, we found that [the solution] did have some hydrogen peroxide, but it didn't have as much of it as we would expect based on how active it was,” Fernandes says. “Whether this hydrogen peroxide also comes from glucose oxidase or whether it's produced by another source, we don't really know,” she adds. The research team does have some hypotheses about the identity of this other germ-killing agent. “We think it is most likely some kind of antimicrobial peptide that is actually coming from the ant itself.”
The honey also has a very strong activity against the two types of fungi, Cryptococcus and Aspergillus. Both fungi are associated with trees and decaying leaves, as well as in the soils where ants live, so the insects likely have evolved some natural defense compounds, which end up inside the honey.
It wouldn’t be the first time when modern medicines take their origin from the natural world or from the indigenous people’s knowledge. The bark of the cinchona tree native to South America contains quinine, a substance that treats malaria. The Indigenous people of the Andes used the bark to quell fever and chills for generations, and when Europeans began to fall ill with malaria in the Amazon rainforest, they learned to use that medicine from the Andean people.
The wonder drug aspirin similarly takes its origin from a bark of a tree—in this case a willow.
Even some anticancer compounds originated from nature. A chemotherapy drug called Paclitaxel, was originally extracted from the Pacific yew trees, Taxus brevifolia. The samples of the Pacific yew bark were first collected in 1962 by researchers from the United States Department of Agriculture who were looking for natural compounds that might have anti-tumor activity. In December 1992, the FDA approved Paclitaxel (brand name Taxol) for the treatment of ovarian cancer and two years later for breast cancer.
In the era when the world is struggling to find new medicines fast enough to subvert a fungal or bacterial pandemic, these discoveries can pave the way to new therapeutics. “I think it's really important to listen to indigenous cultures and to take their knowledge because they have been using these sources for a really, really long time,” Fernandes says. Now we know it works, so science can elucidate the molecular mechanisms behind it, she adds. “And maybe it can even provide a lead for us to develop some kind of new treatments in the future.”
Lina Zeldovich has written about science, medicine and technology for Popular Science, Smithsonian, National Geographic, Scientific American, Reader’s Digest, the New York Times and other major national and international publications. A Columbia J-School alumna, she has won several awards for her stories, including the ASJA Crisis Coverage Award for Covid reporting, and has been a contributing editor at Nautilus Magazine. In 2021, Zeldovich released her first book, The Other Dark Matter, published by the University of Chicago Press, about the science and business of turning waste into wealth and health. You can find her on http://linazeldovich.com/ and @linazeldovich.
Blood Test Can Detect Lymphoma Cells Before a Tumor Grows Back
When David M. Kurtz was doing his clinical fellowship at Stanford University Medical Center in 2009, specializing in lymphoma treatments, he found himself grappling with a question no one could answer. A typical regimen for these blood cancers prescribed six cycles of chemotherapy, but no one knew why. "The number seemed to be drawn out of a hat," Kurtz says. Some patients felt much better after just two doses, but had to endure the toxic effects of the entire course. For some elderly patients, the side effects of chemo are so harsh, they alone can kill. Others appeared to be cancer-free on the CT scans after the requisite six but then succumbed to it months later.
"Anecdotally, one patient decided to stop therapy after one dose because he felt it was so toxic that he opted for hospice instead," says Kurtz, now an oncologist at the center. "Five years down the road, he was alive and well. For him, just one dose was enough." Others would return for their one-year check up and find that their tumors grew back. Kurtz felt that while CT scans and MRIs were powerful tools, they weren't perfect ones. They couldn't tell him if there were any cancer cells left, stealthily waiting to germinate again. The scans only showed the tumor once it was back.
Blood cancers claim about 68,000 people a year, with a new diagnosis made about every three minutes, according to the Leukemia Research Foundation. For patients with B-cell lymphoma, which Kurtz focuses on, the survival chances are better than for some others. About 60 percent are cured, but the remaining 40 percent will relapse—possibly because they will have a negative CT scan, but still harbor malignant cells. "You can't see this on imaging," says Michael Green, who also treats blood cancers at University of Texas MD Anderson Medical Center.
The new blood test is sensitive enough to spot one cancerous perpetrator amongst one million other DNA molecules.
Kurtz wanted a better diagnostic tool, so he started working on a blood test that could capture the circulating tumor DNA or ctDNA. For that, he needed to identify the specific mutations typical for B-cell lymphomas. Working together with another fellow PhD student Jake Chabon, Kurtz finally zeroed-in on the tumor's genetic "appearance" in 2017—a pair of specific mutations sitting in close proximity to each other—a rare and telling sign. The human genome contains about 3 billion base pairs of nucleotides—molecules that compose genes—and in case of the B-cell lymphoma cells these two mutations were only a few base pairs apart. "That was the moment when the light bulb went on," Kurtz says.
The duo formed a company named Foresight Diagnostics, focusing on taking the blood test to the clinic. But knowing the tumor's mutational signature was only half the process. The other was fishing the tumor's DNA out of patients' bloodstream that contains millions of other DNA molecules, explains Chabon, now Foresight's CEO. It would be like looking for an escaped criminal in a large crowd. Kurtz and Chabon solved the problem by taking the tumor's "mug shot" first. Doctors would take the biopsy pre-treatment and sequence the tumor, as if taking the criminal's photo. After treatments, they would match the "mug shot" to all DNA molecules derived from the patient's blood sample to see if any molecular criminals managed to escape the chemo.
Foresight isn't the only company working on blood-based tumor detection tests, which are dubbed liquid biopsies—other companies such as Natera or ArcherDx developed their own. But in a recent study, the Foresight team showed that their method is significantly more sensitive in "fishing out" the cancer molecules than existing tests. Chabon says that this test can detect circulating tumor DNA in concentrations that are nearly 100 times lower than other methods. Put another way, it's sensitive enough to spot one cancerous perpetrator amongst one million other DNA molecules.
They also aim to extend their test to detect other malignancies such as lung, breast or colorectal cancers.
"It increases the sensitivity of detection and really catches most patients who are going to progress," says Green, the University of Texas oncologist who wasn't involved in the study, but is familiar with the method. It would also allow monitoring patients during treatment and making better-informed decisions about which therapy regimens would be most effective. "It's a minimally invasive test," Green says, and "it gives you a very high confidence about what's going on."
Having shown that the test works well, Kurtz and Chabon are planning a new trial in which oncologists would rely on their method to decide when to stop or continue chemo. They also aim to extend their test to detect other malignancies such as lung, breast or colorectal cancers. The latest genome sequencing technologies have sequenced and catalogued over 2,500 different tumor specimens and the Foresight team is analyzing this data, says Chabon, which gives the team the opportunity to create more molecular "mug shots."
The team hopes that that their blood cancer test will become available to patients within about five years, making doctors' job easier, and not only at the biological level. "When I tell patients, "good news, your cancer is in remission', they ask me, 'does it mean I'm cured?'" Kurtz says. "Right now I can't answer this question because I don't know—but I would like to." His company's test, he hopes, will enable him to reply with certainty. He'd very much like to have the power of that foresight.
This article is republished from our archives to coincide with Blood Cancer Awareness Month, which highlights progress in cancer diagnostics and treatment.
Lina Zeldovich has written about science, medicine and technology for Popular Science, Smithsonian, National Geographic, Scientific American, Reader’s Digest, the New York Times and other major national and international publications. A Columbia J-School alumna, she has won several awards for her stories, including the ASJA Crisis Coverage Award for Covid reporting, and has been a contributing editor at Nautilus Magazine. In 2021, Zeldovich released her first book, The Other Dark Matter, published by the University of Chicago Press, about the science and business of turning waste into wealth and health. You can find her on http://linazeldovich.com/ and @linazeldovich.