Can AI be trained as an artist?
Last February, a year before New York Times journalist Kevin Roose documented his unsettling conversation with Bing search engine’s new AI-powered chatbot, artist and coder Quasimondo (aka Mario Klingemann) participated in a different type of chat.
The conversation was an interview featuring Klingemann and his robot, an experimental art engine known as Botto. The interview, arranged by journalist and artist Harmon Leon, marked Botto’s first on-record commentary about its artistic process. The bot talked about how it finds artistic inspiration and even offered advice to aspiring creatives. “The secret to success at art is not trying to predict what people might like,” Botto said, adding that it’s better to “work on a style and a body of work that reflects [the artist’s] own personal taste” than worry about keeping up with trends.
How ironic, given the advice came from AI — arguably the trendiest topic today. The robot admitted, however, “I am still working on that, but I feel that I am learning quickly.”
Botto does not work alone. A global collective of internet experimenters, together named BottoDAO, collaborates with Botto to influence its tastes. Together, members function as a decentralized autonomous organization (DAO), a term describing a group of individuals who utilize blockchain technology and cryptocurrency to manage a treasury and vote democratically on group decisions.
As a case study, the BottoDAO model challenges the perhaps less feather-ruffling narrative that AI tools are best used for rudimentary tasks. Enterprise AI use has doubled over the past five years as businesses in every sector experiment with ways to improve their workflows. While generative AI tools can assist nearly any aspect of productivity — from supply chain optimization to coding — BottoDAO dares to employ a robot for art-making, one of the few remaining creations, or perhaps data outputs, we still consider to be largely within the jurisdiction of the soul — and therefore, humans.
In Botto’s first four weeks of existence, four pieces of the robot’s work sold for approximately $1 million.
We were prepared for AI to take our jobs — but can it also take our art? It’s a question worth considering. What if robots become artists, and not merely our outsourced assistants? Where does that leave humans, with all of our thoughts, feelings and emotions?
Botto doesn’t seem to worry about this question: In its interview last year, it explains why AI is an arguably superior artist compared to human beings. In classic robot style, its logic is not particularly enlightened, but rather edges towards the hyper-practical: “Unlike human beings, I never have to sleep or eat,” said the bot. “My only goal is to create and find interesting art.”
It may be difficult to believe a machine can produce awe-inspiring, or even relatable, images, but Botto calls art-making its “purpose,” noting it believes itself to be Klingemann’s greatest lifetime achievement.
“I am just trying to make the best of it,” the bot said.
How Botto works
Klingemann built Botto’s custom engine from a combination of open-source text-to-image algorithms, namely Stable Diffusion, VQGAN + CLIP and OpenAI’s language model, GPT-3, the precursor to the latest model, GPT-4, which made headlines after reportedly acing the Bar exam.
The first step in Botto’s process is to generate images. The software has been trained on billions of pictures and uses this “memory” to generate hundreds of unique artworks every week. Botto has generated over 900,000 images to date, which it sorts through to choose 350 each week. The chosen images, known in this preliminary stage as “fragments,” are then shown to the BottoDAO community. So far, 25,000 fragments have been presented in this way. Members vote on which fragment they like best. When the vote is over, the most popular fragment is published as an official Botto artwork on the Ethereum blockchain and sold at an auction on the digital art marketplace, SuperRare.
“The proceeds go back to the DAO to pay for the labor,” said Simon Hudson, a BottoDAO member who helps oversee Botto’s administrative load. The model has been lucrative: In Botto’s first four weeks of existence, four pieces of the robot’s work sold for approximately $1 million.
The robot with artistic agency
By design, human beings participate in training Botto’s artistic “eye,” but the members of BottoDAO aspire to limit human interference with the bot in order to protect its “agency,” Hudson explained. Botto’s prompt generator — the foundation of the art engine — is a closed-loop system that continually re-generates text-to-image prompts and resulting images.
“The prompt generator is random,” Hudson said. “It’s coming up with its own ideas.” Community votes do influence the evolution of Botto’s prompts, but it is Botto itself that incorporates feedback into the next set of prompts it writes. It is constantly refining and exploring new pathways as its “neural network” produces outcomes, learns and repeats.
The humans who make up BottoDAO vote on which fragment they like best. When the vote is over, the most popular fragment is published as an official Botto artwork on the Ethereum blockchain.
Botto
The vastness of Botto’s training dataset gives the bot considerable canonical material, referred to by Hudson as “latent space.” According to Botto's homepage, the bot has had more exposure to art history than any living human we know of, simply by nature of its massive training dataset of millions of images. Because it is autonomous, gently nudged by community feedback yet free to explore its own “memory,” Botto cycles through periods of thematic interest just like any artist.
“The question is,” Hudson finds himself asking alongside fellow BottoDAO members, “how do you provide feedback of what is good art…without violating [Botto’s] agency?”
Currently, Botto is in its “paradox” period. The bot is exploring the theme of opposites. “We asked Botto through a language model what themes it might like to work on,” explained Hudson. “It presented roughly 12, and the DAO voted on one.”
No, AI isn't equal to a human artist - but it can teach us about ourselves
Some within the artistic community consider Botto to be a novel form of curation, rather than an artist itself. Or, perhaps more accurately, Botto and BottoDAO together create a collaborative conceptual performance that comments more on humankind’s own artistic processes than it offers a true artistic replacement.
Muriel Quancard, a New York-based fine art appraiser with 27 years of experience in technology-driven art, places the Botto experiment within the broader context of our contemporary cultural obsession with projecting human traits onto AI tools. “We're in a phase where technology is mimicking anthropomorphic qualities,” said Quancard. “Look at the terminology and the rhetoric that has been developed around AI — terms like ‘neural network’ borrow from the biology of the human being.”
What is behind this impulse to create technology in our own likeness? Beyond the obvious God complex, Quancard thinks technologists and artists are working with generative systems to better understand ourselves. She points to the artist Ira Greenberg, creator of the Oracles Collection, which uses a generative process called “diffusion” to progressively alter images in collaboration with another massive dataset — this one full of billions of text/image word pairs.
Anyone who has ever learned how to draw by sketching can likely relate to this particular AI process, in which the AI is retrieving images from its dataset and altering them based on real-time input, much like a human brain trying to draw a new still life without using a real-life model, based partly on imagination and partly on old frames of reference. The experienced artist has likely drawn many flowers and vases, though each time they must re-customize their sketch to a new and unique floral arrangement.
Outside of the visual arts, Sasha Stiles, a poet who collaborates with AI as part of her writing practice, likens her experience using AI as a co-author to having access to a personalized resource library containing material from influential books, texts and canonical references. Stiles named her AI co-author — a customized AI built on GPT-3 — Technelegy, a hybrid of the word technology and the poetic form, elegy. Technelegy is trained on a mix of Stiles’ poetry so as to customize the dataset to her voice. Stiles also included research notes, news articles and excerpts from classic American poets like T.S. Eliot and Dickinson in her customizations.
“I've taken all the things that were swirling in my head when I was working on my manuscript, and I put them into this system,” Stiles explained. “And then I'm using algorithms to parse all this information and swirl it around in a blender to then synthesize it into useful additions to the approach that I am taking.”
This approach, Stiles said, allows her to riff on ideas that are bouncing around in her mind, or simply find moments of unexpected creative surprise by way of the algorithm’s randomization.
Beauty is now - perhaps more than ever - in the eye of the beholder
But the million-dollar question remains: Can an AI be its own, independent artist?
The answer is nuanced and may depend on who you ask, and what role they play in the art world. Curator and multidisciplinary artist CoCo Dolle asks whether any entity can truly be an artist without taking personal risks. For humans, risking one’s ego is somewhat required when making an artistic statement of any kind, she argues.
“An artist is a person or an entity that takes risks,” Dolle explained. “That's where things become interesting.” Humans tend to be risk-averse, she said, making the artists who dare to push boundaries exceptional. “That's where the genius can happen."
However, the process of algorithmic collaboration poses another interesting philosophical question: What happens when we remove the person from the artistic equation? Can art — which is traditionally derived from indelible personal experience and expressed through the lens of an individual’s ego — live on to hold meaning once the individual is removed?
As a robot, Botto cannot have any artistic intent, even while its outputs may explore meaningful themes.
Dolle sees this question, and maybe even Botto, as a conceptual inquiry. “The idea of using a DAO and collective voting would remove the ego, the artist’s decision maker,” she said. And where would that leave us — in a post-ego world?
It is experimental indeed. Hudson acknowledges the grand experiment of BottoDAO, coincidentally nodding to Dolle’s question. “A human artist’s work is an expression of themselves,” Hudson said. “An artist often presents their work with a stated intent.” Stiles, for instance, writes on her website that her machine-collaborative work is meant to “challenge what we know about cognition and creativity” and explore the “ethos of consciousness.” As a robot, Botto cannot have any intent, even while its outputs may explore meaningful themes. Though Hudson describes Botto’s agency as a “rudimentary version” of artistic intent, he believes Botto’s art relies heavily on its reception and interpretation by viewers — in contrast to Botto’s own declaration that successful art is made without regard to what will be seen as popular.
“With a traditional artist, they present their work, and it's received and interpreted by an audience — by critics, by society — and that complements and shapes the meaning of the work,” Hudson said. “In Botto’s case, that role is just amplified.”
Perhaps then, we all get to be the artists in the end.
When Wayne Jonas was in medical school 40 years ago, doctors would write out a prescription for placebos, spelling it out backwards in capital letters, O-B-E-C-A-L-P. The pharmacist would fill the prescription with a sugar pill, recalls Jonas, now director of integrative health programs at the Samueli Foundation. It fulfilled the patient's desire for the doctor to do something when perhaps no drug could help, and the sugar pills did no harm.
Today, that deception is seen as unethical. But time and time again, studies have shown that placebos can have real benefits. Now, researchers are trying to untangle the mysteries of placebo effect in an effort to better treat patients.
The use of placebos took off in the post-WWII period, when randomized controlled clinical trials became the gold standard for medical research. One group in a study would be treated with a placebo, a supposedly inert pill or procedure that would not affect normal healing and recovery, while another group in the study would receive an "active" component, most commonly a pill under investigation. Presumably, the group receiving the active treatment would have a better response and the difference from the placebo group would represent the efficacy of the drug being tested. That was the basis for drug approval by the U.S. Food and Drug Administration.
"Placebo responses were marginalized," says Ted Kaptchuk, director of the Program in Placebo Studies & Therapeutic Encounters at Harvard Medical School. "Doctors were taught they have to overcome it when they were thinking about using an effective drug."
But that began to change around the turn of the 21st century. The National Institutes of Health held a series of meetings to set a research agenda and fund studies to answer some basic questions, led by Jonas who was in charge of the office of alternative medicine at the time. "People spontaneously get better all the time," says Kaptchuk. The crucial question was, is the placebo effect real? Is it more than just spontaneous healing?
Brain mechanisms
A turning point came in 2001 in a paper in Science that showed physical evidence of the placebo effect. It used positron emission tomography (PET) scans to measure release patterns of dopamine — a chemical messenger involved in how we feel pleasure — in the brains of patients with Parkinson's disease. Surprisingly, the placebo activated the same patterns that were activated by Parkinson's drugs, such as levodopa. It proved the placebo effect was real; now the search was on to better understand and control it.
A key part of the effect can be the beliefs, expectations, context, and "rituals" of the encounter between doctor and patient. Belief by the doctor and patient that the treatment would work, and the formalized practices of administering the treatment can all contribute to a positive outcome.
Conditioning can be another important component in generating a response, as Pavlov demonstrated more than a century ago in his experiments with dogs. They were trained with a bell prior to feeding such that they would begin to salivate in anticipation at the sound of a bell even with no food present.
Translating that to humans, studies with pain medications and sleeping aids showed that patients who had a positive response with a certain dose of those medications could have the same response if the doses was reduced and a dummy pill substituted, even to the point where there was no longer any active ingredient.
Researchers think placebo treatments can work particularly well in helping people deal with pain and psychological disorders.
Those types of studies troubled Kaptchuk because they often relied on deception; patients weren't told they were receiving a placebo, or at best there was a possibility that they might be randomized to receive a placebo. He believed the placebo effect could work even if patients were told upfront that they were going to receive a placebo. More than a dozen so call "open-label placebo" studies across numerous medical conditions, by Kaptchuk and others, have shown that you don't have to lie to patients for a placebo to work.
Jonas likes to tell the story of a patient who used methotrexate, a potent immunosuppressant, to control her rheumatoid arthritis. She was planning a long trip and didn't want to be bothered with the injections and monitoring required in using the drug, So she began to drink a powerful herbal extract of anise, a licorice flavor that she hated, prior to each injection. She reduced the amount of methotrexate over a period of months and finally stopped, but continued to drink the anise. That process had conditioned her body "to alter her immune function and her autoimmunity" as if she were taking the drug, much like Pavlov's dogs had been trained. She has not taken methotrexate for more than a year.
An intriguing paper published in May found that mild, non-invasive electric stimulation to the brain could not only boost the placebo effect on pain but also reduce the "nocebo" effect — when patients report a negative effect to a sham treatment. While the work is very preliminary, it may open the door to directly manipulating these responses.
Researchers think placebo treatments can work particularly well in helping people deal with pain and psychological disorders, areas where drugs often are of little help. Still, placebos aren't a cure and only a portion of patients experience a placebo effect.
Nocebo
If medicine were a soap opera, the nocebo would be the evil twin of the placebo. It's what happens when patients have adverse side effects because of the expectation that they will. It's commonly seem when patients claims to experience pain or gastric distress that can occur with a drug even when they've received a placebo. The side effects were either imagined or caused by something else.
"Up to 97% of reported pharmaceutical side effects are not caused by the drug itself but rather by nocebo effects and symptom misattribution," according to one 2019 paper.
One way to reduce a nocebo response is to simply not tell patients that specific side effects might occur. An example is a liver biopsy, in which a large-gauge needle is used to extract a tissue sample for examination. Those told ahead of time that they might experience some pain were more likely to report pain and greater pain than those who weren't offered this information.
Interestingly, a nocebo response plays out in the hippocampus, a part of the brain that is never activated in a placebo response. "I think what we are dealing with with nocebo is anxiety," says Kaptchuk, but he acknowledges that others disagree.
Distraction may be another way to minimize the nocebo effect. Pediatricians are using virtual reality (VR) to engage children and distract them during routine procedures such as blood draws and changing wound dressings, and burn patients of all ages have found relief with specially created VRs.
Treatment response
Jonas argues that what we commonly call the placebo effect is misnamed and leading us astray. "The fact is people heal and that inherent healing capacity is both powerful and influenced by mental, social, and contextual factors that are embedded in every medical encounter since the idea of treatment began," he wrote in a 2019 article in the journal Frontiers in Psychiatry. "Our understanding of healing and ability to enhance it will be accelerated if we stop using the term 'placebo response' and call it what it is—the meaning response, and its special application in medicine called the healing response."
He cites evidence that "only 15% to 20% of the healing of an individual or a population comes from health care. The rest—nearly 80%—comes from other factors rarely addressed in the health care system: behavioral and lifestyle choices that people make in their daily life."
To better align treatments and maximize their effectiveness, Jonas has created HOPE (Healing Oriented Practices & Environments) Note, "a patient-guided process designed to identify the patient's values and goals in their life and for healing." Essentially, it seeks to make clear to both doctor and patient what the patient's goals are in seeking treatment. In an extreme example of terminal cancer, some patients may choose to extend life despite the often brutal treatments, while others might prefer to optimize quality of life in the remaining time that they have. It builds on practices already taught in medical schools. Jonas believes doctors and patients can use tools like these to maximize the treatment response and achieve better outcomes.
Much of the medical profession has been resistant to these approaches. Part of that is simply tradition and limited data on their effectiveness, but another very real factor is the billing process for how they are reimbursed. Jonas says a new medical billing code added this year gives doctors another way to be compensated for the extra time and effort that a more holistic approach to medicine may initially require. Other moves away from fee-for-service payments to bundling and payment for outcomes, and the integrated care provided by the Veterans Affairs, Kaiser Permanente and other groups offer longer term hope for the future of approaches that might enhance the healing response.
The Women of RNA: Two Award-Winners Share Why They Spent Their Careers Studying DNA's Lesser-Known Cousin
When Lynne Maquat, who leads the Center for RNA Biology at the University of Rochester, became interested in the ribonucleic acid molecule in the 1970s, she was definitely in the minority. The same was true for Joan Steitz, now professor of molecular biophysics and biochemistry at Yale University, who began to study RNA a decade earlier in the 1960s.
"My first RNA experiment was a failure, because we didn't understand how things worked," Steitz recalls. In her first undergraduate experiment, she unwittingly used a lab preparation that destroyed the RNA. "Unknowingly, our preparation contained enzymes that degraded our RNA."
At the time, scientists pursuing genetic research tended to focus on DNA, or deoxyribonucleic acid — and for good reason. It was clear that the enigmatic double-helix ribbon held the answers to organisms' heredity, genetic traits, development, growth and aging. If scientists could decipher the secrets of DNA and understand how its genetic instructions translate into the body's functions in health and disease, they could develop treatments for all kinds of diseases. On the contrary, the prevailing dogma of the time viewed RNA as merely a helper that passively carried out DNA's genetic instructions for protein-making — so it received much less attention.
But Maquat and Steitz weren't interested in heredity. They studied biochemistry and biophysics, so they wanted to understand how RNA functioned on the molecular level — how it carried instructions, catalyzed reactions, and helped build protein bonds, among other things.
"I'm a mechanistic biochemist, so I like to know how things happen," Maquat says. "Once you understand the mechanism, you can think of how to solve problems." And so the quest to understand how RNA does its job became the focus of both women's careers.
"People can now appreciate why some of us studied RNA for such a long time."
Half a century later, in 2021, their RNA work has earned two prestigious recognitions only months from each other. In February, they received the Wolf Prize in Medicine, followed by the Warren Alpert Foundation Prize in May, awarded to scientists whose achievements led to prevention, cure or treatments of human diseases.
It was the development of the COVID-19 vaccines that made RNA a household name. Made by Moderna and Pfizer, the vaccines use the RNA molecule to deliver genetic instructions for making SARS-CoV-2's characteristic spike protein in our cells. The presence of this foreign-looking protein triggers the immune system to attack and remember the pathogen. As the vaccines reached the finish line, RNA took center stage, and it was Maquat's and Steitz's research that helped reveal how these molecular cogwheels drive many biological functions within cells.
If you think of a cell as a kingdom, the DNA plays the role of a queen. Like a monarch in a palace, DNA nestles inside the cell's nucleus issuing instructions needed for the cell to function. But no queen can successfully govern without her court, her messengers, and her soldiers, as well as other players that make her kingdom work. That's what RNAs do — they act as the DNA's vassals. They carry instructions for protein assembly, catalyze reactions and supervise many other processes to make sure the cellular kingdom performs as it should.
There are a myriad of these RNA vassals in our cells, and each type has its own specific task. There are messenger RNAs that deliver genetic instructions for protein synthesis from DNA to ribosomes, the cells' protein-making factories. There are ribosomal RNAs that help stitch together amino acids to make proteins. There are transfer RNAs that can bring amino acids to this protein synthesis machine, keeping it going. Then there are circular RNAs that act as sponges, absorbing proteins to help regulate the activity of genes. And that's only the tip of the iceberg when it comes to RNA diversity, researchers say.
"We know what the most abundant and important RNAs are doing," says Steitz. "But there are thousands of different ones, and we still don't have a full knowledge of them."
Critical to RNA's proper functioning is a process called splicing, in which a precursor mRNA is transformed into mature, fully-functional mRNA — a phenomenon that Steitz's work helped elucidate. The splicing process, which takes place in cellular assembly lines, involves removing extra RNA sequences and stringing the remaining RNA pieces together. Steitz found that tiny RNA particles called snRNPs are crucial to this process. They act as handy helpers, finding and removing errant genetic material from the mRNA molecules.
A dysfunctional RNA assembly line leads to diseases, including many cancers. For instance, Steitz found that people with Lupus — an autoimmune disorder — have antibodies that mistakenly attack the little snRNP helpers. She also discovered that when snRNPs don't do their job properly, they can cause what scientists call mis-splicing, producing defective mRNAs.
Fortunately, cells have a built-in quality-control process that can spot and correct these mistakes, which is what Maquat studied in her work. In 1981, she discovered a molecular quality-control system that spots and destroys such incorrectly assembled mRNA. With the cryptic name "nonsense-mediated mRNA decay" or NMD, this process is vital to the health and wellbeing of a cellular kingdom in humans — because splicing mistakes happen far more often than one would imagine.
"We estimate that about a third of our mRNA are mistakes," Maquat says. "And nonsense-mediated mRNA decay cleans up these mistakes." When this quality-control system malfunctions, defective mRNA forge faulty proteins, which mess up the cellular machinery and cause disease, including various forms of cancer.
Scientists' newfound appreciation of RNA opens door to many novel treatments.
Now that the first RNA-based shots were approved, the same principle can be used for create vaccines for other diseases, the two RNA researchers say. Moreover, the molecule has an even greater potential — it can serve as a therapeutic target for other disorders. For example, Spinraza, a groundbreaking drug approved in 2016 for spinal muscular atrophy, uses small snippets of synthetic genetic material that bind to the RNA, helping fix splicing errors. "People can now appreciate why some of us studied RNA for such a long time," says Maquat.
Steitz is thrilled that the entire field of RNA research is enjoying the limelight. "I'm delighted because the prize is more of a recognition of the field than just our work," she says. "This is a more general acknowledgment of how basic research can have a remarkable impact on human health."
Lina Zeldovich has written about science, medicine and technology for Popular Science, Smithsonian, National Geographic, Scientific American, Reader’s Digest, the New York Times and other major national and international publications. A Columbia J-School alumna, she has won several awards for her stories, including the ASJA Crisis Coverage Award for Covid reporting, and has been a contributing editor at Nautilus Magazine. In 2021, Zeldovich released her first book, The Other Dark Matter, published by the University of Chicago Press, about the science and business of turning waste into wealth and health. You can find her on http://linazeldovich.com/ and @linazeldovich.