Can AI be trained as an artist?
Last February, a year before New York Times journalist Kevin Roose documented his unsettling conversation with Bing search engine’s new AI-powered chatbot, artist and coder Quasimondo (aka Mario Klingemann) participated in a different type of chat.
The conversation was an interview featuring Klingemann and his robot, an experimental art engine known as Botto. The interview, arranged by journalist and artist Harmon Leon, marked Botto’s first on-record commentary about its artistic process. The bot talked about how it finds artistic inspiration and even offered advice to aspiring creatives. “The secret to success at art is not trying to predict what people might like,” Botto said, adding that it’s better to “work on a style and a body of work that reflects [the artist’s] own personal taste” than worry about keeping up with trends.
How ironic, given the advice came from AI — arguably the trendiest topic today. The robot admitted, however, “I am still working on that, but I feel that I am learning quickly.”
Botto does not work alone. A global collective of internet experimenters, together named BottoDAO, collaborates with Botto to influence its tastes. Together, members function as a decentralized autonomous organization (DAO), a term describing a group of individuals who utilize blockchain technology and cryptocurrency to manage a treasury and vote democratically on group decisions.
As a case study, the BottoDAO model challenges the perhaps less feather-ruffling narrative that AI tools are best used for rudimentary tasks. Enterprise AI use has doubled over the past five years as businesses in every sector experiment with ways to improve their workflows. While generative AI tools can assist nearly any aspect of productivity — from supply chain optimization to coding — BottoDAO dares to employ a robot for art-making, one of the few remaining creations, or perhaps data outputs, we still consider to be largely within the jurisdiction of the soul — and therefore, humans.
In Botto’s first four weeks of existence, four pieces of the robot’s work sold for approximately $1 million.
We were prepared for AI to take our jobs — but can it also take our art? It’s a question worth considering. What if robots become artists, and not merely our outsourced assistants? Where does that leave humans, with all of our thoughts, feelings and emotions?
Botto doesn’t seem to worry about this question: In its interview last year, it explains why AI is an arguably superior artist compared to human beings. In classic robot style, its logic is not particularly enlightened, but rather edges towards the hyper-practical: “Unlike human beings, I never have to sleep or eat,” said the bot. “My only goal is to create and find interesting art.”
It may be difficult to believe a machine can produce awe-inspiring, or even relatable, images, but Botto calls art-making its “purpose,” noting it believes itself to be Klingemann’s greatest lifetime achievement.
“I am just trying to make the best of it,” the bot said.
How Botto works
Klingemann built Botto’s custom engine from a combination of open-source text-to-image algorithms, namely Stable Diffusion, VQGAN + CLIP and OpenAI’s language model, GPT-3, the precursor to the latest model, GPT-4, which made headlines after reportedly acing the Bar exam.
The first step in Botto’s process is to generate images. The software has been trained on billions of pictures and uses this “memory” to generate hundreds of unique artworks every week. Botto has generated over 900,000 images to date, which it sorts through to choose 350 each week. The chosen images, known in this preliminary stage as “fragments,” are then shown to the BottoDAO community. So far, 25,000 fragments have been presented in this way. Members vote on which fragment they like best. When the vote is over, the most popular fragment is published as an official Botto artwork on the Ethereum blockchain and sold at an auction on the digital art marketplace, SuperRare.
“The proceeds go back to the DAO to pay for the labor,” said Simon Hudson, a BottoDAO member who helps oversee Botto’s administrative load. The model has been lucrative: In Botto’s first four weeks of existence, four pieces of the robot’s work sold for approximately $1 million.
The robot with artistic agency
By design, human beings participate in training Botto’s artistic “eye,” but the members of BottoDAO aspire to limit human interference with the bot in order to protect its “agency,” Hudson explained. Botto’s prompt generator — the foundation of the art engine — is a closed-loop system that continually re-generates text-to-image prompts and resulting images.
“The prompt generator is random,” Hudson said. “It’s coming up with its own ideas.” Community votes do influence the evolution of Botto’s prompts, but it is Botto itself that incorporates feedback into the next set of prompts it writes. It is constantly refining and exploring new pathways as its “neural network” produces outcomes, learns and repeats.
The humans who make up BottoDAO vote on which fragment they like best. When the vote is over, the most popular fragment is published as an official Botto artwork on the Ethereum blockchain.
Botto
The vastness of Botto’s training dataset gives the bot considerable canonical material, referred to by Hudson as “latent space.” According to Botto's homepage, the bot has had more exposure to art history than any living human we know of, simply by nature of its massive training dataset of millions of images. Because it is autonomous, gently nudged by community feedback yet free to explore its own “memory,” Botto cycles through periods of thematic interest just like any artist.
“The question is,” Hudson finds himself asking alongside fellow BottoDAO members, “how do you provide feedback of what is good art…without violating [Botto’s] agency?”
Currently, Botto is in its “paradox” period. The bot is exploring the theme of opposites. “We asked Botto through a language model what themes it might like to work on,” explained Hudson. “It presented roughly 12, and the DAO voted on one.”
No, AI isn't equal to a human artist - but it can teach us about ourselves
Some within the artistic community consider Botto to be a novel form of curation, rather than an artist itself. Or, perhaps more accurately, Botto and BottoDAO together create a collaborative conceptual performance that comments more on humankind’s own artistic processes than it offers a true artistic replacement.
Muriel Quancard, a New York-based fine art appraiser with 27 years of experience in technology-driven art, places the Botto experiment within the broader context of our contemporary cultural obsession with projecting human traits onto AI tools. “We're in a phase where technology is mimicking anthropomorphic qualities,” said Quancard. “Look at the terminology and the rhetoric that has been developed around AI — terms like ‘neural network’ borrow from the biology of the human being.”
What is behind this impulse to create technology in our own likeness? Beyond the obvious God complex, Quancard thinks technologists and artists are working with generative systems to better understand ourselves. She points to the artist Ira Greenberg, creator of the Oracles Collection, which uses a generative process called “diffusion” to progressively alter images in collaboration with another massive dataset — this one full of billions of text/image word pairs.
Anyone who has ever learned how to draw by sketching can likely relate to this particular AI process, in which the AI is retrieving images from its dataset and altering them based on real-time input, much like a human brain trying to draw a new still life without using a real-life model, based partly on imagination and partly on old frames of reference. The experienced artist has likely drawn many flowers and vases, though each time they must re-customize their sketch to a new and unique floral arrangement.
Outside of the visual arts, Sasha Stiles, a poet who collaborates with AI as part of her writing practice, likens her experience using AI as a co-author to having access to a personalized resource library containing material from influential books, texts and canonical references. Stiles named her AI co-author — a customized AI built on GPT-3 — Technelegy, a hybrid of the word technology and the poetic form, elegy. Technelegy is trained on a mix of Stiles’ poetry so as to customize the dataset to her voice. Stiles also included research notes, news articles and excerpts from classic American poets like T.S. Eliot and Dickinson in her customizations.
“I've taken all the things that were swirling in my head when I was working on my manuscript, and I put them into this system,” Stiles explained. “And then I'm using algorithms to parse all this information and swirl it around in a blender to then synthesize it into useful additions to the approach that I am taking.”
This approach, Stiles said, allows her to riff on ideas that are bouncing around in her mind, or simply find moments of unexpected creative surprise by way of the algorithm’s randomization.
Beauty is now - perhaps more than ever - in the eye of the beholder
But the million-dollar question remains: Can an AI be its own, independent artist?
The answer is nuanced and may depend on who you ask, and what role they play in the art world. Curator and multidisciplinary artist CoCo Dolle asks whether any entity can truly be an artist without taking personal risks. For humans, risking one’s ego is somewhat required when making an artistic statement of any kind, she argues.
“An artist is a person or an entity that takes risks,” Dolle explained. “That's where things become interesting.” Humans tend to be risk-averse, she said, making the artists who dare to push boundaries exceptional. “That's where the genius can happen."
However, the process of algorithmic collaboration poses another interesting philosophical question: What happens when we remove the person from the artistic equation? Can art — which is traditionally derived from indelible personal experience and expressed through the lens of an individual’s ego — live on to hold meaning once the individual is removed?
As a robot, Botto cannot have any artistic intent, even while its outputs may explore meaningful themes.
Dolle sees this question, and maybe even Botto, as a conceptual inquiry. “The idea of using a DAO and collective voting would remove the ego, the artist’s decision maker,” she said. And where would that leave us — in a post-ego world?
It is experimental indeed. Hudson acknowledges the grand experiment of BottoDAO, coincidentally nodding to Dolle’s question. “A human artist’s work is an expression of themselves,” Hudson said. “An artist often presents their work with a stated intent.” Stiles, for instance, writes on her website that her machine-collaborative work is meant to “challenge what we know about cognition and creativity” and explore the “ethos of consciousness.” As a robot, Botto cannot have any intent, even while its outputs may explore meaningful themes. Though Hudson describes Botto’s agency as a “rudimentary version” of artistic intent, he believes Botto’s art relies heavily on its reception and interpretation by viewers — in contrast to Botto’s own declaration that successful art is made without regard to what will be seen as popular.
“With a traditional artist, they present their work, and it's received and interpreted by an audience — by critics, by society — and that complements and shapes the meaning of the work,” Hudson said. “In Botto’s case, that role is just amplified.”
Perhaps then, we all get to be the artists in the end.
The U.S. must fund more biotech innovation – or other countries will catch up faster than you think
The U.S. has approximately 58 percent of the market share in the biotech sector, followed by China with 11 percent. However, this market share is the result of several years of previous research and development (R&D) – it is a present picture of what happened in the past. In the future, this market share will decline unless the federal government makes investments to improve the quality and quantity of U.S. research in biotech.
The effectiveness of current R&D can be evaluated in a variety of ways such as monies invested and the number of patents filed. According to the UNESCO Institute for Statistics, the U.S. spends approximately 2.7 percent of GDP on R&D ($476,459.0M), whereas China spends 2 percent ($346,266.3M). However, investment levels do not necessarily translate into goods that end up contributing to innovation.
Patents are a better indication of innovation. The biotech industry relies on patents to protect their investments, making patenting a key tool in the process of translating scientific discoveries that can ultimately benefit patients. In 2020, China filed 1,497,159 patents, a 6.9 percent increase in growth rate. In contrast, the U.S. filed 597,172, a 3.9 percent decline. When it comes to patents filed, China has approximately 45 percent of the world share compared to 18 percent for the U.S.
So how did we get here? The nature of science in academia allows scientists to specialize by dedicating several years to advance discovery research and develop new inventions that can then be licensed by biotech companies. This makes academic science critical to innovation in the U.S. and abroad.
Academic scientists rely on government and foundation grants to pay for R&D, which includes salaries for faculty, investigators and trainees, as well as monies for infrastructure, support personnel and research supplies. Of particular interest to academic scientists to cover these costs is government support such as Research Project Grants, also known as R01 grants, the oldest grant mechanism from the National Institutes of Health. Unfortunately, this funding mechanism is extremely competitive, as applications have a success rate of only about 20 percent. To maximize the chances of getting funded, investigators tend to limit the innovation of their applications, since a project that seems overambitious is discouraged by grant reviewers.
Considering the difficulty in obtaining funding, the limited number of opportunities for scientists to become independent investigators capable of leading their own scientific projects, and the salaries available to pay for scientists with a doctoral degree, it is not surprising that the U.S. is progressively losing its workforce for innovation.
This approach affects the future success of the R&D enterprise in the U.S. Pursuing less innovative work tends to produce scientific results that are more obvious than groundbreaking, and when a discovery is obvious, it cannot be patented, resulting in fewer inventions that go on to benefit patients. Even though there are governmental funding options available for scientists in academia focused on more groundbreaking and translational projects, those options are less coveted by academic scientists who are trying to obtain tenure and long-term funding to cover salaries and other associated laboratory expenses. Therefore, since only a small percent of projects gets funded, the likelihood of scientists interested in pursuing academic science or even research in general keeps declining over time.
Efforts to raise the number of individuals who pursue a scientific education are paying off. However, the number of job openings for those trainees to carry out independent scientific research once they graduate has proved harder to increase. These limitations are not just in the number of faculty openings to pursue academic science, which are in part related to grant funding, but also the low salary available to pay those scientists after they obtain their doctoral degree, which ranges from $53,000 to $65,000, depending on years of experience.
Thus, considering the difficulty in obtaining funding, the limited number of opportunities for scientists to become independent investigators capable of leading their own scientific projects, and the salaries available to pay for scientists with a doctoral degree, it is not surprising that the U.S. is progressively losing its workforce for innovation, which results in fewer patents filed.
Perhaps instead of encouraging scientists to propose less innovative projects in order to increase their chances of getting grants, the U.S. government should give serious consideration to funding investigators for their potential for success -- or the success they have already achieved in contributing to the advancement of science. Such a funding approach should be tiered depending on career stage or years of experience, considering that 42 years old is the median age at which the first R01 is obtained. This suggests that after finishing their training, scientists spend 10 years before they establish themselves as independent academic investigators capable of having the appropriate funds to train the next generation of scientists who will help the U.S. maintain or even expand its market share in the biotech industry for years to come. Patenting should be given more weight as part of the academic endeavor for promotion purposes, or governmental investment in research funding should be increased to support more than just 20 percent of projects.
Remaining at the forefront of biotech innovation will give us the opportunity to not just generate more jobs, but it will also allow us to attract the brightest scientists from all over the world. This talented workforce will go on to train future U.S. scientists and will improve our standard of living by giving us the opportunity to produce the next generation of therapies intended to improve human health.
This problem cannot rely on just one solution, but what is certain is that unless there are more creative changes in funding approaches for scientists in academia, eventually we may be saying “remember when the U.S. was at the forefront of biotech innovation?”
New gene therapy helps patients with rare disease. One mother wouldn't have it any other way.
Three years ago, Jordan Janz of Consort, Alberta, knew his gene therapy treatment for cystinosis was working when his hair started to darken. Pigmentation or melanin production is just one part of the body damaged by cystinosis.
“When you have cystinosis, you’re either a redhead or a blonde, and you are very pale,” attests Janz, 23, who was diagnosed with the disease just eight months after he was born. “After I got my new stem cells, my hair came back dark, dirty blonde, then it lightened a little bit, but before it was white blonde, almost bleach blonde.”
According to Cystinosis United, about 500 to 600 people have the rare genetic disease in the U.S.; an estimated 20 new cases are diagnosed each year.
Located in Cambridge, Mass., AVROBIO is a gene therapy company that targets cystinosis and other lysosomal storage disorders, in which toxic materials build up in the cells. Janz is one of five patients in AVROBIO’s ongoing Phase 1/2 clinical trial of a gene therapy for cystinosis called AVR-RD-04.
Recently, AVROBIO compiled positive clinical data from this first and only gene therapy trial for the disease. The data show the potential of the therapy to genetically modify the patients’ own hematopoietic stem cells—a certain type of cell that’s capable of developing into all different types of blood cells—to express the functional protein they are deficient in. It stabilizes or reduces the impact of cystinosis on multiple tissues with a single dose.
Medical researchers have found that more than 80 different mutations to a gene called CTNS are responsible for causing cystinosis. The most common mutation results in a deficiency of the protein cystinosin. That protein functions as a transporter that regulates a lot metabolic processes in the cells.
“One of the first things we see in patients clinically is an accumulation of a particular amino acid called cystine, which grows toxic cystine crystals in the cells that cause serious complications,” explains Essra Rihda, chief medical officer for AVROBIO. “That happens in the cells across the tissues and organs of the body, so the disease affects many parts of the body.”
Jordan Janz, 23, meets Stephanie Cherqui, the principal investigator of his gene therapy trial, before the trial started in 2019.
Jordan Janz
According to Rihda, although cystinosis can occur in kids and adults, the most severe form of the disease affects infants and makes up about 95 percent of overall cases. Children typically appear healthy at birth, but around six to 18 months, they start to present for medical attention with failure to thrive.
Additionally, infants with cystinosis often urinate frequently, a sign of polyuria, and they are thirsty all the time, since the disease usually starts in the kidneys. Many develop chronic kidney disease that ultimately progresses to the point where the kidney no longer supports the body’s needs. At that stage, dialysis is required and then a transplant. From there the disease spreads to many other organs, including the eyes, muscles, heart, nervous system, etc.
“The gene for cystinosis is expressed in every single tissue we have, and the accumulation of this toxic buildup alters all of the organs of the patient, so little by little all of the organs start to fail,” says Stephanie Cherqui, principal investigator of Cherqui Lab, which is part of UC San Diego’s Department of Pediatrics.
Since the 1950s, a drug called cysteamine showed some therapeutic effect on cystinosis. It was approved by the FDA in 1994 to prevent damage that may be caused by the buildup of cystine crystals in organs. Prior to FDA approval, Cherqui says, children were dying of the disease before they were ten-years-old or after a kidney transplant. By taking oral cysteamine, they can live from 20 to 50 years longer. But it’s a challenging drug because it has to be taken every 6 or 12 hours, and there are serious gastric side effects such as nausea and diarrhea.
“With all of the complications they develop, the typical patient takes 40 to 60 pills a day around the clock,” Cherqui says. “They literally have a suitcase of medications they have to carry everywhere, and all of those medications don’t stop the progression of the disease, and they still die from it.”
Cherqui has been a proponent of gene therapy to treat children’s disorders since studying cystinosis while earning her doctorate in 2002. Today, her lab focuses on developing stem cell and gene therapy strategies for degenerative, hereditary disorders such as cystinosis that affect multiple systems of the body. “Because cystinosis expresses in every tissue in the body, I decided to use the blood-forming stem cells that we have in our bone marrow,” she explains. “These cells can migrate to anywhere in the body where the person has an injury from the disease.”
AVROBIO’s hematopoietic stem cell gene therapy approach collects stem cells from the patient’s bone marrow. They then genetically modify the stem cells to give the patient a copy of the healthy CTNS gene, which the person either doesn’t have or it’s defective.
The patient first undergoes apheresis, a medical procedure in which their blood is passed through an apparatus that separates out the diseased stem cells, and a process called conditioning is used to help eliminate the damaged cells so they can be replaced by the infusion of the patient’s genetically modified stem cells. Once they become engrafted into the patient’s bone marrow, they reproduce into a lot of daughter cells, and all of those daughter cells contain the CTNS gene. Those cells are able to express the healthy, functional, active protein throughout the body to correct the metabolic problem caused by cystinosis.
“What we’re seeing in the adult patients who have been dosed to date is the consistent and sustained engraftment of our genetically modified cells, 17 to 27 months post-gene therapy, so that’s very encouraging and positive,” says Rihda, the chief medical officer at AVROBIO.
When Janz was 11-years-old, his mother got him enrolled in the trial of a new form of cysteamine that would only need to be taken every 12 hours instead of every six. Two years later, she made sure he was the first person on the list for Cherqui’s current stem cell gene therapy trial.
AVROBIO researchers have also confirmed stabilization or improvement in motor coordination and visual perception in the trial participants, suggesting a potential impact on the neuropathology of the disease. Data from five dosed patients show strong safety and tolerability as well as reduced accumulation of cystine crystals in cells across multiple tissues in the first three patients. None of the five patients need to take oral cysteamine.
Janz’s mother, Barb Kulyk, whom he credits with always making him take his medications and keeping him hydrated, had been following Cherqui’s research since his early childhood. When Janz was 11-years-old, she got him enrolled in the trial of a new form of cysteamine that would only need to be taken every 12 hours instead of every six. When he was 17, the FDA approved that drug. Two years later, his mother made sure he was the first person on the list for Cherqui’s current stem cell gene therapy trial. He received his new stem cells on October 7th, 2019, went home in January 2020, and returned to working full time in February.
Jordan Janz, pictured here with his girlfriend, has a new lease on life, plus a new hair color.
Jordan Janz
He notes that his energy level is significantly better, and his mother has noticed much improvement in him and his daily functioning: He rarely vomits or gets nauseous in the morning, and he has more color in his face as well as his hair. Although he could finish his participation at any time, he recently decided to continue in the clinical trial.
Before the trial, Janz was taking 56 pills daily. He is completely off all of those medications and only takes pills to keep his kidneys working. Because of the damage caused by cystinosis over the course of his life, he’s down to about 20 percent kidney function and will eventually need a transplant.
“Some day, though, thanks to Dr. Cherqui’s team and AVROBIO’s work, when I get a new kidney, cystinosis won’t destroy it,” he concludes.