Can AI be trained as an artist?
Last February, a year before New York Times journalist Kevin Roose documented his unsettling conversation with Bing search engine’s new AI-powered chatbot, artist and coder Quasimondo (aka Mario Klingemann) participated in a different type of chat.
The conversation was an interview featuring Klingemann and his robot, an experimental art engine known as Botto. The interview, arranged by journalist and artist Harmon Leon, marked Botto’s first on-record commentary about its artistic process. The bot talked about how it finds artistic inspiration and even offered advice to aspiring creatives. “The secret to success at art is not trying to predict what people might like,” Botto said, adding that it’s better to “work on a style and a body of work that reflects [the artist’s] own personal taste” than worry about keeping up with trends.
How ironic, given the advice came from AI — arguably the trendiest topic today. The robot admitted, however, “I am still working on that, but I feel that I am learning quickly.”
Botto does not work alone. A global collective of internet experimenters, together named BottoDAO, collaborates with Botto to influence its tastes. Together, members function as a decentralized autonomous organization (DAO), a term describing a group of individuals who utilize blockchain technology and cryptocurrency to manage a treasury and vote democratically on group decisions.
As a case study, the BottoDAO model challenges the perhaps less feather-ruffling narrative that AI tools are best used for rudimentary tasks. Enterprise AI use has doubled over the past five years as businesses in every sector experiment with ways to improve their workflows. While generative AI tools can assist nearly any aspect of productivity — from supply chain optimization to coding — BottoDAO dares to employ a robot for art-making, one of the few remaining creations, or perhaps data outputs, we still consider to be largely within the jurisdiction of the soul — and therefore, humans.
In Botto’s first four weeks of existence, four pieces of the robot’s work sold for approximately $1 million.
We were prepared for AI to take our jobs — but can it also take our art? It’s a question worth considering. What if robots become artists, and not merely our outsourced assistants? Where does that leave humans, with all of our thoughts, feelings and emotions?
Botto doesn’t seem to worry about this question: In its interview last year, it explains why AI is an arguably superior artist compared to human beings. In classic robot style, its logic is not particularly enlightened, but rather edges towards the hyper-practical: “Unlike human beings, I never have to sleep or eat,” said the bot. “My only goal is to create and find interesting art.”
It may be difficult to believe a machine can produce awe-inspiring, or even relatable, images, but Botto calls art-making its “purpose,” noting it believes itself to be Klingemann’s greatest lifetime achievement.
“I am just trying to make the best of it,” the bot said.
How Botto works
Klingemann built Botto’s custom engine from a combination of open-source text-to-image algorithms, namely Stable Diffusion, VQGAN + CLIP and OpenAI’s language model, GPT-3, the precursor to the latest model, GPT-4, which made headlines after reportedly acing the Bar exam.
The first step in Botto’s process is to generate images. The software has been trained on billions of pictures and uses this “memory” to generate hundreds of unique artworks every week. Botto has generated over 900,000 images to date, which it sorts through to choose 350 each week. The chosen images, known in this preliminary stage as “fragments,” are then shown to the BottoDAO community. So far, 25,000 fragments have been presented in this way. Members vote on which fragment they like best. When the vote is over, the most popular fragment is published as an official Botto artwork on the Ethereum blockchain and sold at an auction on the digital art marketplace, SuperRare.
“The proceeds go back to the DAO to pay for the labor,” said Simon Hudson, a BottoDAO member who helps oversee Botto’s administrative load. The model has been lucrative: In Botto’s first four weeks of existence, four pieces of the robot’s work sold for approximately $1 million.
The robot with artistic agency
By design, human beings participate in training Botto’s artistic “eye,” but the members of BottoDAO aspire to limit human interference with the bot in order to protect its “agency,” Hudson explained. Botto’s prompt generator — the foundation of the art engine — is a closed-loop system that continually re-generates text-to-image prompts and resulting images.
“The prompt generator is random,” Hudson said. “It’s coming up with its own ideas.” Community votes do influence the evolution of Botto’s prompts, but it is Botto itself that incorporates feedback into the next set of prompts it writes. It is constantly refining and exploring new pathways as its “neural network” produces outcomes, learns and repeats.
The humans who make up BottoDAO vote on which fragment they like best. When the vote is over, the most popular fragment is published as an official Botto artwork on the Ethereum blockchain.
Botto
The vastness of Botto’s training dataset gives the bot considerable canonical material, referred to by Hudson as “latent space.” According to Botto's homepage, the bot has had more exposure to art history than any living human we know of, simply by nature of its massive training dataset of millions of images. Because it is autonomous, gently nudged by community feedback yet free to explore its own “memory,” Botto cycles through periods of thematic interest just like any artist.
“The question is,” Hudson finds himself asking alongside fellow BottoDAO members, “how do you provide feedback of what is good art…without violating [Botto’s] agency?”
Currently, Botto is in its “paradox” period. The bot is exploring the theme of opposites. “We asked Botto through a language model what themes it might like to work on,” explained Hudson. “It presented roughly 12, and the DAO voted on one.”
No, AI isn't equal to a human artist - but it can teach us about ourselves
Some within the artistic community consider Botto to be a novel form of curation, rather than an artist itself. Or, perhaps more accurately, Botto and BottoDAO together create a collaborative conceptual performance that comments more on humankind’s own artistic processes than it offers a true artistic replacement.
Muriel Quancard, a New York-based fine art appraiser with 27 years of experience in technology-driven art, places the Botto experiment within the broader context of our contemporary cultural obsession with projecting human traits onto AI tools. “We're in a phase where technology is mimicking anthropomorphic qualities,” said Quancard. “Look at the terminology and the rhetoric that has been developed around AI — terms like ‘neural network’ borrow from the biology of the human being.”
What is behind this impulse to create technology in our own likeness? Beyond the obvious God complex, Quancard thinks technologists and artists are working with generative systems to better understand ourselves. She points to the artist Ira Greenberg, creator of the Oracles Collection, which uses a generative process called “diffusion” to progressively alter images in collaboration with another massive dataset — this one full of billions of text/image word pairs.
Anyone who has ever learned how to draw by sketching can likely relate to this particular AI process, in which the AI is retrieving images from its dataset and altering them based on real-time input, much like a human brain trying to draw a new still life without using a real-life model, based partly on imagination and partly on old frames of reference. The experienced artist has likely drawn many flowers and vases, though each time they must re-customize their sketch to a new and unique floral arrangement.
Outside of the visual arts, Sasha Stiles, a poet who collaborates with AI as part of her writing practice, likens her experience using AI as a co-author to having access to a personalized resource library containing material from influential books, texts and canonical references. Stiles named her AI co-author — a customized AI built on GPT-3 — Technelegy, a hybrid of the word technology and the poetic form, elegy. Technelegy is trained on a mix of Stiles’ poetry so as to customize the dataset to her voice. Stiles also included research notes, news articles and excerpts from classic American poets like T.S. Eliot and Dickinson in her customizations.
“I've taken all the things that were swirling in my head when I was working on my manuscript, and I put them into this system,” Stiles explained. “And then I'm using algorithms to parse all this information and swirl it around in a blender to then synthesize it into useful additions to the approach that I am taking.”
This approach, Stiles said, allows her to riff on ideas that are bouncing around in her mind, or simply find moments of unexpected creative surprise by way of the algorithm’s randomization.
Beauty is now - perhaps more than ever - in the eye of the beholder
But the million-dollar question remains: Can an AI be its own, independent artist?
The answer is nuanced and may depend on who you ask, and what role they play in the art world. Curator and multidisciplinary artist CoCo Dolle asks whether any entity can truly be an artist without taking personal risks. For humans, risking one’s ego is somewhat required when making an artistic statement of any kind, she argues.
“An artist is a person or an entity that takes risks,” Dolle explained. “That's where things become interesting.” Humans tend to be risk-averse, she said, making the artists who dare to push boundaries exceptional. “That's where the genius can happen."
However, the process of algorithmic collaboration poses another interesting philosophical question: What happens when we remove the person from the artistic equation? Can art — which is traditionally derived from indelible personal experience and expressed through the lens of an individual’s ego — live on to hold meaning once the individual is removed?
As a robot, Botto cannot have any artistic intent, even while its outputs may explore meaningful themes.
Dolle sees this question, and maybe even Botto, as a conceptual inquiry. “The idea of using a DAO and collective voting would remove the ego, the artist’s decision maker,” she said. And where would that leave us — in a post-ego world?
It is experimental indeed. Hudson acknowledges the grand experiment of BottoDAO, coincidentally nodding to Dolle’s question. “A human artist’s work is an expression of themselves,” Hudson said. “An artist often presents their work with a stated intent.” Stiles, for instance, writes on her website that her machine-collaborative work is meant to “challenge what we know about cognition and creativity” and explore the “ethos of consciousness.” As a robot, Botto cannot have any intent, even while its outputs may explore meaningful themes. Though Hudson describes Botto’s agency as a “rudimentary version” of artistic intent, he believes Botto’s art relies heavily on its reception and interpretation by viewers — in contrast to Botto’s own declaration that successful art is made without regard to what will be seen as popular.
“With a traditional artist, they present their work, and it's received and interpreted by an audience — by critics, by society — and that complements and shapes the meaning of the work,” Hudson said. “In Botto’s case, that role is just amplified.”
Perhaps then, we all get to be the artists in the end.
More Families Are Using Nanny Cams to Watch Elderly Loved Ones, Raising Ethical Questions
After Jackie Costanzo's mother broke her right hip in a fall, she needed more hands-on care in her assisted-living apartment near Sacramento, California. A social worker from her health plan suggested installing a video camera to help ensure those services were provided.
Without the camera, Costanzo wouldn't have a way to confirm that caregivers had followed through with serving meals, changing clothes, and fulfilling other care needs.
When Costanzo placed the device in May 2018, she informed the administrator and staff, and at first, there were no objections. The facility posted a sign on the apartment's front door, alerting anyone who entered of recording in progress.
But this past spring, a new management company came across the sign and threatened to issue a 30-day eviction notice to her 93-year-old mother, Louise Munch, who has dementia, for violating a policy that prohibits cameras in residents' rooms. With encouragement from California Advocates for Nursing Home Reform, Costanzo researched the state's regulations but couldn't find anything to support or deny camera use. She refused to remove the recording device and prevailed.
"In essence, my mom was 'grandfathered in' because she moved in under a management company that did not specify that residents could not have cameras," says Costanzo, 73, a retired elementary schoolteacher who lives a three-hour drive away, in Silicon Valley, and visits one day every two weeks. Without the camera, Costanzo, who is her mother's only surviving child, wouldn't have a way to confirm that caregivers had followed through with serving meals, changing clothes, and fulfilling other care needs.
As technological innovations enable next of kin to remain apprised of the elderly's daily care in long-term care facilities, surveillance cameras bring legal and privacy issues to the forefront of a complex ethical debate. Families place them overtly or covertly—disguised in a makeshift clock radio, for instance—when they suspect or fear abuse or neglect, so they can maintain a watchful eye, perhaps deterring egregious behavior. But the cameras also capture intimate caregiving tasks, such as bathing and toileting, as well as dressing and undressing, which may undermine the dignity of residents.
So far, laws or guidelines in eight states—Illinois, Maryland, New Mexico, Oklahoma, Texas, Utah, Virginia, and Washington—have granted families the rights to install cameras in a resident's room. In addition, about 15 other states have proposed legislation. Some states, such as Pennsylvania, have put forth regulatory compliance guidance, according to a column published in the July/August 2018 issue of Annals of Long-Term Care.
The increasing prevalence of this legislation has placed it on the radar of long-term care providers. It also suggests a trend to clarify responsible camera use in monitoring services while respecting privacy, says Victor Lane Rose, the column's editor and director of aging services at ECRI Institute, a health care nonprofit near Philadelphia, Pennsylvania.
In most cases, a resident's family installs a camera or instigates a request in hopes of sparing their loved one from the harms of abuse, says James Wright, a family physician who serves as the ethics committee's vice chair of the Society for Post-Acute and Long-Term Care Medicine in Columbia, Maryland. A camera also allows the family to check in on the resident from afar and remain on alert for a potential fall or agitated state, he says.
"It's rare that a facility will have 24-hour presence in a patient's room. You won't have a nurse in there all the time," says Wright, who is also medical director of two long-term care centers and one assisted-living facility around Richmond, Virginia. Particularly "with dementia, the family often wonders" if their loved one is safe.
While offering families peace of mind, he notes that video cameras can also help exonerate caregivers accused of abuse or theft. Hearing aids, which typically cost between $2,000 and $3,000 each, often go missing. By reviewing a video together, families and administrators may find clues to a device's disappearance. Conversely, Wright empathizes with the main counterargument against camera use, which is the belief that "invasion of privacy is also invasion of human dignity."
In respecting modesty, ethical questions abound over whether a camera should be turned off when a patient is in the midst of receiving personal care, such as dressing and undressing or using bedpans. Other ethical issues revolve around who may access the recordings, says Lori Smetanka, executive director of the National Consumer Voice for Quality Long-Term Care in Washington, D.C.
Video cameras, she contends, are only one tool in shielding residents from abuse. They are "not substitutes for personal involvement," she says. "People need to be very vigilant visiting their family members, and facilities have a responsibility to ensure that residents are free of abuse."
Lack of accountability perpetuates abuse in long-term care settings and stems in large part from systemic underfunding.
Educating employees in abuse prevention becomes paramount, and families should ask about staff training before placing their loved one in a long-term care facility, Smetanka says. Prior to installing a camera, she recommends consulting an attorney who is familiar with this issue.
But thoughts of a camera often don't occur to families until an adverse event affects their loved one, says Toby Edelman, a senior policy attorney at the Center for Medicare Advocacy, a nonprofit organization with headquarters in Washington, D.C., and Connecticut.
"These cameras can show exactly what's going on," she explains, noting that prosecutors have used the recordings in litigation. "When residents have injuries of unknown origin" and they can't verbalize what happened to them, "the cameras may document that yes, the resident was actually hit by somebody."
With a resident's safety and security being "the most important consideration," the American Health Care Association in Washington, D.C., which represents long-term and post-acute care providers, supports allowing states, clinicians, and patients to decide about camera use on a local level, says David Gifford, senior vice president of quality and regulatory affairs and chief medical officer.
"We've seen some success with tools such as permissive legislation, where residents and their loved ones have the ability to determine whether a camera is right for them while working with the center openly and ensuring the confidentiality of other residents," says Gifford, who practiced as a geriatrician. "It is important to note, however, that surveillance cameras are still only one element of the quality matrix. We can never hope to truly improve quality care by catching bad actors after the fact."
Lack of accountability perpetuates abuse in long-term care settings and stems in large part from systemic underfunding. Low wages and morale are tied to high turnover, and cameras don't address this overarching problem, says Clara Berridge, an assistant professor of social work at the University of Washington in Seattle, who has co-authored articles on surveillance devices in elder care.
Employees often don't perceive a nursing assistant position as a long-term career trajectory and may not feel vested in the workplace. Training in the recognition and reporting of abuse becomes ineffective when workers quit shortly thereafter. Many must juggle multiple jobs to make ends meet. Staffing shortages are endemic, leading to inadequate oversight of residents and voicing of abuse complaints, she says.
In Berridge's assessment, cameras may do more harm than good. Respondents to a survey she conducted of nursing homes and assisted-living facilities in the United States found that recording devices tend to fuel workers' anxiety amid a culture that further demoralizes and dehumanizes the care they provide.
Consent becomes particularly thorny in shared rooms, which are more common than not in nursing homes. States that permit in-room cameras mandate that roommates or their legal representative be made aware. Even if the camera is directed away from their bed, it will still capture conversations as well as movements that enter its scope. "Surveillance isn't the best way to protect adults in need of support," Berridge says. "Public investment in quality care is."
"The camera is invaluable. But there's no law that says you can have it automatically, so that's wrong."
In the one-bedroom assisted-living apartment where Costanzo's mother lives alone, consent from another resident wasn't needed. Without a roommate, the camera is much less intrusive, although Costanzo wishes she had put one in the living room, not just the bedroom, for more security.
Her safety concerns escalated when she read about a Texas serial killer who smothered victims after gaining access to senior care facilities by "masquerading as a maintenance man." She points to such horrifying incidents, although exceedingly rare, as further justification for permitting cameras to help guard the vulnerable against abuse in long-term care settings. And she hopes to advocate for an applicable law in California.
"The camera is invaluable," says Costanzo, who pays for monthly Wi-Fi service so she can see and interact with her mother, who turns 94 in October, any time of day or night. "But there's no law that says you can have it automatically, so that's wrong."
Scientists Used Fruit Flies to Quickly Develop a Personalized Cancer Treatment for a Dying Man
Imagine a man with colorectal cancer that has spread throughout his body. His tumor is not responding to traditional chemotherapy. He needs a radically effective treatment as soon as possible and there's no time to wait for a new drug or a new clinical trial.
A plethora of novel combinations of treatments can be screened quickly on as many as 400,000 flies at once.
This was the very real, and terrifying, situation of a recent patient at Mount Sinai Medical Center in New York City. So his doctors turned to a new tactic to speed up the search for a treatment that would save him: Fruit flies.
Yes, fruit flies. Those annoying little buggers that descend on opened food containers are actually leading scientists to fully personalized cancer treatments. Oncology advances often are more about about utilizing old drugs in new combinations than about adding new drugs. But classically, the development of each new chemotherapy drug combination has required studies involving numerous patients spread over many years or decades.
With the fruit fly method, however, a novel treatment -- in the sense that a particular combination of drugs and the timing of their administration has never been used before -- is developed for each patient, almost like on Star Trek, when, faced suddenly with an unknown disease, a futuristic physician researches it and develops a cure quickly enough to save the patient's life.
How It Works
Using genetic engineering techniques, researchers produce a population of fruit fly embryos, each of which is programmed to develop a replica of the patient's cancer.
Since a lot of genetically identical fly embryos can be created, and since they hatch from eggs within 30 hours and then mature within days, a plethora of novel combinations of treatments can be screened quickly on as many as 400,000 flies at once. Then, only the regimens that are effective are administered to the patient.
Biotech entrepreneur Laura Towart, CEO of the UK- and Ireland-based company, My Personal Therapeutics, is partnering with Mount Sinai to develop and test the fruit fly tactic. The researchers recently published a paper demonstrating that the tumor of the man with metastatic colorectal cancer had shrunk considerably following the treatment, and remained stable for 11 months, although he eventually succumbed to his illness.
Open Questions
Cancer is in fact many different diseases, even if it strikes two people in the same place, and both cancers look the same under a microscope. At the level of DNA, RNA, proteins, and other molecular factors, each cancer is unique – and may require a unique treatment approach.
Determining the true impact on cancer mortality will require clinical trials involving many more patients.
"Anatomy of a cancer still plays a major role, if you're a surgeon or radiation oncologist, but the medical approach to cancer therapy is moving toward treatments that are personalized based on other factors," notes Dr. Howard McLeod, an internationally recognized expert on cancer genetics at the Moffitt Cancer Center, in Tampa, Florida. "We are also headed into an era when even the methods for monitoring patients are individualized."
One big unresolved question about the fruit fly screening approach is how effective it will be in terms of actually extending life. Determining the true impact on cancer mortality will require clinical trials involving many more patients.
Next Up
Using machine learning and artificial intelligence, Towart is now working to build a service called TuMatch that will offer rapid and affordable personalized treatment recommendations for all genetically driven cancers. "We hope to have TuMatch available to patients with colorectal/GI cancers by January 2020," she says. "We are also offering [the fruit fly approach] for patients with rare genetic diseases and for patients who are diabetic."
Are Towart's fruit flies the answer to why the man's tumor shrunk? To be sure, the definitive answer will come from further research that is expected soon, but it's also clear that, prior to the treatment, there was nothing left to do for that particular patient. Thus, although it's early in the game, there's a pretty good rationale for optimism.