Artificial intelligence art

Artificial intelligence art is any visual artwork created through the use of an artificial intelligence (AI) program.

Artists began to utilize artificial intelligence in the mid to late 20th century, when the discipline was founded. Throughout its history, artificial intelligence art has raised many philosophical concerns related to the human mind, artificial beings, and what can be considered art in a human–AI collaboration. Since the 20th century, AI art has been exhibited in prominent museums and has won prestigious awards.

The increased availability of AI art tools to the general public in the 2020s AI boom provided opportunities for creating AI outside of academia and professional artists. Commentary about AI art in the 2020s has often focused on issues related to copyright, deception, defamation, and its impact on more traditional artists, including technological unemployment.

History

Early history

The concept of automated art dates back at least to the automata of ancient Greek civilization, where inventors such as Daedalus and Hero of Alexandria were described as having designed machines capable of writing text, generating sounds, and playing music. The tradition of creative automatons has been throughout history, such as Maillardet's automaton, created in the early 1800s.

The academic discipline of artificial intelligence was founded at a research workshop at Dartmouth College in 1956, and has experienced several waves of advancement and optimism in the decades since. Since its founding, researchers in the field have raised philosophical and ethical arguments about the nature of the human mind and the consequences of creating artificial beings with human-like intelligence; these issues have previously been explored by myth, fiction and philosophy since antiquity.

1950s to 2000s: Early implementations

Since the founding of AI in the 1950s, artists and researchers have used artificial intelligence to create artistic works. These works were sometimes referred to as algorithmic art, computer art, digital art, or new media.

One of the first significant AI art systems was AARON, developed by Harold Cohen beginning in the late 1960s at the University of California at San Diego. AARON uses a symbolic rule-based approach to generate technical images in the era of GOFAI programming. Cohen developed AARON with the goal of being able to code the act of drawing. In its primitive form, AARON created simple black-and-white drawings. Cohen would later finish the drawings by painting them. Throughout the years, he also began to develop a way for AARON to also paint. Cohen designed AARON to paint using special brushes and dyes that were chosen by the program itself without mediation from Cohen. AARON was exhibited in 1972 at the Los Angeles County Museum of Art.

In both 1991 and 1992, Karl Sims won the Golden Nica award at Prix Ars Electronica for his 3D AI animated videos using artificial evolution. In 2001, Scott Draves won the Fundacion Telefonica Life 4.0 prize for Electric Sheep, which used AI to create an infinite animation by learning from its audience. In 2009, Eric Millikin won the Pulitzer Prize along with several other awards for his artificial intelligence art that was critical of government corruption in Detroit and resulted in the city's mayor being sent to jail.

2010s: Deep learning

In 2014, Ian Goodfellow and colleagues at Université de Montréal developed the generative adversarial network (GAN), a type of deep neural network capable of learning to mimic the statistical distribution of input data such as images. The GAN uses a "generator" to create new images and a "discriminator" to decide which created images are considered successful. Unlike previous algorithmic art which followed hand-coded rules, generative adversarial networks could learn a specific aesthetic by analyzing a dataset of example images.

In 2015, a team at Google released DeepDream, a program that uses a convolutional neural network to find and enhance patterns in images via algorithmic pareidolia. The process creates deliberately over-processed images with a dream-like appearance reminiscent of a psychedelic experience.

In 2018, an auction sale of artificial intelligence art was held at Christie's Auction House in New York where the AI artwork Edmond de Belamy (a pun on Goodfellow's name) sold for $432,500, which was almost 45 times higher than its estimate of $7,000–10,000. The artwork was created by "Obvious", a Paris-based collective. The website Artbreeder, launched in 2018, uses the models StyleGAN and BigGAN to allow users to generate and modify images such as faces, landscapes, and paintings.

In 2019, Stephanie Dinkins won the Creative Capital award for her creation of an evolving artificial intelligence based on the "interests and culture(s) of people of color." Also in 2019, Sougwen Chung won the Lumen Prize for her performances with a robotic arm that uses AI to attempt to draw in a manner similar to Chung.

2020s: Imggen

In the 2020s, text-to-image models, which generate images based on prompts, became a trend.

In 2021, using the generative pre-trained transformer models used in GPT-2 and GPT-3, OpenAI released a series of images created with the text-to-image AI model DALL-E. Later in 2021, EleutherAI released the open source VQGAN-CLIP based on OpenAI's CLIP model.

In 2022, Midjourney was released, followed by Google Brain's Imagen and Parti, which were announced in May 2022, Microsoft's NUWA-Infinity, and the source-available Stable Diffusion, which was released in August 2022. DALL-E 2, a successor to DALL-E, was beta-tested and released. Stability AI has a Stable Diffusion web interface called DreamStudio, plugins for Krita, Photoshop, Blender, and GIMP, and the Automatic1111 web-based open source user interface. Stable Diffusion's main pre-trained model is shared on the Hugging Face Hub.

Tools and processes

Imagery

There are many additional tools available to the artist when working with diffusion models. Not only are they able to define both positive and negative prompts, but they are also afforded a choice in using (or omitting the use of) VAEs, LorAs, hypernetworks, ipadapter, and embeddings/textual inversions. Further, variables including CFG, seed, steps, sampler, scheduler, denoise, upscaler, and encoder are available for adjustment. Additional influence can be exerted during pre inference by means of noise manipulation while traditional post-processing techniques are frequently used post inference.

The open source philosophy encompassing the ecosystem available to most of the general public allows the artist to pick and choose from a functionally infinite number of options. With the proper hardware, they can also train their own.

In addition, procedural "rule-based" generation of images using mathematical patterns, algorithms that simulate brush strokes and other painted effects, and deep learning algorithms such as generative adversarial networks (GANs) and transformers have been developed. Several companies have released apps and websites that allow one to forego all mentioned options entirely while solely focusing on the positive prompt. There also exists programs which transform photos into art-like images in the style of well-known sets of paintings.

There are many options ranging from simple consumer-facing mobile apps to Jupyter notebooks and webuis that require powerful GPUs to run effectively. Additional functionalities include "textual inversion" which refers to enabling the use of user-provided concepts (like an object or a style) learned from a few images. With textual inversion, novel personalized art can be generated from the associated word(s) (the text that has been assigned to the learned, often abstract, concept) and model extensions or fine-tuning (such as DreamBooth).

Impact and applications

AI has the potential for a societal transformation, which may include enabling the expansion of noncommercial niche genres (such as cyberpunk derivatives like solarpunk) by amateurs, novel entertainment, fast prototyping, increasing art-making accessibility, and artistic output per effort and/or expenses and/or time—e.g., via generating drafts, draft-refinitions, and image components (inpainting). Generated images are sometimes used as sketches, low-cost experimentations, inspiration, or illustrations of proof-of-concept-stage ideas. Additional functionalities or improvements may also relate to post-generation manual editing (i.e., polishing), such as subsequent tweaking with an image editor.

Prompt engineering and sharing

Prompts for some text-to-image models can also include images and keywords and configurable parameters, such as artistic style, which is often used via keyphrases like "in the style of [name of an artist]" in the prompt and/or selection of a broad aesthetic/art style. There are platforms for sharing, trading, searching, forking/refining, and/or collaborating on prompts for generating specific imagery from image generators. Prompts are often shared along with images on image-sharing websites such as Reddit and AI art-dedicated websites. A prompt is not the complete input needed for the generation of an image; additional inputs that determine the generated image include the output resolution, random seed, and random sampling parameters.

Related terminology

Synthetic media, which includes AI art, was described in 2022 as a major technology-driven trend that will affect business in the coming years. Synthography is a proposed term for the practice of generating images that are similar to photographs using AI.

Criticism and controversies

Copyright

Legal scholars, artists, and media corporations have considered the legal and ethical implications of artificial intelligence art since the 20th century.

In 1985, intellectual property law professor Pamela Samuelson argued that US copyright should allocate algorithmically generated artworks to the user of the computer program. A 2019 Florida Law Review article presented three perspectives on the issue. In the first, artificial intelligence itself would become the copyright owner; to do this, Section 101 of the US Copyright Act would need to be amended to define "author" as a natural person or a computer. In the second, following Samuelson's argument, the user, programmer, or artificial intelligence company would be the copyright owner. This would be an expansion of the "work for hire" doctrine, under which ownership of a copyright is transferred to the "employer." In the third situation, copyright assignments would never take place, and such works would be in the public domain, as copyright assignments require an act of authorship.

AI-produced images are causing artists to be concerned about AI art potentially devaluing traditionally-made art. There is also the question of whether or not the gathered data can be used to produce a work. In 2020, Galanter introduces the question of determining how to give credit to the artist based on the differences between the artist and their influences.

In 2022, coinciding with the rising availability of consumer-grade AI image generation services, popular discussion renewed over the legality and ethics of AI-generated art. A particular issue is the inclusion of copyrighted artwork and images in AI training datasets, with artists objecting to commercial AI products using their works without consent, credit, or financial compensation. In September 2022, Reema Selhi, of the Design and Artists Copyright Society, stated that "there are no safeguards for artists to be able to identify works in databases that are being used and opt out." Some have claimed that images generated with these models can bear resemblance to extant artwork, sometimes including the remains of the original artist's signature. In December 2022, users of the portfolio platform ArtStation staged an online protest against nonconsensual use of their artwork within datasets; this resulted in opt-out services, such as "Have I Been Trained?" increasing in profile, as well as some online art platforms promising to offer their own opt-out options. According to the US Copyright Office, artificial intelligence programs are unable to hold copyright, a decision upheld at the Federal District level as of August 2023 followed the reasoning from the monkey selfie copyright dispute.

In January 2023, three artists—Sarah Andersen, Kelly McKernan, and Karla Ortiz—filed a copyright infringement lawsuit against Stability AI, Midjourney, and DeviantArt, claiming that it is legally required to obtain the consent of artists before training neural nets on their work and that these companies infringed on the rights of millions of artists by doing so on five billion images scraped from the web. In July 2023, U.S. District Judge William Orrick was inclined to dismiss most of the lawsuits filed by Andersen, McKernan, and Ortiz, but allowed them to file a new complaint. Also in 2023, Stability AI was sued by Getty Images for using its images in the training data.

In March 2024, Tennessee enacted the ELVIS Act, which prohibits the use of AI to mimic a musician's voice without permission. A month later in that year, Adam Schiff introduced the Generative AI Copyright Disclosure Act which, if passed, would require that AI companies to submit copyrighted works in their datasets to the Register of Copyrights before releasing new generative AI systems.

Income and employment stability

As generative AI image software such as Stable Diffusion and DALL-E continue to advance, the potential problems and concerns that these systems pose for creativity and artistry have risen. In 2022, artists working in various media raised concerns about the impact that generative artificial intelligence could have on their ability to earn money, particularly if AI-based images started replacing artists working in the illustration and design industries. In August 2022, digital artist R. J. Palmer stated that "I could easily envision a scenario where using AI, a single artist or art director could take the place of 5-10 entry level artists... I have seen a lot of self-published authors and such say how great it will be that they don’t have to hire an artist." Scholars Jiang et al. state that “Leaders of companies like Open AI and Stability AI have openly stated that they expect generative AI systems to replace creatives imminently,” and add that “This labor displacement is evident across creative industries. For instance, according to an article on Rest of World, a Chinese gaming industry recruiter has noticed a 70% drop in illustrator jobs, in part due to the widespread use of image generators; another studio in China is reported to have laid off a third of its character design illustrators.”

AI-based images have become more commonplace in art markets and search engines because AI-based text-to-image systems are trained from pre-existing artistic images, sometimes without the original artist's consent, allowing the software to mimic specific artists' styles. For example, Polish digital artist Greg Rutkowski has stated that it is more difficult to search for his work online because many of the images in the results are AI-generated specifically to mimic his style. Furthermore, some training databases on which AI systems are based are not accessible to the public, which makes it impossible to know the extent to which their training data contains copyright-protected images. For example, a tool built by Simon Willison allowed people to search 0.5% of the training data for Stable Diffusion V1.1, i.e., 12 million of the 2.3 billion instances from LAION 2B. Artist Karen Hallion discovered that her copyrighted images were used as training data without their consent.

The ability of AI-based art software to mimic or forge artistic style also raises concerns of malice or greed. Works of AI-generated art, such as Théâtre d'Opéra Spatial, a text-to-image AI illustration that won the grand prize in the August 2022 digital art competition at the Colorado State Fair, have begun to overwhelm art contests and other submission forums meant for small artists. The Netflix short film The Dog & The Boy, released in January 2023, received backlash online for its use of artificial intelligence art to create the film's background artwork.

Deception

As with other types of photo manipulation since the early 19th century, some people in the early 21st century have been concerned that AI could be used to create content that is misleading and can be made to damage a person's reputation, such as deepfakes. Artist Sarah Andersen had her art copied and then used to depict Neo-Nazi ideology. She stated that the spread of hate speech online can be worsened by the use of image generators. Jiang et al. add to this sentiment by stating that "tools trained on artists' works and which allow users to mimic their style without their consent or compensation, can cause significant reputational damage [by] spreading messages that they do not endorse." Some also generate images (or videos) for the purpose of catfishing.

AI systems have the ability to create deepfake content, which is often viewed as harmful and offensive. The creation of deepfakes poses a risk to individuals who have not consented to it. This mainly refers to revenge porn, where sexually explicit material is disseminated to humiliate or harm another person. AI-generated child pornography has been deemed a potential danger to society due to its unlawful nature.

After winning the 2023 "Creative" "Open competition" Sony World Photography Awards, Boris Eldagsen stated that his entry was actually created by artificial intelligence. Photographer Feroz Khan commented to the BBC that Eldagsen had "clearly shown that even experienced photographers and art experts can be fooled". Smaller contests have been affected as well; in 2023, a contest run by author Mark Lawrence as Self-Published Fantasy Blog-Off was cancelled after the winning entry was allegedly exposed to be a collage of images generated with Midjourney.

In May 2023, on social media sites such as Reddit and Twitter, attention was given to a Midjourney-generated image of Pope Francis wearing a white puffer coat. Additionally, an AI-generated image of an attack on the Pentagon went viral as part of a hoax news story on Twitter.

In the days before March 2023 indictment of Donald Trump as part of the Stormy Daniels–Donald Trump scandal, several AI-generated images allegedly depicting Trump's arrest went viral online. On March 20th, British journalist Eliot Higgins generated various images of Donald Trump being arrested or imprisoned using Midjourney v5 and posted them on Twitter; two images of Trump struggling against arresting officers went viral under the mistaken impression that they were genuine, accruing more than 5 million views in three days. According to Higgins, the images were not meant to mislead, but he was banned from using Midjourney services as a result. As of April 2024, the tweet had garnered more than 6.8 million views.

In February 2024, the paper Cellular functions of spermatogonial stem cells in relation to JAK/STAT signaling pathway was published using AI-generated images. It was later retracted from Frontiers in Cell and Developmental Biology because the paper "does not meet the standards".

Bias

Another major concern raised about AI-generated images and art is sampling bias within model training data leading towards discriminatory output from AI art models. In 2023, University of Washington researchers found evidence of racial bias within the Stable Diffusion model, with images of a "person" corresponding most frequently with images of males from Europe or North America.

In 2024, Google's chatbot Gemini's AI image generator was criticized for perceived racial bias, with claims that Gemini deliberately underrepresented white people in its results. Users reported that it generated images of white historical figures like the Founding Fathers, Nazi soldiers, and Vikings as other races, and that it refused to process prompts such as "happy white people" and "ideal nuclear family." Google later apologized for "missing the mark" and took Gemini's image generator offline for updates.

Power usage

Researchers from Hugging Face and Carnegie Mellon University reported in a 2023 paper that generating one thousand 1024×1024 images using Stable Diffusion's XL 1.0 base model requires 11.49 kWh of energy and generates 1,594 grams (56.2 oz) of carbon dioxide, which is roughly equivalent to driving an average gas-powered car a distance of 4.1 miles.

Analysis of existing art using AI

In addition to the creation of original art, research methods that use AI have been generated to quantitatively analyze digital art collections. This has been made possible due to the large-scale digitization of artwork in the past few decades. According to CETINIC and SHE (2022), using artificial intelligence to analyse already-existing art collections can provide fresh perspectives on the development of artistic styles and the identification of artistic influences. AI-assisted study of existing art can also aid in the organization of art exhibitions and support the decision-making process for curators and art historians.

Two computational methods, close reading and distant viewing, are the typical approaches used to analyze digitized art. Close reading focuses on specific visual aspects of one piece. Some tasks performed by machines in close reading methods include computational artist authentication and analysis of brushstrokes or texture properties. In contrast, through distant viewing methods, the similarity across an entire collection for a specific feature can be statistically visualized. Common tasks relating to this method include automatic classification, object detection, multimodal tasks, knowledge discovery in art history, and computational aesthetics. Synthetic images can also be used to train AI algorithms for art authentication and to detect forgeries.

Researchers have also introduced models that predict emotional responses to art such as ArtEmis, a large-scale dataset with machine learning models that contain emotional reactions to visual art as well as predictions of emotion from images or text.

Other forms of art

Some prototype cooking robots can dynamically taste.

There is also AI-assisted writing beyond copy editing (such as helping with writer's block, inspiration, or rewriting segments).

Generative AI has been used in video game production beyond imagery, especially for level design (e.g., for custom maps) and creating new content (e.g., quests or dialogue) or interactive stories in video games.

Some AI can also generate videos, either from text, an image, or a video. This is known as a text-to-video model. Examples of this are Runway's Gen-2, OpenAI's Sora, and Google's VideoPoet.

See also

References