Thank you to Art In America magazine and Jason Bailey for featuring my drawing on their January issue, focusing on Generative Art. This represents a milestone in the industry of Art and Technology, and its an truly an honour to be featured here.
I made this drawing in 2017, during a residency in Tokyo. The project was an experiment; entirely self directed, simply to explore a curiosity. At the time, I was just getting started working with robots after many years of making digital and installation art through code and drawing.
The process of moving beyond what was on-screen was liberating; bringing code into a medium like robotics was a way I could return the physicality of art making back into the work.
It felt like a coming home — my years of drawing, which always felt obscured by software tools, felt like it could be playful, adaptive, and imperfect, in a way that could help me grow as an artist.
I’ve learned a lot about technology through making art with machines — which began over 5 years ago. I learned that the human hand is always present, and “the AI” is always a system that reflects some known and unknown aspects of its designer.
There’s a lot of hype surrounding this idea of “AI Art” at the moment. I think it’s largely driven by a subconscious anxiety of how intertwined in our lives digital technology has become. People appear to really like thinking that the AI is making music, making poetry, making painting. Perhaps this is because if “the AI” can make beautiful art, it might somehow make up for the fact that “the AI” is also taking our jobs, making biased policy decisions, and shaping our society in ways we don’t really understand. The ‘AI as artist’ is an alluring idea that helps soothe, perhaps, a fear of these technologies, but it’s one that does nothing to empower us in the face of technological change.
As an alternative to this, I like to think of the potential for AI systems as collaborators. Why? Because we’ve all had collaborators, good and bad. Collaboration is about making with; it can be about control and dominance, but it can also be generosity and kindness. Ultimately, these AI systems can be created in the service of working together, to make something better for both human and machine.
For me, that’s what is promising and hopeful about working with these technologies - in fact, I think it might be an imperative. We should be designing these technologies to make us, and the world, better. To achieve this goal, art and technology are a very natural pairing; they both reflect the culture in which they are made, and both have the power to profoundly shape the future we want to see.
Deeply grateful to @Lumen_prize, for awarding the Art and Technology Prize to Drawing Operations, a project I started in 2015 exploring human and machine collaboration. Thank you Carla Rapoport, and Jack Addis for your support and welcoming me into the Lumens family. . . In these deeply uncertain times, I find myself thinking of #DonnaHaraway’s writing in Staying With The Trouble 🖤: “It matters what worlds world worlds. We become together or not at all." Collaboration, for me, is a way to steer away from "a technophobic spiral, by conceiving of humans & machines as interacting parts of complex adaptive systems." . . .
LATE LAST YEAR, artificial intelligence made a loud splash in the art world. A trio of French students, calling themselves Obvious, put a smeared, unfinished portrait up for auction at Christie’s. Titled Portrait of Edmond de Belamy, the murky image pictured a suited gentleman with a plain white collar. Rendered in three-quarter profile against an indistinct background, the image mimicked conventions from Renaissance and Baroque portraiture. At the corner of the canvas, a mathematical equation ostentatiously replaced the traditional artist signature. Global media trumpeted the achievement as a stunning first. Obvious oiled its promotional engine by misleadingly claiming it to be the first ever work of art created by AI to go under the hammer. Neither claim turned out to be true: significant human labor went into its making, and a set of AI-generated images had gone to auction at San Francisco’s Gray Area in 2016. Offered at $10,000, the image went to an anonymous buyer for a jaw-dropping $432,000.
This portrait embodies significant problems at the heart of AI’s recent cultural ubiquity. Machine learning is being used across art, science, and industry — from music production to space exploration — but its excursions into the creative economy have ignited the most controversy. Commonly, such works lead to the question, “Can a machine be creative?” But this question misses the mark.
AI is the latest, most intangible incarnation of the automated arts. When it comes to creative automatons, history shows we’ve been rehearsing the same hand-wringing about authorship and authenticity for over a century. The history of photography, for example, famously raised alarms about humans surrendering the arts to machines. While automatons are now largely digital, not strictly mechanical, they, too, seem to imperil the ostensibly humanist essence of art. But Portrait of Edmond de Belamy did not spring fully formed from the neural network’s “mind.” Rather, it emerged out of a complex system where human actors continue to play a crucial role, although the story of their absence — and abdication of the arts — persists.
Obvious worked with an artificially intelligent system known as a generative adversarial network, or GAN. An algorithm composed in two parts, a GAN reproduces an agonistic relationship between artist and critic. The Generator creates new images based on a massive data set, and the Discriminator evaluates that image against a human-made image. Obvious trained their GAN on a data set of 15,000 portraits from the 14th to the 20th century. Ensconced in a gilded frame, and gorged on the Western canon, the image convincingly reproduced the aesthetic conventions of portraiture. By signing the image with a segment of the algorithm’s code, Obvious had, in a clever sleight of hand, cast an algorithm as a stand-in for the absent hand of the artist.
But the artist in question was not, actually, a hypothetical person. Obvious had appropriated their code, as well as the training set, from Robbie Barrat. Then a 19-year-old artist and programmer, Barratt had shared his Old Masters GAN on the open-source sharing website, Github, in 2014. An outcry from the small community of AI artists punctuated the auction. They not only emphasized Barrat’s erased contribution but also expressed disbelief that the amateurish image was representing their years of creative experimentation with GANs to the world at large. In their rush to crown the algorithm as the work’s author, Obvious obscured the central roles played by humans in the conception, coding, and curation that yielded the image. If this was an aesthetic Turing test, then we have willfully failed it. But why?
The equation at the traditional site of the artist’s signature offers some clues. This staging culminates a centuries-old conversation about the agency of creative automatons. Signatures boast a special connection to the human hand. Historically, they have imprinted the heft of the human body upon an object, whether artwork or legal contract. They index the identity of the signer: they affirm that someone was really there. By displacing a human signature with an algorithm, Obvious manifested an obscured predecessor in the history of automated art — not the camera, but the player piano. Yes, that classic piece of ragtime furniture.
Popular in the early 20th century, player pianos are a mechanical incarnation of the absent human hands in AI-generated art. Anyone who has seen a player piano in action can attest to its uncanny nature. Once in motion, one can almost imagine an invisible player sitting at the instrument. Piano keys seem to play themselves in cheerful ragtime. Now largely a relic of a different time, the player piano stirred many of the same kinds of questions and debates that AI art does now.
Inventor Edwin Votey built a prototype of his automated player piano system in 1895. It took the form of a large wooden cabinet, and was powered by suction generated by two foot pedals. It was designed to stand in front of an existing piano, where mobile fingers at its rear would orchestrate its keys. Astride the piano, this cabinet of curiosities contained a small paper roll patterned with tiny perforations representing notes to be played. As a roll moved over the tracker bar, a reading device with a row of evenly spaced holes, a valve would open and trigger a pneumatic motor. This would then fire a felt-covered finger on the external player, causing it to hit the corresponding piano key. Though these functions soon entered the instrument itself, with the introduction of the Apollo line of pianos by the inventor Melville Clark in 1901, these operational principles remained the standard for nearly all roll-operated player piano systems.
In the next decade, the machine developed into a ghostly rendition of human performance. In 1904, German organ builder Edwin Welte and his school friend Karl Bockisch launched a new kind of player piano, the Welte-Mignon. Otherwise known as a reproducing piano, these were able to record and reproduce individual performances by replicating the dynamics, rubato, and pedaling of living players. In its heyday, music roll manufacturers recorded the performances of many famous 20th-century pianists, George Gershwin, Liberace, Jelly Roll Morton, Myra Hess, and Thomas “Fats” Waller among them. Specters of performances past, these rolls conjure the apparitions of nearly every major early 20th-century pianist.
For all its commercial success, the automated instrument drew its share of criticism. In 1906, John Philip Sousa published “The Menace of Mechanical Music,” a screed against the sweeping popularity of player pianos and gramophones. Sounding the alarm about these poor copies, Sousa fretted that these “talking and playing machines” “reduce the expression of music to a mathematical system of megaphones, wheels, cogs, disks, cylinders, and all manner of revolving things, which are as like real art as the marble statue of Eve is like her beautiful, living, breathing daughters.” For Sousa, these creative automatons substituted cogs and wheels for the ineffable human soul.
Creaky in their physicality, the player piano seems a far cry from the digital machines so ubiquitous today. However, the rolls that powered their spectral movements form a bridge with the artificially intelligent systems that generate such images as Portrait of Edmond de Belamy. After all, with the sleek miniaturization of contemporary computing, it is easy to forget that the early computers of the 1940s and ’50s were massive calculators operated by punch cards with holes in them. In these pockmarked landscapes, the automated arts, from player pianos to algorithm-generated portraiture, took root.
As William Gaddis decried in Agapē Agape, his 2002 swan song on the ruination of art by technology, “There was the beginning of key-sort and punched cards and IBM and NCR and the whole driven world we’ve inherited from some rinky-dink piano roll.” The narrator of Gaddis’s pseudo-autobiographical novel lies dying in his hospital bed. He pores over his notes in a desperate rush to finish his magnum opus, a history of the player piano. Paralleling his deteriorating body with the decay of culture, he rages against a commercialized world where art has become mere entertainment, and imitation has displaced authenticity.
Gaddis is not alone in placing the player piano in the history of computational creativity. The recent TV show Westworld also winks at this past. The show envisions a world where guests pay to play in a live-action simulation of the Wild West, replete with cowboys, gun battles, and damsels in distress. Euphemistically named “hosts,” androids in period boots and hats, are at the mercy of the human “guests” who come to ravish, maim, or kill them. In the saloon where guests down whiskey and manhandle artificial women, a player piano sits. Phantom hands alight on the keys; tinkly covers of contemporary songs serve as a soundtrack for gun battles and seduction. As robotic copies of humans cavort among originals, these mechanized cover songs evoke the longer lineage of artificial others.
Despite the century that has passed since their debut, player pianos, like androids, remain eerie, ghostly objects on the brink of life. In a pantheon of creative automatons, the player piano endures for the existential questions it provoked. How have these automated instruments transformed the landscape of creativity? And at what cost?
New technologies demand new visions of how we might use them. AI technologies have been accused of reproducing the invisible human structures that inhere in the data from which they learn, including the best and worst of us. Take Portrait of Edmond de Belamy. Not only did it regurgitate musty art historical conventions, but its success affirmed the gatekeeping function of the auction house, as well as the conflation of monetary worth with artistic value. Where Obvious’s image replicates some of the least compelling elements of the art world, other AI artists imagine other ways we might relate to these creative machines.
Against a narrative of imperiled human creativity, some artists working with AI instead frame these new technologies within an ecology of human and machine collaboration. In Drawing Operations, the artist Sougwen Chung stages a series of collaborative drawing performances with a robotic arm. In three chapters, “Mimicry,” “Memory,” and “Future Speculations,” Chung showcases an evolving robotic behavior based on her research with art and AI. Ongoing since 2015, the work highlights the complexities of assigning authorship to AI.
In Generation 1 of the project, Chung explored robotic mimicry in real time. By capturing her drawing gestures with an overhead camera and analyzing them with computer vision software, the robotic arm replicated her movements. The robot precisely duplicated the pace, shape, and style of the original performer. In many ways, we might understand this arm as a 21st-century version of Votey’s player piano, the mechanical fingers of his wooden cabinet playing the keys in the absence of a human performer. But Chung turns this dynamic on its head. After all, her body was actually present, the robotic arm a proxy for her in the same moment. Their gestures synchronized as they made marks on paper, Chung and her robotic double formed a strange duet.
In Generation 2 of Drawing Operations, the robotic arm was linked to a neural network trained on a database of Chung’s extracted gestures from previous drawings. As a result, the robotic arm was able to generate new movements and drawings in Chung’s style, without duplicating them exactly as it had prior. In improvised performances, Chung draws in tandem with the arm, creating abstract line drawings that blur her marks with those of her robotic collaborator. Speculating on artificial creativity, Drawing Operations suggests a more complicated entanglement of human and machine actors than prevailing media narratives suggest. Through these performances, Chung systematically imagines a creative future where automatons might extend human intelligence.
As the perforated rolls of the player piano prefigured the punch cards of early computing, so, too, have they shaped how we talk about creative machines. Like the ghostly hands that played upon pianola keys, AI art stokes deep cultural anxieties about the risks automation poses to human activity. Ultimately, we fear that they will replace us, whether at the factory or at the canvas.
While the story of absent human hands lingers from the heyday of the player piano, the stakes have changed. For all the controversy it stirred, the player piano was ultimately only a reproducing musical robot, unable to deviate from its punched script. It was also an industrial invention, symptomatic of rapidly changing modes of production. In a creative economy, where factory jobs are all but gone, creativity is just another commodity. The risk is not that humans will stop being creative but that they will cede the signs — and compensation — of their labor to algorithmic signatures.
By demonstrating AI artworks’ commercial viability, Portrait of Edmond de Belamy sparked a gold rush. Shortly after the notorious auction, computer scientist Ahmed Elgammal debuted the creative output of AICAN, his GAN variant, in a Chelsea gallery. Unlike Edmond de Belamy, these pieces, collected under the title Faceless Portraits Transcending Time, were pitched as a joint effort between man and machine. While many images from the show ostensibly sold for five figures, AICAN’s financial ambitions are far higher. Based on its success identifying the chronology of the images in its data set, Elgammal and his investors believe it could be used to anticipate — and produce — future art trends. Unchecked, these virtual automatons could theoretically cut human artists out of the art market entirely. The fears of Sousa and Gaddis could, in fact, be realized.
But rather than allow this threat to send us into a technophobic spiral, we could instead, like Sougwen Chung, conceive of humans and machines as interacting parts of complex adaptive systems. Whether creative duet, industrial partnership, global economy, or internet, these complex systems have marked all of human history. So imagined — and so designed — intelligent machines can serve to amplify and augment human activity. It is neither us nor them, but both.
Sougwen Chung, Drawing Operations (2017). Courtesy the artist.
WHO: Chung is a Canadian-born, Chinese-raised, New York-based interdisciplinary artist and former research fellow at MIT’s Media Lab. She’s currently E.A.T.’s artist-in-residence in partnership with the New Museum and Bell Labs. Her work, which spans installation, sculpture, drawing, and performance, explores mark-making by both hand and machine in order to better understand the interactions between humans and computers. She has exhibited at institutions including The Drawing Center in New York and the National Art Center in Tokyo.
WHAT: For her current project, Drawing Operations, Chung uses Google’s TensorFlow, an open-source software library used for machine learning, to classify archives of her own drawings. The software then transfers what it has learned about Chung’s style and approach to a robotic arm that draws alongside her. She’s also working on a few new projects using pix2pix (a neural network trained to produce variations on an image, like the nighttime version of a daytime photo) and sketch-rnn (which tries to continue or complete a digital sketch based on where the human leaves off) to expand on this idea of human and machine collaboration.
WHY: “As an artist working with these tools, the promise of AI offers a new way of seeing,” Chung explains. “Seeing as self reflection, seeing through the ground truth of ones own artworks as data. There is a lot of talk about biases evident in AI systems and that is absolutely true within AI systems trained on art. You could describe visual language as a kind of visual bias, a foregrounding of the subjective view of the artist. By translating that into machine behavior, I am attempting to create a shared intersubjectivity between human and machine.”
Sougwen Chung, Omnia Per Omnia Performance. Photo by Irina Abraham.
"In Omnia Per Omnia Sougwen Chung explores collaborating with robots as opposed to using them as a tool. As I enter the room, the artist and the engineer Andy Cavatorta are fussing around little machines with exposed hardware, which seem to roam free around a white platform. Sougwen and Andy look like two caretakers with the little robots being their wards.
When Sougwen first contacted Andy about the project, he jokingly asked if she wanted him to build her an army of painting robots. To his surprise it was exactly what Sougwen who had designed, coded and engineered the prototypes, intended. The robots are using surveillance footage and have the collective movement of the city power the movement of their swarm. Sougwen is painting the portrait of the city together with the robots. When asked what the difference is between collaborating with humans and robots, Andy and Sougwen laugh and say that humans cooperate. Their hope is that the robotic swarm will keep learning and evolving.
On the day of the performance, Sougwen is surrounded with audience and journalists. Once the robots start moving and the music fades in, everyone turns quiet. The slow and seemingly purposeful movement of the robots, the traces of the blue paint they leave behind, the motion of Sougwen's brush and the expression of total concentration on her face create an atmosphere of a ritual, a spiritual action. The audience is affected by the magic happening in front of them. Watching the artist paint with the robotic swarm creates a true emotion, the way only art can. "
This post was originally published by NEW INC on April 20, 2018 and is based on an interview between Lindsay Howard and Sougwen Chung.
Recently NEW INC, the New Museum’s cultural incubator, and Nokia Bell Labs presented their first exhibition entitled “Only Human” at the Mana Contemporary in Jersey City, New Jersey. The exhibition showcases the work of NEW INC artistsSougwen Chung, Lisa Park and Hammerstep (Jason Oremus and Garrett Coleman) participating in the artist-in-residence program at Bell Labs and their collaboration with Bell Labs researchers to produce new artistic projects inspired or enabled by Bell Labs technologies.
“Only Human” is available to visit Tuesdays - Saturdays at 3PM through June 2nd as part of the Mana Contemporary’s gallery tours. No RSVP is necessary.
On Saturday, May 12th there is a day-long symposium at the Mana Contemporary for a deeper dive into some of the ideas, themes, and technological research that are being explored in the works on view. It will also reflect on the legacy of the Experiments in Art & Technology program (1967–2001), founded by artists Robert Rauschenberg and Robert Whitman, and Bell Labs engineers Billy Klüver and Fred Waldhauer. Please RSVP here.
The interview below with Sougwen Chung provides additional insight into her practice and how her work with Bell Labs researchers has enriched her artistic experience during her year long tenure as an artist-in-residence.
Lindsay Howard: What are some themes you’ve been exploring, and some past projects that relate to the research you’ve been doing at Bell Labs?
Sougwen Chung: I’ve been evolving the theme of human-robot collaboration through the past year at Bell Labs. My work has always centered around the marks made by hand and the marks made by machines—and the machines are constantly evolving. In particular, I’ve been exploring the fields of collaborative robotics, computer vision, and biometrics during my residency. It’s been able to observe how the role of connected machines are expanding in scale and scope through advances in artificial intelligence and the proliferation of mechanical agencies in IOT and in workplace automation. Sometimes it seems like these systems are imbued with a kind of predictive ability that can seem prescient, or at least much higher in intelligence and ability. I have been meditating on some words by Adrienne Rich during my residency:
“We are living in a time of unprecedented complexity, our senses are currently whip-driven by a feverish new pace of technological change. The activities that mark us as human, though, don’t begin, exist in, or end by such a calculus.”
They were written in 2002, when then the activities that mark us as human were much clearer. Now, in 2018, it’s likely more of a space to define, to demarcate. These speculations have driven my curiosity about working with machines—and co-evolving my artistic practice alongside expanding technological complexity. At Bell Labs, I'm exploring computer vision work and the robotic interface as a creative collaborator, from single mechanical unit to robotic swarm.
LH: What is your day-to-day experience like at Bell Labs? Are you in a science laboratory? Are you in a research space? Are you working daily with an engineer or researcher? How do you balance all of that?
SC: I have the privilege of being able to work with a diverse array of collaborators in my practice. This past year at Bell Labs has been the place where it's all come together. I’ve been able to take the divergent influences, ideas, and prototypes that I’ve been working on, and sit and reflect on them in the quiet space of my studio. I’m currently working on the formal prototypes for the robotic units that will make up my commission project. I’ve been working with artists and designers Andy Cavatorta and Scott Peterman, as well as a fabrication studio named Young Buk on the final iteration of the robotic system past the prototype. It’s been a joy to go from making decisions about the design, behavior, and hardware—the organs of this robotic swarm system—to the finished product with this incredible team. I’ve started calling one of the prototypes DOUGLAS because it’s the continuation of my previous drawing collaborations, or drawing with DOUG.
LH: What does DOUG stand for?
SC: It stands for Drawing Operations Unit Generation [1, 2, 3, etc.]. I think we’re on generation four now. ‘LAS’ stands for “Live Autonomous System.” It's definitely a bit of a mouthful and DOUGLAS is a lot friendlier. We’ve been working on design, programming, and fabrication simultaneously, while integrating it with feedback from my collaborator at Bell Labs, Larry O’Gorman, and getting a sense of how it’s starting to move and shape up at Bell Lab’s research facility.
LH: What interested you in Bell Labs Researcher Larry O’Gorman’s work?
SC: Larry started out in the privacy sector. Turns out that he was one of the major contributors to the fingerprinting technology that is ubiquitous in the world now. I thought he was such an interesting person to have a conversation with. From our conversations, I learned more about his career. Currently, he works on designing visual algorithms for public cameras that extract optical flow data from surveillance footage.
LH: When did you decide to formalize this collaboration and partnership?
SC: I think I always knew I would be working in some form with Larry’s research, in part because of the aesthetic qualities that his system could extract from a camera. There were visual features that shared similarities with my work—my gestural abstract work. I felt like there was a lot of harmony there. He actually sent me a paper after our first meeting about his work, and the paper was titled "Towards A Kinder Camera," which I thought was such an unusual sentiment to come from a research facility.
LH: Sounds like he was anticipating, whether or not he knew [it], some sort of ethical responsibility.
SC: In general, the willingness for Larry and for Bell Labs to be a part of that conversation is something I found to be really inspiring and compelling. They have this future human idea which, I think, on some level, is central to the contemporary artistic imagination as well.
LH: What is the future human idea?
SC: It’s not complicated. Very universal, and, again, a privileged dialogue to have. It concerns itself with questions like: What do people want society to look like in ten years? Our generation and the generation before ours have seen the internet turn into what it is today, from dial-up to Uber. We’ve been able to see the internet evolve, and we’re only just starting to reflect back on the past twenty years of this invention and how it’s influenced how we interact with each other, how we communicate, how we move around. Facilities like Bell Labs have an inside view into how digital technologies have shaped our culture, so the awareness that the decisions we make today can have a significant impact on society ten years from now comes naturally to their legacy and vision moving forward. The same can be said for artistic institutions—the idea of shaping the future through community and artistic artifacts while maintaining a cultural archive, a record of what was.
LH: What have you learned about yourself, or what has most shaped your practice by collaborating with robots, and by collaborating with the technologies that you are describing?
SC: Drawing, I think is one of the most humble art forms. Being able to engage with mark-making in collaboration with a robot means not always knowing what I’m doing—and that has been really enlightening. It’s helped me work through and question what narratives we tell when we engage in collaboration with mechanical agents, and technologies in general. In the conversation of AI, that gets really broad—dystopian, utopian, occasionally fraught with controversy. When people think about AI there is a tendency to ascribe, or imagine, considerable agency. Something like an artificial consciousness, however far-reaching that might be. I’m compelled by the human capacity to anthropomorphize our relationship to machines, particularly to robots, and how that can end up being a mirror for how we view ourselves and our own interactions with others. There are didactic models that are encouraged by developments in IOT and voice interfaces. But the collaborative models are more interesting to me. It’s a new stage for examining authorship and agency. It starts to question, who is in control? Who do we want to be in control? Is that the point?
PROPS: What is the role of images in your research?
Sougwen Chung: The role of images in my research is linked to ways in which interface design shapes image-making. For years I've had an interest in exploring how an interface operates as a layer of mediation in creative process.
My practice is motivated by a curiosity regarding this layer of mediation, of “being on the boundary”.1 This approach is part of a philosophy of making that speaks to the aesthetics of the near-future, inherently and by extension. It does so through the mediums of performance, installation, and moving image.
The fields of human-computer interaction, artificial intelligence, and machine learning are complex, involving layers of computational abstractions and a technical lexicon often inaccessible to people outside of their respective research areas. As such, the fields benefit from a narrative context to communicate their significance to a wider audience.
One such context has been in the arena of competitive game-play. The narrative of games provides cultural reach as well as defined parameters of success. Watson, AlphaGo, and Libratus are centerpieces around which computational abstractions of human computer interaction, artificial intelligence, and machine learning can assemble.2 However, the easy communicability of competition reinforces the adversarial dynamic of human vs machine already prevalent in popular cultural discourse.
My recent project, Drawing Operations, presents an alternative. In Drawing Operations, I engage in a drawing performance with a robotic arm as an exploration of human and machine collaboration.
The co-creation of an image between human and machine reframes the conventional narrative assigned to artificial intelligence. It sets the stage for a broader cultural understanding of the field and posits a different set of research goals. Conceptually, collaboration extends the interaction of human and machine to that of a creative partnership (however aspirationally). Additionally, it invites the subjective assessment of an audience as well as inspiring research goals defined by aesthetics, interaction, and craft. The role of images, in this case, creates space for comprehensive exploratory narratives to emerge. (Sometimes counterpoint, sometimes polyphony.)
AI Researcher Fei-Fei Li says that if we want machines to think, we need to teach them to see.3 In addition to teaching machines how to see, as we speculate upon modes of vision and cognition dissimilar to our own, we are also teaching them what to see.
Drawing Operations In Drawing Operations, the role of images is two-fold. As input, they are firing the synapses of the machine which cause it to move. As output, they are objects of aesthetic inquiry.
Input: Behaviour & Process The behavior of the machine is driven by learning algorithms trained on images from a variety of sources. These images are derived from art historical archives as well as the works of contemporary artists. By using computer vision, video analytics, and machine learning, I interpret contemporary and historical image sets to glean meaningful gestural behavior. From there, these behaviors are taken into the context of a collaborative performance. Within this collaboration, Drawing Operations aims to showcase a confluence of biological and mechanical modes of sensing, cognizing, interpreting, and mark-making.
Output: Aesthetics “When the image is new, the world is new.”4 The speculative nature of the project is advanced by the interoperability of visual artifacts as concrete representations of research, sites of aesthetic comprehension, and objects within themselves. (The wreck and not the story of the wreck, the thing itself and not the myth.)5
As aesthetic inquiry: what forms emerge as a result of human and machine collaboration? What new types of information can be encoded in a single image? How do unfamiliar aesthetics stimulate new ways of seeing, sensing and decoding in the viewer?
Conclusion Today, the interface is not simply a mediating apparatus for creation but a speculative agent of co-creation. As a result, in projects like Drawing Operations, the role of the image is multifaceted. The image functions not only as an aesthetic object, but a visual artifact showcasing the intersection of artistic practice and machine learning.
For me, the project's continued evolution teases at some creative possibilities of the near-future. Beyond traditional anthropocentrism, and towards a promiscuously inclusive array of cognizing agents — mechanical and biological, singular and composite, discovered and soon-to-be discovered.
Humans have an innate need to adapt and improve what surrounds them. The strong desire to create a better, more meaningful future can be seen through each culture’s artistic and technological developments. Though we as a species are programmed to aspire, grow, and create, the invention of something truly novel—whether it takes the shape of a groundbreaking artwork or a new technology that revolutionizes daily life—is rare. One, therefore, has to wonder: what are the circumstances that enable creativity and invention?
In the 1950s, Mervin Kelly became the president of Bell Labs, the New Jersey research facility, and set out to transform it into an “institute of creative technology,” an initiative that ultimately led to the invention of the laser, transistor, and solar cell, to name a few.1 He enacted social and architectural strategies to empower researchers, including a mandatory open-door policy that encouraged interactions between engineers, chemists, and mathematicians in a collaborative, interdisciplinary environment. Kelly also offered researchers the latitude to delve into self-directed inquiries for years at a time without a specific result in mind. He even designed some of the hallways at Bell Labs to be exceptionally long so that, in walking from one place to another, one would come across acquaintances, who might inspire fresh ideas or diversions. This synergistic, go-for-broke mindset became firmly embedded in the Bell Labs culture and laid the groundwork for radical breakthroughs and collaborations across disciplines between employees and others beyond the walls of Bell Labs.
Billy Klüver talking about E.A.T. and 9 Evenings to group of artists and engineers in Toronto. Artists requests to the engineers for their 9 Evenings performances are projected on the wall behind him. Photographer Unknown. All rights reserved. Courtesy of E.A.T. and Broadway 1602.A SENSE OF POSSIBILITY EXPANDED ON BOTH SIDES, AND THE ARTISTS GAVE KLÜVER WISH LISTS FOR NEWLY AVAILABLE OR NOT YET IMAGINED TECHNOLOGIES
Billy Klüver joined Bell Labs as an electrical engineer during Kelly’s tenure. He was encouraged to follow his passion for cinema and regularly attended film screenings and exhibitions in New York. Before long, Klüver met Robert Rauschenberg at the Museum of Modern Art, followed by John Cage, Merce Cunningham, Jasper Johns, Yvonne Rainer, and Andy Warhol. Over the course of their conversations, Klüver and the artists realized that their combined skills and resources could lead to the creation of works that merge artistic vision with cutting-edge research at Bell Labs. A sense of possibility expanded on both sides, and the artists gave Klüver wish lists for newly available or not yet imagined technologies. Klüver and the engineers at Bell Labs went into production mode, developing inventions such as modified TV sets and projectors that displayed abstract images in response to a musical tone, a Doppler sonar that translated movement into sound, and FM transmitters that relayed sounds from the human body to loudspeakers. The resulting artworks were not only ambitious but also the first of their kind, pushing both sets of collaborators to consider the creative possibilities and implications of emerging technologies. In October 1966, they were introduced to the public in a performance series, called 9 Evenings: Theatre and Engineering, at the 69th Regiment Armory. A few months later, Klüver and Rauschenberg, in collaboration with Fred Waldhauer and Robert Whitman, launched Experiments in Art and Technology (E.A.T.), a nonprofit organization established to support collaborations between artists and engineers.
Through these seminal collaborations and the decades of partnerships that followed, both artists and engineers experienced the advantages of multidisciplinary collaboration. While engineers tend to address challenges in a reductive, linear way, artists generally create artworks using a broader, more divergent approach that can also inspire tangible contributions to a technologist’s research. The engineer will often take a general or universal question and break it down into small components while an artist can observe something that’s seemingly simple and create a whole universe froman it. Though different, the approaches are complementary because both parties are working in an abstract way toward new discoveries. The goal of ascertaining some new or deeper understanding of the world is a commonality between them.
Early in 2017, Bell Labs partnered with NEW INC, the art, technology, and design incubator founded by the New Museum, to reinstate the E.A.T. program with a year-long artist residency. The artists are granted traditional Bell Labs privileges: access to the company’s research, tools, resources, and fabrication studios as well as the freedom to explore countless avenues of research in order to identify relationships and areas of interest, open collaboration with researchers who are exploring everything from machine learning to multi-touch sensors, and a long-term scope to try, perhaps fail, learn, and try again.IN ORDER TO REACH OUR GREATEST POTENTIAL, IT’S IMPERATIVE THAT WE SUPPORT THOSE WHO ARE OPERATING AT THE EDGE OF POSSIBILITY
Two artists and one artistic collaboration—Sougwen Chung, Lisa Park, and the dance duo Hammerstep—are working with engineers to propel their practices forward. Chung began the residency after years of participating in a drawing collaboration with a robotic arm; through her discussions at Bell Labs, she is increasing the robot’s intelligence and multiplying it in order to study crowd behavior, influence, and empathy. Park is building upon her past work with brainwave sensors and heart-rate monitors to create a holographic installation that responds to human touch. Hammerstep is turning their futuristic dystopian written narrative into an immersive theater experience using interactive projections, motion- and biometric sensing, and low-latency locational technology. While these alternative and often poetic interpretations diverge from the engineer’s initial intentions, they have the ability to reveal aspects of technology—and our relationship to it—that wouldn’t have existed any other way.
On a daily basis, artists and engineers seek original ideas, processes, and devices for the benefit of humankind. Through close examination of past successes, we learn that the ability to take creative or intellectual risks without fear of failure, to participate in multidisciplinary collaborations, and to have freedom to develop seeds of ideas over a long period of time are essential for nurturing invention. Art and technology will define the future of humanity. In order to reach our greatest potential, it’s imperative that we support those who are operating at the edge of possibility.
I love the point you bring up in your discussion about "image-making," specifically teaching machines how and what to see. In many ways, I feel that's often the role of the visual artist for people, as well—to show us what to "see" or how to look at something in a new way. Is that something that motivates you?
In some ways, yes. I’m curious about what is unique about image-making today, and why?
How can it be taken apart?
These concerns are not new, and have precedents in the history of visual art. However, our methods of producing and disseminating images have changed pretty significantly, even over the past 15 years. Our generation has a relationship with images quite different to previous generations, and it continues to evolve quite rapidly.
One example is that images today are captured en masse, and filtered in a way to become part of a collective visual memory, which shapes our collective thinking and behaviour.
This cycle of sensing, capturing, and mediating is implicitly addressed in the collaboration with the robotic arm.
The drawing process is recorded, algorithmically parsed, and then reintroduced as behaviour for a collaborative performance. The experiment is ongoing. I’ve found that the project is generating a curious set of visual experiments that also facilitate my deeper understanding of images, creative process, and self-directed tools.
I’ve found that by evolving this workflow, I’m starting to understand drawing in new ways.
Do you see yourself or technology leading your work? Does the prospect of new technology encourage you to try new things or do ideas formulate and then you find the means to create them?
I’m sensitive to the mediating effect of technology; I try to be very aware of the framework which is presented by whatever technology I am engaging with at the time.
My process is about finding ways to understand the tools available more deeply but also break them apart a bit. Maybe it started from a reaction to perceived constraints and questioning preconceptions of interfaces.
In some earlier works, my leading the form has been a central tenet of my interest in it — but in more recent explorations, I’ve been loosening my grip on that idea of control, so to speak.
The question of control in technology today deeper than simple determinism.
Often the frustration of technology is that we expect it to do one thing and then it inexplicably does another. Either out of human error or some internal problem. Does that ever complicate your work? Does it ever elevate it?
Projects that challenge what is expected of that dynamic — the false binary of intuition and computation, excite me.
As technology advances through machine learning, what keeps the artist and machine separate? The computational and algorithmic nature of technology “learning” new modes means that, at some point possibly in the near future, it could create something functionally no different than “organic” work. Is there a line you draw between what the machine can do on its own and what has to be manipulated?
Our definitions are becoming more elastic, some would say gradually and others too rapidly. As with most things, its a matter of perpspective. Imaging technologies are already pretty advanced, so the authenticity of the form is no longer defined by the perfection of the representation. Put another way, a computer can already replicate a painting, and its already difficult to distinguish a photo of an object from the “real thing”. This is pretty established ground.
What I’m curious about is how these new ways of seeing and learning evolves a creative process. To your question — does the artist and machine need to be separate? What does a composite creative process look like — and what precedents does this evolving hybridity have in art history? In this space, there is a strain of speculation that leads to real invention. We’re living in an unprecedented time, the interplay between speculation and invention is sparking like a live wire. My curiosities are driven by that energy.