Deeply grateful to @Lumen_prize, for awarding the Art and Technology Prize to Drawing Operations, a project I started in 2015 exploring human and machine collaboration. Thank you Carla Rapoport, and Jack Addis for your support and welcoming me into the Lumens family. . . In these deeply uncertain times, I find myself thinking of #DonnaHaraway’s writing in Staying With The Trouble 🖤: “It matters what worlds world worlds. We become together or not at all." Collaboration, for me, is a way to steer away from "a technophobic spiral, by conceiving of humans & machines as interacting parts of complex adaptive systems." . . .
LATE LAST YEAR, artificial intelligence made a loud splash in the art world. A trio of French students, calling themselves Obvious, put a smeared, unfinished portrait up for auction at Christie’s. Titled Portrait of Edmond de Belamy, the murky image pictured a suited gentleman with a plain white collar. Rendered in three-quarter profile against an indistinct background, the image mimicked conventions from Renaissance and Baroque portraiture. At the corner of the canvas, a mathematical equation ostentatiously replaced the traditional artist signature. Global media trumpeted the achievement as a stunning first. Obvious oiled its promotional engine by misleadingly claiming it to be the first ever work of art created by AI to go under the hammer. Neither claim turned out to be true: significant human labor went into its making, and a set of AI-generated images had gone to auction at San Francisco’s Gray Area in 2016. Offered at $10,000, the image went to an anonymous buyer for a jaw-dropping $432,000.
This portrait embodies significant problems at the heart of AI’s recent cultural ubiquity. Machine learning is being used across art, science, and industry — from music production to space exploration — but its excursions into the creative economy have ignited the most controversy. Commonly, such works lead to the question, “Can a machine be creative?” But this question misses the mark.
AI is the latest, most intangible incarnation of the automated arts. When it comes to creative automatons, history shows we’ve been rehearsing the same hand-wringing about authorship and authenticity for over a century. The history of photography, for example, famously raised alarms about humans surrendering the arts to machines. While automatons are now largely digital, not strictly mechanical, they, too, seem to imperil the ostensibly humanist essence of art. But Portrait of Edmond de Belamy did not spring fully formed from the neural network’s “mind.” Rather, it emerged out of a complex system where human actors continue to play a crucial role, although the story of their absence — and abdication of the arts — persists.
Obvious worked with an artificially intelligent system known as a generative adversarial network, or GAN. An algorithm composed in two parts, a GAN reproduces an agonistic relationship between artist and critic. The Generator creates new images based on a massive data set, and the Discriminator evaluates that image against a human-made image. Obvious trained their GAN on a data set of 15,000 portraits from the 14th to the 20th century. Ensconced in a gilded frame, and gorged on the Western canon, the image convincingly reproduced the aesthetic conventions of portraiture. By signing the image with a segment of the algorithm’s code, Obvious had, in a clever sleight of hand, cast an algorithm as a stand-in for the absent hand of the artist.
But the artist in question was not, actually, a hypothetical person. Obvious had appropriated their code, as well as the training set, from Robbie Barrat. Then a 19-year-old artist and programmer, Barratt had shared his Old Masters GAN on the open-source sharing website, Github, in 2014. An outcry from the small community of AI artists punctuated the auction. They not only emphasized Barrat’s erased contribution but also expressed disbelief that the amateurish image was representing their years of creative experimentation with GANs to the world at large. In their rush to crown the algorithm as the work’s author, Obvious obscured the central roles played by humans in the conception, coding, and curation that yielded the image. If this was an aesthetic Turing test, then we have willfully failed it. But why?
The equation at the traditional site of the artist’s signature offers some clues. This staging culminates a centuries-old conversation about the agency of creative automatons. Signatures boast a special connection to the human hand. Historically, they have imprinted the heft of the human body upon an object, whether artwork or legal contract. They index the identity of the signer: they affirm that someone was really there. By displacing a human signature with an algorithm, Obvious manifested an obscured predecessor in the history of automated art — not the camera, but the player piano. Yes, that classic piece of ragtime furniture.
Popular in the early 20th century, player pianos are a mechanical incarnation of the absent human hands in AI-generated art. Anyone who has seen a player piano in action can attest to its uncanny nature. Once in motion, one can almost imagine an invisible player sitting at the instrument. Piano keys seem to play themselves in cheerful ragtime. Now largely a relic of a different time, the player piano stirred many of the same kinds of questions and debates that AI art does now.
Inventor Edwin Votey built a prototype of his automated player piano system in 1895. It took the form of a large wooden cabinet, and was powered by suction generated by two foot pedals. It was designed to stand in front of an existing piano, where mobile fingers at its rear would orchestrate its keys. Astride the piano, this cabinet of curiosities contained a small paper roll patterned with tiny perforations representing notes to be played. As a roll moved over the tracker bar, a reading device with a row of evenly spaced holes, a valve would open and trigger a pneumatic motor. This would then fire a felt-covered finger on the external player, causing it to hit the corresponding piano key. Though these functions soon entered the instrument itself, with the introduction of the Apollo line of pianos by the inventor Melville Clark in 1901, these operational principles remained the standard for nearly all roll-operated player piano systems.
In the next decade, the machine developed into a ghostly rendition of human performance. In 1904, German organ builder Edwin Welte and his school friend Karl Bockisch launched a new kind of player piano, the Welte-Mignon. Otherwise known as a reproducing piano, these were able to record and reproduce individual performances by replicating the dynamics, rubato, and pedaling of living players. In its heyday, music roll manufacturers recorded the performances of many famous 20th-century pianists, George Gershwin, Liberace, Jelly Roll Morton, Myra Hess, and Thomas “Fats” Waller among them. Specters of performances past, these rolls conjure the apparitions of nearly every major early 20th-century pianist.
For all its commercial success, the automated instrument drew its share of criticism. In 1906, John Philip Sousa published “The Menace of Mechanical Music,” a screed against the sweeping popularity of player pianos and gramophones. Sounding the alarm about these poor copies, Sousa fretted that these “talking and playing machines” “reduce the expression of music to a mathematical system of megaphones, wheels, cogs, disks, cylinders, and all manner of revolving things, which are as like real art as the marble statue of Eve is like her beautiful, living, breathing daughters.” For Sousa, these creative automatons substituted cogs and wheels for the ineffable human soul.
Creaky in their physicality, the player piano seems a far cry from the digital machines so ubiquitous today. However, the rolls that powered their spectral movements form a bridge with the artificially intelligent systems that generate such images as Portrait of Edmond de Belamy. After all, with the sleek miniaturization of contemporary computing, it is easy to forget that the early computers of the 1940s and ’50s were massive calculators operated by punch cards with holes in them. In these pockmarked landscapes, the automated arts, from player pianos to algorithm-generated portraiture, took root.
As William Gaddis decried in Agapē Agape, his 2002 swan song on the ruination of art by technology, “There was the beginning of key-sort and punched cards and IBM and NCR and the whole driven world we’ve inherited from some rinky-dink piano roll.” The narrator of Gaddis’s pseudo-autobiographical novel lies dying in his hospital bed. He pores over his notes in a desperate rush to finish his magnum opus, a history of the player piano. Paralleling his deteriorating body with the decay of culture, he rages against a commercialized world where art has become mere entertainment, and imitation has displaced authenticity.
Gaddis is not alone in placing the player piano in the history of computational creativity. The recent TV show Westworld also winks at this past. The show envisions a world where guests pay to play in a live-action simulation of the Wild West, replete with cowboys, gun battles, and damsels in distress. Euphemistically named “hosts,” androids in period boots and hats, are at the mercy of the human “guests” who come to ravish, maim, or kill them. In the saloon where guests down whiskey and manhandle artificial women, a player piano sits. Phantom hands alight on the keys; tinkly covers of contemporary songs serve as a soundtrack for gun battles and seduction. As robotic copies of humans cavort among originals, these mechanized cover songs evoke the longer lineage of artificial others.
Despite the century that has passed since their debut, player pianos, like androids, remain eerie, ghostly objects on the brink of life. In a pantheon of creative automatons, the player piano endures for the existential questions it provoked. How have these automated instruments transformed the landscape of creativity? And at what cost?
New technologies demand new visions of how we might use them. AI technologies have been accused of reproducing the invisible human structures that inhere in the data from which they learn, including the best and worst of us. Take Portrait of Edmond de Belamy. Not only did it regurgitate musty art historical conventions, but its success affirmed the gatekeeping function of the auction house, as well as the conflation of monetary worth with artistic value. Where Obvious’s image replicates some of the least compelling elements of the art world, other AI artists imagine other ways we might relate to these creative machines.
Against a narrative of imperiled human creativity, some artists working with AI instead frame these new technologies within an ecology of human and machine collaboration. In Drawing Operations, the artist Sougwen Chung stages a series of collaborative drawing performances with a robotic arm. In three chapters, “Mimicry,” “Memory,” and “Future Speculations,” Chung showcases an evolving robotic behavior based on her research with art and AI. Ongoing since 2015, the work highlights the complexities of assigning authorship to AI.
In Generation 1 of the project, Chung explored robotic mimicry in real time. By capturing her drawing gestures with an overhead camera and analyzing them with computer vision software, the robotic arm replicated her movements. The robot precisely duplicated the pace, shape, and style of the original performer. In many ways, we might understand this arm as a 21st-century version of Votey’s player piano, the mechanical fingers of his wooden cabinet playing the keys in the absence of a human performer. But Chung turns this dynamic on its head. After all, her body was actually present, the robotic arm a proxy for her in the same moment. Their gestures synchronized as they made marks on paper, Chung and her robotic double formed a strange duet.
In Generation 2 of Drawing Operations, the robotic arm was linked to a neural network trained on a database of Chung’s extracted gestures from previous drawings. As a result, the robotic arm was able to generate new movements and drawings in Chung’s style, without duplicating them exactly as it had prior. In improvised performances, Chung draws in tandem with the arm, creating abstract line drawings that blur her marks with those of her robotic collaborator. Speculating on artificial creativity, Drawing Operations suggests a more complicated entanglement of human and machine actors than prevailing media narratives suggest. Through these performances, Chung systematically imagines a creative future where automatons might extend human intelligence.
As the perforated rolls of the player piano prefigured the punch cards of early computing, so, too, have they shaped how we talk about creative machines. Like the ghostly hands that played upon pianola keys, AI art stokes deep cultural anxieties about the risks automation poses to human activity. Ultimately, we fear that they will replace us, whether at the factory or at the canvas.
While the story of absent human hands lingers from the heyday of the player piano, the stakes have changed. For all the controversy it stirred, the player piano was ultimately only a reproducing musical robot, unable to deviate from its punched script. It was also an industrial invention, symptomatic of rapidly changing modes of production. In a creative economy, where factory jobs are all but gone, creativity is just another commodity. The risk is not that humans will stop being creative but that they will cede the signs — and compensation — of their labor to algorithmic signatures.
By demonstrating AI artworks’ commercial viability, Portrait of Edmond de Belamy sparked a gold rush. Shortly after the notorious auction, computer scientist Ahmed Elgammal debuted the creative output of AICAN, his GAN variant, in a Chelsea gallery. Unlike Edmond de Belamy, these pieces, collected under the title Faceless Portraits Transcending Time, were pitched as a joint effort between man and machine. While many images from the show ostensibly sold for five figures, AICAN’s financial ambitions are far higher. Based on its success identifying the chronology of the images in its data set, Elgammal and his investors believe it could be used to anticipate — and produce — future art trends. Unchecked, these virtual automatons could theoretically cut human artists out of the art market entirely. The fears of Sousa and Gaddis could, in fact, be realized.
But rather than allow this threat to send us into a technophobic spiral, we could instead, like Sougwen Chung, conceive of humans and machines as interacting parts of complex adaptive systems. Whether creative duet, industrial partnership, global economy, or internet, these complex systems have marked all of human history. So imagined — and so designed — intelligent machines can serve to amplify and augment human activity. It is neither us nor them, but both.
My (re)search and art practice featured in Issues in Science in Technology: Future of Work Special Edition.
In Omnia per Omnia, I paint with a multi-robotic system connected to the flow of a city. It is a meditation on the porous relationship between self and world; individual agency within a collective body (intelligence).
Sougwen Chung, Drawing Operations (2017). Courtesy the artist.
WHO: Chung is a Canadian-born, Chinese-raised, New York-based interdisciplinary artist and former research fellow at MIT’s Media Lab. She’s currently E.A.T.’s artist-in-residence in partnership with the New Museum and Bell Labs. Her work, which spans installation, sculpture, drawing, and performance, explores mark-making by both hand and machine in order to better understand the interactions between humans and computers. She has exhibited at institutions including The Drawing Center in New York and the National Art Center in Tokyo.
WHAT: For her current project, Drawing Operations, Chung uses Google’s TensorFlow, an open-source software library used for machine learning, to classify archives of her own drawings. The software then transfers what it has learned about Chung’s style and approach to a robotic arm that draws alongside her. She’s also working on a few new projects using pix2pix (a neural network trained to produce variations on an image, like the nighttime version of a daytime photo) and sketch-rnn (which tries to continue or complete a digital sketch based on where the human leaves off) to expand on this idea of human and machine collaboration.
WHY: “As an artist working with these tools, the promise of AI offers a new way of seeing,” Chung explains. “Seeing as self reflection, seeing through the ground truth of ones own artworks as data. There is a lot of talk about biases evident in AI systems and that is absolutely true within AI systems trained on art. You could describe visual language as a kind of visual bias, a foregrounding of the subjective view of the artist. By translating that into machine behavior, I am attempting to create a shared intersubjectivity between human and machine.”
I always like to start with talking about an artist’s background. Can you talk a bit about your early life and how you came to visual art?
I grew up in an immigrant family in Canada, my parents are originally from Hong Kong. Hong Kong is a place that I fantasize about -- feel this deeply unearned nostalgia and longing for; I wonder if that’s typical for children born into the east asian diaspora.
My father was a classically trained opera singer and my mother a computer programmer, so those influences were always in the background. I always considered my upbringing to be quite normal, growing up with rigorous musical training and access to technology before everyone else, but I recognize it now as a very distinct privilege.
I grew up playing several instruments. I didn’t know it at the time, but learning to communicate through music, through the apparatus of an instrument, probably shaped my outlook on everything. Playing the violin was what resonated me the most.
I look back and think of my time as a violinist as one of my first experiences of artistic escapism -- learning a feedback loop of internalization and externalization through gesture that was at once improvisational and abstract. At the same time, it wasn’t really subjective. You either had the skill and intent to communicate the emotion or you were asleep at the wheel. Or in my case, the bridge.
In tandem, we were among the first to have computers and access to the internet. As a child I was very active in the building of online culture and digital identity, which as a result still fascinates me. Nostalgia is overrated, but I indulge in it a bit when I think about those early internet days. I like what Andre Nusselder says about digital space as a projection of one's desire and fantasy -- especially back then before online spaces were so regulated and before communication became so reliant on platforms. The fantasy before the deep fake. When talking to someone from another continent still felt like magic -- it probably still is, we just forget that it is.
My early life was mostly about learning how to communicating through instruments, musical and technological. Growing with that practice, sharing through performance. In that sense, nothing much has changed.
What’s different is that now these instruments, these tools, are part of much more complex systems. So there’s a feedback loop, a cycle of learning and adaptation through the promise of AI and an evolution of the digital object. In that sense, everything has changed.
How long have you been interested in exploring the relationship between robots and humans? What led you to explore this relationship through visual art?
I’ve been exploring this space for almost 5 years now... Maybe more? It seems like a long time and also a split second in the scheme of things.
I’ve always considered the relationship between robots and humans as a proxy for exploring the broader scope of technology’s influence on our lives. One specific influence is that of AI systems in the world today. My robots bring AI generated behaviours into physical space -- out of the digital simulation. You could say that it’s about bodies, as well as bytes.
I work alongside the AI system to make drawings, creating a feedback loop on a single canvas in real time. I was surprised that by translating these behaviours physically and working alongside them it changed what drawing meant for me. Working with prototypes and behavioral experiments; it’s a reminder of the vitality and fallibility of the process at the same time -- I guess that’s what I’m drawn to. The uncertainty, the unlearning, the unknowing of it all.
When I think of robot production, I think of functionality, and not so much artistic expression and collaboration between robots and humans. What do you believe about the future of human and robot communication? How do technological advancement and artistry intersect?
That’s such a big question. It could go so many different ways, really. I will say that working with robots has contributed to my own thinking and fluency with deep learning systems, rapid prototyping, physical computing. It’s intimidating at first, but rife with aesthetic and conceptual inspiration.
Its further affirmed that technology in general isn’t inherently good or bad, but it isn’t exactly neutral either. However, the stories we tell about technology, especially in pop culture and tech media tends to be pretty reductive and sensationalized. It’s not all Terminator, Tesla, or Twitter.
We probably think of functionality when we think about robots as they were a prominent icon of the industrial revolution -- we associate them with assembly lines and factories. It lends itself to a didactic perception of the role between human and machine, with humans in control.
What if it wasn’t about control? What if it could that be another way? I wonder what would that look like. I think control is interesting only when its incomplete -- and most of the time it is.
On the same vein, the future of human and robotic communication is pretty porous. It’s not so linear or one-way. It’s not really clear who’s at the wheel, sometimes literally. It raises some interesting questions about agency.
At the risk of sounding like a foghorning media theorist, I’ve been thinking of the parallels between technological and artistic processes, in that they both foreground human bias. One process translates the bias of the dataset and the human engineer into the AI system and the other the visual bias of the artist and her visual style onto a canvas.
By intersecting artistic and technological processes, I’ve been working towards discovering particularities about both. They’re just ways of seeing and shaping the world at the end of the day, after all.
In Omnia per Omnia, you reframe the idea of a landscape as a documentation of movement throughout a space––particularly the ebb and flow of bodies throughout a city. This data is gathered through the interplay of technology or surveillance with the organic and improvisational movement of life. How are these robots taught to move––i.e. Is their movement improvisational? What does it mean to reverse the documentation––by having the human hand follow the robot, as opposed to the other way around?
The movements of the robots are linked to urban flow as captured through public cameras. They move according to motion vectors extracted from pre-existing surveillance feeds in New York City. The movements are improvisational in the way that how we move through cities is improvisational… sometimes chaos, sometimes flow. The mechanical units, which I call D.O.U.G._L.A.S. (Drawing Operations Unit Generation 4, Live Autonomous System) start on predictable paths which, by translating the data through physical space, eventually veer off course, creating space for emergent, random behaviours as the composition unfolds on the canvas. When performing with these units, they resemble to me, a swathe, a broad stroke of the brush on by a multi-agent robotic body.
It’s interesting that you describe it as a reversal, I’m not sure if I see it that way. I think culturally we’re moving beyond the linearity of human following machine or vice versa, to something more cyclical.
As the initial movements of the robots in Omnia follow human movement, and I in turn am creating alongside these translated positions, the marks made between human and machine function as a feedback loop. It’s been described as it’s a rhythmic duet, evoking a sense of push and pull; of coexistence. Alternatively it could be seen as meditative, and sometimes disturbing and chaotic. For me, the performance demands a vital focus on the movements of my robotic collaborators. It requires a kind of ego dissociation. Like being part of a collective.
You ask “are we on the onset of a new, collaborative imagination––of radical new intersubjectivities?” What does this mean? What do you foresee this world, in which humans and technology collaborate, looking like?
The promise of some of these new technologies, like computer vision and deep learning, offers an alternative way of seeing. In my work I explore the particularities of that vision by turning it on my drawings, my environment, myself, to see from another perspective.
Seeing can be an act of self reflection, but self-reflection is a closed loop. Learning to see as a machine sees is a vantage point that requires mediation between symbolism and data. In deep there is the concept of ground truth, that which is objective and not derived from inference.
There is a lot of conversation about biases evident in AI systems and that is absolutely true within AI systems trained on art. In that vein, you could describe visual language as a kind of visual bias, a foregrounding of the subjective view of the artist, like I said before. Bias is another form of subjective view. For my part, by translating visual style as bias as subjectivity into machine behavior, and collaborating with machines as an aspiration, I am attempting ask certain questions. What might the ground truth of an artistic practice be? Is that something we want? What might a shared intersubjectivity between human and machine look like?
For my project, Omnia per Omnia, I elaborated on this “intersubjectivity” and expanded it to the idea of a collective. In the work, I painted alongside a system of painting robots whose movements were linked to the flow of New York City. The movement was extracted using an optical flow algorithm trained on surveillance cameras around New York City. The performance is a trio of sorts between myself as a performer, the 20 multi-robotic units and the collective movement of the city, improvising a hybrid choreography. It’s a performance of human and machine and city, an emergent creative process.
With Omnia, the act of improvisational painting takes place through a heightened sense of awareness of the machines. It’s flow is site-specific. The movements are linked to the movements of a crowd, who aren’t aware of their position as catalysts of the robotic swarm.
But what if the crowd were aware of being all watched over by these machines? That’s what I mean by being at the onset of this collective imagination and intersubjectivity. It’s inspired the idea of a collaborative space between multiple bodies that is physical, digital, telepresent, and most importantly, shared. It’s a representation that isn’t about control, but something else.
The project really stemmed from a curiosity -- a willingness to de-privilege the western conception of the individual towards an entangled, intersubjective, radical ecosystem. I’ve been inspired by this reconfiguration of the “I”, through theorists like Yuk Hui’s who explore a new cosmotechnics, and the philosophies of media centred around eastern concepts of relation.
The act of painting in Omnia per Omnia appears as a meditative practice, in which the lines between the maker and the final product are blurred––that is to say, the act of making seems to be more important than the actual outcome. It reminds me of Allan Kaprow’s “Happenings” in that way. Would you say that the making is more about the act than the product?
Yes, I think I would. Perhaps it’s my upbringing in music. The performance vs the recording of the performance, for instance. My approach to making is grounded in my background as a musical performer, as an instrumentalist. Making through collaboration as folie a deux; a shared hallucination between collaborators. Sometimes harmonious, and sometimes discordant, but always intertwined, and relational. Counterpoint.
You speak about the Japanese Gutai art movement, in which “matter never compromises itself with the spirit; (and) the spirit never dominates matter.” Can you talk a bit about this movement’s influence on your work, and how your work builds upon these theories of making?
I love that particular excerpt of the Gutai manifesto. “matter never compromises itself with the spirit; (and) the spirit never dominates matter.” Its deceptively complex and.. Beautiful and provocative.
In my view, Gutai’s theories of making reframed the human’s relationship with the matter of the day. It sought to recentre the human body (as a channel to spirit) as their artists engaged with painting and sculpture (matter) towards freedom of expression. It stemmed from Jiro Yoshihara’s rejection of contemporary propagandistic trends of art, which were supportive of the political contexts that supported the Japanese involvement in WW2.
For me, their group shows represented a collective action in a time of uncertainty. Gutai artists engaged in radical artistic gestures, through a range of media, performances, and sculptures. Their approach to making was focused on a liberation from the past -- an uncoupling from the cognitive patterns derived from a post-war social order. They worked towards creating new meaning and pathways through these artistic gestures, towards “engaging with the uncompromising spirit of matter itself”.
I’m quite inspired by this approach, though the Matter I’m working with today, you could say, differs from the Matter used by Gutai artists in the 60s.
Technology as matter is interactive; it can be linked, sensate, responsive, interactive, intelligent, interconnected in a way which has never existed in human history. The technology of media art like cameras, sensors, computers, for instance. Technology can serve a propagandistic function all the same and work against our best interests all the same, and our relationship to it must be reframed.
While Gutai emerged from the political and psychic disillusionment of war, it’s important to note that it wasn’t a movement that ended on a sense of nihilism. In the face of uncertainty, the movement demonstrated artistic strategies for uncoupling and unlearning, in service of cultural extension, collective action, and hope in the dark. My work builds on, and is inspired by, these Gutai traditions.
We made a film for #OmniaPerOmnia and it was a wild ride. (This is the teaser ~) .
Thank you to @mynameiscutter for sharing the vision, lending your talents to the project, and assembling the star team for our film. @tshung@cleggm @arianne_alizio .
Original score by @aquarianyes and Mickey Partlow @v3rysaid .
Omnia per Omnia pays homage to the flow of cities. This first chapter begins in NYC, my home of a decade and birthplace of the Experiments in Art and Technology initiative.
Sougwen Chung, Omnia Per Omnia Performance. Photo by Irina Abraham.
"In Omnia Per Omnia Sougwen Chung explores collaborating with robots as opposed to using them as a tool. As I enter the room, the artist and the engineer Andy Cavatorta are fussing around little machines with exposed hardware, which seem to roam free around a white platform. Sougwen and Andy look like two caretakers with the little robots being their wards.
When Sougwen first contacted Andy about the project, he jokingly asked if she wanted him to build her an army of painting robots. To his surprise it was exactly what Sougwen who had designed, coded and engineered the prototypes, intended. The robots are using surveillance footage and have the collective movement of the city power the movement of their swarm. Sougwen is painting the portrait of the city together with the robots. When asked what the difference is between collaborating with humans and robots, Andy and Sougwen laugh and say that humans cooperate. Their hope is that the robotic swarm will keep learning and evolving.
On the day of the performance, Sougwen is surrounded with audience and journalists. Once the robots start moving and the music fades in, everyone turns quiet. The slow and seemingly purposeful movement of the robots, the traces of the blue paint they leave behind, the motion of Sougwen's brush and the expression of total concentration on her face create an atmosphere of a ritual, a spiritual action. The audience is affected by the magic happening in front of them. Watching the artist paint with the robotic swarm creates a true emotion, the way only art can. "
This post was originally published by NEW INC on April 20, 2018 and is based on an interview between Lindsay Howard and Sougwen Chung.
Recently NEW INC, the New Museum’s cultural incubator, and Nokia Bell Labs presented their first exhibition entitled “Only Human” at the Mana Contemporary in Jersey City, New Jersey. The exhibition showcases the work of NEW INC artistsSougwen Chung, Lisa Park and Hammerstep (Jason Oremus and Garrett Coleman) participating in the artist-in-residence program at Bell Labs and their collaboration with Bell Labs researchers to produce new artistic projects inspired or enabled by Bell Labs technologies.
“Only Human” is available to visit Tuesdays - Saturdays at 3PM through June 2nd as part of the Mana Contemporary’s gallery tours. No RSVP is necessary.
On Saturday, May 12th there is a day-long symposium at the Mana Contemporary for a deeper dive into some of the ideas, themes, and technological research that are being explored in the works on view. It will also reflect on the legacy of the Experiments in Art & Technology program (1967–2001), founded by artists Robert Rauschenberg and Robert Whitman, and Bell Labs engineers Billy Klüver and Fred Waldhauer. Please RSVP here.
The interview below with Sougwen Chung provides additional insight into her practice and how her work with Bell Labs researchers has enriched her artistic experience during her year long tenure as an artist-in-residence.
Lindsay Howard: What are some themes you’ve been exploring, and some past projects that relate to the research you’ve been doing at Bell Labs?
Sougwen Chung: I’ve been evolving the theme of human-robot collaboration through the past year at Bell Labs. My work has always centered around the marks made by hand and the marks made by machines—and the machines are constantly evolving. In particular, I’ve been exploring the fields of collaborative robotics, computer vision, and biometrics during my residency. It’s been able to observe how the role of connected machines are expanding in scale and scope through advances in artificial intelligence and the proliferation of mechanical agencies in IOT and in workplace automation. Sometimes it seems like these systems are imbued with a kind of predictive ability that can seem prescient, or at least much higher in intelligence and ability. I have been meditating on some words by Adrienne Rich during my residency:
“We are living in a time of unprecedented complexity, our senses are currently whip-driven by a feverish new pace of technological change. The activities that mark us as human, though, don’t begin, exist in, or end by such a calculus.”
They were written in 2002, when then the activities that mark us as human were much clearer. Now, in 2018, it’s likely more of a space to define, to demarcate. These speculations have driven my curiosity about working with machines—and co-evolving my artistic practice alongside expanding technological complexity. At Bell Labs, I'm exploring computer vision work and the robotic interface as a creative collaborator, from single mechanical unit to robotic swarm.
LH: What is your day-to-day experience like at Bell Labs? Are you in a science laboratory? Are you in a research space? Are you working daily with an engineer or researcher? How do you balance all of that?
SC: I have the privilege of being able to work with a diverse array of collaborators in my practice. This past year at Bell Labs has been the place where it's all come together. I’ve been able to take the divergent influences, ideas, and prototypes that I’ve been working on, and sit and reflect on them in the quiet space of my studio. I’m currently working on the formal prototypes for the robotic units that will make up my commission project. I’ve been working with artists and designers Andy Cavatorta and Scott Peterman, as well as a fabrication studio named Young Buk on the final iteration of the robotic system past the prototype. It’s been a joy to go from making decisions about the design, behavior, and hardware—the organs of this robotic swarm system—to the finished product with this incredible team. I’ve started calling one of the prototypes DOUGLAS because it’s the continuation of my previous drawing collaborations, or drawing with DOUG.
LH: What does DOUG stand for?
SC: It stands for Drawing Operations Unit Generation [1, 2, 3, etc.]. I think we’re on generation four now. ‘LAS’ stands for “Live Autonomous System.” It's definitely a bit of a mouthful and DOUGLAS is a lot friendlier. We’ve been working on design, programming, and fabrication simultaneously, while integrating it with feedback from my collaborator at Bell Labs, Larry O’Gorman, and getting a sense of how it’s starting to move and shape up at Bell Lab’s research facility.
LH: What interested you in Bell Labs Researcher Larry O’Gorman’s work?
SC: Larry started out in the privacy sector. Turns out that he was one of the major contributors to the fingerprinting technology that is ubiquitous in the world now. I thought he was such an interesting person to have a conversation with. From our conversations, I learned more about his career. Currently, he works on designing visual algorithms for public cameras that extract optical flow data from surveillance footage.
LH: When did you decide to formalize this collaboration and partnership?
SC: I think I always knew I would be working in some form with Larry’s research, in part because of the aesthetic qualities that his system could extract from a camera. There were visual features that shared similarities with my work—my gestural abstract work. I felt like there was a lot of harmony there. He actually sent me a paper after our first meeting about his work, and the paper was titled "Towards A Kinder Camera," which I thought was such an unusual sentiment to come from a research facility.
LH: Sounds like he was anticipating, whether or not he knew [it], some sort of ethical responsibility.
SC: In general, the willingness for Larry and for Bell Labs to be a part of that conversation is something I found to be really inspiring and compelling. They have this future human idea which, I think, on some level, is central to the contemporary artistic imagination as well.
LH: What is the future human idea?
SC: It’s not complicated. Very universal, and, again, a privileged dialogue to have. It concerns itself with questions like: What do people want society to look like in ten years? Our generation and the generation before ours have seen the internet turn into what it is today, from dial-up to Uber. We’ve been able to see the internet evolve, and we’re only just starting to reflect back on the past twenty years of this invention and how it’s influenced how we interact with each other, how we communicate, how we move around. Facilities like Bell Labs have an inside view into how digital technologies have shaped our culture, so the awareness that the decisions we make today can have a significant impact on society ten years from now comes naturally to their legacy and vision moving forward. The same can be said for artistic institutions—the idea of shaping the future through community and artistic artifacts while maintaining a cultural archive, a record of what was.
LH: What have you learned about yourself, or what has most shaped your practice by collaborating with robots, and by collaborating with the technologies that you are describing?
SC: Drawing, I think is one of the most humble art forms. Being able to engage with mark-making in collaboration with a robot means not always knowing what I’m doing—and that has been really enlightening. It’s helped me work through and question what narratives we tell when we engage in collaboration with mechanical agents, and technologies in general. In the conversation of AI, that gets really broad—dystopian, utopian, occasionally fraught with controversy. When people think about AI there is a tendency to ascribe, or imagine, considerable agency. Something like an artificial consciousness, however far-reaching that might be. I’m compelled by the human capacity to anthropomorphize our relationship to machines, particularly to robots, and how that can end up being a mirror for how we view ourselves and our own interactions with others. There are didactic models that are encouraged by developments in IOT and voice interfaces. But the collaborative models are more interesting to me. It’s a new stage for examining authorship and agency. It starts to question, who is in control? Who do we want to be in control? Is that the point?
The proliferation of computation is bringing about a paradigm shift. From marks made by hand toward marks made by machine, we invite the question: if machines are able to execute actions traditionally performed by humans, will the human hand be replaced?
Contemporary artists with a forward-thinking approach are responding to this question by imagining new pluralities, rejecting the simplistic binary of human or machine. Through explorations of machine automation as an artistic medium, artists are defining paradigms of human and machine as intertwined agencies informing each other. Through co-authorship of organic and computational intelligence, an expanded notion of collaboration sets the stage for vital new inter-subjectivities. Thus, we investigate how might the conception of human agency need to evolve.
Allegories of Human & Machine in Art
Art, by necessity, provokes speculation. Artistic movements have long helped shape a cultural understanding of historical epochs through allegorical lineage, linking the role of the artist to conceptions of agency — from Michelangelo’s hand setting an angel free from marble, to omniscient arm of SEEK that shape the environment of its citizens, to the works of today, exploring the emergent possibilities of human and machine.
During the renaissance, the artist’s hand was considered God’s vessel by religious institutions. Michelangelo's Angel of Arca di San Domenico was a valorized motif, an idealized form waiting to emerge from its captivity in stone – he artist’s skill was secondary. Michaelangelo’s role as an artist was that of a facilitator of religious wonder – an intermediary between the commonality of man and the sublime.
In the case of SEEK, the machine’s hand was omniscient. The machine’s prescient use of computer vision technology introduced a synthetic mode of sensing that contrasted the organic sensory apparatus of its gerbil subjects. The project simulated computational sovereignty via a robotic entity that was responsible for maintaining order in an environment based on a predetermined set of rules that determined environmental changes informed by its population’s response to previous iterations. SEEK was an early example of automation and computational governance provided by a simulation of a self-organizing and self-referential system. SEEK provided a metaphor for how technological systems could potentially govern behavior.
Allegories in art reflect and shape the social values of their historical context. Michelangelo's works in the renaissance can be seen as glorifying religious iconography with the scope of supporting the church’s governing dogma. SEEK and comparable works in media art effectively simulated the regulating influence of systems, a prescient concept in 1969 and a precursor to the networked systems of today. On one side of the spectrum, the fictive machines as a consummate offspring of humankind (combining the tendency towards pareidolia with imaginative delusions of grandeur). On the other, machines that render humankind obsolete affirm a fatalistic mindset. While such ideas might satisfy the tendency towards utopian and dystopian fantasies about technology, they emphasize a reductive binary of human or machine.
Man or Machine
Throughout time, we have witnessed the ways in which cultural shifts provoke anxiety and how, beneath the tectonic plates of the establishment, they dismantle structures. With that in mind, it can be tempting to mythologize the utopian belief in a hypothetical emergent intelligence within the computational operations of the machine as a seraph enclosed within the substrate of bits and bytes. To seek sublime within sublimation. These myths provide comfort and a stabilizing, yet perhaps naive, optimism. But if we operate under this assumption, what becomes of human agency, as embodied by the artist’s hand? Is it no longer required?
The reality of today’s technological landscape is more complex. What is a representative microcosm of the overall megastructure? As alluded to in prescient works like SEEK, the effect of computational governance is at once non-neutral, ubiquitous and invisible. Systems link to form an interconnected network that sees and yet is unseen. The anatomy of the network, its sensors, its data collection and influence, are rarely in the frames of mind of the users it governs. It is these conditions that have contributed to the intractable effect of technology in the ebb and flow of modern life. It is “an accidental megastructure, one that we are building both deliberately and unwittingly and is in turn building us in its own image.”
Man and Machine
Today, artists are caught in a cacophonous feedback loop as they ricochet between the tradition of art (manifesting social consciousness and reflecting on human experience) and the increased presence of machine automation (proliferated in tools, software, sensors, and robots). The modern psyche is on a transitional edge. We are charged with constructing intellectual and philosophical frameworks alongside the technologies that are simultaneously shifting perceptions of human agency.
Take AARON, a computer program designed in 1973 by artist Harold Cohen, as an example. Trained as a painter long before AARON began, Cohen’s project translated singular artistic intention through computers. In this project, the machine operated as an extension of the artist’s vision. Cohen defined the system according to his own aesthetic sensibility. The machine’s rule-based system used familiar motifs within the genre of painting such as people and plants, which eventually shifted towards abstraction in later years. In this project, Cohen programed the machine to paint a series of novel images and acted as the overseer of the work – absolving him from contributing physically to the painting and privileging didactic over dialogical methodology. Cohen prefered to extend his agency through a machine by translating his artistic vision into computational language. AARON executed permeations on a style, but did not invent its own original style beyond Cohen’s predefined set of rules. Thus, the limitations of the output were predetermined. The artist, for Cohen, was a designer of systems and an interrogator of authorship.
The creativity described in Cohen’s work is linear, as it begins with the artist and ends with the machine. The human and machine gestures remain separate because the mechanical operations are executional. Cohen creates the rules for AARON to follow and they never occupy the same space in time. With this approach, Cohen occupies a privileged space in the interaction with the machine because his vision is absolute, perfect, unchallenged, non culpable. While the authorship may be in question, the roles occupied by human and machine are not. Now, Cohen is able to produce posthumously through automated agency. In questioning authorship, we challenge the definition of creativity. But what of collective authorship?
Drawing Operations: Towards Collaboration
Since 1969, it could be said that we are all living in a Blockworld described by SEEK, overseen by an accidental megastructure fed by memes, gps data, satellite imaging, reddit threads, remix culture, and unfortunate flash mobs. The “landscape” of human extension through networked devices has contributed to the widespread digitization and regurgitation of culture.
Ambiguous authorship has introduced incredible complexity into the network. Inspired by Cohen’s AARON, artists have continued to create visual systems that can produce infinite permutations upon a theme. The linearity of Cohen’s human and machine has extended to form a loop. Now, the conversation has grown to explore what happens to agency, originality, and authorship after we extend ourselves through machines, spawning a new generation of artistic inquiry. Notably, the Hong Kong international exhibition Ubiquitous Humanity explored the boundary between human and the mechanical through collaborative and interactive pieces, such as D.O.U.G.
Drawing Operations Unit: Generation 1 (D.O.U.G.)
I introduced Drawing Operations Unit: Generation 1 (D.O.U.G.) as a performance piece investigating automation, autonomy, and collaboration in 2015. In Generation 1, the robot (D.O.U.G_1) and I draw together. The machine mimics my gestures in real time via computer vision, which results in a synchronous collaboration between human and machine. Set as a 20 minute performance, the demonstration of co-creation builds upon the mechanical extension proposed by Harold Cohen’s AARON and suggests a metaphor for human machine symbiosis through behavioral empathy.
In Generation 1: Mimicry, both human and robotic agent are creatively implicated through the act of performance. As D.O.U.G._1 mimics my movements, we create a visual language and cohesion manifested through a sustained awareness of mechanical and human marks. The project is primarily concerned with ambiguity and interoperability in a machine and computer mediated creative process. We create a feedback loop within the performance that allows for a fluid dialog between the mark made by a human hand and a mark made by machine.
Through this work, I suggest that separation between collaborators is inconsequential. The producer and the impersonator are indistinguishable, and there is no leader or follower as they endlessly follow each other. Ultimately, the authorship of the resulting artwork is un-ascribable. The folie-a-deux’s binding creation may only be attributed by compulsory collaboration, a shared (polycentric) improvisation. In Generation 1, my improvised gesture is an extension of experience, emotion, subjective, moment, impulse control, dexterity, and rhythm translation over time. The robot’s movements are circuitry and servos, with its vision in pixels and coordinates, and electricity driving its movement, but also sight, interpretation, “understanding”, “parsing”, “learning”, and “data sets”.
In an evolutionary progression, Generation 2 of Drawing Operations matures the interaction model of artist and machine from mimicry to memory via machine learning. Gestural data extracted from my archives are fed into training models, simulating a neural network. The training models interpret archives of images, which provide a foundation of a rudimentary understanding of style. Effectively, the robotic arm begins to learn from the style of the artist’s hand throughout time. The system learns the stylistic patterns of its human counterpart, and, in a sense, interprets the history of the artist’s archives, learning to “independently” produce its own conclusions. The performance can be thought of as a collaborative simulation between an artist and her own mechanical doppelgänger. By doing so, Drawing Operations embraces indeterminacy in the age of mechanical production by implicating the machine as artistic collaborator, or possibly, originator. Not only does this place the authorship of the artwork in question, but it also speculates on the necessary evolution of our existing conception of collaboration.
Generation 2 teases at future applications of machine learning intertwined with artistic production – not limited to a single artist, but inclusive of a wider breadth of artistic sources. The machine as collaborator may invent a range of styles beyond the imagination of the human artist, as digitized archives of drawing take on a new life. By cataloguing art history as training data, the machine may be able to forecast, produce and thus originate future movements and styles by tracing and speculating the provocation of artistic development over time.
Overall, the pursuit of multi-threaded agency is poised to stimulate new ways of seeing, sensing and decoding the artistic process.In this framework, the process of creation is as much a part of the artwork as the visual outcome, if not more so. In a sense, the artistic agency questioned within the Drawing Operations project is a composited interoperation. It embraces creative ambiguity in the work, rendering full credit beyond the grasp of human or machine, suggesting the obsolescence of the distinction. In exploring a continuously blurring of distinction between self and machine, Drawing Operations fuels speculation upon inclusive models of radical inter-subjectivities.
What’s to come
Beyond the adversarial binary of human versus machine lies a spectrum of intertwined conceptions of biological and mechanical agencies. In this pursuit, emerging themes will be shaped by emergent interconnectivity. Developments in brain computer interfaces are heralding a generation of responsive prosthetics as cognitive extensions of the mind.In competitive gaming through online communities, human players are competing with learning systems inspired by their own playing styles. By doing so, players are evolving new strategies for competing with machines, as well as extending notions of beauty within gameplay.In tandem, by capitalizing on recent advances in audio codecs and digital signal processing, researchers are exploring ways that haptic feedback can be used to restore perceptions, and perhaps create new ones. Multi-player virtual reality may eventually lead to the normalization of post-geographic communal spaces and extend the idea of co-presence within an abstracted space. Works that are manifold and cooperative, inspired by shared sensorial and spatial experiences, will espouse the imagining of complex new inter-subjectivities.
By examining the co-authorship of human and machine in the context of building, artworks offer allegories for how we can potentially navigate technological change, providing models for how the multi-sensorial data collected from humans, machines and the environment applies to the human experience. Beyond adversarial binaries, towards a promiscuously inclusive, multi-species array of cognizing agents — mechanical and biological, singular and composite, discovered and soon-to-be discovered. We are moving from a mark made by hand and a mark made by machine toward a mark made of something else entirely.
 Harari, Yuval N. Sapiens: A Brief History of Humankind. Toronto, Ontario: Signal, McClelland & Stewart, 2016. Print.
 “Io intendo scultura quella che si fa per forza di levare: quella che si fa per via di porre è simile alla pittura.” a basis for the interpretive translation "I saw the angel in the marble and carved until I set it free." "Lettera a Messer Benedetto Varchi." Lettera a Messer Benedetto Varchi - Wikisource. N.p., n.d. Web. 20 June 2017.
 Vardouli, Theodora. "Nicholas Negroponte: An Interview." Open|architectures. N.p., 26 Oct. 2011. Web. 20 June 2017.
 "Angel (Michelangelo)." Wikipedia. Wikimedia Foundation, 11 June 2017. Web. 20 June 2017.
 Vardouli, Theodora. "Nicholas Negroponte: An Interview." Open|architectures. N.p., 26 Oct. 2011. Web. 20 June 2017.
 Kranzberg, Melvin. "Technology and History: "Kranzberg's Laws"." Technology and Culture27.3 (1986): 544. Web.
 Bratton, Benjamin H. The Stack - On Software and Sovereignty. Massachusetts: MIT, 2016. Print.
 "The Further Exploits of AARON, Painter." The Further Exploits of AARON, Painter. Stanford, 22 July 1995. Web. 23 June 2017. SEHR, volume 4, issue 2: Constructions of the Mind
 The 2016 Hong Kong group exhibition Ubiquitous Humanity comprised of a selection of works curated by Takahashi Mizuki addressing the role of technology in expanding human sensitivity (physical, emotional and behavioral), examining the boundaries between the human and mechanical through visual, collaborative, and interactive experiments.
 Metaphorically, “shared psychosis”, is a psychiatric syndrome whereby its symptoms are shared by more than two people by transmission. Berrios, G. E. & Marková, I. S.. “Shared Pathologies”. In Bhugra D & Malhi G (eds) Troublesome disguises. Managing challenging Disorders in Psychiatry. 2nd Edition, London, Wiley, 2015. pp.3-15.
 DOUG’s mechanical mode of production becomes the same tool as the artist’s tool for expression.
 The readability of massive quantities of data is the focus of the field of machine learning, which has been thriving on the mass data sets generated by humans, and the advancement of processing speeds. Machine learning algorithms are being trained on this data, and the techniques are improving in sophistication. The machine learning of today utilizes neural networks to glean behavioral patterns from the collected data of humans through images and text, which, en masse, form encoded impressions of the collective as a whole.
 DOUG heeds an imagining of machine and human that transitions beyond the conventional species-specific binary of human vs machine, by presenting a ulterior model of interaction derived of multiple artists.
 Xu, Zhe, and Emanuel Todorov. "Design of a Highly Biomimetic Anthropomorphic Robotic Hand towards Artificial Limb Regeneration." 2016 IEEE International Conference on Robotics and Automation (ICRA) (2016): n. pag. Web.
 Clark, Andy, and David J. Chalmers. "The Extended Mind." The Extended Mind (2010): 26-42. Web.
 Metz, Cade. "The Rise of Artificial Intelligence and the End of Code." Wired. Conde Nast, 01 May 2017. Web. 20 June 2017.
 Metz, Cade. "The Sadness and Beauty of Watching Google's AI Play Go." Wired. Conde Nast, 03 June 2017. Web. 20 June 2017.