Robotics/AI/XR/Quantum

Machine Unlearning

Vision calibration from Machine Unlearning (2020).
Photography by Elody Libe. Image courtesy of the artist.

2020

In Machine Unlearning, the artist greets the viewer and slowly offers them a unique neural conditioning “treatment”: sonically reproducing the unraveling outputs of an LSTM algorithm as it “unlearns” through whispering, moving backwards in time through its epochs of training.

This aural treatment is couched in a first-person roleplay scenario that grounds the viewer through a series of simple audio visual tests. At no point is the neural network technology “seen” – it is instead performed by a human interlocuter, translated into affective vocality and whispered text. The algorithm was created by media artist Sofian Audry, and trained on the text of Emily Brontë’s novel Wuthering Heights (1847). This novel was chosen in part because of its richly poetic syntax, but also for its feminine vocality and conceptual themes of love and intergenerational trauma. Machine Unlearning is a novel combination of neural network technologies and the popular internet genre “Autonomous Sensory Meridian Response,” or ASMR. ASMR is a social media genre that has developed largely through massive social media metrics in the form of upvotes, clicks, comments, subscribes, and likes in response to audio visual stimuli that creates feelings of mild euphoria, relaxation and pleasure. ASMR fans online seek out specific video content that causes the physiological reaction of “tingles” – tingling sensations across the skin, a mild body high, or simply a means of falling asleep. Gee considers ASMR as a form of psychosomatic body hacking. By combining machine learning with ASMR, Gee draws parallels between cutting edge autonomous/non-conscious algorithms and the autonomous/unconscious functions of the human body. Just as ASMRtists use specific sounds and visual patterns in their videos to “trigger” physical reactions in the viewer, machine learning algorithms also unconsciously respond to patterns perceived through limited senses in order to develop learning (and unlearning) results. The artist’s emphasis on whispering the textual outputs of the algorithm as it slowly “unlearns” allows the listener to grasp the materiality of machine learning processes at a human level, but also a subconscious level: allowing one’s body to be mildly and charmingly “hacked” through soft and gentle play.

The use of the word “intelligence” in the metaphor of AI focuses on higher functions of consciousness that algorithms do not possess. While algorithms have not meaningfully achieved a humanistic consciousness to date, today’s algorithms act autonomously on sensory information, processing data from its environment in unconscious, automatic ways. The human brain also responds unconsciously and automatically to sensory data in its environment, for example, even if you are not conscious of how hot a stove is, if you place your hand on a hot stove, your hand will automatically pull away. These unconscious, physiological actions in the sensory realm points to an area of common experience between algorithms and the human.  For more explanation of these ideas, take a look at the work of postmodern literary critic N. Katherine Hayles in her 2017 book Unthought: The power of the cognitive nonconscious.  In this way I wonder if the expression “autonomous intelligence” makes more sense than “artificial intelligence”, however like posthumanist feminist Rosi Braidotti I am deeply suspicious of the humanist pride that our species takes in the word “intelligence” as something that confers a special status and justification for domination of other forms of life on earth.

Credits

Photography and videography by Elody Libe.

Production Support: Machine Unlearning video installation was produced at Perte de Signal with the support of the MacKenzie Art Gallery for the exhibition To the Sooe (2020) curated by Tak Pham.

The roleplay performance was developed during my artistic residency at Locus SonusÉcole Superieur d’art d’Aix en Provence and Laboratoire PRISM.

Custom LSTM Algorithm created by media artist Sofian Audry

Video

Machine Unlearning (2020)
Videography by Elody Libe

Gallery

This work was first developed as a performance that debuted at Cluster Festival, Winnipeg in 2019.  During live performance, each audience member dons a pair of wireless headphones.  The performance allows the audience members to see the ASMR “result” of the performance for camera, simultaneous with the ability to see my “backstage” manipulation of props and light in real time.

to the sooe

to the sooe (2018)
Sofian Audry and Erin Gee. Photography: Alexandre Saunier

2018

A 3D printed sound object that houses a human voice murmuring the words of a neural network trained by a deceased author.

to the sooe (SLS 3D printed object, electronics, laser-etched acrylic, audio, 2018) is the second piece in a body of work Erin Gee made in collaboration with artist Sofian Audry that explores the material and authorial agencies of a deceased author, a LSTM algorithm, and an ASMR performer.

The work in this series transmits the aesthetics of an AI “voice” that speaks through outputted text through the sounds of Gee’s softly spoken human vocals, using a human body as a relatively low-tech filter for processes of machine automation.  Other works in this series include of the soone (2018), and Machine Unlearning (2018-2019)

to the sooe is a sound object that features a binaural recording of Erin Gee’s voice as she re-articulates the murmurs of a machine learning algorithm learning to speak. Through this work, the artists re-embody the cognitive processes and creative voices of three agents (a deceased author, a deep learning neural net, and an ASMR performer) into a tangible device. These human and nonhuman agencies are materialized in the object through speaking and writing: a disembodied human voice, words etched onto a mirrored, acrylic surface, as well as code written into the device’s silicon memory.

The algorithmic process used in this work is a deep recurrent neural network agent known as “long short term memory” (LSTM). The algorithm “reads” Emily Brontë’s Wuthering Heights character by character, familiarizing itself with the syntactical universe of the text. As it reads and re-reads the book, it attempts to mimic Brontë’s style within the constraints of its own artificial “body”, hence finding its own alien voice.

The reading of this AI-generated text by a human speaker allows the listener to experience simultaneously the neural network agent’s linguistic journey as well as the augmentation of this speech through vocalization techniques adapted from Autonomous Sensory Meridian Response (ASMR). ASMR involves the use of acoustic “triggers” such as gentle whispering, fingers scratching or tapping, in an attempt to induce tingling sensations and pleasurable auditory-tactile synaesthesia in the user. Through these autonomous physiological experiences, the artists hope to reveal the autonomous nature of the listener’s own body, implying the listener as an already-cyborgian aspect of the hybrid system in place.

Credits

Sofian Audry – neural network programming and training

Erin Gee – vocal performer, audio recording and editing, electronics

Grégory Perrin – 3D printing design and laser etching

Exhibition history

Taking Care – Hexagram Campus Exhibition @ Ars Electronica, Linz Sept 5-11 2018. Curated by Ana Kerekes.

Printemps Numérique – McCord Museum Montreal, May 29-June 3 2019. Curated by Erandy Vergara.

To the Sooe – MacKenzie Art Gallery, Regina January 26-April 26, 2020. Curated by Tak Pham.

Sounds

to the sooe (2018)

Gallery

of the soone

of the soone (2018) Print

2018

A disembodied voice invites the listener to partake in a speculative audio treatment that promises to awaken underdeveloped neural passageways through exposure to the non-human processes of neural network language acquisition.

In this work, media artists Erin Gee and Sofian Audry expose listeners to the architectures of an artificial intelligence algorithm through the sounds of an Autonomous Sensory Meridian Response (ASMR) roleplay. ASMR is a genre of audio and videomaking developed by internet aficionados interested in using specific everyday sounds (whispering, soft voice, crinkling and textured sounds) alongside verbal suggestion to “trigger” pleasant tingling reactions in the body of the listener. The artists combined these ASMR principles of sound with artificial intelligence to create a speculative neural conditioning treatment. In of the soone, the listener encounters a soft female voice that whispers a script written by a machine learning algorithm as it slowly loses its neural training and “forgets.” This combination of algorithmic text and ASMR connects the unconscious, automatic processes of artificial intelligence algorithms to the autonomous reactions of the human body to sound, using intimacy to “hack” into the subconscious of the human listener and recondition neural pathways.

Exhibition history

October 2020: Digital Cultures: Imagined Futures Audio Programme curated by Joseph Cutts. Adam Mickiewicz Institute, Warszawa Poland

June 9 to August 19, 2018: Pendoran Vinci. Art and Artificial Intelligence Today  curated by Peggy Schoenegge and Tina Sauerländer. NRW Forum, Düsseldorf, Germany

January 2018: Her Environment @ TCC Gallery, Chicago

Sounds

of the soone (2018)

Gallery

of the soone – print. text 2018. Courtesy of artists.

Project H.E.A.R.T.

Project H.E.A.R.T. (2017)

2017

A biodata-driven VR game where militainment and pop music fuel a new form of emotional drone warfare.

A twist on popular “militainment” shooter video games, Project H.E.A.R.T. invites the viewer to place their fingers on a custom biodata device, and summon their enthusiasm to engage their avatar, Yowane Haku, in “combat therapy.” Fans of the Vocaloid characters may recognize Haku as the “bad copy” of Japanese pop celebrity Hatsune Miku, a holographic personnage that invites her fans to pour their content and songs into her virtual voice.

The biosensing system features a pulse sensor, and a skin conductance sensor of Gee’s design. Through principles of emotional physiology and affective computing, the device gathers data relative to heart rate and blood flow from index finger, and skin conductance from middle and ring fingers of users. The biodata is read by a microcontroller and transferred to Unity VR, thus facilitating emotional interactivity: a user’s enthusiasm (spikes in signal amplitude in skin conductance, elevated heart rate, and shifts in amplitude of the pulse signal) stimulates the holographic pop star to sing in the virtual warzone, thus inspiring military fighters to continue the war, and create more enemy casualties. At the end of the experience the user is confronted with their “score” of traumatized soldiers vs enemies killed, with no information whether this means that they won or lost the “game”.

The user is thus challenged to navigate soldier’s emotional anxieties and summon their positivity to activate Haku’s singing voice as soldiers battle not only against a group of enemies, but also against their own lack of confidence in times of global economic instability.

The landscape of Project H.E.A.R.T. was built from geopolitically resonant sites found on Google Maps, creating a dreamlike background for the warzone. In-game dialogue wavers between self-righteous soldier banter typical of video games, and self-help, bringing the VR participant to an interrogation of their own emotional body in a virtual space that conflates war, pop music, drone technology, and perhaps movement-induced VR nausea.

As Kathryn Hamilton pointed out in her 2017 essay “Voyeur Realism” for New Inquiry,

“VR’s genesis and development is in the military, where it has been used to train soldiers in “battle readiness,” a euphemism for: methods to overcome the innate human resistance to firing at another human being. In the last few years, VR’s usage has shifted 180 degrees from a technology used to train soldiers for war, to one that claims to “amplify” the voices afflicted by war, and to affect “world influencers” who might be able to stop said wars.”

Credits

Narrative Design: Sofian Audry, Roxanne Baril-Bédard, Erin Gee
3D Art: Alex Lee and Marlon Kroll
Animation and Rigging: Nicklas Kenyon and Alex Lee
VFX: Anthony Damiani, Erin Gee, Nicklas Kenyon
Programming: Sofian Audry, Erin Gee, Nicklas Kenyon, Jacob Morin
AI Design: Sofian Audry
Sound Design: Erin Gee, Austin Haughton, Ben Hinckley, Ben Leavitt, Nicolas Ow
BioSensor Hardware Design: Erin Gee and Martin Peach
BioSensor Case Design: Grégory Perrin
BioSensor Hardware Programming: Thomas Ouellet Fredericks, Erin Gee, Martin Peach
Featuring music by Lazerblade, Night Chaser and Austin Haughton
Yowane Haku character designed by CAFFEIN
Yowane Haku Cyber model originally created by SEGA for Hatsune Miku: Project DIVA 2nd (2010)
Project H.E.A.R.T. also features the vocal acting talents of Erin Gee, Danny Gold, Alex Lee, Ben McCarthy, Gregory Muszkie, James O’Calloghan, and Henry Adam Svec.

Thanks to the support of the Canada Council for the Arts and AMD Radeon, this project was commissioned by Trinity Square Video for the exhibition Worldbuilding, curated by John G Hampton and Maiko Tanaka.

This project would have not been possible without the logistical and technical support of the following organizations:

Technoculture Art and Games Lab (Concordia University)

Concordia University

ASAP Media Services (University of Maine)

Exhibition history

November-December 2017  Worldbuilding @ Trinity Square Video, Toronto

February-March 2018 Future Perfect @ Hygienic Gallery, New London Connecticut

April 26-28, 2018 @ Digifest, Toronto

June 7-17, 2019 @ Elektra Festival, Montreal

January 2020 @ The Artist Project, Toronto

October 2020 @ Festival LEV Matadero, Spain

Links

Project H.E.A.R.T. official website
Worldbuilding Exhibition Website
Review in Canadian Art
My research blog: Pop and Militainment
Featured on Radiance VR

Video

Project H.E.A.R.T (2017)
Installation and Gameplay

Gallery

Erin Gee - Swarming Emotional Pianos

Swarming Emotional Pianos

Swarming Emotional Pianos (2012 – ongoing)
Aluminium tubes, servo motors, custom mallets, Arduino-based electronics, iCreate platforms
Approximately 27” x 12” x 12” each

2012

A looming projection of a human performer surrounded by six musical chime robots: their music is driven by the shifting rhythms of the performer’s emotional body, transformed into data and signal that activates the motors of the ensemble.

Swarming Emotional Pianos is a robotic installation work that features performance documentation of an actress moving through extreme emotions in five minute intervals. During these timed performances of extreme surprise, anger, fear, sadness, sexual arousal, and joy, Gee used her own custom-built biosensors to capture the way that each emotion affects the heartbeat, sweat, and respiration of the actress. The data from this session drives the musical outbursts of the robotics surrounding the video documentation of the emotional session. Visitors to this work are presented with two windows into the emotional state of the actress: both through a large projection of her face, paired with stereo recording of her breath and sounds of the emotional session, and through the normally inaccessible emotional world of physiology, the physicality of sensation as represented by the six robotic chimes.

Micro bursts of emotional sentiment are amplified by the robots, providing an intimate and abstract soundtrack for this “emotional movie”. These mechanistic, physiological effects of emotion drive the robotics, illustrating the physicality and automation of human emotion. By displaying both of these perspectives on human emotion simultaneously, I am interested in how the rhythmic pulsing of the robotic bodies confirm or deny the visibility and performativity of the face. Does emotion therefore lie within the visibility of facial expression, or in the patterns of bodily sensation in her body? Is the actor sincere in her performance if the emotion is felt as opposed to displayed?

Custom open-source biosensors that collect heartrate and signal amplitude, respiration amplitude and rate, and galvanic skin response (sweat) have been in development by Gee since 2012.  Click here to access her GitHub page if you would like to try the technology for yourself, or contribute to the research.

Credits

Thank you to the following for your contributions:

In loving memory of Martin Peach (my robot teacher) – Sébastien Roy (lighting circuitry) – Peter van Haaften (tools for algorithmic composition in Max/MSP) – Grégory Perrin (Electronics Assistant)

Jason Leith, Vivian Li, Mark Lowe, Simone Pitot, Matt Risk, and Tristan Stevans for their dedicated help in the studio

Concordia University, the MARCS Institute at the University of Western Sydney, Innovations en Concert Montréal, Conseil des Arts de Montréal, Thought Technology, and AD Instruments for their support.

Videos

Swarming Emotional Pianos (2012-2014)
Machine demonstration March 2014 – Eastern Bloc Lab Residency, Montréal

Swarming Emotional Pianos (2012-2014)
Machine demonstration March 2014 – Eastern Bloc Lab Residency, Montréal

Gallery

Swarming Emotional Pianos