A twist on popular “militainment” shooter video games, Project H.E.A.R.T. invites the viewer to place their fingers on a custom biodata device, and summon their enthusiasm to engage their avatar, Yowane Haku, in “combat therapy.” Fans of the Vocaloid characters may recognize Haku as the “bad copy” of Japanese pop celebrity Hatsune Miku, a holographic personnage that invites her fans to pour their content and songs into her virtual voice.
The biosensing system features a pulse sensor, and a skin conductance sensor of Gee’s design. Through principles of emotional physiology and affective computing, the device gathers data relative to heart rate and blood flow from index finger, and skin conductance from middle and ring fingers of users. The biodata is read by a microcontroller and transferred to Unity VR, thus facilitating emotional interactivity: a user’s enthusiasm (spikes in signal amplitude in skin conductance, elevated heart rate, and shifts in amplitude of the pulse signal) stimulates the holographic pop star to sing in the virtual warzone, thus inspiring military fighters to continue the war, and create more enemy casualties. At the end of the experience the user is confronted with their “score” of traumatized soldiers vs enemies killed, with no information whether this means that they won or lost the “game”.
The user is thus challenged to navigate soldier’s emotional anxieties and summon their positivity to activate Haku’s singing voice as soldiers battle not only against a group of enemies, but also against their own lack of confidence in times of global economic instability.
The landscape of Project H.E.A.R.T. was built from geopolitically resonant sites found on Google Maps, creating a dreamlike background for the warzone. In-game dialogue wavers between self-righteous soldier banter typical of video games, and self-help, bringing the VR participant to an interrogation of their own emotional body in a virtual space that conflates war, pop music, drone technology, and perhaps movement-induced VR nausea.
As Kathryn Hamilton pointed out in her 2017 essay “Voyeur Realism” for New Inquiry,
“VR’s genesis and development is in the military, where it has been used to train soldiers in “battle readiness,” a euphemism for: methods to overcome the innate human resistance to firing at another human being. In the last few years, VR’s usage has shifted 180 degrees from a technology used to train soldiers for war, to one that claims to “amplify” the voices afflicted by war, and to affect “world influencers” who might be able to stop said wars.”
Photography by Toni Hafkenscheid. Images of Worldbuilding exhibition courtesy of Trinity Square Video, 2017.
November-December 2017 Worldbuilding @ Trinity Square Video, Toronto
February-March 2018 Future Perfect @ Hygienic Gallery, New London Connecticut
April 26-28, 2018 @ Toronto Digifest, Toronto
Narrative Design: Sofian Audry, Roxanne Baril-Bédard, Erin Gee
3D Art: Alex Lee and Marlon Kroll
Animation and Rigging: Nicklas Kenyon and Alex Lee
VFX: Anthony Damiani, Erin Gee, Nicklas Kenyon
Programming: Sofian Audry, Erin Gee, Nicklas Kenyon, Jacob Morin
AI Design: Sofian Audry
Sound Design: Erin Gee, Austin Haughton, Ben Hinckley, Ben Leavitt, Nicolas Ow
BioSensor Hardware Design: Erin Gee and Martin Peach
BioSensor Case Design: Grégory Perrin
BioSensor Hardware Programming: Thomas Ouellet Fredericks, Erin Gee, Martin Peach
Featuring music by Lazerblade, Night Chaser and Austin Haughton
Yowane Haku character designed by CAFFEIN
Yowane Haku Cyber model originally created by SEGA for Hatsune Miku: Project DIVA 2nd (2010)
Project H.E.A.R.T. also features the vocal acting talents of Erin Gee, Danny Gold, Alex Lee, Ben McCarthy, Gregory Muszkie, James O’Calloghan, and Henry Adam Svec.
Thanks to the support of the Canada Council for the Arts and AMD Radeon, this project was commissioned by Trinity Square Video for the exhibition Worldbuilding, curated by John G Hampton and Maiko Tanaka.
This project would have not been possible without the logistical and technical support of the following organizations: