Affective VR Workshop
Welcome
Welcome to the Emotional Data and Unity/VR workshop! This workshop is hosted by Erin Gee, a Canadian artist and researcher who has worked in emotional biodata and art since 2012. She has created work for emotional biodata and robotics (Swarming Emotional Pianos, 2014), children’s choir (Song of Seven, 2017), and now for VR with her latest work Project H.E.A.R.T (2017).
She is an active promoter of open-source and feminist culture, and publishes all of her technical work (Arduino code/Unity code) under the GNU GPL 3.0.
What is the GNU GPL?
The GNU General Public License is a free, copyleft license for software and other kinds of works.
TLDR: You may use, modify, redistribute this code for free. If you redistribute, you must acknowledge my original authorship, and you must always allow other people to also modify and redistribute for free. Any violation of this agreement means that you are breaking my copyright! If you do modify aspects of this code, please share the love and contribute to the GitHub.
If you are not sure how to contribute to a GitHub project, feel free to contact me at erin dot marie dot gee at gmail dot com (or just the contact form on my website) and I’ll set you up!
BIOSENSORS IN GAMES
April 13th, 2011
Jacob Aron first reported on a variety of games that were taking advantage of biosensing technologies in an article published in New Scientist.
Aron, Jacob. (2011). “Emotional Video Gaming Makes the Action Real.” New Scientist. Accessed November 15th 2017.
October 2016 – BfB Labs’ emotionally responsive game “Champions of the Shengha,” is dependent on a user’s emotional control — measured by a heart rate sensor — for success.
October 2016 – Nevermind is an adventure game where you explore strange worlds and solve puzzles to unlock a mystery that lurks within each “patient’s” inner psyche. The Windows and Mac versions of Nevermind use biofeedback technology to detect your feelings of stress while playing, dynamically responding to those feelings to affect gameplay. You can also play the game without this technology. http://nevermindgame.com/
BIOSENSORS IN CINEMA
Published by The Verge on 18 Jul 2017.
Lauren Goode goes inside Dolby’s little-known biophysical labs, where the company has been embarking on a five-year project to track people’s emotional responses as they watch movies and TV shows.
Biosensors are used by Dolby to study viewers’ emotional responses to
- Aural frequency ranges
- Dynamic color ranges
- Audio volume as well as screen brightness
- Music and sound effects
VR and Empathy – A reading list
Ebert, Roger. (June 23, 2005.) “Ebert’s Hall of Fame Remarks.” Roger Ebert’s Journal. Accessed November 15th 2017.
Bye, Kent. (January 31, 2017). “VR as the Ultimate Empathy Machine with Gabo Arora.” Voices of VR Podcast. Accessed November 13th 2017.
Hamilton, Kathryn. (February 23, 2017). “Voyeur Reality.” The New Inquiry. Accessed November 15th 2017.
Yang, Robert. (April 5, 2017). “If you walk in someone else’s shoes, then you have taken their shoes”: empathy machines as appropriation machines.” radiator design blog. Accessed November 15th 2017.
Scientific Resources
In this section you will find information on the science behind how emotion is generated by the brain, and how it can be “read” by sensing instruments.
What is Emotion
Emotion is a multi-component response to an emotionally potent event, causing changes in subjective feeling quality (psychological dimension), expressive social behavior (behavioral dimension), and physiological activation (physiological dimension) of the subject. (Kreiberg 2009)
Psychological/neurological frameworks for understanding emotion itself are articulated very well by Dr Lisa Feldman-Barrett, a scientist at Northeastern University. She speaks particularly of the complexity of the human body, which might be experiencing a host of physiological effects from the ANS system, and how these are interpreted and perhaps also constructed by language.
On her website you will find many plain-language resources for emotional understanding from a contemporary neuroscientific point of view under the heading “Articles.”
How Scientists detect Emotion
The physiological techniques presented at this workshop were made according to psychologist Dr. Sylvia Kreibig‘s 2010 review of 134 publications dating from the 1970s to the 2000s. While the techniques viewed in this document are not weighted for “success” of the cited studies, and the literature clearly shows that there is no one “technique” for applying these skills, this document that has been cited over 1000 times in scientific literature since its publication, including over 200 times in 2017.
Source:
Continued research
If you would like to continue keeping up with scientific research in this area, an academic journal search using the databases PsycINFO, PsycARTICLES, and PubMed is recommended with the following search terms:
[emotion] and [autonomic nervous system or cardiovascular or cardiac or heart or respiration or respiratory or electrodermal or skin conductance]
Unity Tutorials
This section is dedicated to technical resources, tutorials and code to get you started playing around with the biosensors.
Mirza VFX teaches amazing tutorials in particle creation. Plug your biosensors in to manipulate beautiful particle-driven environments.
(Inspiration: Mufson, Beckett. (Aug 29, 2014) “A Team Of Artists Are 3D-Printing Their Emotions As Abstract House Decorations.” Creators Project. Accessed November 15th 2017.)
GitHub
Hardware Resources
Materials list for Erin Gee’s BiodataTrio PCB board.
Don’t forget to prepare your PulseSensor before you use it! Click here for instructions on how to apply the protective decal and provide a gluegun seal. The rest of the advice you can take or leave, but these two steps are ESSENTIAL to the longevity and accuracy of your device!
When buying cables to build your sensors – Look for cable that features 2 signals that are shielded. The biosensors are picking up on very, very sensitive electrical information from the body – any disruption in the electric field could make your readings less accurate. Especially if the sensor cables are placed near to one another!
To prevent this, you can either buy shielded cable (like this) and solder the silver shielding to your ground connection (pick whatever you like for the other two, maybe red for power and black for signal?)
Or if you’re in a pinch, you can just twist the ground wire around the signal wire that you are trying to protect from outside interference.
Here’s a link to my Respiration Belt Instructable. After a few years I didn’t find that a respiration belt was as interesting to me because the belts are awkward to strap into for the average person, but if you’d like to go for it, here is a simple way to make it happen! This signal is perhaps best amplified, and you might need to calculate the relative drift of the elastic and account for it as an offset in order to capture things like when someone is holding their breath.