Atau Tanaka and I presented our work on EMG-EEG hybridization for Brain-Body Digital Music Instrument (BBDMI) during this year’s ICLI 2022, on June 20th – 23rd, 2022. It was a blast! The presentations and performances explored digital art, computation and their relationship to human thought, experience and art. The conference was a hybrid live-remote one, including the performances. As such a conference behoves, it turned out to be a fully embodied experience, thanks to the fantastic organizational and technical skills of the organizers and supporting staff.
The conference has now been fully documented and recorded on the ICL conference website. I really recommend you take a stroll through the many interesting talks and performances. You can find Atau’s and my contribution under the showcases here, and embedded below.
For our part, this was a chance to explore ways in which we can hybridize EEG and EMG signals as part of the BBDMI project. It also posed a unique user case scenario and performative configuration in which Atau was performing a sound piece he composed for EMG, using Max/MSP to extract EMG features from two Myo armbands, and created live sound synthesis and modulation. At the same time, he streamed that audio to a JackTrip virtual server, to which I was connected from Paris. That stereo audio stream was then send to my modular synthesizer. At the same time, the EMG features were also send over internet as OSC messages, which I retrieved on my computer in Pure Data, and then send as DC-audio to my modular synthesizer (using the Expert Sleepers’ ES-9 module). At the same time, I was recording my EEG in real-time with the Mentalab Explore, using our EEGsynth to extract EEG features (oscillatory band power) that I then send to my modular synthesizer as DC-audio as well. Everything thus came together in the modular synth, where I live-patched, using EMG features (Atau) and EEG features (me), to modulate Atau’s audio stream (e.g. using the AJH Synths’s Gemini 2412 Dual Voltage Controlled Filter module), as well as other EEG/EMG controlled sound sources. The resultant stereo audio stream was then routed back, through a Eventide Space pedal, into my desktop computer running Open Broadcaster Software (OBS), in which I mixed live EEG signals, live visualization from the EEGsynth, the Pure Data patch receiving Atau’s OSC signals, and video of the modular synth patching from a second video camera. This was then all streamed as an audio-video collage to the ICLI conference in Lison, and online, using Zoom. You can find the EEGsynth patch on the EEGsynth repository, together with some additional technical details, e.g. about the hardware used.
As I said, for streaming low-latency high-quality audio over internet we used Jack and JackTrip. We would not have been able to achieve this without the support of Synthia Payne, who patiently and enthousiastically helped me set it up on Windows, and provided us with a deeper understanding and expertise in dealing with all manner of glitches and confusions. Jack and JackTrip are remarkable tools for high-quality and low-latency audio that allows online/virtual musical collaborations. I don’t know how they do it, or how it works, but it sure does.
We would like to thank and congratulate Adriana Sá and CICANT at Universidade Lusófona in Lisbon, Portugal, for organizing such an amazing event, and of course all the other participants for their enriching thoughts, presentations, performances and discussions. Many thanks and appreciation also for the rechnical staff: Nikolas Gomes, Afonso Sekkas, André Silva and Margarida Saraiva, and the film-maker: Stella Carneiro. As you can see, the website brings together more than just documentation, but rather the beginning of a platform for further discussion and exploration. In the words of Adriana Sá:
My main goal is to give you the opportunity to see/hear all contributions, at your own pace – and perhaps discern unexpected threads amongst the wealth of different questions and approaches. I’m thinking of that as a pre-preparation for the following milestone in this research project. Its format and contents are of course not really defined yet, but as you know, the idea is to make a peer-reviewed online publication, gathering essays, sonic compositions, photo narratives, videos and and so on.
Adriana Sá in email correspondance
We are proud to have been able to share our work at ICLI, and to feel a part of this great community of explorers.