As I mentioned two posts ago, dealing properly with events has been next on our list. Using events for musical purposes has been on the back-burner previously, largely because we wanted to develop more uses of real-time analysis of continuous EEG signals. But in truly interdisciplinary fashion, the EEGsynth is now being used in a scientific project that includes real-time analysis (and feedback) of heart-rate (yah!). Hopefully Robert will be able to reveal more of that project soon. It has made us reconsider a robust implementation of events by providing some basic user-case scenarios for its use. That’s pretty much the way we work: first imagine a use, and then implement it consistent within the general philosophy and architecture of the EEGsynth. Furthermore, since my own modular synth has grown, I’ve also become more interested in using gates, triggers, beats and rhythms. It might bear repeating that the relationship between the EEGsynth and modular synthesizers is more than practical: The EEGsynth is an interface between the brain-body and sound synthesis, developed as a modular system: something like a hybrid between a soft-synth and a brain-computer-interface. We want to merge your mind with music, patching both ways. So, when we talk EEGsynth, we want to talk modular. We therefor need to translate terms, and get from ‘events’ to the modular language of ‘triggers’, ‘gates’ and ‘latches’.
I use the term ‘event’ to describe the identification of a meaningful moment in time. We can go Bergsonian in this (and perhaps we should at some point), but in this case I only mean it in the boring sense of EEG analysis, e.g. event-related potentials (ERPs): electrical potentials that the brain generates when something specific happens, for instance in direct response to a particular kind of picture, or a sound, or the moment you decide to press a button. Those brain responses are often impossibly hard to decode based on a single occurrence, and are typically repeated hundreds of times to extract them from the ‘noise’ of background brain activity. Approaches using machine-learning algorithms might be a able to catch some ERPs sometimes, and this concept was explored in a very sophisticated way in the sonic art-neuroscience performance Ringing Minds (2014), by Tim Mullen, David Roosenboom, and others. They did not use repetitions over time, but rather multiple brains at the same time to extract significant events from the brains of the participants. Another solution might be to use brains that produce very large brain events spontaneously, such as those of people with epilepsy (as my current research involved exactly that, you can expect some exciting work on that this year). For now, EEG practically just does not provide much to work with when it comes to events, and instead we should look at other electrophysiological signals such as electrocardiography (ECG). As you can see in the animated GIF, with ECG we can clearly identify ‘events’, namely heart-beats, detected by the R-peak of the QRS complex. We know the time between heart-beats, the inter-beat-interval, or it’s inverse, the heart-rate, has physiological meaning (heart-rate variability is a reliable indicator of physiological health), has variation over time, and can be brought to some degree under conscious control via biofeedback, altogether making it a very interesting signal to explore in performances and music.
Now to step into the world of modular synthesizers, where we deal with ‘triggers’, ‘gates’ and ‘latches’. Whether it be the pressing of a key on a keyboard, the output of a clock, the passing of an audio-threshold, when something has to start, a trigger is used. A trigger is a short electrical pulse (typically 5-10v for about 3ms) that can start an envelope, proceed to the next step in a sequencer, provide a tick for another clock, etc. What is important in terms of implementation in the EEGsynth, is to realize that a trigger has no duration. So as a way to code an event, it only codes for the beginning (or end) of an event. Gates, however, do have duration (I know, it’s becoming increasingly Bergsonian). They are used exactly for that purpose: to code for the duration of events by being open (5-10v) during the event, and closed (0v) otherwise. A gate is used e.g. to code the output of logical operators (TRUE/FALSE), whether or not an LFO is cycling or not, whether a sample-loop is playing, etc. Because of the way triggers are implemented, gates can often be used instead, and many modular wigglers might not care about the difference (if it works, it works). For the EEGsynth, though, the implementation is very different indeed. Before I spend some time on that, a quick word on latches. Latches are less common in modular parlance, but again interesting as another example of implementing binary-temporal codes (my term), as a flip-flop gate, or toggle: it will turn from on to off, and from off to on in response to a trigger/gate, typically a manual button-press. It is therefor a combination of a gate (describing its current state) and a trigger (events that change it’s state).
Now we have finally set the stage: with on the one side the modular world of gates and trigger, and on the other the EEGsynth. As you might know (and if not, you can read up on our documentation here), we implement the other side of modular synthesizer, it’s control signals (or rather Control Voltages, or CVs), in a (Redis) database. This database has key-value pairs, with the name of the control signal (e.g. ‘alpha’) and a value between 0 and 1. These values are read and updated by the modules in real-time. At any point in time, you are able to read the state of all the (CV) control values by reading the content of that database. It would be the same as taking a multimeter and measuring the currents on all the wires on a modular synthesizer at the same time! The reason that this is such a good way of implementing control values, is that it’s asynchronous: modules don’t have to read and update the values continuously, only whenever they need the information, or whenever they have a value to update. This is important to understand. It’s the core of the EEGsynth, and why it is able to run many modules at the same time, at different rates (some are complex and slow, some are simple and fast, some care, some don’t), without having to wait for each other: they always get the latest update, whenever they want. I’m sure there is a great metaphor there (Robert – I am looking at you), but let me continue asynchronously, while you make one up in your mind, and I’ll see you further down this sentence when you’re ready. At this point you might intuit that this way of working with control signals (CV) would work just as well for gates. And you would be absolutely right! But what about triggers? The whole point of triggers is to be precise in time – their only function is to indicate when something happened (or has to happen).
The solution to this is to use another way of communicating – rather than having a general repository to put or get information, we can use a pub/sub way of messaging between modules. ‘Pub’ stands for publishing, while ‘sub’ stands for subscribe. So rather than modules updating a value in a general database, they can give a shout out “Heartbeat trigger detected, read all about it!”, to whichever module subscribed to it. This makes the communication near instant, i.e. synchronous. A module that needs to respond to a heartbeat would therefor not have to keep looking at the database as often as possible, slowing everybody down, but just subscribe to the module publishing those events, and wait until they get news so that they can respond instantly. And that, my friend, is how we implement triggers. Let me wrap up for now. In the animated GIF you see the measurement of the ECG in the bottom (using the module plotsignal), with the events plotted above with the new module plottrigger. In addition, we just release a clockdivider and clockmultiplier, to speed up or divide those clock signals. In a next post I will show how you can use it with the sequencer module to create more complex and interesting rhythms. Until then – subscribe!