#csound #interactive_instruments #myo #ngimu #verkstedhallen #dansit
All of the interactive instruments were programmed using Csound. The data from the VIBRA-hub were received via an Ethernet connection for minimal latency, using the OSC implementation in Csound (OSClisten opcode). The setup in Csound was flexible in that data from all sensors could be routed to the different instruments, and also in that every instrument could be set up with different parameters to create variations in their sonic qualities. However, due to the particular nature and structure of the EMG data, some instruments were designed particularly for the data the Myo provided.
Four simple instrument sketches were developed prior to the first workshop, whereas the majority of the instrument development happened in response to what happened in the workshops, in line with the exploratory nature of the workshops. Therefore, many of them also have an ad hoc quality, often prioritizing quick and workable solutions over clear concepts, processing methods and code. A total of eight instruments were used in the workshops, where five used different methods of sound synthesis and three used different kinds of processing of sampled sounds. The instruments varied in complexity, from the very simple to the more complex. These eight instruments are described in the following, along with video examples of their use:
WarbleSine
MultiSine
MinceTracer
Squine
Pulzer
Noizer
Bipp-a-chu
Flow
1) WarbleSine
Simple additive sine-wave synthesis instrument with a warbling quality played with the Myo armband
The eight RMS values from the EMG electrodes were firstly mapped to the amplitude of eight oscillators, each playing a partial related to the same fundamental frequency. The fundamental could be change from instance to instance for variation. The warbling nature of the RMS signals gave the instrument a warbling quality, also with small glitches when the values changed rapidly. This instrument was introduced at the first day of the first VIBRA workshop. Although the instrument felt very crude and simplistic, the dancers liked it. It was slightly modified before the second day, when Bayes filtering (using the pipo Bayesfilter that comes with the MuBu package) was added, and the warbling had a smoother quality to it.
Video: The dancers are Tone Pernille Østern and Arnhild Staal Pettersen, who in this clip improvise freely for the first time with this instrument. Thus, this is the first and more "glitchy" versions of the instrument. Apologies for poor sound quality and for talking during the recording.
2) MultiSine
Simple additive sine-wave synthesis instrument played with the Myo armband
This instrument could be considered a variant of WarbleSine, but due to the different playability and character, it was eventually considered an instrument in its own right. In the same way as the WarbleSine the signals from the eight EMG electrodes of the Myo armband were Bayes filtered using the pipo Bayesfilter.The filtered signals were then passed through an onset-detector, which produced triggers when the value crossed a certain threshold(0.06). Here, a mechanism which prevented rapid re-triggering was implemented. When an onset was detected, the values would be scaled so as to create a momentary impulse/peak. The decay of these impulses were then smoothed with a decay time of four seconds, thus creating an impulse - resonance morphology for each value (Smalley, 1997). These values were then emplyed to control the amplitude of 8 oscillators, all tuned to one randomly chosen partial (between 0 and 5) of a fundamental frequency. The fundamental frequency could be changed between different instances of the instrument.
The instrument tended to be experienced by experienced by the dancers as having relatively indirect sense of causality. Moreover, the dancers found it somewhat static and minimalistic in character, something which sometimes created a nearly hypnotic effect. It was only used during the first workshop, including the showing before an audience at the end of the workshop.
Video: The dancers here are Tone Pernille Østern, who does her solo first, and Elen Øien, who overlaps with her, and then continues with her solo. This clip is from the showing of the first workshop.
3) MinceTracer
Sample playback instrument with partial extraction and spectral blurring played with the Myo armband
The instrument was based on sample playback using the mincer opcode, which is a phase vocoder/FFT-based opcode which gives independent control over transposition and playback position in the file. Playback was subsequently processed d with a partial-tracing algorithm, which retains the N strongest partials. After smoothing the output of this process, the FFT was resynthesized and sent to output.
In this instrument, three different kinds of parameters and treatments of these are used to control different aspects of this process:
An intensity value derived from the normalized accelerometer values (based on IRCAM's algorithm in the RIoT intensity object). This value was used to control the number of partials that were allowed through the partial tracing algorithm.
The sum of Bayes filtered RMS values for all EMG sensors of the Myo armband. This value was mapped to the global amplitude of the instrument.
A scaled x-axis value from the gyro is used with an accumulator to control the reading time pointer to the playback of the sound file.
The dancers thought this instrument had a "scary" quality. It was only used in the first VIBRA workshop.
Video: Two subsequent solos featuring Tone Pernille Østern and Elen Øien from the first VIBRA workshop in March 2018.
4) Squine
Synthesis instrument using a mostly bandlimited shape-shifting square-pulse-saw-sinewave oscillator with hardsync
The squinewave opcode used in this instrument generates a variable shape waveform that can morph freely between classical shapes sine, square, pulse and saw by manipulating a skew and a clip parameter, where the former determines its symmetry and the latter the "squareness" of the waveform. The pitch of the oscillator was limited to two tones a fifth apart, but with a gliding transition in between.
This instrument uses the same three parameters that the MinceTracer instrument, but with the following mapping:
1) The intensity value is here used to control both the amplitude of the instrument
2) The summed EMG values are here used to control the pitch of the oscillator.
3) The gyro value was used to control the skew of the waveform.
The result was an instrument with a bee-like quality with quite limited expressivity. It was tested by a couple of the dancers, but was soon found annoying and abandoned.
Video: The video shows a playful try-out by Luis Della Mea.
5) Pulzer
Synthesis instrument with an event generator creating notes for a sawtooth wave oscillator
The Pulzer instrument generated rhythmic streams of notes with varying pitches and tempo. It had a possibility of choosing between different variants with different pitch sets with either a minor, major or more ambiguous flavor. It could be played both the Myo and NGIMU sensors.
This instrument uses:
1) The intensity value (as described above) to choose among a pitch set of 8 different pitches, with higher intensity giving a higher pitch.
2) The gyro value was used to control the tempo of the note stream. Changes in the gyro values - caused by general movement - would cause an increase in tempo, up to a point in which tempo would suddently fall down to a base level again.
The Pulzer instrument was used in both workshop and stood out with a quite lively and playful output. Although it didn't have large expressive qualities, it created a contrast to many of the other instruments that the dancers appeared to appreciate.
Video: A duet featuring Luis Della Mea and Arnhild Staal Pettersen from the public showing og the second VIBRA workshop in May 2018.
6) Noizer
Filtered white noise (subtractive) instrument
Euler angle values in 3 dimensions are smoothed, and the delta (1st derivative = angular velocity) is used to control the amplitude of four white noise generators and after some scaling and offset with different values for each filter, it further controls the cutoff frequencies of four lowpass filters (2 lpf18 and 2 moogladder). The x-axis here controls two noise generators and two filters, but with different settings for scaling/offset and filter resonance.
The normalized vector of the three euler angles is mapped through a transfer function, which also acts as a gate that only will render values above 0 with a certain amount of movement. The output of this gate function controls the output amplitude of the instrument.
The signals from the four low-pass filters were then mixed and sent to the output.
The result was a instrument with very clear and direct sense of causality, quick temporal response, which invited the dancers to move with fast and forceful movements. This was among the most popular instruments among the dancers, especially in the first workshop.
The instrument was used in the introductory part of the showing of both the first and the second VIBRA workshops.
Video: A duet featuring Luis Della Mea and Elen Øien from the first VIBRA workshop in March 2018, where Luis plays the Pulzer instrument and Arnhild, the Noizer, a filtered white noise (subtractive) instrument.
7) Bipp-a-chu
Multi-layer sample playback instrument with time-stretching, pitch shifting and auto-convolution
This instrument was developed before and during the second VIBRA workshop.
The delta of the euler angle values (1st derivative = angular velocity) in three dimensions are smoothed with a portamento filter (port opcode). The absolute value of the velocity values are then accumulated, and each time this value crosses a threshold, it creates a trigger at the same time as the accumulated value is reset to zero. The result is that when the dancer moves more (in any direction), the trigger will fire more rapidly. The trigger is run through an envelope follower with adjustable attack and decay time, so that each trigger will produce an attack-decay envelope. The seven variants/presets for these values tested in the workshops lie within these ranges: attack time 0.001-0.1s, decay time 0.15-0.2 s, in other words largely sharp attacks and medium fast decays.
The sound samples used for processing in this instrument is chosen randomly among 12 different sound files, all recorded music excerpts from Norwegian radio broadcasts. This created a variation between every time we initiated the instrument, and thereby made the sound quality less predictable to the dancers.
The playback is executed with six instances of the mincer opcode, which is a phase vocoder based playback engine with controllable time pointer and pitch. For this instrument, the accumulated absolute value of the angular velocity of the Y-angle is used to control the time pointer of the playback. However, for each of the six instances, the time pointers were slightly out of sync (0-0.2 s.).
The transposition of the six instances was controlled independently, but did not vary over time. However, seven sets of transposition values were stored as presets/variations of the instrument, some of these in dominantly octave and fifth ratios, others with inharmonic relationships between the transpositions.
The mix of the six playback instances was then treated with three effects: distortion, auto-convolution (the signal is convoluted with itself) and reverberation. The mix of the effects also varied with the seven different presets.
The result was a relatively complex sounding instrument with two layers: 1) a foreground layer where the triggers create a form of harmonically colored stuttering sound, with the frequency of the stutters varied with the amount of movement. 2) a background layer with timbral and harmonic similarities with the foreground layer, but with a much more smooth envelope and often long decaying tails. The sonic complexity made it into a favorite for the dancers at the second VIBRA workshop, and it was used in two of the solos (Elen and Luis).
Video: Solo with Luis Della Mea from the showing of the second VIBRA workshop in May 2018 using the Bipp-a-chu instrument.
8) Flow
Noise instrument based on band-pass filtering of recorded noise samples
The flow instrument was a flowing noise instrument controlled by the Myo armband developed during the last day of the second VIBRA workshop, and it was used in the showing (beginning and end parts). Even if it was considered incomplete and perhaps having relatively limited expressive potential, it produced a wind-like sound that the dancers liked.
Technically, it was implemented as a bit like the Noizer instruments with four layers of filtered noise, only that here, the noise was generated by sample playback of recorded radio static and that it was accompanied by eight layers of sampled music (using mincer as in the Bipp-a-chu instruments) from radio, all with different individual transpositions and a very slowly increasing time pointer, almost freezing the sample playback.
Those layers of sound were run through eight different filters, four bandpass and four low-pass, with the resonant/center frequencies individually mapped to the individual electrodes of the Myo sensor. The result was then pair-wise ring-modulated (one band-pass with one low-pass) before it was mixed and reverberation was added.
Video: Demo of the Flow instrument using a Myo sensor showing the sonic possibilities of the instrument.
https://photos.app.goo.gl/FE9M6kpBExsvjpHV9
The instrument is also featured at the opening of the second VIBRA workshop showing: