Experience replay is an improvised performance for guitar and live electronics. It is based on a digital system for free improvisation developed by the author. The performance consists of a guitar player interacting with a system composed of two granular synthesizers which process the sound of the guitar. The parameters of the synthesizers are mapped to perceptual features computed from analysis of the live sounds of the guitar and the two synthesizers.
This system is part of the author’s artistic exploration of free improvisation and algorithmic automation. The system acts as a “musical playground” with different options for processing incoming sound. This system is continuously updated by the author, integrating machine listening algorithms, experimenting with mapping strategies [1] and incorporating additional electronic and digital instruments. The version of the system presented in this performance is composed of two parallel granular synthesizers which process the live sound of the guitar. The two synthesizers are coupled through control-rate modulation signals based on perceptual features. A signal-flow diagram of the system is shown in Figure 1.
Figure 1. Signal-flow diagram of the improvisation system. Live audio from the guitar is recorded and processed in real-time by two parallel granular synthesis modules. Sound descriptors are continuously extracted from the guitar sound. The descriptors are used as modulation signals which are mapped to the parameters of the granular synthesizers. The sonic output of each granular sysnthesizer is also analyzed using machine listening, and the resulting descriptors are use to modulate selected parameters of the other synthesizer, creating a coupled feedback system. While playing, the musician can regulate the high-level behaviour of the system using a MIDI controller.
In this system, the sound of the guitar is processed by two granular synthesis modules coded in Pure Data (PD). The granular synthesizers can either process the incoming live sound or a recorded portion of it by recombining and playing back short samples (sound grains). The granular modules are controlled by low-level parameters such as grain size, grain density and pitch, and several randomization options regulating grain-level features such as pan, delay and distortion [2].
The sound of the guitar is continuously analyzed using machine listening algorithms, adopting the Flucoma library in its PD implementation [3]. Specifically, loudness, spectral shape and chroma descriptors are extracted in real-time and used as control-rate modulation signals mapped to the low-level parameters of the granular synthesis modules. In this way, the synthesizers are responsive to perceptual qualities of the guitar sound, and the guitar acts as both the sound source for the synthesizers and a controller through the properties of its sound. Different mappings are used for the two granular modules, generating a complex response to the guitar. High-level properties of the system can be directly adjusted by the musician using a MIDI controller, which allows to regulate the degree to which the modulation signals affect the synthesis parameters, i.e. how much the system is automated.
Perceptual features are also used to analyze the sounds produced by the synthesizers. Specifically, the two granular modules are coupled through a control-rate feedback loop based on descriptors extracted from the sonic outputs of the two granular units. In this way, the perceptual characteristics of the sound of each of the two synths affects the sonic output of the other, resulting in a coupled system [4]. This strategy makes it difficult to predict how the system will react to the variety of the guitar’s sonic input and to the resulting modulation at a specific point in time. The mappings create a structure of complex co-creative relationships between the live musician and the synthesis models [5], which provides an interesting playground for the musician to improvise.
Figure 2. Hardware setup used in Experience Replay.
Technical/Stage Requirements
Technical requirements for the venue:
Table
Small mixer (4 channels or more)
Stereo or multichannel diffusion
Cables
Electric guitar
Provided by the artist:
MIDI controller and MIDI keyboard
Digital audio interface
Laptop
Program Notes
Experience Replay is an improvised performance for guitar and live electronics. The sounds generated by the guitar throughout the performance are continuously recorded, re-combined and processed by a system composed of two granular synthesis modules. Machine listening algorithms are used to extract perceptual qualities of the sounds produced by the guitar and the two synthesis modules, generating modulation signals that are used to affect the synthesis parameters. The modulation signals are used to create a feedback loop between the two synthesis units. This generates a complex and unpredictable co-creative system which responds to the guitar, providing an engaging improvisation playground for the performer.
Media
Figure 3. Performance set up.
Audio excerpts from improvised sessions:
Audio 1. Excerpt from an improvised session.
Audio 2. Excerpt from an improvised session.
Video recording of a full improvised set:
VIdeo 1. Example of a full improvised set.
Acknowledgements
This work is supported by the Department of musicology at the University of Oslo as part of the author’s doctoral research fellowship. The improvisation system is based on several online resources and open-source Pure Data code. The author would like to thank the Pure Data community online, without whom this project would not have been possible.
Creative Commons Attribution 4.0 International License (CC-BY 4.0)
