Conceived as an experiment for dynamically growing networks as a response to an incoming audio stream, ‘Streamlines’ is a software, piece and performance created using SuperCollider, the Keras API and Cytoscape.js. The piece is based on an inference process that has been trained using synthetic sound sources mapping to custom data structures that are designed to appear as nodes in a network. During a performance, a stream of live-coded sonorities is produced that is analyzed and then used to make predictions of suitable nodes. These nodes then form part of a dynamically growing network of short, articulated sequences that form a counterpart to the synthetic sound. ‘Streamlines’ runs in a web browser where the visual outcomes of the simulations are displayed. The piece reconsiders the relationship between generative activity using the learning and inference processes and that is subsequently ‘interpreted’ by sound processes in SuperCollider, creating a feedback of causes and interaction.
Running the inference process while growing the network becomes the carrier of musical development. The interaction enforces a way of thinking that revolves around the dynamic building of graphs and sonic behaviors. An attitude that considers musical output as something that emerges from an interaction with a trained model. The idea is that evolving processes are set in motion where the trained mappings influence the live-coding activity and where the creator/composer becomes an active observer of the network growth and inference. The output can only sometimes be controlled in detail but is instead interpreted and further processed.
Live-coded synthesis as a mode of composition is what drives many of the possible applications of ‘Streamlines’. The observer (live-coder) produces rich computer-generated sound where interaction emerges through the inference of the analysis and the treatment of the growing network.
Besides the live-coded synthesis and building of networks, two other important modes of control are supported. Configuring the context (initial states) and interfering (or blocking) the generated output. For all of the interaction modes, both manual commands can be used as well as audio-driven operations. This way of working introduces an operational space within which highly detailed synthesis instructions algorithms interact and clash with blind generators of computational behaviors that are produced from the learning and through the dynamic growth of the networks. The relationship of influence goes in both directions and across different time scales.
The piece relates to the theme of AIMC 2024 through its use of dynamics networks as an adjacent field. For network growth, the starting point was to implement a classic network algorithm, The Barabási–Albert model. It is designed to capture the growth and preferential attachment mechanisms observed in many real-world networks, where a few nodes accumulate a disproportionately large number of connections while most nodes have only a few connections (Barabási, Albert 1999). Instead of making the graphs move towards equality they would instead boost certain nodes and create clusters within the graph. Playing around with variations of the basic model, a few variants have been implemented such as the Krapivsky and Redner Model where rather than remaining forever part of a network once added, it introduces the possibility of edge deletion, in addition to the creation of new edges and nodes (Krapivsky, Redner 2003). Therefore, relations change during the growth process and every step during the growth presents a unique state of the network. The Keras learning processes use a rather standard regression process that comes to life really with its relation to the network growth.
‘Streamlines’ questions the concepts of generative activity, learning, inference, and network growth, through an ongoing reconfiguration and live-coding.
Type of submission
For ‘Streamlines’, the ideal situation is a dark space, with the audience either seated, standed or moving slowly. Light should be rather dim where the focus can be on the projected screen but mostly on the sound.
The performance is suitable for Performance 2, at an Oxford University performance space and Performance 3 at the Old Fire Station.
Technical/Stage Requirements
Tech Rider:
Laptop
1 MIDI controllers
4-channel sound card
Program Notes
"Streamlines" is an inventive blend of music, technology, and visual art, using SuperCollider, Keras API, and Cytoscape.js. This project dynamically grows networks in response to live-coded sounds, mapping them to visual nodes in a network. It's an exploration of how synthetic sounds can interact with and influence a growing digital ecosystem, displayed in real-time in a web browser. At its core, "Streamlines" experiments with the relationship between sound generation and network evolution. The piece invites both creators and observers to engage with an evolving digital landscape, where music and visual art coalesce, highlighting the creative potential of integrating machine learning and audio analysis in artistic expressions.
Media
3. Video & Audio Material
The performance/piece involves an interaction between a live-coding environment and a dual simulation it communicates with through a web browser.
The system is under construction but here below are three examples of how it looks, behaves and sounds.
Example #1, Screenshot of the underlying system in an older configuration with two browsers and a live-coding environment.
Example #2, Examples of generated networks that are visualized after a live-coding session
Creative Commons Attribution 4.0 International License (CC-BY 4.0)
