harmonic improvisation

Process

We began by visiting a cafe near Northeastern at different times of the day to observe how people behave and how those different behaviors add up to the emergent property we sense as the mood of a space. We found that the most important elements to determining the mood of the space related to how many people were coming in and out of the space, how many people were in the space, how much people were interacting, and how people were grouped.

We began to construct a system model with these elements in mind as inputs to the system. We categorized them as follows:

• flowrate: entry and exit from the cafe

• accumulation: amount of people in the cafe

• volume: loudness of conversation

• clusters: grouping of people

In order for a system to be improvisational, it must have a learning element to it. We sketched 2nd order cybernetic systems, but struggled to derive meaning from them. We had read that in Gordon Pask's Musicolour, the system would, over time, get bored and change it's goal. We knew this was an element of the system we needed to incorporate but didn't know how it might manifest.

We were fortunate to have been put in touch with Paul Pangaro, a former student of Pask and leader in cybernetics, to discuss our project. Among other things, he instructed us to begin to define the system goal and consider how the system might work before incorporating the learning element.

We defined the goal of the system as, "to drive conviviality." We set system boundaries, being the location of doors as the entry and exit points, the dimensions of the space, the location of tables, and a capacity for people in the space.

With a goal of driving conviviality, the system has targets for each of its inputs that would be characteristic of a convivial environment. This might be a low flowrate of people, a high accumulation of people that is below the capacity for the space, a moderate loudness in proportion to the number of people in the space, and a high degree of grouping of people.

How Does this Sound?

With an understanding of how the system might process the mood of the cafe, we had to determine how that would lead to system behavior in the form of playing music. How could the system know how to play a song in order to make people engage in conversation or stay a little longer?

While we read some research on the affect of music on the behavior of people, we knew that in order for the system to be able to play popular recorded music as is typically played in a cafe, the system would have to index every song in its library with information on how it might affect the variables considered.

We instead decided the system to compose its own music. The system is designed to play phrases built upon three elements - swelling chords, rhythmic bass, and granular synthesis of live recorded audio.

swelling chords

Each phrase is to obey most rules of classical music, beginning by establishing a tonal center, building tension, and moving to a cadence. The system shifts through moods in the same way composers tend to - through modulation. Once the system has achieved its goal, it remains within the key.

rhythmic bass

While the swelling chords reveal the harmonic information about the music, the rhythmic bass reveals the tempo and groove of the music. The bass is what listeners will nod their heads to and feel through syncopated rhythms. The rhythmic bass will play in patterns within a phrase, changing for new phrases which give listeners a new groove, characteristic of a new mood.

granular synthesis

While the swelling chords and rhythmic bass are able to adjust the mood as the system understands it through its inputs, the system is able to provide a unique texture to the sound by sampling noises live in the room, selecting those that pass a loudness gate within a frequency range characteristic of conversation, and playing them in random slices. This method, known as granular synthesis, signals to listeners that the music system is responding to the people in the room and encourages patrons to consider making noises that might show up in the next phrase played by the system.

Selective Hearing

Everything discussed so far works for a goal of driving conviviality, but would not be improvisational if that goal could not adjust. As such, the system model, shown below, includes a second order loop that defines the goal of the inner loop. For example, if the system finds that it has achieved, "conviviality," and is maintaining it by repeating the same phrase, the system will set for it a new goal, a new definition of what conviviality is, possibly by changing the metric for volume of conversation or the number of people in the space. In Musicolour, this was considered the system getting, "bored," and shifting it's focus. In Harmonic Improvisation, the system might consider conviviality to be a tired goal and instead seek to drive a more quiet and intimate environment. In any case, the behavior of the system is non-deterministic and is sure to surprise those in the cafe.

What's in the Queue?

The concept of Harmonic Improvisation, while intriguing and thought-provoking, could prove more powerful with further development and possible recontextualization. The sound of Harmonic Improvisation is currently constrained by a small set of rules that limit the compositional possibilities, but more importantly, the range of sounds that people can hear. A reconsideration of how sound gets composed, possibly by allowing the system to experiment and learn its own rules for what is "pleasurable" music, could lead to a more compelling and engaging experience. In addition, by providing the system with a wider range of instruments with which to compose, even with the current set of rules, could make the experience with the system more intriguing.

A possibly more interesting direction for Harmonic Improvisation is its application in other contexts beyond the cafe. The idea of a system that seeks to understand human behavior and creates music around that behavior could apply to group contexts such as a workplace, a grocery store, or city bus, or to personal experiences such as running, studying, and driving. The inputs for these systems would be specifically defined to the context and could have different compositional rules for the output or possibly output popular music based on an index of a given catalogue of music.

For music streaming services that seek to allow the music they provide to interact with their users in a manner like this could consider attributing certain characteristics to songs in their catalogue and allowing their service to improvise with their listeners. Rather than restricting users to listen to songs that exist within a playlist or seek variety through the shuffle button, an improvisational model of a streaming service might propose the first song and queue each successive song based on one of several inputs from the user, such as their heartbeat, their location, or their phone activity.