Code of Music

Midterm

Accessible Sequencing

Single button sequencer with “living” notes that move around and disappear when they are done.

Audience

The audience for this is people who know nothing about music, children, and people with disabilities. It uses bright colors, very simple language, and most importantly it is controlled with only a single button. Musical keys are preprogrammed and every note is played rhythmically in time so it always sounds good,

Even with the ultra-simple input, my goal for anything I build is that it is more than a toy. Someone can pick it up and feel like they can be expressive with it - like any acoustic instrument. They might play for 15 or 20 minutes or come back to it later to make music. My hope is that I never make something such that one thinks oh that’s neat, but does not feel like they can really derive any joy from using it to make sounds.

Single Button Input

Any button press adds a single note to a random position on the screen (weighted towards lower notes). Of course, if all the user can do is add notes, then the texture would only grow. So, I came up with this idea of “lifespan” - notes only stay on the grid for a while and eventually they disappear. Two more ideas grew naturally from that. One - notes get quieter as they grow older. Musically, this means the user can add accented notes by letting a bunch of notes age and then adding one or a few loud ones, and more finely craft crescendos/decrescendos than most sequencers. Two - since notes are “alive” they might have different personalities. One type moves from side to side (changing rhythm), one moves up and down (changing pitch), and one stays still. This leads to constant variation without any user input, but in order to keep the music going, the user must put energy into the system. More energy - more amplitude, harmony, and rhythm.

Other Features

Xylophone library:

For the sounds, I use a xylophone sample pack I found on this random website samplepacks.com. The library contains five different samples for each note on the xylophone which my app loads and draws from. While I looked at the waveforms of the audio files to confirm that they are indeed different, in practice, one can’t really tell the difference. Furthermore, In order the soften the attack and make each note a little more like a marimba played with yarn mallets, I use a slight fade-in time for each note.

Another interesting point - the library is named in a weird way so I had to write my own function to parse all the file names so that they are correctly played. (A (1).wav is actually F4.)

Sound Effects

I played around a bit with different effects. In the end I went with panning to match the layout of xylophone (low notes on the left) and a pretty heavy reverb using the Tone.js Freeverb object. Furthermore, I added a slight humanize feature so that pitches might be played a little ahead or behind the beat. (This is a fun parameter to play around with - if the range is too big, one hears utter cacophony.)

Animation

The interface looks and feels fluid - objects move and colors change gradually. I made heavy use of the lerp() function which returns an interpolation between a starting value and an ending value. Over many of these homework assignments, I came up with this idea of a easily-lerped object. Rather than a single value, it includes three - two target states and a current state that interpolates between them. Since in some cases the target state might change (e.g. each key has its own color palette), I use a slightly more complex data structure in this assignment. For example, the background color variable in sketch looks like this:

let bcg_col = {
    true: [
        {r: 80, g: 20, b: 80},
        {r: 20, g: 80, b: 60},
        {r: 170, g: 140, b: 10},
        {r: 22, g: 45, b: 80}
    ],
    false: [
        {r: 30, g: 30, b: 30},
        {r: 30, g: 30, b: 30},
        {r: 30, g: 30, b: 30},
        {r: 30, g: 30, b: 30}
    ],
    curr: {r: 30, g: 30, b: 30}
}

Planning

Image of class worksheet

Problem

When more notes are added, and especially when the volume is high, I hear annoying clicking noises similar to those heard when a sound source is moved to zero amplitude without any ramp time. I spent well over an hour or so trying to figure this out to no avail. I tried:

The problem persisted. Unfortunately, it seems worse than the Melody assignment this is built off of after I switched to the Players object from the simpler Player object. I think the problem goes deeper down to Tone.js or even the Web Audio API. Maybe once too many buffers are triggered at once or consecutive buffers are triggered, the browser can’t handle it and clicking sounds appear. It could also be that the volume is getting too loud with so many layers of sound on top of each other. Either way, this was an annoying issue that I could not solve.

Play it. (Supported by Chrome.)

View Source in Github.

A blog for displaying Willie Payne’s progress in the NYU ITP Course “The Code of Music.”