In April 2005, I was invited to produce live visuals for Bit Shifter, a musician making electronic music using Nintendo Gameboys as synthesizers. I used Processing with its newly released OpenGL renderer which allowed my dense projects to run fullscreen at an improved frame rate.

I designed an interface for controlling the Processing software with a set of 4 Griffin Powermate knobs. They are framed by a laser-cut plexiglass base which helps to keep them from sliding around while they are being used and helps spread the light which is emitted from their blue LEDs. I use the left-most Powermate to control the camera. Turning the knob left or right adjusts the distance of the camera from the point of interest. Press-turn left and press-turn right adjusts the rotation of the camera around the point of interest. The actions controlled by the other three knobs are customized for each project. Some projects only use 2 knobs, one for the camera and one to control color information.

Unfortunately, the source code provided below does not support using Powermates. Instead they rely on keyboard input and some presets. The library which allows Processing and Powermates to chat with each other is still under development and the version I currently use only works with Mac OSX (the upgrade to OSX Tiger 10.4 introduced some new bugs).

I think it's important to note these projects run around 20 to 40 fps at full screen, which for me is 1680x1050. The method used for preparing these videos with audio is a bit convoluted and the results arent as lush as I would like. They were made by spitting out a still image for every 1/24th of a second of audio analyzed, which makes live manipulation nearly impossible. The joy in using these pieces is being able to play the visuals along with the music and not rely on computer driven audio analysis to do all the work. Beat matching and transitions are best done by hand and can result in dramatic visuals.

Event photos - Jared Tarbell. Event video footage - Casey Reas.

Here is some video footage of the Bridge event. For this performance, I used a Mac G4 laptop running a beta release of Processing. The Apple iSight was used for video and audio capture. Amit Pitaru's Sonia library was used for audio analysis. 4 Griffin Powermate knobs were hooked up to the laptop to provide additional controls.

Ripples is a 2D array of objects which undulate to ripples created by live audio input. The audio affects the objects which lie on the horizontal axis of the array, with lower frequencies near the center and higher ones near the outside. All of the other objects move with the ripples created by the audio axis. In order to give the user some extra control, there are key commands which trigger waves manually so you can match the primary beat for a more believable effect.

Similar in behavior to Ripples, this audio visualizer gets all of its color from a live webcam used as a texture. When performing with this project, I use the powermates to control camera position, spacing of the nodes, and the threshold value for whether the nodes produce standard audio spectrum analysis, or spread as ripples.

Spheroid features hundreds of copies of webcam input rendered around a center point in a 3D environment. User input controls the rotation offset, distance from the center, and orientation. Through these manipulations, many complex intertwined spherical masses can be created. This piece is still a bit hard to control. There are some presets that can be triggered with the keyboard, but for the best effects, you must also be able to manipulate the webcam to increase the variety of the colors and textures shown within this project.