"The audio in this lava lamp video was created by analyzing the frames in real-time with the GMS or gestural music sequencer. Each frame is processed to find the brightest pixel. The x coordinate is converted into pitch while the y coordinate is translated into dynamics. The application was also designed to use probability distributions on pitch and durations. Most of the brightness is in the center, but when bright globs float down the right side higher pitches can be heard in the piece."
Josh Clos has produced a documentary about the Gestural Music Sequencer or GMS recently. His short video interview illustrates what the GMS does and how he has been using it to compose music in real-time.
