Project Scriabin - The Color of Sound
Project Scriabin
Designer/Developer - Darren Woodland Jr.
This project will map musical pitches with the corresponding visual light spectrum (colors) in an attempt to create a musical composition from the visuals of a photo or work of art. It is an attempt to visualize the music of imagery that can possibly inform on its characteristics or structure.
The original idea for the project was derived from the medical condition of synesthesia, where some individuals can “see” sound. In particular, from Alexander Scriabin, a Russian composer, and pianist who has sound-color synesthesia. The novelty of the project comes from the idea that colors have emotions associated with them and that tonal values or chords can also be considered warm/cool or happy/sad.
The user will be able to use a second screen painting application, in this instance, Adobe Photoshop to place color on the screen in a grid and create notes and chords on a simulated musical measure. The finished product (musical composition) would bring an element of surprise in that it is unknown how it will sound. There will also be surprise in the process, from the creation of imagery to match certain chords or how the colors can be mapped to specific tonal values.
As I studied the life and work of Alexander Scriabin I became increasingly interested in the condition of synesthesia and sensory modalities. I began to look into how the human brain processes aural and visual information. I wanted to explore how using visual and aural information at different points and at different intensities throughout the process of creation affected the users experience.
Research: I began the process by researching Alexander Scriabin’s life and work. I then moved to understand how sound and color have been used in relation to one another and how the brain processes the two.
TouchDesigner: The bulk of the work was done in the visual programming application TouchDesigner. I first began the process by using my webcam and cropping it into a 5 X 3 grid, so 15 “individual cameras.”
Ableton Live: The final step was sending out individual MIDI signals from TouchDesigner to Ableton Live. In Ableton I set up vocal voices to sing a simulated C- Major Bach Chorale created using the colors in Photoshop.
The next phase of Project Scriabin is to expand upon the association of sound and color by viewing the coupling through the lens of cultural differences between Western and Eastern music. The bulk of the project will remain as a TouchDesigner application and the interfacing mechanism will still be a second screen painting application (Adobe Photoshop) but with a second mode to differentiate between the east/west sound-color pairings.
To explore what differences the two musical spectrums could possibly have on the structure and aesthetics of an expressive object I constructed a three-dimensional visualization of the tonal outputs from the painting application.
The audio visualization is designed to change its form based on various structural elements of both the aural and visual inputs. The governing principles for audio visualization are as follows.
-
The RGB color values of any given note-color pairing control the external forces of the audiovisual wave, like wind speed and turbulence.
-
The frequency and wavelength control the number of particles generated in the rows and columns of the audiovisual waveform respectively.
-
The rotation speed and the rate at which the waves oscillate are controlled by the value/velocity of the midi note that is sent to Ableton Live.
-
The color of the particle field will directly correlate to the color present in the sound-color pairings.
