Meet the Cyborg Teaching Musicians How to Play Color, Not Sheet Music

Neil Harbisson doesn’t see colors, he listens to them. The day I talk to the Spanish-born artist, he is wearing a discordant array of notes: F (red), G (yellow) and C (blue). “It’s not a normal major chord. It’s a bit like “da da da,” he sings.

It’s certainly an unconventional way to approach a wardrobe— Harbisson likes to say he dresses to sound good not look good— but it works for the artist, who was born with achromatopsia, a condition that left him completely colorblind.

For the past 10 years, Harbisson has been wearing an electronic eye, his “eyeborg,” to transform his grayscale world into color. This device, implanted directly into his skull, is essentially a camera that captures colors and turns it into sound frequencies that Harbisson can listen to via bone conduction. In Harbisson’s cybernetic world, every color has a corresponding note: Red is F, orange is F sharp, G is yellow, C is blue, A is green and so on. He listens to Warhols, paints with sounds and writes music based on what he sees around him.

It’s a totally foreign way of perceiving the world; almost unimaginable for people who have lived their lives using their senses in the traditional way. But in a recent project, Harbisson taught a group of musicians to hear color the way he does.

Working with the Barcelona’s Palau de la Musica youth choir and the Catalan Quartic String Quartet, Harbisson taught the musicians to play music based not on notes, but on colors. “I wanted to see if musicians would be able to perform light instead of a score,” he says. To give the concert context, Harbisson and Vodafone created software that would capture the Palau de la Musica’s vibrant blues, greens and oranges. From there, he created a score based on the colors recorded that would be performed in the venue without sheet music.

Harbisson taught his method to the musicians, explaining that instead of sheet music, they’d get flashes of colored light to tell them which note to play while on stage. This isn’t unlike learning a new language, he explains. At first it might take minutes to process and translate a color into music, but eventually it becomes so natural you begin thinking in that language without realizing you’re doing so. “After a while, they were looking at things and actually thinking about how they would play the colors in their room or in the street,” he says.

On the night of the performance, Harbisson set up shop as conductor. Instead of holding a stick, he wielded an iPad that allowed him to control the lights being projected onto the stage. As the colors switched from orange to blue to green, the choir and quartet reacted by singing and playing different notes. The F Sharp Major harmonies coming from the musicians were reflected in the colors all around them in the building. A happy coincidence, considering the Palau’s colors could have sounded like a train wreck. “That’s an actual risk,” says Harbisson. “I guess we were lucky.”

If you want to hear color like Harbisson, you can download the Eyeborg app for Android. It’s out on iOS next month.