At Campus Party Europe, we spoke to Neil Harbisson, who uses an electronic eye to allow him to see colours using sound, about extending human senses using technology.
Neil Harbisson stands on stage dressed in bright blue, pink and yellow. There’s a tiny camera on a flexible cable arched over his forehead. “To me, a girl with ginger hair, blue eyes, dressed in pink, sounds like F# hair, C# eyes, dressed in F,” he tells the crowd.
Born with achromatopsia, which means he sees the world in black, white and grayscale, Harbisson became fascinated by the concept of colour. He first studied fine arts, then – after hearing that Newton, centuries ago, related colour to a musical scale – he began to think about colour more in terms of music.
In the early 2000s, he started a collaboration with cybernetics expert Adam Montandon to create the eyeborg, a camera that picks up colour, converts it to sound and transmits it to Harbisson by pressing on the bone at the base of his head (so it doesn’t interfere with his regular hearing). In September this year, he will have an operation in a hospital in Barcelona to put the chip that makes the device work inside his skull.
“I want to stop using electricity,” he adds. “I want to use blood circulation to charge the chip… I’m sure this will be possible.”
As well as develop the software and hardware of the eyeborg to make it smaller and more portable, and pick up saturation as well as hue, Harbisson had to learn which sounds corresponded to which colours – starting with the basics, expanding out to 360 colours and more (ultraviolet, infrared). It took some time for him to get used to it but now hearing colours seems as familiar to him as seeing colours is to anyone else.
“After five months of hearing colour constantly, the software and my brain united,” he recalls. “I woke up one morning and I’d been dreaming colors. I noticed in my dreams, I’d hear different notes for different colours… That’s when I started to really feel and understand the word cyborg.”
“Now, galleries are like concert halls and supermarkets are like nightclubs.”
Harbisson shares his rich new world of perception through visual art, performances and public talks. “I used to dress to look good, now I dress to sound good,” he explains. His outfit at Campus Party – blue, pink, yellow – sounds to him like a major chord.
He has painted the dominant colours of songs (above – Beethoven’s “Fur Elisa”, left, Vivaldi’s “Spring”, right) and of European cities, and created sound portraits of celebrities. “Nicole Kidman sounds very good,” he says – so does Prince Charles.
From mobile apps to apps for the body
In 2010, Harbisson and Spanish artist Moon Ribas, who uses speedborg ear pieces to sense and measure movement, set up the Cyborg Foundation to help others who wish to extend their senses using technology, and to promote and defend cyborg rights.
“We are now at a point where we will stop using technology as an external tool, and more as a part of our bodies,” he predicts. “It is our generation that can do this – stop creating apps for mobile phones, start creating apps for our own body.”
It could be argued that touchscreens, smartphones and the Google Glass smart-eyewear project are already blurring the lines between humans and technology. Harmisson says that’s not enough to qualify as cyborgism, as the examples above are still external tools.
Checking a smartphone map system to know where to go, for example, is very different from sensing where to go. “The next step after the glasses needs to be to sense vibrations to know where to go – knowing if I turn left, right, I will feel vibrations in my body,” he says. “Then you don’t need to block a sense, vision. The main thing is not to block any senses that we already have.”
The mind races to other possible applications – infrared sensors to help rescuers search in earthquake rubble or ultraviolet sensors for those who want to pick and choose their sunbathing days carefully, for example. Harbisson says there’s a lot of interest in sensing ultraviolet and also in using technology to extend human hearing rather than just repair it.
At Campus Party Europe, surrounded by tech enthusiasts, programmers and robot football players, this is one vision for the future that seems vividly possible.