Cyborg Neil Harbisson Can Hear Colors, Share Senses with Friends.

from Cyborg Neil Harbisson Can Hear Colors, Share Senses with Friends.

by Yarrow Maurer

Neil Harbisson was born with achromatopsia which left him only able to see in shades of black and white. However, after inventing the “eyeborg,” Harbisson has been able to hear colors and experience the world in a new way. Initially, the device worked externally using headphones, a camera, and a small computer that he carried in a backpack. A chip translated a limited spectrum of colors into notes on a musical scale. The device has undergone many revisions since then to reduce its size and also to vastly expand the number of colors Harbisson is capable of perceiving (which now include infrared and ultraviolet!) as well as saturation. Recently, Harbisson unveiled the latest revisions to his device which allow him to hear colors directly through bone conductivity thanks to a recent surgical implantation. Harbisson also demonstrated and experienced a new addition for the first time in front of the audience that allows him to experience sensations from colors that his friends are seeing from across the globe! From Motherboard:

Unsurprisingly, it took a lot of effort to find a doctor willing to go ahead with the procedure, but he finally had the operation in Barcelona last December. He showed photographs of the surgeons drilling into his head as he sat with his chin to his chest. The antenna is embedded in the occipital bone at the back of his head, with a separate hole for audio input—essentially, a jack drilled into his skull that transmits sound into his head through bone conduction.

At the end of the antenna, a modified camera detects both hue and saturation (more vibrant colours make a louder noise), and the whole setup is controlled by a chip. In a phone call, Harbisson talked me through the surgery. They had to remove a patch of his hair permanently to reduce infection, and reduce the thickness of the skin. “Then they opened the skin and they drilled—it was just drilling different holes for the antenna and the audio entry,” he said. It took around eight weeks to heal.

For Harbisson, the new antenna understandably feels even more a part of his body than before. There’s no pressure against the back of his head, the sound quality is better, and it feels like a body part. “If you touch the camera or the antenna it’s like touching a tooth or a nail—I feel it, basically, which is weird, because I didn’t feel that before,” he said.

The technology in the latest version of the device also included a very juicy little addition: thanks to a Bluetooth connection and a custom developed app, Harbisson can now hear colours that other people are seeing.

He demonstrated this new capability—which he hadn’t tried before—by hooking up to friends in Barcelona and New York (a third connection in Melbourne appeared to have overslept). They called Harbisson through the Eyeborg app, then used their smartphone cameras to look at different coloured objects. Harbisson could then hear the hues directly in his head.

He also experimented with getting one of the students present to point their viewfinder at different objects on a table and stream what they were seeing to his antenna. Going on the sound, he could correctly identify objects like his blue travel card, multicoloured tie, and burgundy passport.

I asked Harbisson about this experience and he said it was “very special.” “When someone was pointing at the passport I was actually visualising the passport,” he said. “It was not only a sense of colour, it was actually the object for me. So this is something new to me, to visualise things that are not in front of me and share someone else’s vision.”

Read more.