Teaching the blind to see through sound

A new soundscape language has been created by an Israeli researcher that allows blind people to 'see' objects, in much the same way a bat does.

Sounds can delight the ears and the soul, but according to Israeli researcher Dr. Amir Amedi, sounds can also give sight to the blind. After all, if bats can echolocate and "see" with sound, then why not people, too?

Amedi, of the Hebrew University of Jerusalem, has an eye-opening approach. Using sounds which can be associated with the basic shapes of physical objects, Amedi and his Dutch colleague Peter Mejeri have effectively trained a number of congenitally blind people to "see."

The patients do not "see" via the optic nerve in the eye, but use their visual cortex directly. The sounds that Amedi's new software and algorithms teach to the blind are, in effect, a new language called "soundscapes." Soundscapes could enable those blind from birth, or newly blind, to "see" who and what is in a room.

Hopefully, in time, those who have trained their brains will even be able to see objects that they haven't been trained to see, such as paintings.

Seeing like a bat

"Theoretically, it could be possible to create high-tech audio glasses that convert visual information into audio signals that the blind can interpret," says Amedi.

"Instead of being ‘blind as a bat,' one would effectively ‘see' like a bat - using sound instead of vision to guide you on your way," he explains to ISRAEL21c.

The Soundscape language works with the help of a webcam and a microphone. Amedi plays a sound signal for his blind subjects and presents them with an object - something simple like a thick diagonal bar.

The person is asked to feel the object and draw it. The resulting picture represents one letter of the new alphabet. He then introduces various sound-object combinations together, such as two diagonals crossing.

"The most attractive thing we found, and already published in Nature Neuroscience, is that there is a division of labor in the brain," Amedi tells ISRAEL21c.

"There are different parts of the cortex which are devoted to different functions in the temporal lobe, occipital lobe, etc.," he says, adding that within each lobe there are sub-lobes, devoted to different tasks, such as localizing objects and color vision.

Waking up the brain

"What we found in congenitally blind people who never saw, when we started to work on this, is that new languages, these auditory soundscapes, are being activated in the auditory cortex. After they start learning, they start to recruit their visual cortex. In someone who hasn't seen in two decades, the visual cortex starts to recover.

"It's like waking them up after all these years. They can retrain their visual cortex," Amedi says.

"The blind use a webcam to transform what we see into sound. Learning an algorithm that converts the pictures to sounds is the trick," reveals Amedi, whose lab is in the Department of Medical Neurobiology, Institute for Medical Research Israel-Canada, at the Hebrew University.

In effect, the tool functions as "artificial vision." Once they master it, blind people should eventually be able to read and write and locate objects in their immediate environment. It wouldn't be unreasonable to imagine a day when they could walk into a room and "see" how many people are in it.

So far, the researchers have taught blind subjects to recognize objects in a room, such as shoes, and to read short words like "dad." In theory, they could even be trained to recognize the colors of objects, but "that's a bit more tricky," says Amedi.

The new universal language?

Depending on a person's motivation and ability, the basis for the new "seeing through sound" language could be learned in as few as 10 to 20 hours. Amedi is already working with about 10 people in Israel and is awaiting permission to work with children who are blind. Those under six especially, have the ability to quickly learn and retain new languages.

Amedi is constantly working to improve the algorithms and streamline the tool. But he confesses that it is sometimes an uphill process, and not just from a technical point of view: "It's challenging in itself to have a dialogue with the blind, to get them involved and to convince them to invest their time," he tells ISRAEL21c.

"It's not like a flick of a button and that's it. They need to learn another language. It's challenging in the beginning, but when you get it can be very useful and has a lot of potential."

If the tool becomes commercialized, along with the business plan Amedi intends to include an outline for a training camp so that people can learn the new soundscape language together, in groups.

Amedi maintains that unlike sign language or Braille, the soundscape language could be a universal language. He says that it can be taught to people from any culture, race or religion. Who knows? We may find that it's the only language that blind people - and maybe all of us - have in common.

Israel21C

Set as favorite

Email This