Eye Music with Universal Design

This article shows some of the rapid research-to-technology that is taking place where neuroscience and product development take place through Universal Design. 

Keep in mind that this seemingly archane emergence of Sensory Substitution Devices (SSDs) is just a "productization" of the same insight that bring us Sensory Trails in botanical gardens, Sensory Audits of tourist destinations and intineraries, and Social Stories (Narciso, 2011) used to orient museum visitors with autism.

Seeing Eye-Music.jpg


Researchers at the Hebrew University of Jerusalem have developed a new device that converts images into music to help individuals without vision reach for objects in space.

Sensory substitution devices (SSDs) use sound or touch to help the visually impaired perceive the visual scene surrounding them. The ideal SSD would assist not only in sensing the environment but also in performing daily activities based on this input. For example, accurately reaching for a coffee cup, or shaking a friend's hand.

In a new study, Hebrew scientists trained blindfolded sighted participants to perform fast and accurate movements using their new SSD, called EyeMusic.

The EyeMusic employs pleasant musical tones and scales to help the visually impaired "see" using music. This non-invasive SSD converts images into a combination of musical notes, or "soundscapes.

The device was developed by the senior author Prof. Amir Amedi and his team at the Edmond and Lily Safra Center for Brain Sciences (ELSC) and the Institute for Medical Research Israel-Canada at the Hebrew University.

The EyeMusic scans an image and represents pixels at high vertical locations as high-pitched musical notes and low vertical locations as low-pitched notes according to a musical scale that will sound pleasant in many possible combinations.

The image is scanned continuously, from left to right, and an auditory cue is used to mark the start of the scan. The horizontal location of a pixel is indicated by the timing of the musical notes relative to the cue (the later it is sounded after the cue, the farther it is to the right), and the brightness is encoded by the loudness of the sound.

The EyeMusic's algorithm uses different musical instruments for each of the five colours: white (vocals), blue (trumpet), red (reggae organ), green (synthesized reed), yellow (violin); Black is represented by silence.

Prof. Amedi said "The notes played span five octaves and were carefully chosen by musicians to create a pleasant experience for the users."

"We demonstrated in this study that the EyeMusic, which employs pleasant musical scales to convey visual information, can be used after a short training period (in some cases, less than half an hour) to guide movements, similar to movements guided visually," explained lead investigators Dr. Shelly Levy-Tzedek, an ELSC researcher at the Faculty of Medicine, Hebrew University, Jerusalem, and Prof. Amir Amedi.

"The level of accuracy reached in our study indicates that performing daily tasks with an SSD is feasible, and indicates a potential for rehabilitative use," they added

The study tested the ability of 18 blindfolded sighted individuals to perform movements guided by the EyeMusic, and compared those movements to those performed with visual guidance. At first, the blindfolded participants underwent a short familiarization session, where they learned to identify the location of a single object (a white square) or of two adjacent objects (a white and a blue square)

In the test sessions, participants used a stylus on a digitizing tablet to point to a white square located either in the north, the south, the east or the west. In one block of trials they were blindfolded (SSD block), and in the other block (VIS block) the arm was placed under an opaque cover, so they could see the screen but did not have direct visual feedback from the hand.

The endpoint location of their hand was marked by a blue square. In the SSD block, they received feedback via the EyeMusic. In the VIS block, the feedback was visual.

"Participants were able to use auditory information to create a relatively precise spatial representation," notes Dr Levy-Tzedek.

The study lends support to the hypothesis that representation of space in the brain may not be dependent on the modality with which the spatial information is received, and that very little training is required to create a representation of space without vision, using sounds to guide fast and accurate movements.

"SSDs may have great potential to provide detailed spatial information for the visually impaired, allowing them to interact with their external environment and successfully make movements based on this information, but further research is now required to evaluate the use of our device in the blind," concluded Dr. Levy-Tzedek.

Their results were published in the July issue of Restorative Neurology and Neuroscience.



Recent Entries

Eye Music with Universal Design
This article shows some of the rapid research-to-technology that is taking place where neuroscience and product development take place through…
As Ecuador Gears up for Inclusion in Tourism
Here at the Rolling Rains Report you will continue to hear breaking news on developments toward Inclusive tourism in Ecuador.…
Advocating Constitutional Change for Inclusion in India
From The Hindu:The Constitution must be amended to prohibit discrimination against the disabled and to bring them into the mainstreamIn…
Ceara, Brazil: Third Forum on Disability and Accessibility
There is no better post for Independence day than to announce that the revolutionary spirit of full citizenship for all…
Sobre Interdependencia no Dia Antes do Dia de Independencia (Portuguese)
Interesante. Para o debate deixamos aceitar que "A distancia entre o sonho e a conquista chama-se atitude." Pois bem, isso so…
On Interdependence - on the Day before Independence Day 2012
Interesting.For the sake of the debate let's accept the overlay on Nick's photo below "The distance between the dream…
India: D.R.G. member resigns from Ministry of Civil Aviation Committee
From D.N.I.S. News Network, India: Shivani Gupta, nominated by Disabled Rights Group (D.R.G.) to the Ministry of Civil Aviation Committee drafting…
Acessibilidade: Legado da Rio+20 (Portuguese)
"A partir do Modelo Brasileiro de Governança da Acessibilidade e de sua execução para a Rio+20, a ONU passará a…