A Boston University neuroscientist is currently studying how the human brain regions interact, with the hopes of combining mind and machine to ultimately improve the lives of thousands of people in the US alone, who live with complete paralysis–unable to move or even speak–yet have normal cognitive brain activity. The condition is referred to in the medical world as locked-in syndrome, in which a person with normal cognitive brain activity suffers severe total paralysis, either from injury or an illness such as Lou Gehrig’s disease (ALS).
“Locked-in people are unable to move at all except possibly their eyes, and so they’re left with no means of communication but they are fully conscious,” explained Frank Guenther, the neuroscientist heading up this new technology.
Guenther works with the National Science Foundation’s (NSF) Center of Excellence for Learning in Education, Science and Technology (CELEST), whose purpose is to synthesize experimental modeling and technological approaches to research and understand how the brain learns. “People who have no other means of communication can start to control a computer that can produce words for them or they can manipulate what happens in a robot and allow them to interact with the world,” Guenther said.
Guenther’s research team demonstrated two experiments. Assistant research professor Jonathan Brumberg conducted one experiment in which a volunteer used a speech synthesizer to make vowel sounds just by thinking about moving her hand or foot, all while remaining completely still and not speaking.
Brumberg explained, “We use an EEG cap to read the brain signals coming from her brain through her scalp. Depending on what body part she imagines moving, the cursor moves in different directions on the screen.” He added that he could, “Translate those brain activities into audio signals that can be used to drive a voice synthesizer. We’ve mapped the “uw” sound to a left hand movement, the “aa” sound to right hand movement, and the “iy” sound to a foot movement.”
The cursor moved across the computer screen as the volunteer sat completely still. Three circles on the screen represented each of the sounds. The volunteer had to move the cursor into the center of any of the circles to get the synthesizer to make the correct vowel sound. The subject imagined moving her left hand to make the cursor move right into the center of the “uw” circle, causing the synthesizer to make a synthetic “uw” sound.
The experience has also been conducted on locked-in patients with amazing results, according to Brumberg. “We started with helping a locked-in patient regain an ability to make certain vowel sounds and that was amazing. He hasn’t been able to talk in years and the first time he made a movement with our formant synthesizer, he nearly, you know, jumped out of his chair with excitement,” he said. “Although the patient has no actual voluntary movement, involuntary motor actions are often seen when the patient gets excited.”
According to Guenther, this technology is promising for locked-in patients as well as other people with communication problems. “We hope these technologies would be applied to people that have other communication disorders that cause them to be unable to speak,” he said. “This sort of thing would allow them to produce synthetic speech, which could be used to talk to the people around them and mention their needs.”
In the second experiment, a graduate student used brainwaves to operate a robot. Checkerboards on the computer screen flashed at different frequencies that were subtle to the naked eye but as Guenther explains, “The neurons in his visual cortex start firing in synchrony with the checkerboard he’s looking at and so we can pick up the frequency and from that, determine which choice he was trying to make, left, right, forward or backward, for example.”
For locked-in patients, he adds, “If they’re pointing their eyes at a visual screen, they can focus their attention on one of the different frequencies and they can manipulate what happens in a robot or in a computer.”
Guenther believes it’s only a matter of time before these technologies are available for commercial use–the culmination of a vision that combines technology and biology to give locked-in patients a “way out”.