You think it, you say it.
Most people have an internal dialog with themselves, a voice that only they can hear. If we actually spoke aloud all the words flying through our heads, we’d quickly find ourselves picked up by nice men in a white van. Even reading these sentences, it’s likely your mind “hears” the words as you read them. Brian Pasley at the University of California, Berkeley wanted to prove that “thinking” words to yourself and hearing words created similar neural patterns in the brain. Pasley and his colleagues wondered whether they could identify where some of the most vital aspects of speech are processed by the brain, and whether they could simply record the neural activity and later reconstruct the original sound in a computer.
The team played spoken words to 15 volunteers already undergoing surgery for brain tumors or epilepsy. Electrodes were attached to their exposed brains and recorded neural activity from the superior and middle temporal gyri, a brain region near the ear involved with sound processing. From these recordings, Pasley’s team was able to partially decode which aspects of speech were related to what kind of brain activity.
Exactly how the brain converts the spoken word into meaningful information is still a partial mystery.
Correlating these speech frequencies with neural activity, Pasley’s team used an algorithm to interpret the data and create a spectrogram, a graphical representation showing sound frequency over time. They tested the algorithm by comparing spectrograms reconstructed solely from neural activity with a spectrogram of the original sound. “Imagine a musician watching a piano being played with no sound,” says Pasley, “If a pianist were watching a piano being played on TV with the sound off, they would still be able to work out what the music sounded like because they know what key plays what note.”
The result was speech generated solely from the subject’s brain waves. While quite a bit of imagination is required to understand the extremely garbled words, it’s clear that this was a successful first effort. Within the coming years, it may be possible for those with total paralysis (like Stephen Hawking) or locked-in-syndrome to “speak” simply by thinking words instead of saying them.