The first real-time brain-scanning speller will allow people in an apparent vegetative state (unable to speak or move) to communicate, according to Maastricht University scientists. (video)
The new technology builds on earlier uses of fMRI (functional magnetic resonance imaging) brain scans by Adrian Owen and colleagues to assess consciousness by enabling patients to answer yes and no questions. fMRI tracks brain activity by measuring blood flow.
“The work led me to wonder whether it might even become possible to use fMRI, mental tasks, and appropriate experimental designs to freely encode thoughts, letter-by-letter, and therewith enable back-and-forth communication in the absence of motor behavior,” said Bettina Sorger of Maastricht University in The Netherlands.
Letter-encoding via brain patterns
So Sorger’s team came up with a letter-encoding technique that requires almost no pre-training. Participants in their study voluntarily selected letters on a screen, which guided the letter encoding; for each specific character, participants were asked to perform a particular mental task for a set period of time. That produced 27 distinct brain patterns corresponding to each letter of the alphabet and the equivalent of a space bar, which could be automatically decoded in real-time using newly developed data analysis methods.
In each communication experiment, participants held a mini-conversation consisting of two open questions and answers. Everyone the researchers tested was able to successfully produce answers within a single one-hour session.
“By exploiting spatiotemporal characteristics of hemodynamic (blood-flow) responses, evoked by performing differently timed mental imagery tasks, our novel letter encoding technique allows translating any freely chosen answer (letter-by-letter) into reliable and differentiable single-trial fMRI signals,” the researchers say.
“Most importantly, automated letter decoding in real time enables back-and-forth communication within a single scanning session. Because the suggested spelling device requires only little effort and pretraining, it is immediately operational and possesses high potential for clinical applications, both in terms of diagnostics and establishing short-term communication with nonresponsive and severely motor-impaired patients.”
The results substantially extend earlier uses of fMRI, which allowed individuals to answer the equivalent of multiple-choice questions having four or fewer possible answers, by enabling free-letter spelling. That could make all the difference for people who are completely paralyzed and unable to benefit from other means of alternative communication, Sorger says.
Ultimately, she says their goal is to transfer the fMRI technology they’ve developed to a more portable and affordable method for measuring blood flow, such as functional near-infrared spectroscopy (fNIRS).