Using AI, Tel Aviv University researcher hopes to turn patients’ thoughts into speech

Dr. Ariel Tankus said that what he is doing, in a word, is “mindreading.”

Tankus, of Tel Aviv University’s Medical and Health Sciences and Sourasky Medical Center (Ichilov Hospital), said that he and his team of researchers programmed a computer to “read” a person’s mind and verbalize his thoughts by decoding the neuron signals of his brain.

The findings offer hope to those unable to speak due to brainstem stroke, brain injury, or Amyotrophic Lateral Sclerosis (ALS), also known as Lou Gehrig’s disease.

The groundbreaking research appeared in June in the prestigious journal, Neurosurgery.

Tankus teamed up with Dr. Ido Strauss of Tel Aviv University’s School of Medical and Health Sciences and the director of the Functional Neurosurgery Unit at Ichilov Hospital.

For their research, they worked with a patient with epilepsy who agreed to have electrodes implanted deep in his brain. The implanted electrodes are a few centimeters long, with eight to nine micro-wires, as “small and thin as the thickness of a hair,” Tankus said.

Surgeons implanted the electrodes to determine the “very strong electrical currents” that might cause the patient’s epileptic seizures. The electrodes then enable surgeons to locate the specific area in the brain where they can operate.

For Tankus, this was “maybe the only opportunity” to record the activity of one single cell in the brain.

In the first part of the experiment, Tankus asked the patient to repeat two sounds, called phonemes, which are the basic building blocks of speech. The sounds were an “a” pronounced “ah,” and an “e,” pronounced “eh.”

These phonemes are the start of constructing a language that will allow the computer “to speak for the patient,” Tankus said.

The researchers then recorded the activity in the brain each time the patient said the vowel sounds.

Tankus said the electrodes were implanted in parts of the brain — the anterior cingulate cortex and the orbitofrontal cortex — that are associated with cognitive and emotional processes, but are not generally associated with speech.

“I discovered these neurons in the cortex and how they are connected to speech,” Tankus said.

He then used artificial intelligence algorithms to “train the computer” to make a distinction between the two sounds by recognizing the brain’s activity.

In the second part of the experiment, the patient silently imagined saying “a” or “e.” The computer then predicted, in real-time, what the patient said based only on “several single neurons in the brain,” Tankus said. “The computer spoke for the patient.”

“The research is aimed towards helping completely paralyzed persons to speak again,” Tankus said. Patients with ALS cannot communicate, “not even with an eye blink, ” Tankus said.

In the future, when a patient is in the early stages of ALS and can still speak, the electrodes would be implanted in their brain. Then, the AI algorithms that the research team developed could decode the brain’s patterns. The computer would then be programmed to understand what the patient is trying to say.

Although more research needs to be done, Tankus believes that even with ALS, a person’s brain has similar to normal activity, but “simply cannot express itself.”

“In the future, the computer will understand and speak for the patient,” Tankus said.

If a paralyzed person could “tell us yes or no when we ask questions like, ‘Are you hungry?’ or ‘Are you in pain?’ the computer could help the person answer,” Tankus said.

“That could make a great change for a completely paralyzed, locked-in patient,” he said.