Last week, the New York Times reported the first successful “limb reanimation” in a person with quadriplegia. Ian Burkhart, 24, had broken his neck as a teen in a diving accident. His spine was damaged at the fifth cervical vertebra, leaving him paralyzed from the shoulders down. Using nerve bypass technology that transmits his thoughts directly to his hand muscles, he has regained control over his right hand and fingers.
The system effectively bypasses his spinal injury. This is the first time a brain-computer interface has been used to help an individual move his own hands. The research was published in in the journal Nature, and explains how machine learning and MATLAB were used in this project.
This remarkable breakthrough is the result of years of research and a cross-discipline project. Engineers, bioengineers, neuroscientists, and surgeons contributed to the prototype medical system and clinical study. The device, called NeuroLife, was invented at Battelle Institute. The engineers at Battelle worked with physicians and neuroscientists from Ohio State University Wexner Medical Center to develop the research approach and perform the clinical study.
In 2014, Ohio State surgeons implanted a chip in Ian’s brain. The Utah Array chip from Blackrock Microsystems consists of 96 microelectrode sensors that record the firing of individual neurons. The team used brain imaging to identify and isolate the part of Mr. Burkhart’s brain that controls hand movements. During the surgery, the team repeatedly tested exposed brain tissue to pinpoint the correct location for the chip.
After the surgery, Mr. Burkhart spent hours upon hours watching an avatar of a hand on a screen, concentrating on the thoughts it would take to make the movement. During these sessions, the activity in his brain was recorded and analyzed. The goal was to understand which signals in his brain corresponded with the hand movements he was mentally emulating.
The firing patterns in his brain were transmitted to a computer and recorded. The amount of data this requires is immense. Three million samples of neural activity are collected each second. Approximately one gigabyte of data is generated from neurons in the motor cortex every three minutes.
The researchers used machine learning to decode these patterns. Through repetition, the firing patterns were analyzed and used to develop an algorithm to control the muscles in his hand. The algorithm was adjusted after almost every training session. Machine learning aided the recalibration as Ian’s mind adjusted to the movement and apparatus.
How MATLAB was used in this project
From the journal Nature:
“The digitized data were then transmitted to a personal computer where they were decoded to determine which motion was being imagined and then encoded to evoke the desired response from the muscles in the forearm. The computer communicated with the custom high-definition neuromuscular electrical stimulator (NMES) that drove the electrode sleeve wrapped around the forearm.
“The signal processing and decoding/control algorithms were all run on a personal computer using MATLAB. The digitized data from the Neuroport was processed every 100 ms.”
The computer decoded the signals from the brain and translated them to specific movements. The team used a NMES sleeve that applied electrical stimulus to his hand muscles. The sleeve has 130 electrodes that rest on the skin’s surface and transmit signals through Ian’s arm to activate his muscles.
After a year of training, Ian was able to learn through repetition to pour from a bottle, and stir a cup. He was also able to learn to play a guitar video game!
More work is needed
- For starters, the system isn’t portable. The Neuroport requires a direct connection to the computer. Additionally, the amount of data produced exceeds current wireless capabilities. To truly enhance independence, a wireless system will be required.
- There are also limits to the movements that can be accomplished. The implant only monitors up to 96 of the millions of neurons in the motor cortex. For complex hand movements, people use far more than a hundred neurons. In future studies, the team plans to increase number of electrodes.
- The sleeve that transmits electrical pulses to stimulate the muscles in the forearm will likely need to be adjusted as well. The current voltage levels required to transmit through the skin can interfere with the signals being recorded in the brain. In the future, researchers may implant the electrodes in the forearm in order to reduce the voltage used.
As amazing as the technology is, I have to say that I am equally amazed by the brave man that was willing to dedicate so much of his time and effort in an attempt to further the research and help others. Mr. Burkhart agreed to go through multiple brain surgeries even though he knew the reclaimed mobility would only be temporary. The system only works while he’s at the lab and the chip will be removed later this summer. The hope is to create a system that will be able to help people like Ian gain independence in their everyday life.
“…even if it’s something I can never take home in my lifetime, I’m glad I’ve had the opportunity to take part in this study… I know that I’ve done a lot of work to help other people as well,” Ian Burkhart said in his interview with Nature.
12 CommentsOldest to Newest
Great story Lisa! I am the statistician on this project and would be happy to discuss how we use Matlab on this project further if you’re interested.
Thanks David! I’m glad you like the post. Your project is amazing! We would definitely like to learn more about your project.
That’s an amazing project. Coming from a related background, I could imagine how challenging the work should be and the joy outburst when it worked. Kudos to the team! Will also be glad to know more details of the work. Congrats!!
Wow! Awesome work! That means in near future a computer will read a human mind.
I’d also love to hear more about how machine learning was used in this project and whether any biologically inspired neural network architectures were applied (e.g. CNN etc.) or developed as part of this research.
It’s great work by team. Congratulations on your sincere effort. Looking forward for your future work and results.
Thank you, Raghu. And Congrats to Battelle. This research was featured in the Wall Street Journal this week as one of the “6 New Medical Technologies Worth Watching”. Here’s the link: http://www.wsj.com/articles/six-new-medical-technologies-worth-watching-1466992921
What a leap for BCI, congratulations for such achievement.
On the other hand, for wireless connectivity, increasing the number of electrodes would increase size ratio. instead what if we could do this-
We are collecting GBs of neuronal data every second, and then applying machine learning algos to analyze pattern.
What if instead of taking millions of neuronal input every time, we sought out those neurons which are required for the particular muscle movement and omit rest of the data. In this case, we would only require more data in the initial phase, but then later the data could be predicted and reduced.
Also for the skin electrodes, if we will be using wireless technology in the future, we could use different frequencies to input data of the brain and the skin electrodes minimising the risk of interference with the two signals.
This work really sounds good.
I am doing work on the assessment of Cognitive parameters through several electronic methods…. It’s very helpful for me.
Thank you Lisa for sharing such a nice information.
If you’d like to read more, here’s a user story on this project.
this is beyond huge. congrats to the team who did this, and to the patient whose very persistence made this possible. As we can read the neural activity of more neurons and we enable the wireless transmission of more data from neurons to electrodes implanted in the muscles, this could enable many people who suffered accidents to leave a more fulfilling life. It will also help us understand better how the brain works. Kudos to the entire team.
What a marvelous use of technology…very happy to find that MATLAB has been an integral part of it..