Institute of Biomedical Engineering (BME)

A helping hand for neurorehabilitation

May 19, 2021

Dr. Jose Zariffa and his research team are developing tools that enable clinicians to evaluate rehabilitation success and track recovery. The goal is to create a personalized process for the improvement of rehabilitation from spinal cord injuries and stroke, accelerating the affected individuals back on the road to recovery.

What is neurorehabilitation?

After a spinal cord injury or a stroke, the neural commands that control movement are often interrupted, and the goal of rehabilitation is to help restore the lost function. In best cases, the individual regains complete recovery to their motor functions, but often the results can be sub-optimal. At this point, any improvement that rehabilitation can provide can have a real impact on independence and quality of life.

“There is a balance of two processes: recovery and compensation,” said Dr. Jose Zariffa, an associate professor at the Institute of Biomedical Engineering (BME) and Senior Scientist at KITE, the research arm of the Toronto Rehabilitation Institute – University Health Network. “In recovery, the individual’s motor function improves, while in the case of compensation, they maximize what they can do within the constraints of the remaining function by learning to perform tasks in a different way.”

During rehabilitation, an interdisciplinary team works with each patient to help them regain the ability to independently carry out activities that are important to them. The process and results can vary greatly based on the nature of the injury, the person’s goals, and the treatments available. Being able to accurately assess the abilities of each person over time is key to this process.

The current practice of measuring function in rehabilitation involves direct testing by a clinician. “The shortcoming of this method”, said Dr. Zariffa, “is that it is difficult to track function during daily life in the home and community, and therefore to evaluate the true impact of the care that was delivered.”

You say egocentric, I say Point of View (POV)

One of the most important determinants of independence after neurological injuries is hand function. Dr. Zariffa and his team are interested in giving clinicians and researchers, for the first time, the ability to measure how an individual is using their hands in real activities of daily living at home.

Members of the Adaptive Neurorehabilitation Systems Lab who are leading the egocentric video research. Clockwise from top left: Dr. Andrea Bandini, Meng-Fen Tsai, Adesh Kadambi, and Mehdy Dousty (Photos: Zariffa lab)

“Our solution is to equip the injured individual with a wearable camera where they can film their day-to-day interactions with objects at home, with their hands in the field of view.” Said Dr. Zariffa, “From these egocentric videos, we can extract information regarding the recovery process, which can then be used to evaluate the impact of a therapy or returned to the clinicians such as physiotherapists to modify their treatment regimen.”

While humans can intuitively interpret a grasping motion, translating these actions it into an algorithm is a complex topic. For example, recognizing a hand in a complex background is in itself a challenge, and a moving hand further blurs the miniscule detail involved in interpreting the data.

By using artificial intelligence (AI) and deep learning algorithms, Dr. Zariffa and his team are expecting to translate a large set of video footage into biometric data that can be understood by clinicians.

“From the videos, we can get a sense of a person’s independent hand use, and put their motor function in context.” Said Dr. Zariffa. “A lot of this information would be extremely valuable to a therapist, and can help them decide what to work on with the person in order to either achieve a more functional performance or to help the person work towards reestablishing movement patterns that are similar to what they had preinjury.”

This research was the winner of the $100,000 grand prize for the MaRS and Praxis Spinal Cord Rehab Innovation Challenge earlier in 2021.

Challenges and next steps

An issue with the data extraction is the scarcity of information and data available. In order to train a large enough data set using AI for maximum accuracy, a substantial amount of patient information is required. Dr. Zariffa and his team have collected several large egocentric video datasets of individuals with a wide range of hand function in laboratory and home environments, resulting in dozens of hours of video and hundreds of thousands of annotated images.

The future of neurorehabilitation, as imagined by Dr. Zariffa, is to improve our understanding of the interplay between all the factors involved, then use this information to determine the best possible treatment for each person.

“If we can make sense of patient data such as their demographics, injury, and use technology to track function in more detail, we can pick the best avenue for this person,” said Dr. Zariffa, “Now we’re finally at a stage where we’re starting to have tools that are going to enable us to make that happen.”