Clustering Recurrent Hand Postures of Individuals with Spinal Cord Injury in Wearable Camera Video

Sumitro, Elizabeth 1, 2 ; Likitlersuang, Jirapat 1, 2 ; Kalsi-Ryan, Sukhvinder 3, 4 ; Zariffa, José 1, 2

1. Institute of Biomaterials and Biomedical Engineering, University of Toronto; 2. Toronto Rehabilitation Institute, University Health Network; 3. Krembil Neuroscience Spine Program, Toronto Western Hospital, University Health Network; 4. Department of Physical Therapy, University of Toronto

Background: Recovery of hand function is a priority for individuals with tetraplegia after a spinal cord injury (SCI). Existing hand function assessments are limited to a clinical setting. Use of wearable cameras allows for monitoring at home or in the community, providing data that is more representative of everyday hand function. Manual analysis of the resulting lengthy videos is infeasible, but automatic summarization could enable interpretation by a clinician.

Objectives: This study seeks to develop an algorithm that can identify recurrent hand postures of individuals with SCI in wearable camera video and summarize its findings in a report. The report is intended for use by a clinician to evaluate recovery of hand function.

Methods: The clustering algorithm employed was based on the Determinantal Point Process. Image features were extracted by representing each hand using a masked histogram of orientated gradients descriptor, which has been shown to work well for videos of able-bodied individuals. We validated this approach using 1,125 hand images from 3 individuals with SCI.

Results: 80.3% of hand images were assigned to a correct cluster. 51.8% of clusters formed were redundant. Some infrequent postures were not detected, corresponding to 13.6% of the images.

Conclusions: Clustering hand postures in individuals with SCI is feasible and has applications in the automated summarization of wearable camera video. Further algorithm optimization is needed to reduce cluster redundancy and the number of undetected postures. This work will facilitate the evaluation of interventions for recovery of hand function, thus helping to restore independence after SCI.