A reliable and efficient machine learning pipeline for american sign language gesture recognition using EMG sensors

Abstract

Sign languages has extensive applications among differently-abled to communicate with their surroundings. With the development of different sensing technologies, several new human-computer interaction techniques (HCI) have been established to recognize hand gestures. Computer vision-based methods have shown significant utility for such applications. However, these methods are strongly dependent on the lighting conditions. The surface electromyography (sEMG) technique is invariant to lighting conditions and can easily reflect human motion intention. In this work, sEMG based sign language recognition model was developed using an efficient machine learning pipeline. Two sEMG datasets were recorded for predefined hand gestures using wireless sensors. These signals were mainly acquired against 24 manual alphabets (ASL-24) and ten digits(ASL-10) of American Sign Language (ASL). The collected data sets were preprocessed, and around 450 well-established feature was extracted from each sEMG channel. We applied an ensemble feature selection approach combining four diverse filter-based feature selection methods (ANOVA, Chi-square, Mutual Info, ReliefF). A newly proposed feature combiner that exploits feature–feature and feature–class correlation thresholds is used to combine feature subsets formed across the ensemble. The resulting features comprise reduced & most representative feature subsets and are further used in the pipeline for classifying ASL gestures. Using the CatBoost algorithm, the pipeline presented excellent average classification accuracy(99.91% on ASL-24) and other performance parameters for recognizing ASL gestures. The pipeline was also applied and validated on a benchmark dataset (Ninapro database 5, exercise A) and achieved similar outcomes. The result highlights the feasibility of using sEMG based approach as better options to computer-vision-based techniques to build an accurate and robust Sign Language Recognition system (SLRS). Moreover, efforts were made to find the optimal number of sensors and features for recognition task on (ASL-10 dataset) without impacting the overall reliability and accuracy of the system. The experiments results can be used to enhance the performance of various wearable sEMG sensor based HCI applications. © 2022, The Author(s), under exclusive licence to Springer Science+Business Media, LLC, part of Springer Nature.