Enhancing Emotion Recognition: Machine Learning with Phasic Spectrogram Texture Features

Abstract

This study aims to develop an emotion detection system that involves the phasic decomposition of electrodermal activity (EDA) signals, segmentation, texture feature extraction, and machine learning algorithms. Initially, the EDA signals are obtained from the continuously annotated signals of emotions dataset and decomposed into tonic and phasic signals using convex optimization approach to electrodermal activity decomposition method. The phasic signals are divided into two halves, and each half is further divided into five non-overlapping segmented windows. Spectrograms are generated using the short-time Fourier transform for each segmented window of the phasic signal, and texture features are extracted using the gray-level co-occurrence matrix and gray-level run length matrix. We built machine learning models such as logistic regression, random forest (RF), and extreme gradient boosting using texture features extracted from the spectrogram of each segment of the first, and second half of phasic component of EDA signals. The results of our experiment using a process pipeline that included second-half phasic segments of EDA, spectrogram, texture features, and RF classifier yielded the highest classification results. We achieved an average accuracy, sensitivity, specificity, precision, and f1-score of 78.5%, 52%, 84.61%, 57.31%, and 53.36%, respectively. The results demonstrate the potential of this approach for emotion detection from EDA signals and can be useful in psychology, human-computer interaction, and health monitoring. © 2023 IEEE.