Emotion Classification Through Optimal Segments of EDA and Texture Analysis of Time-Encoded Images With Artificial Intelligence

Abstract

In this study, we focus on investigating the time-encoded images of electrodermal activity (EDA) segments to identify significant patterns for an emotion recognition system. Initially, the EDA signals were procured from two openly accessible datasets, namely, continuously annotated signals of emotions (CASE) and wearable stress and affect detection (WESAD). These signals were then preprocessed and decomposed into phasic signals through a convex optimization approach. Subsequently, we divided the phasic signals into two equal segments, each further subdivided into nine equal windows with a 50% overlap. Moreover, we generated time-encoded image representations of these windowed phasic signals using a Gramian angular summation field (GASF), Markov transition field (MTF), recurrence plot (RP), and a fusion of these images. In addition, we extracted 85 textural features based on the gray-level co-occurrence matrix (GLCM), gray-level run-length matrix (GLRLM), fractal dimension texture analysis (FDTA), Zernike's moments (ZMs), Hu's moments (HMs), and first-order statistics (FOSs). Four machine learning (ML) models, including logistic regression (LR), random forest (RF), extreme gradient boosting (XGB), and multilayer perceptron (MLP), were developed to classify two-class emotions associated with arousal and valence from CASE, as well as three-class emotions (amusement, neutral, and stress) from WESAD, considering three different approaches: the first half, the second half, and the whole phasic signal. The highest classification accuracy achieved was 79.79% for two-class arousal and 71.71% for two-class valence on the CASE. In contrast, our models demonstrated an outstanding emotional classification accuracy of 98.4% for the three-class emotion in the WESAD. These outcomes highlight the potential of our proposed methodology for analyzing emotions in healthcare, with the ability to accurately classify emotions holding promising implications for improving patient care, mental health assessment, and overall well-being. © 1963-2012 IEEE.