Learning models to forecast toxicity in conversation threads by identifying potential toxic users

Abstract

While most conversations on social media platforms promote freedom of expression, some can take a harmful turn, potentially impacting users’ mental well-being. Therefore, it’s imperative to monitor and moderate these discussions. Significant efforts have been made to classify conversations as either toxic or non-toxic. However, post-detection toxicity methods are insufficient. In the proposed work, our focus lies on predicting the toxicity level of conversations before they cause harm. We aim to preemptively address toxic conversations by forecasting the degree of toxicity in advance. Our approach integrates essential user behavior features to optimize toxicity forecasting. Leveraging various learning models such as State Space, Convolution Neural Network (CNN), Long-short Term Memory (LSTM), Bi-directional Long-short Term Memory (Bi-LSTM), and Ensemble models, we analyze features like the proportion of toxic replies and the similarity of toxicity patterns to those of the most toxic users in the dataset. Our proposed method significantly surpasses state-of-the-art models, demonstrating around six-fold improvement. Additionally, it achieves an 82.31% reduction in mean average error and a 78.3% reduction in root mean squared error compared to the state-of-the-art. © The Author(s), under exclusive licence to Springer-Verlag GmbH Germany, part of Springer Nature 2024.