In this month’s research spotlight, COSMOS highlights recent research that uses epidemiological models to investigate the spread and mitigation of online toxicity. Specifically, the paper authored by Niloofar Yousefi and Dr. Nitin Agarwal titled “Studying the Influence of Toxicity Intensity on Its Propagation Using Epidemiological Models” uses the Susceptible-Toxic-Recovered-Susceptible (STRS) model to study toxicity. The paper will be published at the 30th Americas Conference on Information Systems (AMCIS) held August 15-17, 2024 in Salt Lake City, Utah.

Toxicity is a rising concern in the age of social media. This study explores the ways in which toxicity spreads like a disease, using epidemiological models that separate populations into groups based on whether individuals are susceptible to, infected/affected by, or recovered from an illness like disease, toxicity, or misinformation. Such models can also consider specific conditions like being extremely toxic or quarantined to further improve their efficacy in mitigating toxicity. 

In particular, the research comparatively uses the STRS model, focusing on comparing error rates with earlier epidemiological models to evaluate the improvement STRS provides in modeling. Namely, the research compared STRS to the earlier models of SIR (Susceptible-Infectious-Recovered) and SIS (Susceptible-Infectious-Susceptible). STRS stands out in that it accounts for the rate at which the recovered can again become susceptible to infection/toxicity. Using datasets for COVID-19, violent protests in Brazil’s Capitol riot on January 8, 2023, and social movements in Peru during 2022 and 2023, the authors compared the error rates for the three different models. 

The authors found that the STRS model outperformed the two others in capturing toxicity diffusion dynamics, indicated by a lower margin of error. Additionally, a part of this analysis accounted for the difference between moderate and high toxicity cases. Interestingly, the model showed a lower error rate for the COVID-19 dataset for moderately toxic users, while the model had lower error rates for highly toxic users in Brazil’s violent protest and Peru’s social movement dataset. This shows that, for certain contexts, different intensities of toxic individuals will more closely conform to the STRS model. In both cases, the separation between moderate and high toxicity improved the accuracy of the model significantly.

Older epidemiological models like SIS were made for general disease trends, but newer models like the STRS model, by accounting for special conditions that spread like toxicity and its intensity, don’t just outperform the older ones; the STRS model confirms that toxicity spread is affected by its intensity, and can be used to adapt even more the current understanding of toxicity as epidemiological. In short, this research will help policymakers better understand and predict the dynamics of toxicity spread. It also confirms that different contexts reflect different spreads of toxicity, captured in how certain intensities of toxicity conformed to the model predictions.

Dr. Agarwal said, “This research provides valuable insights into toxicity spread dynamics, informing strategies for managing online discourse, thereby enhancing the resiliency of our communities. By employing a more accurate model and considering toxicity intensity, we enhance our understanding and modeling of its propagation. Another key contribution of this research lies in categorizing individuals based on their toxicity scores, providing insights into how toxicity levels influence content dissemination. For policymakers, this means having a more granular view of where intervention efforts should be focused. By aligning intervention efforts with the patterns of toxicity identified in this research, policymakers can optimize the allocation of resources and maximize their impact in fostering healthier and more resilient online environments.”