Research Spotlight: Multimedia Datasets

In this month’s newsletter, we delve into the themes of papers presented by cosmographers, both of which were presented and published recently at the SBP-BRiMS 2023 conference. Both delve into multimedia datasets, specifically YouTube and Instagram, although for different purposes. “The Role of Instagram’s Multimedia on Social Unrest in Brazil and Peru: A Connective Action Analysis Based on Diffusion of Innovation and Cognitive Mobilization Theories,” by Mainuddin Shaik, Niloofar Yousefi, Nitin Agarwal, and Billy Spann, attempts to see how multimedia objects such as images, videos, and reels  on Instagram can more effectively mobilize social unrest and mobs. While “Analyzing Narrative Evolution about South China Sea Dispute on YouTube: An Exploratory Study using GPT-3,” by Mayor Inna Gurung, Hayder Al Rubaye, Nitin Agarwal, and Ahmed Al-Taweel, studies the YouTube platform to investigate specific narratives and their evolution. 

The narrative analysis research looks at the South China Sea dispute and uses machine learning, NLP, and AI to extract and visualize narratives from YouTube videos related to the dispute.  Mayor Inna said, “the visualization approach empowers users to track narratives. Users also have the flexibility to edit these narratives for future iterations of our multiple-shot narrative extraction work.”

The cognitive mobilization research evaluates the theory of Diffusion of Innovation to examine the dissemination of multimedia (Instagram) objects during recent protests in Brazil and Peru, and how that influences participation of protesters in such events. The study analyzes 78,478 images, 12,967 sidecars, and 8,810 videos from Instagram posts related to recent protests in Brazil and Peru,  most significantly in the context of protesters coming together in social unrest.

Findings reveal distinct multimedia usage patterns and their effectiveness in different stages of a social movement, highlighting the importance of understanding the unique characteristics of each multimedia type to drive connective action. This study contributes to the understanding of modern activism and the strategic incorporation of multimedia in mobilizing collective action.

Dr. Agarwal said, “both the studies highlighted here demonstrate that our approaches are always grounded in theories and evaluated in real-world contexts for meaningful impacts to science as well as society.”