Video Characterization

Video Characterization

• Overview
• Characterizing tactics
• Dynamic Social Campaigns

Overview

In recent years, several frameworks have been proposed to communicate and characterize disinformation campaigns and incidents of information manipulation and interference online, while some have been incorporated into the existing institutional frameworks by government agencies and industry. Some of the proposed frameworks aimed at systematic documentation and communication of reported campaigns across institutions, while others focused on the characterization of behaviors, tactics, techniques, and procedures (TTPs) employed by threat actors. Also, a few frameworks characterize the narrative and rhetorical characteristics of influence in general. The existing frameworks offer great value in overall understanding and timely detection of influence operations. However, almost all the proposed frameworks cannot address the emerging developments in the modern information environment and pose a number of limitations to the characterization of high-risk hostile influence operations. In this study, we emphasize the use of multimedia environments, including video, audio, real-time broadcasting, and real-time audio chat environments for online influence operations as a high-priority area of focus for an additional computational and analytical framework. All in all, we posit that the expansion of the current set of frameworks with a multi-modal approach toward such social media platforms would improve the efforts to monitor, track, detect, and mitigate the emerging threats in the information environment.

The state-of-the-art characterization frameworks have several limitations as mentioned below.

1. Most critical limitation of existing characterization frameworks stems from the fact that they do not consider the dynamic characterization of information actors (producers and consumers) and the information campaign process. To develop a rich understanding of the OIE, all three elements are critical, i.e., tactics, information actors, and the campaign’s dynamics. Our characterization framework considers the information actor and the campaign dynamics to develop effective countermeasures.

2. Existing characterization frameworks exclude comprehensive evaluation of non-textual data, platform affordances, and metadata such as audio, video, engagement statistics, recommendation information, and dynamic network information. Further, time series data analysis needs to be considered for a near real time evaluation. Our characterization framework considers all the above-mentioned data elements as well as time series information.

3. There is not much focus on impact assessment in existing frameworks. These frameworks mainly focus on defining TTPs and identifying/detecting these and not on impact assessment. In our approach, we address these gaps. Further, our approaches have rigorous theoretical and mathematical grounding.

We are developing a characterization framework for multimedia-rich social platforms. The framework addresses the gaps in the current frameworks (discussed above) and presents novel ways of characterization of multimedia-rich social platform-based behaviors. The framework also provides analytical tools to not only identify such behaviors but also measure their impact.

Our characterization framework considers five approaches:

• Characterizing multimedia OIE tactics and impact assessment;
• Characterizing dynamics of social/campaign process for better characterization of campaign phases;
• Characterizing information actors (producers and consumers) dynamically;
• Characterizing network coordination structures; and
• Characterizing mobs and their dynamics.

These characterization approaches afford understanding of multimedia OIE from a multidimensional and dynamic perspective to develop countermeasures/ intervention strategies and further measure response effectiveness. These approaches have rigorous mathematical and theoretical grounding for better explainability, interpretability, and transparency. The taxonomy trees illustrate our characterization approaches.