Virginia Tech® home

Vicarious Offensive Language Identification

Researchers will study content moderation to better understand human and machine perception of what others might find offensive by exploring how one group thinks another group will perceive potentially offensive content. The goal is to develop more inclusive, transparent, and trustworthy content moderation processes. 

Funded by the CCI Northern Node

Project Investigator

Principal Investigator (PI): Marcos  Zampieri, George Mason University School of Computing Department of Information Sciences and Technology

Rationale and Background

Previous studies have focused on asking raters, “Do you find this content offensive?” 

Researchers conducted exploratory studies that showed hot-button issues such as reproductive rights and gun control influenced raters’ ability to predict vicarious offenses. Assigning random annotators also often produced results that reflected their own biases.

The studies showed the need to consider such factors as gender and age when selecting annotators.

Methodology

The research will focus on four areas:

  • How aligned are offense predictions of human moderators across different political beliefs in the socio-political context of the United States? The study will expand on previous research involving Independents, Democrats, and Republicans.
  • How well does the concept of vicarious offense translate to demographic groups? Researchers will examine data from previous studies.
  • How do state-of-the-art large language models (LLMs) perform in vicarious offense? Recent research indicates personas can be induced in LLMs, threatening their ability to moderate content. Researchers will evaluate LLMls’ performance and contrast it with human performance.
  • How can the notion of vicarious offensive contribute to robust, fair, and efficient moderation? Aligning posts with the right moderators is vital.

Projected Outcomes

  • Findings will be the foundation for the pursuit of an NSF CAREER Award to be submitted in 2025.
  • Financial support will be sought from companies that seek proposals on content moderation, such as Meta, Amazon, and Google. 
  • Using proposed models for content moderation will alleviate the psychological burden on social media moderators exposed to large amounts of offensive content.