Propaganda, Fallacies, and Media Integrity

This research has been supported in part by the AI Innovation Institute (AI3) at Stony Brook University.

AI3 Seed Grant (01.2025 - 06.2026)

Ritwik Banerjee, Principal Investigator
Ruobing Li, School of Communication and Journalism, Co-Investigator
Chenlu Wang, Doctoral Researcher  →  Research Scientist, Meta
Weimin Lyu, Doctoral Researcher  →  Applied Scientist, Amazon
Parth Thapliyal, M.S. Researcher
Jelwin Rodrigues, M.S. Researcher
Abhishek Kalugade, M.S. Researcher
Harsh Gupta, M.S. Researcher
Taisiia Sabadyn, Undergraduate Researcher
Ritesh Sunil Chavan, Undergraduate Researcher
Samridh Samridh, Undergraduate Researcher

Collaborators

Dikshya Mohanty, Doctoral Researcher
Chaoyuan Zuo, Lecturer (tenure-track), School of Journalism & Communication, Nankai University

Large language models have achieved striking performance on NLP benchmarks while failing systematically at a class of tasks that requires pragmatic competence rather than surface-level semantic matching: detecting propaganda techniques and fallacious argumentation in online discourse. This is not a minor gap. Propaganda operates through framing, selective emphasis, and emotional manipulation — none of which leaves a straightforward lexical signature. Fallacies like whataboutism, appeal to majority, and false causality derive their persuasive power precisely from their resemblance to valid argument forms. A model that cannot distinguish the structure of an argument from its surface appearance will miss both, allowing complex misinformation to pass undetected.

This project investigates the computational roots of this failure and develops targeted responses. It is organized around three objectives.

Diagnosing LLM Limitations

The first objective characterizes why current models fail, not just that they fail. Using a multi-pronged diagnostic approach — thematic analysis of error patterns, adversarial testing with near-minimal pairs, and attention mechanism analysis — we map the specific aspects of pragmatic context that transformer-based models are unable to integrate. Prior work by the PI on whataboutism detection demonstrated that cross-attention weights carry recoverable signals about discourse-level intent that standard classification heads do not exploit; this project extends that diagnostic lens to the broader landscape of propaganda techniques and argumentation fallacies, including their co-occurrence, which amplifies persuasive power in ways that models trained on isolated instances cannot recognize.

Developing Improved Detection Models

The second objective develops detection models informed by these diagnostics. The key methodological insight is that sociocultural context external to a single document — captured through cross-document discourse signals from social media commentary and adjacent reporting — provides the pragmatic grounding that in-document context alone cannot supply. Preliminary results show that integrating this broader context through parametric methods and attention-based distance metrics outperforms retrieval-augmented generation systems while requiring substantially less training data. The class distillation framework developed in parallel work provides an efficient training paradigm particularly suited to the small-target-class structure of propaganda and fallacy detection tasks [Wang et al. 2025].

A Corpus for Narrative Divergence and Information Integrity

Understanding propaganda and information distortion at scale requires corpora that capture how the same events are narrated across different national media ecosystems — not just within a single genre or language. We constructed dnipro (Diverse Narratives and International Perspectives on the Russo-Ukrainian Offensive), a longitudinal, multinational, and multilingual corpus spanning 31 months of news coverage of the Russo-Ukrainian war, drawn from sources in the United States, United Kingdom, Ukraine, Russia, and China, in English, Russian, and Mandarin Chinese [Mohanty et al. 2026]. dnipro is released on Zenodo as a public resource¹ and supports research on narrative divergence, counter-narrative analysis, and the cross-national propagation of contested claims — questions that are at the heart of computational approaches to propaganda analysis and information integrity.

¹   Mohanty, D., Sabadyn, T., Rodrigues, J., Wang, C., Kalugade, A., & Banerjee, R. (2026). Diverse Narratives and International Perspectives on the Russo-Ukrainian Offensive (DNIPRO) (Version v1) [Data set]. Zenodo. doi: 10.5281/zenodo.18470677

Scientific Discourse and Cross-Genre Verification

A parallel thread addresses misinformation propagation through the scientific literature pipeline, where peer-reviewed findings are translated into journalism and social media under pressures that distort both accuracy and emphasis. We developed and evaluated systems for the CLEF CheckThat! 2025 shared task, which targets scientific claim verification by bridging social media discourse, science journalism, and the primary scientific literature [Thapliyal et al. 2025]. Complementary work on LLM-driven biomedical named entity recognition [Gupta and Banerjee 2025] contributes to the information extraction infrastructure that underlies cross-genre claim verification, while also probing the capabilities and limitations of LLMs in specialized biomedical domains.

Media Literacy and Education

The third objective translates research findings into educational materials for students in journalism and communication — populations that are methodologically sophisticated but underserved by technical AI literacy curricula. Working with the School of Communication and Journalism, the Center for News Literacy, and the Alan Alda Center for Communicating Science, the project develops lecture modules and case studies integrating these findings into core courses and outreach programs, contributing to a broader public understanding of how AI can and cannot be used to identify manipulation in media.

Publications

Mohanty, D., Sabadyn, T., Rodrigues, J., Wang, C., Kalugade, A., and Banerjee, R. 2026. A Longitudinal, Multinational, and Multilingual Corpus of News Coverage of the Russo-Ukrainian War. Proceedings of the Fifteenth Language Resources and Evaluation Conference (LREC 2026), European Language Resources Association (ELRA), 6452–6471.     

Wang, C., Lyu, W., and Banerjee, R. 2025. Class Distillation with Mahalanobis Contrast: An Efficient Training Paradigm for Pragmatic Language Understanding Tasks. Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), Association for Computational Linguistics, 29428–29442.     

Thapliyal, P.M., Chavan, R.S., Samridh, S., Zuo, C., and Banerjee, R. 2025. SCIRE at CheckThat! 2025: Bridging social media, scientific discourse, and scientific literature. Working Notes of CLEF 2025 - Conference and Labs of the Evaluation Forum, CEUR-WS.org, 1256–1264.     

Gupta, H.P. and Banerjee, R. 2025. SCIRE at BioASQ 2025: LLM-Driven Biomedical Named Entity Recognition for GutBrainIE 2025. Working Notes of CLEF 2025 - Conference and Labs of the Evaluation Forum, CEUR-WS.org, 281–291.     

This project page is hosted and maintained by the principal investigator, Dr. Ritwik Banerjee.