Pragmatic Language Understanding

This research has been supported in part by the Secure & Trustworthy Cyberspace (SaTC) program of the U.S. National Science Foundation (NSF) and the AI Innovation Institute (AI3) at Stony Brook University.

SES-1834597 (05.2018 - 03.2022)  |  CNS-2335686 (10.2023 - 09.2025)  |  AI3 Seed Grant (01.2025 - 06.2026)

Ritwik Banerjee, Principal Investigator
  Chenlu Wang, Doctoral Researcher  →  Research Scientist, Meta
  Khiem Phi, Doctoral Researcher
  Noushin Salek Faramarzi, Doctoral Researcher  →  NLP Researcher, Boeing AI
  Weimin Lyu, Doctoral Researcher  →  Applied Scientist, Amazon

Collaborators

Indrakshi Ray, Professor of Computer Science, Colorado State University at Fort Collins
  Sina Mahdipour Saravani, M.S. Researcher, Colorado State University  →  Ph.D. Student, University of Utah

Overview

Pragmatic language phenomena — sarcasm, metaphor, sexism — often share a structural challenge that standard NLP classification setups handle poorly: the target class is small, semantically coherent, and precisely defined, but it sits inside a large and highly heterogeneous background of superficially similar discourse. A model that optimizes for accuracy on such an imbalanced distribution is learning the semantic patterns instead of the intent-driven phenomena. The same failure mode appears at the discourse level: whataboutism deploys the topical changes in the surface forms, but so do ordinary (and sometimes helpful) comparative questions and legitimate contextualizations. A classifier anchored to semantics alone will systematically conflate a genuine rhetorical deflection with discourse that merely resembles it. Detecting pragmatic phenomena requires attending to the pragmatic function, not the semantic form.

Three lines of work in our group converge on this problem. The first established what does not help. We investigated NeXtVLAD — a locally aggregated descriptor architecture borrowed from computer vision that had produced a striking 93.1% F1 on the FigLang2020 sarcasm detection task, fourteen points above the next-best system. The result appeared to signal a major architectural advance for figurative language identification. Controlled ablation experiments, however, demonstrated that the gain is not statistically significant when proper baselines are held constant, and the resource-intensive NeXtVLAD component adds no measurable benefit for figurative language identification [Saravani et al. 2021]. The right response to a hard pragmatic task is not a larger or more elaborate model — it is a more principled training objective.

The “Tu Quoque” Fallacy of Argumentation

Whataboutism offered a test case for this principle. The phenomenon — a rhetorical deflection that shifts attention rather than addressing the immediate concern — is pragmatically broader than the tu quoque fallacy, and often distinct from propaganda. But these distinctions had been systematically conflated in prior work, which treated the “what about” surface pattern as a reliable proxy for the underlying fallacious argumentation. We introduce two manually annotated datasets (Twitter/X and YouTube) that make these distinctions explicit, and propose a training method that mines hard negatives by exploiting attention-weight signals — targeting the subtle cases where whataboutism overlaps with (but is not identical to) related rhetorical devices [Phi et al. 2024]. The method achieves 4% and 10% absolute improvements over prior state-of-the-art on the two collections, respectively. The deeper result is methodological: discourse-level signals of function, not semantic forms, are what allow a model to track the strategic intent of an utterance rather than its propositional content.

Class Distillation

Our subsequent work on class distillation (ClaD) provides the training paradigm that generalizes this insight across pragmatic language tasks. ClaD is built around two components: a loss function grounded in Mahalanobis distance that exploits the geometric structure of class distributions rather than treating all dimensions as equally informative, and an interpretable β-decision algorithm optimized explicitly for clean separation between the target class and the heterogeneous background of “everything else” [Wang et al. 2025]. On three benchmark tasks — sexism, metaphor, and sarcasm detection — ClaD with small language models matches or outperforms both standard fine-tuned classifiers and several large language models with orders of magnitude more parameters.

Ablation experiments confirm that both components are necessary: removing either the Mahalanobis contrast loss or the β-decision algorithm causes F1 drops of 46–81% depending on the task. For pragmatic language tasks where the target class has clear distributional structure, explicit geometric modeling of that manifold is more valuable than raw model scale.

Publications

Wang, C., Lyu, W., and Banerjee, R. 2025. Class Distillation with Mahalanobis Contrast: An Efficient Training Paradigm for Pragmatic Language Understanding Tasks. Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), Association for Computational Linguistics, 29428–29442.     

Phi, K., Salek Faramarzi, N., Wang, C., and Banerjee, R. 2024. Paying Attention to Deflections: Mining Pragmatic Nuances for Whataboutism Detection in Online Discourse. Findings of the Association for Computational Linguistics: ACL 2024, Association for Computational Linguistics, 12628–12643.     

Saravani, S.M., Banerjee, R., and Ray, I. 2021. An Investigation into the Contribution of Locally Aggregated Descriptors to Figurative Language Identification. Proceedings of the Second Workshop on Insights from Negative Results in NLP, Association for Computational Linguistics, 103–109.     

This project page is hosted and maintained by the principal investigator, Dr. Ritwik Banerjee.