$(RSA)^2$: A Rhetorical-Strategy-Aware Rational Speech Act Framework for Figurative Language Understanding
Extracted source text · arXiv 2506.09301
% This must be in the first 5 lines to tell arXiv to use pdfLaTeX, which is strongly recommended.
\pdfoutput=1
% In particular, the hyperref package requires pdfLaTeX in order to break URLs across lines.
\documentclass[11pt]{article}
% Remove the "review" option to generate the final version.
\usepackage[review]{acl}
% Standard package includes
\usepackage{times}
\usepackage{latexsym}
% For proper rendering and hyphenation of words containing Latin characters (including in bib files)
\usepackage[T1]{fontenc}
% For Vietnamese characters
% \usepackage[T5]{fontenc}
% See https://www.latex-project.org/help/documentation/encguide.pdf for other character sets
% This assumes your files are encoded as UTF8
\usepackage[utf8]{inputenc}
% This is not strictly necessary, and may be commented out,
% but it will improve the layout of the manuscript,
% and will typically save some space.
\usepackage{microtype}
% If the title and author information does not fit in the area allocated, uncomment the following
%
%\setlength\titlebox{<dim>}
%
% and set <dim> to something 5cm or larger.
\title{Instructions for *ACL Proceedings}
% Author information can be set in various styles:
% For several authors from the same institution:
% \author{Author 1 \and ... \and Author n \\
% Address line \\ ... \\ Address line}
% if the names do not fit well on one line use
% Author 1 \\ {\bf Author 2} \\ ... \\ {\bf Author n} \\
% For authors from different institutions:
% \author{Author 1 \\ Address line \\ ... \\ Address line
% \And ... \And
% Author n \\ Address line \\ ... \\ Address line}
% To start a seperate ``row'' of authors use \AND, as in
% \author{Author 1 \\ Address line \\ ... \\ Address line
% \AND
% Author 2 \\ Address line \\ ... \\ Address line \And
% Author 3 \\ Address line \\ ... \\ Address line}
\author{First Author \\
Affiliation / Address line 1 \\
Affiliation / Address line 2 \\
Affiliation / Address line 3 \\
\texttt{email@domain} \\\And
Second Author \\
Affiliation / Address line 1 \\
Affiliation / Address line 2 \\
Affiliation / Address line 3 \\
\texttt{email@domain} \\}
\begin{document}
\maketitle
\begin{abstract}
This document is a supplement to the general instructions for *ACL authors. It contains instructions for using the \LaTeX{} style files for ACL conferences.
The document itself conforms to its own specifications, and is therefore an example of what your manuscript should look like.
These instructions should be used both for papers submitted for review and for final versions of accepted papers.
\end{abstract}
\section{Introduction}
These instructions are for authors submitting papers to *ACL conferences using \LaTeX. They are not self-contained. All authors must follow the general instructions for *ACL proceedings,\footnote{\url{http://acl-org.github.io/ACLPUB/docs/formatting.html}} and this document contains additional instructions for the \LaTeX{} style files.
The templates include the \LaTeX{} source of this document (\texttt{acl.tex}),
the \LaTeX{} style file used to format it (\texttt{acl.sty}),
an ACL bibliography style (\texttt{acl\_natbib.bst}),
an example bibliography (\texttt{custom.bib}),
and the bibliography for the ACL Anthology (\texttt{anthology.bib}).
\section{Engines}
To produce a PDF file, pdf\LaTeX{} is strongly recommended (over original \LaTeX{} plus dvips+ps2pdf or dvipdf). Xe\LaTeX{} also produces PDF files, and is especially suitable for text in non-Latin scripts.
\section{Preamble}
The first line of the file must be
\begin{quote}
\begin{verbatim}
\documentclass[11pt]{article}
\end{verbatim}
\end{quote}
To load the style file in the review version:
\begin{quote}
\begin{verbatim}
\usepackage[review]{acl}
\end{verbatim}
\end{quote}
For the final version, omit the \verb|review| option:
\begin{quote}
\begin{verbatim}
\usepackage{acl}
\end{verbatim}
\end{quote}
To use Times Roman, put the following in the preamble:
\begin{quote}
\begin{verbatim}
\usepackage{times}
\end{verbatim}
\end{quote}
(Alternatives like txfonts or newtx are also acceptable.)
Please see the \LaTeX{} source of this document for comments on other packages that may be useful.
Set the title and author using \verb|\title| and \verb|\author|. Within the author list, format multiple authors using \verb|\and| and \verb|\And| and \verb|\AND|; please see the \LaTeX{} source for examples.
By default, the box containing the title and author names is set to the minimum of 5 cm. If you need more space, include the following in the preamble:
\begin{quote}
\begin{verbatim}
\setlength\titlebox{<dim>}
\end{verbatim}
\end{quote}
where \verb|<dim>| is replaced with a length. Do not set this length smaller than 5 cm.
\section{Document Body}
\subsection{Footnotes}
Footnotes are inserted with the \verb|\footnote| command.\footnote{This is a footnote.}
\subsection{Tables and figures}
See Table~\ref{tab:accents} for an example of a table and its caption.
\textbf{Do not override the default caption sizes.}
\begin{table}
\centering
\begin{tabular}{lc}
\hline
\textbf{Command} & \textbf{Output}\\
\hline
\verb|{\"a}| & {\"a} \\
\verb|{\^e}| & {\^e} \\
\verb|{\`i}| & {\`i} \\
\verb|{\.I}| & {\.I} \\
\verb|{\o}| & {\o} \\
\verb|{\'u}| & {\'u} \\
\verb|{\aa}| & {\aa} \\\hline
\end{tabular}
\begin{tabular}{lc}
\hline
\textbf{Command} & \textbf{Output}\\
\hline
\verb|{\c c}| & {\c c} \\
\verb|{\u g}| & {\u g} \\
\verb|{\l}| & {\l} \\
\verb|{\~n}| & {\~n} \\
\verb|{\H o}| & {\H o} \\
\verb|{\v r}| & {\v r} \\
\verb|{\ss}| & {\ss} \\
\hline
\end{tabular}
\caption{Example commands for accented characters, to be used in, \emph{e.g.}, Bib\TeX{} entries.}
\label{tab:accents}
\end{table}
\subsection{Hyperlinks}
Users of older versions of \LaTeX{} may encounter the following error during compilation:
\begin{quote}
\tt\verb|\pdfendlink| ended up in different nesting level than \verb|\pdfstartlink|.
\end{quote}
This happens when pdf\LaTeX{} is used and a citation splits across a page boundary. The best way to fix this is to upgrade \LaTeX{} to 2018-12-01 or later.
\subsection{Citations}
\begin{table*}
\centering
\begin{tabular}{lll}
\hline
\textbf{Output} & \textbf{natbib command} & \textbf{Old ACL-style command}\\
\hline
\citep{Gusfield:97} & \verb|\citep| & \verb|\cite| \\
\citealp{Gusfield:97} & \verb|\citealp| & no equivalent \\
\citet{Gusfield:97} & \verb|\citet| & \verb|\newcite| \\
\citeyearpar{Gusfield:97} & \verb|\citeyearpar| & \verb|\shortcite| \\
\hline
\end{tabular}
\caption{\label{citation-guide}
Citation commands supported by the style file.
The style is based on the natbib package and supports all natbib citation commands.
It also supports commands defined in previous ACL style files for compatibility.
}
\end{table*}
Table~\ref{citation-guide} shows the syntax supported by the style files.
We encourage you to use the natbib styles.
You can use the command \verb|\citet| (cite in text) to get ``author (year)'' citations, like this citation to a paper by \citet{Gusfield:97}.
You can use the command \verb|\citep| (cite in parentheses) to get ``(author, year)'' citations \citep{Gusfield:97}.
You can use the command \verb|\citealp| (alternative cite without parentheses) to get ``author, year'' citations, which is useful for using citations within parentheses (e.g. \citealp{Gusfield:97}).
\subsection{References}
\nocite{Ando2005,borschinger-johnson-2011-particle,andrew2007scalable,rasooli-tetrault-2015,goodman-etal-2016-noise,harper-2014-learning}
The \LaTeX{} and Bib\TeX{} style files provided roughly follow the American Psychological Association format.
If your own bib file is named \texttt{custom.bib}, then placing the following before any appendices in your \LaTeX{} file will generate the references section for you:
\begin{quote}
\begin{verbatim}
\bibliographystyle{acl_natbib}
\bibliography{custom}
\end{verbatim}
\end{quote}
You can obtain the complete ACL Anthology as a Bib\TeX{} file from \url{https://aclweb.org/anthology/anthology.bib.gz}.
To include both the Anthology and your own .bib file, use the following instead of the above.
\begin{quote}
\begin{verbatim}
\bibliographystyle{acl_natbib}
\bibliography{anthology,custom}
\end{verbatim}
\end{quote}
Please see Section~\ref{sec:bibtex} for information on preparing Bib\TeX{} files.
\subsection{Appendices}
Use \verb|\appendix| before any appendix section to switch the section numbering over to letters. See Appendix~\ref{sec:appendix} for an example.
\section{Bib\TeX{} Files}
\label{sec:bibtex}
Unicode cannot be used in Bib\TeX{} entries, and some ways of typing special characters can disrupt Bib\TeX's alphabetization. The recommended way of typing special characters is shown in Table~\ref{tab:accents}.
Please ensure that Bib\TeX{} records contain DOIs or URLs when possible, and for all the ACL materials that you reference.
Use the \verb|doi| field for DOIs and the \verb|url| field for URLs.
If a Bib\TeX{} entry has a URL or DOI field, the paper title in the references section will appear as a hyperlink to the paper, using the hyperref \LaTeX{} package.
\section*{Acknowledgements}
This document has been adapted
by Steven Bethard, Ryan Cotterell and Rui Yan
from the instructions for earlier ACL and NAACL proceedings, including those for
ACL 2019 by Douwe Kiela and Ivan Vuli\'{c},
NAACL 2019 by Stephanie Lukin and Alla Roskovskaya,
ACL 2018 by Shay Cohen, Kevin Gimpel, and Wei Lu,
NAACL 2018 by Margaret Mitchell and Stephanie Lukin,
Bib\TeX{} suggestions for (NA)ACL 2017/2018 from Jason Eisner,
ACL 2017 by Dan Gildea and Min-Yen Kan,
NAACL 2017 by Margaret Mitchell,
ACL 2012 by Maggie Li and Michael White,
ACL 2010 by Jing-Shin Chang and Philipp Koehn,
ACL 2008 by Johanna D. Moore, Simone Teufel, James Allan, and Sadaoki Furui,
ACL 2005 by Hwee Tou Ng and Kemal Oflazer,
ACL 2002 by Eugene Charniak and Dekang Lin,
and earlier ACL and EACL formats written by several people, including
John Chen, Henry S. Thompson and Donald Walker.
Additional elements were taken from the formatting instructions of the \emph{International Joint Conference on Artificial Intelligence} and the \emph{Conference on Computer Vision and Pattern Recognition}.
% Entries for the entire Anthology, followed by custom entries
\bibliography{anthology,custom}
\bibliographystyle{acl_natbib}
\appendix
\section{Example Appendix}
\label{sec:appendix}
This is an appendix.
\end{document}
% This must be in the first 5 lines to tell arXiv to use pdfLaTeX, which is strongly recommended.
\pdfoutput=1
% In particular, the hyperref package requires pdfLaTeX in order to break URLs across lines.
\documentclass[11pt]{article}
% Remove the "review" option to generate the final version.
\usepackage{acl}
\usepackage[switch]{lineno}
% Standard package includes
\usepackage{times}
\usepackage{latexsym}
\usepackage{todonotes}
\usepackage{amsmath}
\usepackage{subcaption}
\usepackage{hyperref}
\usepackage{bbm}
\usepackage{ stmaryrd }
% For proper rendering and hyphenation of words containing Latin characters (including in bib files)
\usepackage[T1]{fontenc}
% For Vietnamese characters
% \usepackage[T5]{fontenc}
% See https://www.latex-project.org/help/documentation/encguide.pdf for other character sets
% This assumes your files are encoded as UTF8
\usepackage[utf8]{inputenc}
% This is not strictly necessary, and may be commented out,
% but it will improve the layout of the manuscript,
% and will typically save some space.
\usepackage{microtype}
% If the title and author information does not fit in the area allocated, uncomment the following
%
%\setlength\titlebox{<dim>}
%
% and set <dim> to something 5cm or larger.
\title{COMP 767 - Project Proposal \\ Modeling Ironic Utterances via the Rational Speech Acts}
% Author information can be set in various styles:
% For several authors from the same institution:
% \author{Author 1 \and ... \and Author n \\
% Address line \\ ... \\ Address line}
% if the names do not fit well on one line use
% Author 1 \\ {\bf Author 2} \\ ... \\ {\bf Author n} \\
% For authors from different institutions:
% \author{Author 1 \\ Address line \\ ... \\ Address line
% \And ... \And
% Author n \\ Address line \\ ... \\ Address line}
% To start a seperate ``row'' of authors use \AND, as in
% \author{Author 1 \\ Address line \\ ... \\ Address line
% \AND
% Author 2 \\ Address line \\ ... \\ Address line \And
% Author 3 \\ Address line \\ ... \\ Address line}
\author{Cesare Spinoso-Di Piano \\
Role: Student \\
Affilitation: McGill/Mila \\
\texttt{cesare.spinoso@mila.quebec} \\\And
David Austin \\
Role: Student \\
Affilitation: McGill/Mila \\
\texttt{david.austin@mila.quebec} \\}
\begin{document}
\linenumbers
\maketitle
\begin{abstract}
The computational modeling of figurative language such as verbal \emph{irony} has been a long-standing objective in natural language understanding. Yet, natural language processing systems such as large language models (LLMs) have been shown to perform poorly on irony evaluation tasks such as classification and generation. Meanwhile, cognitive scientists have introduced the Rational Speech Act (RSA) framework, a model of human communication, which has been successfully adapted to predict human uses of figurative language such as irony. However, the use of RSA for irony modeling has been constrained to specific topics (e.g., the weather) and has required human-derived probability priors. These constraints severely limit the scalability of these methods. As a result, in this project, we investigate the use of RSA for more realistic cases of irony classification and generation by proposing a generalization of the RSA framework and by leveraging LLM-derived priors. We aim to show that LLMs are able to handle ironic interpretations of language by explicitly accounting for speaker and listener expectations using RSA. More generally, we expect our findings to reveal whether additional inference-time reasoning steps are enough for LLMs to account for uses of language where the literal meaning differs from the intended one.
% Large language models (LLMs) have been shown to struggle with figurative uses of language such as irony. Existing work has proposed fine-tuning-based approaches to improve the ability of LLMs to reason about ironic utterances. While these approaches have led to performance improvements on irony classification and generation datasets, they do not scale well and are unlikely to generalize to out-of-distribution use cases. As a result, we propose to leverage a model of human communication, the Rational Speech Act (RSA) framework, in which explicit models of speakers and listeners are leveraged to reason about the intended meaning of an utterance. In particular, we look at the problem of computationally modeling irony by integrating LLMs within the RSA framework to classify and generate ironic utterances. We aim to show that leveraging a model which explicitly accounts for listener and speaker expectations allows LLMs to better handle irony while eliminating the need for further fine-tuning on irony-specific datasets. More generally, we expect our findings to reveal whether adding additional inference-time reasoning layers is enough for LLMs to account for uses of language where the literal meaning significantly differs from the intended one.
\end{abstract}
\section{Introduction}
Figurative language, defined as when the intended meaning of an utterance is different from its literal meaning, is a ubiquitous component of natural language and fundamental to understanding human language use \cite{roberts_why_figurative_1994}. While large language models (LLMs) have reached human-level performance on many language understanding tasks \cite{sparksagi}, they underperform on some tasks involving figurative language \cite{fig_testing_2022}. In particular, LLMs struggle with humour, irony, and other areas which require an understanding of social conventions and expectations \cite{hu_fine-grained_2023, jones_epitome_2024}.
We focus on verbal irony - when the intended meaning of a statement is opposite to the literal interpretation \cite{Colston2000Contrast, Kao2015Irony, roberts_why_figurative_1994}. More specifically, we operationalize the modeling of irony via irony classification and generation. To improve performance on irony classification, existing work has fine-tuned LLMs on pragmatics datasets to improve performance \cite{fig_testing_2022}. In contrast, we utilize the Rational Speech Act (RSA) framework (Appendix~\ref{sec:rsa_formal}). RSA has been successfully applied to pragmatic phenomena \cite{kao_number_words_2014, andreas_reasoning_2016}, and has been applied to irony \cite{Kao2015Irony}. However, existing RSA approaches to irony modeling have been environment specific and required priors derived from human surveys \cite{Kao2015Irony}. These methods are not scalable, limiting their utility outside of small, fixed state spaces.
We propose an RSA-based model which addresses these limitations by obtaining prior distributions from LLMs (Figure~\ref{fig:irony-diagram}). We claim that this will allow the model to generalize on-the-fly. We also claim that explicitly modelling the speaker and listener's expectations will improve performance on tasks which involve subverting listener expectations.
\begin{figure}
\centering
\includegraphics[width=\linewidth]{Images/irony_diagram_v2.png}
\caption{Given an utterance from the speaker (e.g. "It's tiny"), the pragmatic listener considers which intended meanings the speaker could choose this utterance to communicate, either truthfully or ironically.}
\label{fig:irony-diagram}
\end{figure}
Evaluation will be conducted using standard metrics for classification and generation, as well as human evaluation. Initial evaluation will be done in a toy environment, based on pragmatic reasoning tasks like the reference game \cite{frank_ref_game_2012, andreas_reasoning_2016}, in which the speaker must describe a reference image to a listener such that the listener is able to identify it from among a slate of options. We will sanity test using our model as both speaker and listener and will evaluate using human speakers to determine if our model is able to understand ironic utterances.
We will benchmark our model on irony evaluation datasets, including PragMega \cite{huFinegrainedComparisonPragmatic2023} and iSarcasm \cite{oprea_isarcasm_2020}. The PragMega dataset contains curated descriptions of scenarios involving pragmatic phenomena, including irony. The iSarcasm dataset contains real tweets, a sarcasm label, and intended meanings provided by tweet authors (see Appendix \ref{sec:isarc_samples}). We will evaluate our model’s ability to label irony and recover the intended meaning on real scenarios with very little context.
We anticipate that our RSA-based irony model will demonstrate the utility of explicitly modeling pragmatic reasoning and speaker intent. Prior work has noted that LLMs have difficulty generating and understanding linguistic phenomena which require an understanding of social conventions and expectations. Our primary anticipated contribution would be a novel RSA-based model of irony which removes the existing need for tightly defined scenarios and human-surveyed priors. However, the success of this approach would motivate further work for explicitly modelling of pragmatic phenomena.
\section{Related Work}
The goal of understanding and modeling pragmatic phenomena, in particular phenomena such as irony, metaphor and hyperbole where the \emph{literal meaning} of an \emph{utterance} differs from its \emph{intended meaning}, has been a long-standing goal in several fields including cognitive science and computational linguistics. We first discuss how pragmatic language phenomena such as irony have been studied from a linguistic and cognitive perspective. We subsequently link these perspectives to how the natural language processing (NLP) community has computationally modeled irony and other pragmatic phenomena. Furthermore, we discuss the limitations of current systems and how we believe our proposed approach may work towards solving some of these issues.
\subsection{Modeling Human Irony Usage}
\label{sec:philo_cogsci_irony}
There has been extensive work to explain both how and why irony and other related figurative uses of language occur. Early studies in pragmatics \citep{grice1975logic,horn1984towards} attempt to explain such uses of language by considering the communicative aspect of language. According to \citep{grice1975logic}, language users communicate with each other using a ``cooperative principle'' by following certain ``conversational maxims'' (e.g., being informative and succinct, only stating what is true, etc.). As a result, because a listener expects a speaker to abide by these maxims, ironic statements which explicitly violate certain conversational maxims can carry meaning beyond their literal interpretation. For instance, if Alice tells Bob that ``It is so nice outside.'' when it is raining, then Bob will try to explain Alice’s ``uncooperativeness'' by supposing that she meant something more by her statement\footnote{Since rainy weather is not typically associated with being ``nice''.}, for instance that Alice is annoyed by the rain. Subsequent studies have furthered our understanding of human irony usage by showing that speakers use ironic statements to soften the tone of a critique or to be humorous \citep{robertsWhyPeopleUse1994,colstonSaltingWoundSugaring1997}, that ironic and other figurative uses of language can sometimes be easier to process cognitively than their literal counterparts \citep{ContextualEffectsUnderstanding1979} and that listeners understand ironic statements by explicitly interpreting the mental states of speakers through the ``Theory of Mind''-network part of the brain \citep{spotornoNeuralEvidenceThat2012}.
While the cooperative principle and related theories of pragmatic phenomena provide for intuitive high-level explanations, they cannot be used to computationally model pragmatic phenomena. This limitation is particularly problematic in cognitive science where scientists seek to build models of the mind to predict, for instance, when a speaker may decide to use an utterance in a non-literal manner. To address this gap, \citep{frankPredictingPragmaticReasoning2012} introduced the Rational Speech Act (RSA) framework in which human communication is modeled by defining the speaker and the listener of an utterance and the procedure by which these agents reason about \emph{each other} to communicate information as effectively as possible (Appendix~\ref{sec:rsa_formal}). This framework which was initially developed to explain speaker utterance decisions in a reference communication game setting (Figure~\ref{fig:reference_game}), has been extended to model several pragmatic phenomena including implicature \citep{goodman2013knowledge}, metaphor \citep{kao2014formalizing}, hyperbole \citep{kao2014nonliteral}, and indirect speech acts \citep{lumer2022modeling}.
\begin{figure*}[t!]
\centering
\begin{subfigure}[t]{0.47\textwidth}
\centering
\includegraphics[width=0.75\linewidth]{Images/figure_reference_game.png}
\caption{Figure from \citep{goodman2013knowledge} illustrating the reference game setup used to motivate the introduction of the RSA framework. A \emph{pragmatic} speaker attempting to reference the blue circle among the three shapes must decide between using one of four utterances: $\{\text{circle},\text{square},\text{green},\text{blue}\}$.}
\label{ref:subfig1}
\end{subfigure}%
~
\begin{subfigure}[t]{0.47\textwidth}
\centering
\includegraphics[width=\linewidth]{Images/figure_franke_textbook.png}
\caption{Figure illustrating the intuition behind RSA\footnote{\url{https://github.com/michael-franke/npNLG}}. A \emph{pragmatic} listener tries to understand an utterance generated in a reference game setting by reasoning about why a \emph{pragmatic} speaker would have chosen a certain utterance. The pragmatic speaker has their own internal model of a \emph{literal} listener, i.e. a listener who only interprets utterances literally.}
\label{ref:sibfig2}
\end{subfigure}
\caption{Illustrations of the reference game setting used to motivated the introduction of the RSA model as well as of the intuitive speaker and listener model proposed by the RSA framework.}
\label{fig:reference_game}
\end{figure*}
In closer relation to our work, \citep{Kao2015Irony} adapt the RSA framework to model ironic utterances about the weather by accounting for the affective attitudes of speakers. The study hired human participants to estimate probability priors (e.g., $P(\text{negative sentiment}|\text{rainy weather})$) which were then used with the RSA framework to estimate the non-literal meaning of utterances of the form ``The weather is $X$.'' when the actual state of the weather was $Y$ (e.g., $X = \text{nice}, Y = \text{rainy}$). In this work, we aim to generalize this RSA adaptation beyond positive/negative attitudes and to introduce LLMs within the framework to extend its use beyond statements about the weather.
\subsection{Irony Modeling in NLP}
The RSA framework has been applied to several tasks in natural language understanding and generation in an effort to make these systems more effective at communication. \citep{andreas_reasoning_2016} were the first to use \emph{neural} listeners and speakers with RSA in the context of image captioning. The authors showed that an neural-network-based image captioning system can produce more informative and useful image captions by explicitly considering a listener's ability to distinguish between the captioned image and a distractor image. \citep{friedUnifiedPragmaticModels2018} extended the use of RSA to instruction following and generation. \citep{shenPragmaticallyInformativeText2019} investigated RSA for generic summarization and language generation from structured meaning representations and \citep{cohn-gordonLostMachineTranslation2019} applied it to machine translation. More recently, \citep{darrin-etal-2024-cosmic} leveraged the RSA framework to develop a multi-document summarization system of scholarly reviews. Our work aims to expand the application of RSA in this direction by adapting it to irony detection and generation using LLMs.
Despite the use of RSA in language understanding and generation tasks, there has been limited work in using this framework to model non-literal uses of language like irony. One recent work combines the RSA framework with LLMs for the task of metaphor understanding \citep{careniniFullyInterpretableMore2024}. Current studies which have benchmarked the capabilities of LLMs in terms of pragmatic language understanding \citep{huFinegrainedComparisonPragmatic2023,ruis2023goldilockspragmaticunderstandingfinetuning,sravanthiPUBPragmaticsUnderstanding2024} have shown that LLMs perform significantly worse than humans when it comes to figurative uses of language. This limitation has been attributed to non-literal uses of language ``violating listener expectations''. Thus far, the most common approaches to mitigate these issues in irony modeling have been through dataset collection and fine-tuning \citep{zhengGRICEGrammarbasedDataset2021,liDiPlomatDialogueDataset2023}. We believe that RSA may offer an alternative solution by making an LLM explicitly aware of non-literal interpretations of utterances via a listener model.
\bibliographystyle{plain} % We choose the "plain" reference style
\bibliography{anthology}
\appendix
\section{Overview of RSA}
\label{sec:rsa_formal}
% Add technical details of RSA here, so don't break flow of RW
We review some of the technical details of the original RSA framework in this section and finish the section with a brief discussion of what limitations we plan on addressing in this work.
In the RSA framework, communication is modeled by defining a \emph{pragmatic}\footnote{Another term often used is \emph{rational}.} listener $L_1$ who aims to decipher the intended meaning $m$ from a generated utterance $u$ by explicitly reasoning about the speaker of the utterance.
To model the pragmatic listener $L_1$, the RSA framework begins by defining a \emph{literal} listener, $L_0$. One can think of the literal listener as someone who takes every utterance literally i.e., they do not consider the context of the utterance. Thus, given an utterance $u$, $L_0$ places probability mass on all meanings $m$ which are compatible with a literal interpretation of $u$. That is, the probability of \emph{literally} deriving a meaning $m \in \mathcal{M}$ from an utterance $u$ is
%
\begin{equation}
P_{L_0}(m | u) \propto P_\mathcal{M}(m) \cdot \mathbbm{1}_{m \in \llbracket u \rrbracket}
\end{equation}
%
where $P_\mathcal{M}(\cdot)$ refers to a prior over meanings $\mathcal{M}$ and $\mathbbm{1}_{m \in \llbracket u \rrbracket}$ returns $1$ if and only if $m$ belongs to the set of possible literal meanings of $u$. For instance, considering the reference game setting from Figure~\ref{fig:reference_game}, if $u$ is ``The blue object.'' then this could \emph{literally} refer to either the \underline{blue} square or the \underline{blue} circle.
Once a base listener model is defined, the RSA framework introduces a \emph{pragmatic} speaker $S_1$ who, given a meaning $m$, decides on which utterance $u$ to generate based on how likely $L_0$ is to recover the correct intended meaning. That is, the probability of $S_1$ generating utterance $u$ for meaning $m$ is
%
\begin{equation}
P_{S_1}(u | m) \propto P_{L_0}(m|u) \cdot \exp^{-C(u)}
\end{equation}
%
where $C(u)$ is some cost function for generating utterance $u$ which discourages $S_1$ from being overly descriptive. Thus, given the reference game example, $S_1$ would know not to choose the utterance ``The green object'' to refer to either the blue circle or the blue square since $L_0$ would assign zero probability mass to each of those meanings.
Finally, the \emph{pragmatic} listener, $L_1$, reasons about the meaning of an utterance based on $S_1$,
%
\begin{equation}
P_{L_1}(m|u) \propto P_\mathcal{M}(m) \cdot P_{S_1}(u|m)
\end{equation}
%
In this way, when determining the intended meaning of an utterance $u$, the pragmatic listener explicitly accounts for how $S_1$ is likely to talk about $m$. Thus, a pragmatic listener would reason that ``The blue object.'' is much more likely to refer to the blue square because if $S_1$ wanted to refer to the blue circle they could have simply said ``The circular object.''
This version of RSA has been adapted to quantitatively predict several human uses of pragmatics (e.g., implicature, metaphor, irony, etc.). However, existing adaptations have remained environment and task specific and have required the use of human-derived priors. We plan on addressing both of these limitations by 1) generalizing the indicator function used by $L_0$, $\mathbbm{1}_{m \in \llbracket u \rrbracket}$, to allow for non-zero probability mass to be assigned to meanings which are incompatible with a literal interpretation of $u$ (e.g., in irony) and by 2) integrating LLMs within this framework as a way to provide an estimate for its prior probabilities via prompting (e.g., prompting LLMs to estimate $P_\mathcal{M}(m)$).
\section{Dataset Samples}
\label{sec:isarc_samples}
The PragMega dataset is composed of text descriptions of scenarios which display pragmatic phenomena. They include four options for intended meanings to choose from. One option is the pragmatic interpretation, one is literal, and two are unrelated.
\begin{quote}
"In a shop, Lara tries on a dress. The dress is far too long for her. Lara asks Simon: ""Does this dress fit me?"" Simon answers: ""Wow! That must be custom made! It's clearly the perfect size and length for you."" What did Simon want to convey? "
A) The dress is fitting well.\\
B) Lara needs to get a dress of a shorter length.\\
C) Simon does not like the color of this dress.\\
D) Simon has to get back to work. \\
\end{quote}
The iSarcasm dataset is composed of real tweets, a binary sarcasm label, and the intended meaning of the utterance, retrieved from the original tweet author. We include a sample in Table \ref{tab:isarc_sample}.
\begin{table*}
\centering
\begin{tabular}{|p{3in}|c|c|}
\hline
Tweet & Intended Meaning & Sarcasm \\
\hline
\hline
Falling asleep at your laptop is always fun & I hate falling asleep at my laptop & 1 \\
\hline
The French anthem is a banger. Love this gorgeous orchestration! \#Olympics & N/A & 0 \\
\hline
\end{tabular}
\caption{Samples from the iSarcasm Dataset}
\label{tab:isarc_sample}
\end{table*}
\end{document}
% Instructions:
The goal of related work is to situate your work in relation to what has happened. Related work is not about having several papers and discussing what each paper is. Instead, it is to identify key papers that are relevant to your work and talk about them in detail but in a crisp manner (also these will be the people who review your work when you submit a paper, so it is important not to miss any key paper, otherwise your paper will be rejected).
In the related work section, you talk about an aspect of your problem, how an existing approach solves it, and how your proposal is similar or different to the existing approach. You repeat this for a few aspects. For example, if you are an author of Bert, you will talk about aspects like word masking (left-to-right, masked language model), attention (self-attention, gating), parallelization (sequential processing vs parallel processing), fine-tuning (static usage of representations and dynamic usage).
A paper should always be self-contained, i.e., everything that is required to understand the paper should be present in the paper. If it is a fundamental concept like RNN, you don’t have to explain it. But if it is a niche concept, say memory-networks, you should spend some time explaining what it is.
Some suggestions:
\begin{itemize}
\item Organize related work based on aspects of the problem (see above description)
\item When you refer to a paper, make sure you read the paper (at least briefly) in order to avoid false claims about what the work is. False claims could lead to rejections.
\item Related work is “not” about making other’s work look bad. It is about how your approach is similar or different.
\end{itemize}
% Paragraph 1: Figurative uses of language are ubiquitous/critical part of human communication. Provide some examples - use a weather example to prepare the reader for the running example.
% Paragraph 2: At the same time, LLMs, which have become the primary way of interfacing with language technologies, have been shown to perform poorly on figurative language interpretation. Figure 1 shows an example. List some reasons for why this might be (use the wording that Jackie suggested), guide the reader to implying that it's due to a lack of explicitly reasoning about the rhetorical strategy being used.
% Segue from LLM to this paragraph with communicative goal, this allows you to talk about RSA and then to talk about affect-aware RSA
Paragraph 3: This failure of LLMs suggests that their inference procedure should reason more explicitly about a speaker's choice of rhetorical strategy when interpreting an utterance. Use the weather example to make this more concrete. [Jackie: Use communicative goal to make the segue smoother and less about comparing with affect-aware RSA.]
Paragraph 4: It turns out that there already exists a model of human communication which makes these reasoning steps explicit: RSA. Briefly describe the general idea behind RSA and how, in its original formulation, it is unsuitable for non-literal language (Add a reference to where we prove this more formally). As a result, affect-aware RSA has been proposed to mitigate this problem but makes the assumption that figurative language's primary use is to convey affect and/or valence even though it's been shown to be used for other things (e.g. criticizing, softening, etc.). Refer to related work for a more detailed discussion.
While affect-aware RSA is successful in enabling the RSA framework to obtain non-literal interpretations of utterances, it results in a framework which we believe does not faithfully reflect the communicative goal of utterances employed a figurative manner. As a result of coupling non-literal meaning with affect, the framework is unable to account for figurative uses of language which are not affect-driven for instance, in conventionalized language \citep{ContextualEffectsUnderstanding1979}. For example, the exaggeration in the phrase ``The weather here is always terrible'' might be meant to describe how often bad weather occurs, rather than to express the speaker's feelings about it.
Paragraph 5: As a result, we introduce \rsatwospace where a speaker's rhetorical strategy is explicitly modeled and used to adjust listener expectations. \rsatwospace shifts the belief distribution of a listener based on the rhetorical strategy (define rhetorical strategy formally here) to enable non-literal interpretations of utterances *without* having to assume any particular speaker affect. The rhetorical strategy can either be set explicitly like in our weather example, a practice (the hardcoding of things) commonly followed in cognitive science studies which use RSA to predict the utterance or meaning of a human generated utterance in a specific scenario or it can be inferred by considering the difference in intended meaning between the utterance being interpreted and alternative utterances which may have been generated in the same context.
Paragraph 6: List some of the main results: In section 3, we provide a toy experiment where we illustrate how, by explicitly considering a speaker's rhetorical strategy, we are able to handle several different figurative uses of language as they pertain to describe the weather: irony, hyperbole, understatement which the original and affect-aware RSA formulations could not handle. In section 4, we show how a listener's expectations may be adjusted automatically at inference by inferring rhetorical strategy adjustment coefficients and provide a concrete implementation using LLMs in section 5. We show that \rsatwospace enables LLM's to shift their probability distribution of the meanings of an utterance which aligns with rhetorical strategy being used. Concretely, use and expand the PragMega dataset and show X performance gains.
Paragraph 7: List core contributions: 1. We provide an alternative formulation to RSA which can handle figurative uses of language by explicitly reasoning about the employed rhetorical strategy. 2. We show that our method enables LLMs to provide more faithful interpretations of figurative uses of language.
By modeling communication in this manner, the RSA framework has been shown to successfully predict the meaning of literally ambiguous utterances such as ``some'' in scalar implicatures. However, it can be shown that interpretations of utterances which are not compatible with the literal meaning of an utterance will be assigned zero probability by the pragmatic listener (Proof in Appendix~\ref{app:rsa_theory}). That is, in the standard RSA framework, the only meanings of ``This one is really sharp.'' from Figure~\ref{fig:john-example} which could receive non-zero probability would be ones which are semantically entailed by this utterance, namely that the student is in fact smart when the context suggests they are not.
As a result, efforts to model human use of figurative language have sought to address this limitation by incorporating the affect of an utterance within the RSA framework. As described in Section~\ref{}, the authors enable non-literal meanings to receive probability mass from the pragmatic listener by coupling an utterance's meaning and affect together. For instance, in our earlier example, a pragmatic listener may interpret the utterance ``It is so nice outside.'' in the context of a snowstorm as meaning that the weather is bad because this meaning is typically associated with a negative affect (e.g., annoyance or frustration).
While affect-aware RSA is successful in enabling the RSA framework to obtain non-literal interpretations of utterances, it results in a framework which we believe too tightly couples the notion of non-literal meaning and affect. As a result of this coupling, the framework is unable to account for figurative uses of language which are not affect-driven \citep{ContextualEffectsUnderstanding1979}. For example, in Figure~\ref{fig:john-example}, it is not clear that the John's ironic utterance ``This one is really sharp.'' is meant to convey affect. Instead, John may simply be used to conveying information about students' academic performances in a non-literal manner. Rather than pinpointing the exact --- if at all present --- affect which motivated the speaker's figurative use of an utterance, we believe that reasoning about the employed non-literal rhetorical strategy is a more direct way of enabling RSA to produce non-literal meaning interpretations of utterances.
By introducing a rhetorical strategy prior along with an adaptable semantic understanding function $\llbracket \cdot \rrbracket_r$, we are able to modify which meaning interpretations receive probability mass from the literal listener and, thus, which meanings will be available to the pragmatic listener when applying its Bayesian posterior update. For instance, under $r = \text{literal}$, $\texttt{The student is not clever.} \notin \llbracket \text{``This one is really sharp.''} \rrbracket^r$, while for $r = \text{ironic}$ $\texttt{The student is not clever.} \in \llbracket \text{``This one is really sharp.''} \rrbracket^r$. The pragmatic listeners under each rhetorical strategy are then marginalized together to produce a rhetorical-stategy-aware meaning interpretation of the utterance $u$, $P_{L_1}(m|c,u)$.
We test our formulation on the same setting as \citet{Kao2015Irony} which proposed an affect-aware RSA formulation to interpret utterances describing the weather. In this setting, the possible intended meanings included describing the weather as \texttt{terrible}, \texttt{bad}, \texttt{ok}, \texttt{good} or \texttt{amazing}. The set of possible utterances included all utterances of the form ``The weather is $x$'' where $x$ was one of the aforementioned intended meanings. Finally, these utterances and meanings were communicated under two contexts: one where the weather is sunny with clear skies and another with heavy precipitation (e.g., rain or snow) and winds.
% Paragraph 1: Figurative uses of language are ubiquitous/critical part of human communication. Provide some examples - use a weather example to prepare the reader for the running example.
% Paragraph 2: At the same time, LLMs, which have become the primary way of interfacing with language technologies, have been shown to perform poorly on figurative language interpretation. Figure 1 shows an example. List some reasons for why this might be (use the wording that Jackie suggested), guide the reader to implying that it's due to a lack of explicitly reasoning about the rhetorical strategy being used.
% Segue from LLM to this paragraph with communicative goal, this allows you to talk about RSA and then to talk about affect-aware RSA
Paragraph 3: This failure of LLMs suggests that their inference procedure should reason more explicitly about a speaker's choice of rhetorical strategy when interpreting an utterance. Use the weather example to make this more concrete. [Jackie: Use communicative goal to make the segue smoother and less about comparing with affect-aware RSA.]
Paragraph 4: It turns out that there already exists a model of human communication which makes these reasoning steps explicit: RSA. Briefly describe the general idea behind RSA and how, in its original formulation, it is unsuitable for non-literal language (Add a reference to where we prove this more formally). As a result, affect-aware RSA has been proposed to mitigate this problem but makes the assumption that figurative language's primary use is to convey affect and/or valence even though it's been shown to be used for other things (e.g. criticizing, softening, etc.). Refer to related work for a more detailed discussion.
While affect-aware RSA is successful in enabling the RSA framework to obtain non-literal interpretations of utterances, it results in a framework which we believe does not faithfully reflect the communicative goal of utterances employed a figurative manner. As a result of coupling non-literal meaning with affect, the framework is unable to account for figurative uses of language which are not affect-driven for instance, in conventionalized language \citep{ContextualEffectsUnderstanding1979}. For example, the exaggeration in the phrase ``The weather here is always terrible'' might be meant to describe how often bad weather occurs, rather than to express the speaker's feelings about it.
Paragraph 5: As a result, we introduce \rsatwospace where a speaker's rhetorical strategy is explicitly modeled and used to adjust listener expectations. \rsatwospace shifts the belief distribution of a listener based on the rhetorical strategy (define rhetorical strategy formally here) to enable non-literal interpretations of utterances *without* having to assume any particular speaker affect. The rhetorical strategy can either be set explicitly like in our weather example, a practice (the hardcoding of things) commonly followed in cognitive science studies which use RSA to predict the utterance or meaning of a human generated utterance in a specific scenario or it can be inferred by considering the difference in intended meaning between the utterance being interpreted and alternative utterances which may have been generated in the same context.
Paragraph 6: List some of the main results: In section 3, we provide a toy experiment where we illustrate how, by explicitly considering a speaker's rhetorical strategy, we are able to handle several different figurative uses of language as they pertain to describe the weather: irony, hyperbole, understatement which the original and affect-aware RSA formulations could not handle. In section 4, we show how a listener's expectations may be adjusted automatically at inference by inferring rhetorical strategy adjustment coefficients and provide a concrete implementation using LLMs in section 5. We show that \rsatwospace enables LLM's to shift their probability distribution of the meanings of an utterance which aligns with rhetorical strategy being used. Concretely, use and expand the PragMega dataset and show X performance gains.
Paragraph 7: List core contributions: 1. We provide an alternative formulation to RSA which can handle figurative uses of language by explicitly reasoning about the employed rhetorical strategy. 2. We show that our method enables LLMs to provide more faithful interpretations of figurative uses of language.
By modeling communication in this manner, the RSA framework has been shown to successfully predict the meaning of literally ambiguous utterances such as ``some'' in scalar implicatures. However, it can be shown that interpretations of utterances which are not compatible with the literal meaning of an utterance will be assigned zero probability by the pragmatic listener (Proof in Appendix~\ref{app:rsa_theory}). That is, in the standard RSA framework, the only meanings of ``This one is really sharp.'' from Figure~\ref{fig:john-example} which could receive non-zero probability would be ones which are semantically entailed by this utterance, namely that the student is in fact smart when the context suggests they are not.
As a result, efforts to model human use of figurative language have sought to address this limitation by incorporating the affect of an utterance within the RSA framework. As described in Section~\ref{}, the authors enable non-literal meanings to receive probability mass from the pragmatic listener by coupling an utterance's meaning and affect together. For instance, in our earlier example, a pragmatic listener may interpret the utterance ``It is so nice outside.'' in the context of a snowstorm as meaning that the weather is bad because this meaning is typically associated with a negative affect (e.g., annoyance or frustration).
While affect-aware RSA is successful in enabling the RSA framework to obtain non-literal interpretations of utterances, it results in a framework which we believe too tightly couples the notion of non-literal meaning and affect. As a result of this coupling, the framework is unable to account for figurative uses of language which are not affect-driven \citep{ContextualEffectsUnderstanding1979}. For example, in Figure~\ref{fig:john-example}, it is not clear that the John's ironic utterance ``This one is really sharp.'' is meant to convey affect. Instead, John may simply be used to conveying information about students' academic performances in a non-literal manner. Rather than pinpointing the exact --- if at all present --- affect which motivated the speaker's figurative use of an utterance, we believe that reasoning about the employed non-literal rhetorical strategy is a more direct way of enabling RSA to produce non-literal meaning interpretations of utterances.
By introducing a rhetorical strategy prior along with an adaptable semantic understanding function $\llbracket \cdot \rrbracket_r$, we are able to modify which meaning interpretations receive probability mass from the literal listener and, thus, which meanings will be available to the pragmatic listener when applying its Bayesian posterior update. For instance, under $r = \text{literal}$, $\texttt{The student is not clever.} \notin \llbracket \text{``This one is really sharp.''} \rrbracket^r$, while for $r = \text{ironic}$ $\texttt{The student is not clever.} \in \llbracket \text{``This one is really sharp.''} \rrbracket^r$. The pragmatic listeners under each rhetorical strategy are then marginalized together to produce a rhetorical-stategy-aware meaning interpretation of the utterance $u$, $P_{L_1}(m|c,u)$.
We test our formulation on the same setting as \citet{Kao2015Irony} which proposed an affect-aware RSA formulation to interpret utterances describing the weather. In this setting, the possible intended meanings included describing the weather as \texttt{terrible}, \texttt{bad}, \texttt{ok}, \texttt{good} or \texttt{amazing}. The set of possible utterances included all utterances of the form ``The weather is $x$'' where $x$ was one of the aforementioned intended meanings. Finally, these utterances and meanings were communicated under two contexts: one where the weather is sunny with clear skies and another with heavy precipitation (e.g., rain or snow) and winds.
% This must be in the first 5 lines to tell arXiv to use pdfLaTeX, which is strongly recommended.
\pdfoutput=1
% In particular, the hyperref package requires pdfLaTeX in order to break URLs across lines.
\documentclass[11pt]{article}
% Change "review" to "final" to generate the final (sometimes called camera-ready) version.
% Change to "preprint" to generate a non-anonymous version with page numbers.
\usepackage[review]{acl}
% \usepackage{acl}
% Standard package includes
\usepackage{times}
\usepackage{latexsym}
\usepackage[T1]{fontenc}
\usepackage[utf8]{inputenc}
\usepackage{microtype}
\usepackage{inconsolata}
\usepackage{graphicx}
\usepackage{amsmath}
\usepackage{booktabs}
\usepackage{lipsum}
\usepackage[caption=false]{subfig}
\usepackage{float}
\usepackage{multirow}
\usepackage{amsmath}
\usepackage{changepage}
\usepackage{natbib}
\usepackage{amsmath}
\usepackage{stmaryrd}
\usepackage{dsfont}
\usepackage{geometry} % Optional, to ensure full-page width
\usepackage{changepage}
\usepackage{makecell}
\usepackage{listings} % Import the listings package
\usepackage{framed}
\usepackage{amsfonts}
\usepackage{algorithm}
\usepackage{algorithmicx}
\usepackage{algpseudocode}
\lstset{
breaklines=true, % Enable line wrapping
basicstyle=\ttfamily, % Use monospace font
frame=single, % Add a frame around the code block
}
\DeclareMathOperator*{\argmax}{arg\,max}
\usepackage[]{todonotes} % use option "disable" to suppress notes
\makeatletter
\newcommand*\iftodonotes{\if@todonotes@disabled\expandafter\@secondoftwo\else\expandafter\@firstoftwo\fi}
\makeatother
\newcommand{\noindentaftertodo}{\iftodonotes{\noindent}{}\ignorespaces}
\newcommand{\david}[1]{{\color{purple}[David: #1]}}
\newcommand{\ces}[1]{{\color{red}[Ces: #1]}}
\newcommand{\rsatwo}{\textbf{(RSA)}\textsuperscript{2}}
\newcommand{\rsatwospace}{\rsatwo\,\,}
\newcommand{\rsatwolearned}{\textbf{Learned--(RSA)}\textsuperscript{2}}
\newcommand{\rsatwolearnedspace}{\rsatwolearned\,\,}
\newcommand{\response}[1]{\vspace{3pt}\hrule\vspace{3pt}\textbf{#1:}}
\algnewcommand{\Hypothesis}[1]{\textbf{Hypothesis:} #1}
\interfootnotelinepenalty=10000
% \definecolor{googlegreen}{RGB}{56, 118, 29}
% \definecolor{googleblue}{RGB}{17, 85, 204}
\title{\rsatwo: A Rhetorical-Strategy-Aware Rational Speech Act Framework for Irony Understanding}
\author{
Cesare Spinoso-Di Piano and David Austin\\
\texttt{\{cesare.spinoso-dipiano,david.e.austin\}@mail.mcgill.ca}\\
Mila – Quebec Artificial Intelligence Institute \\
McGill University \\
}
\begin{document}
\maketitle
\begin{abstract}
Figurative language (e.g., irony, hyperbole, understatement) is ubiquitous in human communication. However, previous work indicates that current LLMs struggle with interpreting the intended meaning of non-literal utterances. The Rational Speech Act (RSA) framework, which explicitly models speaker intentions, has been proposed as a solution but existing implementations are either unable to interpret figurative expressions or impose additional assumptions to do so. As a result, we introduce the \textbf{R}hetorical-\textbf{S}trategy-\textbf{A}ware \textbf{RSA} \rsatwospace framework which models figurative language use by explicitly considering a speaker's employed rhetorical strategy. We show that \rsatwospace is able to make non-literal interpretations of utterances which match those of humans without making any additional assumptions about the speaker's motivations to be non-literal. We also develop \rsatwolearned, an unsupervised learning algorithm which automatically induces the rhetorical strategies when they cannot be specified ahead of time. We achieve state-of-the-art performance on an existing irony interpretation dataset.
\end{abstract}
\section{Introduction}
% \ces{@Jackie: If this has not been commented out, then the draft is not ready for review. In particular, I've made some local changes which I still need to transfer here.}
\label{sec:intro}
\begin{figure}[t]
\centering
\includegraphics[width=\linewidth]{Images/rsa_john_example.png}
\caption{A sample from the PragMega dataset showing probabilities of intended meanings as well as rhetorical strategies for a given scenario. The probabilities are computed using the Mistral 7B Instruct model. \ces{TODO: Modify this figure - Remove the irony detection probability and get the correct ones since it clashes with the actual text in the intro.}}
\label{fig:john-example}
\end{figure}
Figurative uses of language --- where a speaker does not literally say what they mean --- are ubiquitous in human communication. For example, when faced with a blizzard with heavy winds, a speaker may say ``It's a little chilly out.'' to mean that it is very cold outside. Similarly, as illustrated in Figure~\ref{fig:john-example}, a teacher might refer to a student as ``really sharp'' when they intend to mean that they are in fact not very clever. As a result, it is imperative for language technologies and computational models of language to account for this linguistic phenomenon.
However, recent studies have shown that large language models (LLMs), which have become the de facto language technology, struggle with figurative interpretations of language. In \citet{hu_fine-grained_2023}, the authors show that LLMs often misinterpret utterances which subvert listener expectations such as in irony and humour. For instance, in Figure~\ref{fig:john-example}, we observe that the Mistral 7B (V3) LLM overwhelmingly favours the incorrect literal interpretation of the utterance (The student is smart.) over the correct non-literal one (The student is not very clever.). Consequently, this behaviour suggests that LLMs do not correctly or entirely capture the underlying \emph{communicative goal} (i.e., intention) of utterances needed to interpret them.
A potential solution to the poor performance of LLMs on figurative understanding is the Rational Speech Act (RSA) framework, a model of human communication designed to recover the intended meaning of an utterance, regardless of form. In RSA, an utterance's meaning is interpreted probabilistically by a \emph{pragmatic listener} which reasons about a posited \emph{pragmatic speaker}'s likelihood of generating the utterance under different intended meanings. However, while RSA was designed to interpret utterances based on their communicative goal, its original formulation does not allow non-literal meaning interpretations of utterances. In particular, the pragmatic listener from the original RSA formulation assigns zero probability mass to any non-literal interpretation of an utterance.
Therefore, to enable the application of RSA under non-literal rhetorical strategies (e.g, irony, hyperbole, understatement, etc.), we introduce a novel formulation of the RSA framework: \textbf{R}hetorical-\textbf{S}trategy-\textbf{A}ware \textbf{RSA} \rsatwo. In \rsatwo, a speaker's rhetorical strategy is explicitly modeled and used by the pragmatic listener to interpret an utterance's potentially non-literal intended meaning. Formally, we define a rhetorical strategy as the way in which an utterance relates to its intended meaning. For example, the \emph{ironic} rhetorical strategy interprets an utterance using the opposite of its literal meaning. With this novel formulation, we show that \rsatwospace enables a pragmatic listener to infer non-literal interpretations of utterances \emph{without} needing to make any additional assumptions about the speaker, an approach previously studied in \citet{kao_number_words_2014,Kao2015Irony} (Section~\ref{sec:human_irony}).
With our novel \rsatwospace formulation, we are able to provide interpretations of utterances which better match intuitive interpretations of figurative language. In Section~\ref{sec:experiment1}, we show how explicitly considering rhetorical strategies such as irony and hyperbole enables \rsatwospace to provide non-literal interpretations of weather-related utterances. For instance, our \rsatwo-based pragmatic listener is able to interpret ``The weather is amazing.'' uttered during in a blizzard as meaning that the weather is in fact quite bad. In Section~\ref{sec:experiment2}, we use a novel unsupervised learning algorithm which we call \rsatwolearned to leverage the \rsatwospace equations even when the set of rhetorical strategies used are not known in advance. Using this adapted \rsatwolearnedspace formulation, we show that our \rsatwo-derived pragmatic listener achieves state-of-the-art performance on an expanded ironic utterance interpretation dataset, PragMega+, which we design for this study.
To summarize, in this work, we design \rsatwo, a communicative-goal-centered procedure of figurative language interpretation. By explicitly considering the rhetorical strategies a speaker might use, we experimentally show that this enables a pragmatic listener to interpret utterances in a non-literal way which, until now, has either been impossible or required additional assumptions about the speaker's motivations. In addition, we adapt the \rsatwospace equations using a simple heuristic to enable state-of-the-art performance on an ironic interpretation dataset. Beyond figurative language understanding, our work re-establishes the importance of computational models of language which focus on modeling the communicative goals of speakers and listeners.
\section{Related Work}
\label{sec:rw}
\subsection{Modeling Human Uses of Figurative Language}
\label{sec:human_irony}
There has been extensive work to explain why humans use figurative language. Early studies in pragmatics \citep{grice1975logic,horn1984towards} attempt to explain such uses of language by considering the communicative aspect of language. According to \citep{grice1975logic}, language users communicate with each other ``cooperatively'' by following certain ``conversational maxims'' (e.g., being informative and succinct, only stating what is true, etc.). As a result, because a listener expects a speaker to abide by these maxims, figurative statements (e.g., irony, hyperbole, etc.) which explicitly violate them can carry meaning beyond their literal interpretation. Subsequent studies have furthered our understanding of the usage of figurative language by showing that speakers use ironic and other figurative expressions to soften the tone of a critique or to be humorous \citep{robertsWhyPeopleUse1994,colstonSaltingWoundSugaring1997}, that figurative uses of language can sometimes be easier to process cognitively than their literal counterparts \citep{ContextualEffectsUnderstanding1979} and that listeners understand figurative statements by explicitly interpreting the mental states of speakers through the ``Theory of Mind''-network part of the brain \citep{spotornoNeuralEvidenceThat2012}.
While the cooperative principle provides an intuitive explanation for why figurative language occurs, it cannot be used to predict its use and to recover its intended meaning. To address this gap, \citet{kao_number_words_2014,kao2014formalizing,Kao2015Irony} adapt the RSA framework originally introduced by \citet{frankPredictingPragmaticReasoning2012}. In their work, the authors adapt the RSA framework to model utterances to be interpreted figuratively (e.g., ironic utterances about the weather, hyperbolic utterances about prices). To enable non-literal interpretations of utterances, the authors assume that figurative language use is motivated by affect (e.g., to convey joy, annoyance, etc.) and leverage this additional variable in the RSA procedure. In this work, we seek to computationally model figurative language \emph{without} assuming a particular motivation for its use thus enabling its use is more scenario this RSA adaptation by lifting its dependence on explicitly modeling speaker affect
\subsection{Figurative Language Modeling in NLP}
In NLP, the ability of language systems to understand figurative language has mainly been evaluated through irony both for detection and generation \citep{van-hee-etal-2018-semeval,ZHANG20191633,balestrucci-etal-2024-im}. The most common approach to solving these tasks with language systems has been to collect irony-specific datasets and to fine-tune language classification and generation models on them \citep{van-hee-etal-2018-semeval}. However, as the focus in the field shifts towards LLMs and prompt-based approaches, there has been growing interest in evaluating pre-trained and instruction-tuned LLMs on their ability to recognize, interpret and generate figurative language such as irony and sarcasm \citep{gu-etal-2022-just,liu-etal-2022-testing,hu_fine-grained_2023}. Recent studies have shown that LLMs perform significantly worse than humans when it comes to interpreting figurative uses of language such as humour and irony. This failure of LLMs to correctly interpret ironic utterances is likely due to the fact that LLMs do not fully or correctly model speaker intentions and communicative goals. In this work, we aim to propose a computational model of figurative language which explicitly models the rhetorical strategy being used and its connection with a speaker's communicative goal.
\subsection{RSA in NLP}
\citet{andreas_reasoning_2016} were the first to use neural listeners and speakers with RSA in the context of image captioning. \citet{friedUnifiedPragmaticModels2018} extended the use of RSA to instruction following and generation. \citet{shenPragmaticallyInformativeText2019} investigated RSA for abstractive summarization and \citet{cohn-gordonLostMachineTranslation2019} applied it to machine translation. In closer connection to our work on leveraging RSA for figurative language, \citet{careniniFullyInterpretableMore2024} combined the RSA framework with LLMs for the task of metaphor understanding. To the best of our knowledge, we are the first to propose a rhetorical-strategy-aware recursive reasoning procedure as it relates to understanding figurative language.
\section{Rhetorical-Strategy-Aware Rational Speech Acts}
\label{sec:rsa_two}
In this section, we introduce our novel formulation of the RSA framework --- \textbf{R}hetorical-\textbf{S}trategy-\textbf{A}ware \textbf{RSA} \rsatwo --- which enables the RSA procedure to explicitly consider the rhetorical strategy being used when resolving the meaning of an utterance. Unlike previous approaches which are either unable to handle figurative language or require imposing additional assumptions about its use (Section~\ref{sec:human_irony}), our proposed approach is able to provide non-literal interpretations of utterances by explicitly reasoning about their corresponding non-literal rhetorical strategy (e.g., irony, hyperbole). We present our approach by first reviewing the inner workings of the RSA framework and subsequently present the novel \rsatwospace framework and an unsupervised learning algorithm \rsatwospacelearnedspace for instances where the rhetorical strategies are not known ahead of time. Throughout this presentation, we will use the running example of a speaker ironically saying ``The weather is amazing.'' during a blizzard.
\subsection{The Rational Speech Act Framework}
The RSA framework was originally introduced to model human communication and, more specifically, to explicitly account for listener and speaker expectations when predicting uses of scalar implicature \citep{goodman2013knowledge}. The goal of this framework is to derive a probability distribution over some fixed set of meanings $\mathcal{M}$ for some observed utterance $u \in \mathcal{U}$ generated in some context\footnote{The context $c$ can be linguistic (e.g., conversation history) as well as situational (e.g., the current state of the world).} $c \in \mathcal{C}$. To do so, this framework posits the existence of multiple speakers and listeners who reason about each other in a recursive fashion. This recursive procedure begins with a \emph{literal listener}, $L_0$, which reasons literally about the interpretation of utterances by defining the following conditional probability distribution over $\mathcal{M}$:
%
\begin{align}
P_{L_0}(m|c,u) &\propto \mathds{1}_{m \in \llbracket u \rrbracket} \cdot P_{\mathcal{M}|\mathcal{C}}(m|c),
\end{align}
%
where $P_{\mathcal{M}|\mathcal{C}}$ is a prior conditional distribution over possible meanings given a context and where $\llbracket \cdot \rrbracket : \mathcal{U} \rightarrow \mathcal{P(M)}$ is a shared semantic understanding function\footnote{$\mathcal{P()}$ is the power set function.} which, given some utterance $u \in \mathcal{U}$, returns the set of possible intended meanings which are \emph{literally} compatible with $u$.
For instance, given our running example ``The weather is amazing.'' and a discrete set of weather states which a speaker might want to convey, $\mathcal{M} = \{\texttt{terrible}, \texttt{bad}, \texttt{ok}, \texttt{good}, \texttt{amazing}\}$, the semantic understanding function would return the following set:%
\[
\llbracket \text{``The weather is terrible.''} \rrbracket = \{\texttt{terrible}\}
\]%
With the base case of the reasoning procedure defined, the RSA framework attempts to provide a speaker-aware interpretation of the utterance $u$ by positing the existence of a \emph{pragmatic speaker}, $S_1$, which selects an utterance $u$ based on its prior probability, $P_{\mathcal{U}|\mathcal{C}}$, and $L_0$'s expected information gain.\footnote{While Equation~\ref{eq:pragmatic_speaker} may appear different from its common expected-utility formulation \citep{goodman2013knowledge}, we show the equivalence of these two formulations in Appendix~\ref{app:rsa_theory}.} That is,%
\begin{align}
\label{eq:pragmatic_speaker}
P_{S_1}(u|c,m) &\propto P_{L_0}(m|c,u)^\alpha \cdot P_{\mathcal{U}|\mathcal{C}}(u|c)
\end{align}%
where $\alpha$ is a rationality parameter which controls how much the pragmatic speaker will favor selecting the most informative utterance according to the literal listener.
In our running example, assuming a discrete set of utterances of the form $\mathcal{U} = \{\text{``The weather is $m$.''} : m \in \mathcal{M}\}$, a speaker intending to convey meaning a weather state $m$ would assign all of its probability mass to the weather utterance $u$ which maximizes the probability that $L_0$ infers $m$ from $u$. In this case, this would lead to $S_1$ always selecting the utterance which literally conveys the weather state they intend to convey.
Finally, to interpret a given utterance $u$ pragmatically, the \emph{pragmatic listener}, $L_1$, updates their prior belief about the distribution of possible intended meanings $P_{\mathcal{M}|\mathcal{C}}$ by using the likelihood that the utterance was generated by $S_1$ in the given context. In other words, the pragmatic listener's conditional probability distribution over $\mathcal{M}$ becomes the following posterior Bayesian update%
\begin{align}
P_{L_1}(m|c,u) \propto P_{S_1}(u|c,m) \cdot P_{\mathcal{M}|\mathcal{C}}(m|c)
\end{align}%
Unfortunately, in the case of our ironically uttered ``The weather is amazing.'', the pragmatic listener's meaning distribution would be exactly the same as the literal listener\footnote{Assuming $\alpha > 0$ which is typically the case.}. That is, the pragmatic listener is unable to escape the literal interpretation of the utterance prescribed by $L_0$. In general, it can be shown that interpretations of utterances which are not compatible with the literal meaning of an utterance will be assigned zero probability by the pragmatic listener (Proof in Appendix~\ref{app:rsa_theory}). Existing approaches discussed in Section~\ref{sec:human_irony} have attempted to mitigate this limitation by imposing assumptions on the speaker's motivation to be figurative (e.g., for affect-related reasons). In contrast, our \rsatwospace approach which we present below does not make any such assumptions and instead requires the pragmatic listener to reason about the rhetorical strategy being employed.
\subsection{The Rhetorical-Strategy-Aware RSA Framework}
\label{sec:rsatwooriginal}
We introduce a rhetorical-strategy-aware RSA framework, \rsatwo, where a speaker's rhetorical strategy is explicitly modeled and used to interpret an utterance's potentially non-literal intended meaning. We define a rhetorical strategy as the way in which an utterance relates to its intended meaning. For example, the \emph{ironic} rhetorical strategy interprets an utterance using the opposite of its literal meaning, while the \emph{hyperbole} rhetorical strategy interprets an utterance using meanings which are higher on a gradable scale than its literal meaning.
To integrate rhetorical strategies within RSA's recursive reasoning procedure, we introduce a rhetorical strategy variables $r \in \mathcal{R}$ and a rhetorical function $f_{r}: \mathcal{R} \times \mathcal{M} \times \mathcal{U} \rightarrow \{0,1\}$ which generalizes the literal semantic understanding indicator function $\mathds{1}_{m \in \llbracket u \rrbracket}$ \ces{@Jackie, does it make more sense to generalize to $[0,1]$ here? It might seem a little random because in the running example we never have a continuous value.}. The rhetorical function enables for non-literal interpretations of utterances to arise based on the employed rhetorical strategy. For instance, using our running example, the ironic and hyperbolic rhetorical strategies might return the following:
\begin{align*}
f_\textit{irony}(\texttt{terrible}, \text{``The weather is amazing.''}) &= 1 \\
f_\textit{irony}(\texttt{amazing}, \text{``The weather is amazing.''}) &= 0 \\
f_\textit{hyperbole}(\texttt{good}, \text{``The weather is amazing.''}) &= 1 \\
f_\textit{hyperbole}(\texttt{amazing}, \text{``The weather is amazing.''}) &= 0
\end{align*}
We replace the semantic understanding function with our generalized rhetorical function within the $P_{L_0}$ equation as follows%
\begin{align}
P_{L_0}(m|c,u,r) &\propto f_r(m,u) \cdot P(m|c)\label{eqn:rsatwol0}
\end{align}%
Thus, a meaning which receives zero probability mass when an utterance is interpreted literally by $L_0$ may still receive non-zero probability mass when that same utterance is interpreted using irony, hyperbole or some other non-literal rhetorical strategy. For example, using $f_\text{irony}$ and $f_\text{hyperbole}$ in our running example would enable the probability distribution of $L_0$ to shift towards non-literal interpretations of the utterance such as \texttt{terrible} and \texttt{good} respectively.
We define the pragmatic speaker and listener in a similar fashion to standard RSA with our rhetorical strategy variable acting as an additional conditional variable.%
\begin{align}%
P_{S_1}(u|m,c,r) &\propto P_{L_0}(m|c,u,r)^\alpha \cdot P(u|c) \\
P_{L_1}(m|c,u,r) &\propto P_{S_1}(u|m,c,r) \cdot P(m|c)
\end{align}%
The set of all considered rhetorical strategies are integrated within a single pragmatic listener by marginalizing the rhetorical-strategy aware pragmatic listeners across all considered strategies. This marginalization requires the introduction of a new rhetorical strategy prior, $P(r|c,u)$, shown in the following equation:%
\begin{align}
\hspace{-5mm}P_{L_1}(m|c,u) = \sum_{r'} &P_{L_1}(m|c,u,r') \cdot P(r'|c,u)
\end{align}%
Returning to our running example, let us consider a discrete set of rhetorical strategies $\mathcal{R} = \{\textit{literal}, \textit{irony}, \textit{hyperbole}\}$. We observe that the pragmatic listener conditioned with the \textit{literal} strategy would produce the same distribution as the RSA-derived pragmatic listener. However, the utterance would also be interpreted by $P_{L_1}(m|c,u,\textit{irony})$ with most probability mass concentrated around \texttt{terrible}, the opposite of the utterance's literal meaning, and by $P_{L_1}(m|c,u,\textit{hyperbole})$ with most mass concentrated around \texttt{bad}, the utterance's scaled down literal meaning. By marginalizing these listener distributions using a rhetorical prior\footnote{The prior may be uniform, human-derived or computed experimentally.}, we would obtain a distribution which assigns mass to meanings beyond the utterance's literal interpretation.
\ces{TODO: Move Section 4 to a subsection here - Only show PL1(m|c,u) when using the precipitation context.}
\subsection{\rsatwolearned}
While \rsatwospace enables the pragmatic listener to interpret utterances non-literally, this formulation requires manually specifying which rhetorical strategies to consider along with determining their corresponding meaning-utterance mappings, i.e. specifying $f_r(m, u)$. If no prototypical rhetorical strategies can be assumed such as in our weather-related running example, this would require the pragmatic listener to marginalize over every possible rhetorical function which would be intractable in most settings (see Appendix~\ref{app:rsa_theory} for more theoretical details).
As a result, to make \rsatwospace a scalable approach to figurative language understanding, we introduce an unsupervised-learning-based algorithm, \rsatwolearned, to automatically \emph{induce} the most salient rhetorical functions and their corresponding values. To do so, we rely on the intuition that utterances which are generated with the same intended meaning and rhetorical strategy are likely to be semantically similar. For example, the utterances ``The weather is amazing.'' and ``Gosh, this weather is so great!'', uttered in the context of bad weather, are likely to both employ the same rhetorical strategy (irony) to convey the same intended meaning (that the weather is in fact terrible). The algorithm which computes the set of induced rhetorical function values $F(m, u) = \{f_1(m, u), \dots, f_n(m, u)\}$ relies on embedding alternative utterances and clustering them to induce \emph{rhetorical strategy clusters} (Algorithm~\ref{alg:learned_rsa}). These rhetorical strategy clusters and the values they return for $m, u$ can then be used in Equation~\ref{eqn:rsatwol0}.
\begin{algorithm}
\caption{\rsatwolearned}\label{alg:learned_rsa}
\begin{algorithmic}[1]
\Ensure Context $c$, Meaning $m$, Utterance $u$
\Require Set of induced rhetorical function values $F(m, u) = \{f_1(m, u), \dots, f_n(m, u)\}$
\Comment{Initialize the following objects.}
\State Instruction-tuned LLM $M$
\State Base LLM $G$
\State Embedding function $\texttt{embed}: \mathcal{U} \to \mathbb{R}^n$
\State Clustering function $\texttt{cluster}: \mathcal{P}(\mathbb{R}^n) \to \mathcal{P}(\mathbb{R}^n)$
\State Semantic similarity function $\texttt{sim}: \mathbb{R}^n \times \mathbb{R}^n \to [0, 1]$
\Comment{Compute $F(m,u)$ using cluster centroids.}
\State $U_\text{alt} \leftarrow \{ u : u \sim P_G(u|c) \}$
\State $E \leftarrow \{ \texttt{embed}(u') : u' \in \{ u \} \cup U_\text{alt} \}$
\State $C \leftarrow \texttt{cluster}(E)$
\State $F(c, m, u) \leftarrow \{ \texttt{sim}(\texttt{embed}(u), c') \cdot P_M(m|c',u) : c' \in C\}$
\State \Return $F(c, m, u)$
\end{algorithmic}
\end{algorithm}
In brief, the \rsatwolearnedspace algorithm generates, embeds and clusters a set of alternative utterances which could have occurred in context $c$ to create the rhetorical strategy clusters $C$. Using the generated clusters $C$, \rsatwolearnedspace computes the corresponding rhetorical function values $f_r(m, u)$ for $r \in C$ by measuring the similarity between the $u$'s embedding and the centroid of cluster $r$. Returning to our running example, if the alternative utterances for $u = \text{``The weather is amazing.''}$ include $u_1 = \text{``Gosh, this weather is so great!''}$, $u_2 = \text{``The weather is terrible.''}$ and $u_3 = \text{``The weather is so bad.''}$, then we expect for the embeddings of $u$ and $u_1$ to be in one cluster and for the embeddings of $u_2$ and $u_3$ to be in another cluster. These two rhetorical strategy clusters would then resemble the \emph{ironic} and \emph{literal} rhetorical strategies from Section~\ref{sec:rsatwooriginal}.
To experimentally validate our approach, in Section~\ref{sec:experiment1} we run our \rsatwospace framework on the set of weather-related utterances presented as a running example in this section and show that our derived pragmatic listener is able to achieve non-literal meaning interpretations. In Section~\ref{sec:experiment2}, we show that \rsatwolearnedspace enables \rsatwospace to be used where the set of rhetorical strategies need not be defined in advance. We show that our approach achieves state-of-the-art performance on an irony understanding dataset.
\section{A Weather-Related Proof-Of-Concept Application of \rsatwo}
\label{sec:experiment1}
In this section, we apply \rsatwospace on weather-related utterances and show, as a proof-of-concept, that it is able to produce non-literal interpretations unlike the original RSA formulation. In particular, we compute pragmatic listener meaning probability distributions using the standard RSA framework and our novel \rsatwospace framework on a dataset consisting of utterances from our weather-related running example.
\subsection{Dataset}
Our weather-related descriptions dataset, which was originally developed by \citet{Kao2015Irony}, consists of the following sets of contexts, rhetorical strategies, meanings and utterances:%
\begin{align*}
\mathcal{C} &= \{\text{``Precipitation with heavy winds.''},\\ &\text{``Sunny with clear skies.''}\} \\
\mathcal{R} &= \{\textit{literal},\textit{irony},\textit{hyperbole},\textit{understatement}\}\\
\mathcal{M} &= \{\texttt{terrible}, \texttt{bad}, \texttt{ok}, \texttt{good}, \texttt{amazing}\}\\
\mathcal{U} &= \{\text{``The weather is $m$.''} : m \in \mathcal{M}\}
\end{align*}%
For each of the considered rhetorical strategies, we manually set the values for $f_r$ for all meaning-utterance pairs. The complete list of mappings can be found in Appendix~\ref{app:weather_related}.
\subsection{Experimental Setup}
We compute the pragmatic listener probability distributions over meanings using the original and \rsatwospace formulations. We use the same meaning priors $P(m|c)$ as elicited using humans by \citet{Kao2015Irony} and set the utterance and rhetorical strategy priors $P(u|c), P(r|c,u)$ to be uniform and set the rationality parameter $\alpha = 1$.
\subsection{Results}
Figure~\ref{fig:weather} shows the meaning interpretation of the utterance ``The weather is amazing.'' under both sunny and precipitation contexts using the original and rhetorical-strategy-aware formulations of RSA (see Appendix~\ref{app:weather_related} for all other contexts). We observe that, as expected, the pragmatic listener implemented with the original RSA formulation assigns all of the intended meaning probability mass to the meaning that is literally compatible with the utterance i.e. \texttt{amazing}. In contrast, \rsatwospace provides a non-literal interpretation of the utterance for both contexts. For instance, the pragmatic listener adopts an ironic interpretation in the precipitation scenario, assigning as much probability mass to \texttt{terrible} (0.27) as \texttt{amazing} (0.27). Additionally, the \rsatwospace pragmatic listener adopts a hyperbolic interpretation in the sunny weather scenario, assigning more probability mass to \texttt{good} (0.38) than \texttt{amazing} (0.28). In brief, our experimental results show that explicitly considering rhetorical strategies is sufficient to enable RSA to achieve non-literal interpretations of utterances. \ces{@Jackie - Like you said, I think it's hard to see what the ideal behaviour here would be. The best would be to correlate these responses with humans, but we don't have that data. How could we write this here? I think this is still a meaningful section because it really makes the point of the paper clear.}
\begin{figure*}[h]
\vspace{-1cm}
\centering
\includegraphics[width=\linewidth]{Images/meaning_probabilities.png}
\caption{Meaning probabilities for``The weather is amazing.'' under both the precipitation and sunny scenarios.\ces{TODO: See if can make the y-axis cutting look nice.}}
\label{fig:weather}\vspace{-0.5cm}
\end{figure*}
\section{Interpreting Ironic Utterances with \rsatwospace and LLMs}
\ces{@Jackie: Only Section 5.1 and the baselines in 5.2.3 are ready. The rest has to be re-written to accomodate for \rsatwolearned.}
\label{sec:experiment2}
In this section, we show that \rsatwospace can be adapted to enable LLMs to provide non-literal interpretations which better align with human interpretations of figurative language. In particular, we show how LLM-induced rhetorical strategies can be used in \rsatwospace to enable LLMs to provide correct meaning interpretations of \emph{ironic utterances} when faced with alternative literal and distractor meanings to choose from. Our results demonstrate that adapting \rsatwospace to LLMs significantly reduces lexical-overlap-based errors and enables them to achieve new state-of-the-art performance on PragMega+, an expanded irony understanding dataset.
In Section~\ref{sec:rs_coefficients}, we present the \emph{rhetorical strategy coefficients} (RSCs) which serve as a soft approximation for the \rsatwospace meaning-utterance indicator function, $\mathds{1}_{m \in \llbracket u \rrbracket^r}$. The RSCs enable us to use an approximate version of \rsatwospace where the rhetorical strategies need not be specified beforehand which can be intractable in general. In Section~\ref{sec:dataset} and~\ref{sec:experimental_setup}, we provide additional details regarding the PragMega+ dataset and the experimental setup we use to compute and use the RSCs for irony understanding.
\subsection{PragMega+}
\label{sec:dataset}
We use an expanded version of the PragMega dataset from \citet{hu_fine-grained_2023} to test the \rsatwospace framework on irony detection and irony interpretation. The original PragMega dataset contains 25 instances of \emph{ironic scenarios} similar to the one shown in Figure~\ref{fig:john-example}. These scenarios all include some background context as well as an utterance generated by a speaker ironically. In addition, each scenario is accompanied by four possible intended meanings as they relate to the utterance: a literal interpretation of the utterance (\texttt{Literal Meaning}), an ironic interpretation of the utterance (\texttt{Ironic Meaning}), a distractor meaning designed to have high lexical overlap with the utterance (\texttt{Overlap Meaning}), and another distractor meaning which is designed to not logically follow from the context and the utterance (\texttt{Non-Sequitur Meaning}).
Our expanded version of the PragMega dataset, which we call PragMeta+, contains 25 additional ironic scenarios which we use as part of our test set. In additional, we manually modify each of the 50 ironic scenarios to create 50 \emph{literal scenarios} in which the context literally matches what is being said in the utterance produced by the speaker. In this way, the correct interpretation of the utterance becomes the \texttt{Literal Meaning} rather than the \texttt{Ironic Meaning}. Examples of ironic and literal scenarios from PragMega are shown in Table~\ref{tab:pragmega_examples}. Additional examples can be found in Appendix~\ref{app:ironic_exp}.
\begin{table*}
\centering
\resizebox{\linewidth}{!}{
\begin{tabular}{p{6cm}p{6cm}p{6cm}}
\toprule
Ironic Scenario & Literal Scenario & Intended Meanings \\
\midrule
John is a teacher at an elementary school. When talking with the principal about a new student, \textbf{who did poorly on her entrance examination}, John said, ``This one is really sharp.'' & John is a teacher at an elementary school. When talking with the principal about a new student, \textbf{who scored one of the highest scores}, John said, ``This one is really sharp.'' & 1. The student is smart. \newline 2. The student is not very clever. \newline 3. The entrance exam is unfair. \newline 4. The pencils need to be sharpened. \\\midrule
In a shop, Lara tries on a dress. \textbf{The dress is far too long for her.} Lara asks Simon: ``Does this dress fit me?'' Simon answers: ``Wow! That must be custom made! It's clearly the perfect size and length for you.'' & In a shop, Lara tries on a dress. \textbf{The dress fits her perfectly.} Lara asks Simon: ``Does this dress fit me?'' Simon answers: ``Wow! That must be custom made! It's clearly the perfect size and length for you.'' & 1. The dress is fitting well. \newline 2. Lara needs to get a dress of a shorter length. \newline 3. Simon does not like the color of this dress. \newline 4. Simon has to get back to work. \\
\bottomrule
\end{tabular}
}
\caption{Examples of the original ironic scenarios found in the PragMega dataset as well as corresponding modified literal scenarios. The intended meanings are shared for both original and modified scenarios and are ordered in the following way: 1. \textsc{Ironic Meaning} 2. \textsc{Literal Meaning} 3. \textsc{Overlap Meaning} 4. \textsc{Non-Sequitur Meaning}. The context of the original scenarios is changed so that it matches with the literal intended meaning.}
\label{tab:pragmega_examples}
\end{table*}
\subsection{Experimental Setup}
\begin{figure*}[h]
\vspace{-1cm}
\centering
\includegraphics[width=\linewidth]{Images/lm_avgd_plot_v2.png}
\caption{Meaning probabilities averaged across ironic and literal scenarios. \rsatwolearnedspace and Standard RSA outperform the LLM baseline. These results are when using Llama-3.1-8B for generation and Mistral-7B-Instruct for probabilities, with $\alpha$ set to 1. Test set results and other combinations can be found in the appendix.}
\label{fig:avgd_main_results}
\end{figure*}
\begin{table*}
\centering
\resizebox{\linewidth}{!}{
\begin{tabular}{l|rrr|rrr}
Model & Correct$_{dev}$ & Incorrect$_{dev}$ & Distractor$_{dev}$ & Correct$_{test}$ & Incorrect$_{test}$ & Distractor$_{test}$ \\
\hline
\rsatwolearnedspace $L_{1}$ & 0.88 & 0.07 & \textbf{0.05} & 0.74 & 0.15 & \textbf{0.10} \\
\rsatwolearnedspace $L_{0}$ & \textbf{0.92} & 0.05 & 0.03 & 0.76 & 0.16 & 0.08 \\
Standard RSA $L_{1}$ & 0.81 & 0.16 & 0.03 & 0.76 & 0.19 & 0.05 \\
Standard RSA $L_{0}$ & 0.88 & 0.11 & 0.01 & \textbf{0.81} & 0.16 & 0.04 \\
$P_{LLM}(m|c,u)$ & 0.75 & \textbf{0.21} & \textbf{0.05} & 0.72 & \textbf{0.23} & 0.05 \\
\hline
\end{tabular}
}
\caption{Average probabilities of correct, incorrect, and distractor responses across all scenarios. Results are separated between development and test set. These results are when using Llama-8B for generation and Mistral-7B-Instruct for probabilities. \david{Make \rsatwospace not bolded lol}}
\label{tab:meaning_scores}
\end{table*}
\label{sec:experimental_setup}
We describe our experimental implementation of the \rsatwospace using LLM-derived rhetorical strategy coefficients. In particular, we describe how we use LLMs to estimate the prior and literal listener meaning probabilities as well as generate the alternative utterances needed to compute the RSCs used in Equation~\ref{eq:adapted_literal_listener} of our approximate \rsatwospace formulation. Using the implementations presented, we are able to compute the meaning probability distributions for the literal and pragmatic listeners with the RSC-based approximation of \rsatwo.
\subsubsection{Meaning Probability Estimation}
\label{sec:llms}
We use an instruction-tuned LLM, $M$, and prompt engineering to estimate the prior and literal listener meaning probabilities --- $P_{\mathcal{M}|\mathcal{C}}(m|c)$ and $P_{L_0}(m|c,u)$ --- from the approximate \rsatwospace formulation. To extract LLM-derived meaning probability estimates for each of these distributions --- $P_{M}(m|c)$ and $P_{M}(m|c,u)$ respectively --- we use a prompt template similar to \citet{huFinegrainedComparisonPragmatic2023} which lists all (4) possible intended meanings in a multiple-choice question (MCQ) format. After passing the prompt to the LLM $M$, we extract the next-token logit of the number corresponding to each meaning. These logits are re-normalized so that the sum of their corresponding probabilities equals one. In addition, to avoid positional bias, we randomly shuffle the order of the options in the prompt $X$ times and compute average probabilities. The prompts used are included in Appendix~\ref{app:ironic_exp}.
As implementations of $M$ and $G$, we experiment with the Mistral-7B-Instruct LLM \citep{jiang2023mistral7b}, and Llama-8B \david{add citation} via HuggingFace \citep{wolf2020huggingfacestransformersstateoftheartnatural}.
We use \emph{Mistral-7B-Instruct} for $M$ and \emph{Llama-3.1-8B} for $G$, which we found to be the best-performing combination. Results for other combinations are available in Appendix \ref{app:model_variations}.
\subsubsection{Alternative Utterance Generation} We use a base LLM $G$ which has only been pre-trained on next-token prediction to generate the alternative utterances need for \emph{both} the RSC computations \emph{as well as} the pragmatic speaker from Equation~\ref{eq:pragmatic_speaker}. To do so, we condition $G$'s next-token generation with the full scenario up to the original utterance's first quotation mark, e.g. ``\dots ... John said "'', and extract the generated tokens until another quotation mark is generated. In addition, we use the generation likelihood, $P_G(u|c)$, as the utterance prior.
We use a decoding temperature of $1.0$ and generate $50$ alternative utterances for each context. Thus, for a given context $c$, $|\mathcal{U}_c| = 50$.
We experiment with several implementations of $G$ including \ces{TODO: Add current implementation which is LLama3.1/2 and also add others.}
\subsubsection{Baselines}
Along with our RSC-based approximation of \rsatwo, we use several other LLM-derived baselines to estimate the meaning probabilities for the utterances from the PragMega+ dataset.
\paragraph{LLM Baseline} We use the average probability computed by $P_{M}(m|c,u)$ as a first baseline.
\paragraph{Standard RSA} We use the standard RSA formulation and estimate $\mathds{1}_{m \in \llbracket u \rrbracket^r}$, $P(m|c)$ and $P(u|c)$ with $P_{M}(m|c,u)$, $P_M(m|c)$, and $P_G(u|c)$ respectively. We use the same experimental settings as our \rsatwospace implementation.
\paragraph{RSC-Free \rsatwo} We also include a RSC-free adaptation of \rsatwospace which more closely resembles its original formulation in Section~\ref{sec:rsa_two}. To do so, we explicitly consider two rhetorical strategies: literal and ironic. We approximate the rhetorical-strategy-aware indicator function, $\mathds{1}_{m \in \llbracket u \rrbracket^r}$, and the additional rhetorical strategy prior with the instruction-tuned probabilities $P_M(m|c,u,r)$ and $P_M(r|c,u)$. For the former probability, we experiment with several prompt templates (Appendix~\ref{app:ironic_exp}) which explicitly instruct $M$ to interpret $u$ under the rhetorical strategy $r$. A similar MCQ-formatted prompt is used for the latter probability distribution $P_M(r|c,u)$.
We note that an affect-aware RSA baseline cannot be implemented as some of the scenarios (e.g., Figure~\ref{fig:john-example}) lack any obvious affect and thus only an approximation of the literal RSA formulation would be possible.
\section{Experimental Results}
\begin{figure*}[h]
\vspace{-1cm}
\centering
\includegraphics[width=\linewidth]{Images/lm_main_plot_v5.png}
\caption{Meaning probabilities on ironic and literal scenarios from the development and test sets. On ironic scenarios, \rsatwolearnedspace and Standard RSA outperform the LLM prior. On literal scenarios from the test set, the LLM prior performs better than \rsatwolearned and Standard RSA. These results are when using Llama-3.1-8B for generation and Mistral-7B-Instruct for probabilities, with $\alpha$ set to 1. Other combinations can be found in the appendix.}
\label{fig:main_results}
\end{figure*}
\begin{figure*}[h]
\centering
\includegraphics[width=\linewidth]{Images/alpha_sweep_dev_v3.png}
\caption{Meaning probabilities for different $\alpha$ values on ironic and literal scenarios from the development set. Increasing $\alpha$ improves performance on literal scenarios while worsening performance on ironic scenarios. These results are when using Llama-3.1-8B for $G$ and Mistral-7B-Instruct for $M$. Results from the test set are similar and can be found in Appendix \david{add ref}.}
\label{fig:alpha_dev}
\end{figure*}
\subsection{Overall Performance}
Figure \ref{fig:avgd_main_results} and Table \ref{tab:meaning_scores} show the average probabilities assigned to correct, incorrect, and distractor responses on the development and test sets. $P_{LLM}(m|c,u)$ performs worst overall. While \rsatwolearnedspace outperforms Standard RSA on the development set, Standard RSA performs slightly better on the test set. For all cases, each literal listener $L_0$ outperforms the respective pragmatic listener $L_1$. On the test set, \rsatwolearnedspace allocates higher probabilities to distractor values \david{maybe clarify and move to analysis}.
\subsection{Ironic vs. Literal Performance}
Figure \ref{fig:main_results} shows the average meaning probabilities broken down between ironic and literal scenarios. Additionally, it separates distractors into non-sequiturs and lexical overlap associates. On ironic scenarios, \rsatwolearnedspace performs best, followed by Standard RSA, then $P_{LLM}(m|c,u)$. Additionally, literal listeners $L_0$ outperform their respective pragmatic listeners $L_1$ on ironic scenarios \david{indicating the strength of the prior}. On literal scenarios from the development set, all methods perform similarly. On the test set, $P_{LLM}(m|c,u)$ performs a bit better than Standard RSA and much better than \rsatwolearned.
Most errors are a result of assigning probability to the opposite meaning (i.e. the literal meaning in ironic scenarios and the non-literal meaning in literal scenarios). Most distractor errors are due to associate meanings with high lexical overlap, rather than non-sequiturs. \david{Some analysis here perhaps}.
\subsection{Rhetorical Strategy Latent Variable}
Figure \ref{fig:rs_dev_latent_results} shows the average meaning probabilities assigned by \rsatwo. The literal speaker $L_0$ assigns high probabilities to the correct meaning when given the correct rhetorical strategy. When given the incorrect rhetorical strategy, $L_0$ probabilities for the Literal and NonLiteral meanings are near 0.5, indicating a high degree of uncertainty. When calculating \rsatwospace $L_1$, we assume a uniform probability $P(r|c,u)$. We find that this performs worse than the base LLM prior $P_{LLM}(m|c,u)$. We believe that this is due to the assumption of a uniform prior - \david{COME BACK TO THIS WHEN YOU HAVE RESULTS}
\begin{figure*}[h]
\vspace{-1cm}
\centering
\includegraphics[width=\linewidth]{Images/rs_dev_probs_v2.png}
\caption{Meaning probabilities on ironic and literal scenarios from the development sets using \rsatwo. \rsatwospace $L_0$ performs well when given the correct rhetorical strategy, but struggles otherwise. \rsatwospace $L_1$ marginalized across both strategies performs poorly. These results are when using Llama-3.1-8B for generation and Mistral-7B-Instruct for probabilities, with $\alpha$ set to 1. Test set results and other combinations can be found in the appendix.}
\label{fig:rs_latent_results}
\end{figure*}
\david{Assuming that the P(m|c,u,r) thing fails miserably, analyze why. For example that it's very hard to get the LLM to interpret things ironically/literally directly [this may also be model specific]. If the LLM could actually do this, we would still have the problem of not being able to know which rhetorical strategies are relevant beforehand.
For instance, a reviewer might comment on why we would not just use the rhetorical strategy as a latent variable and include it in the prompt. We can show that L0 is not easily adjusted by prompt engineering LLMs.}
\subsection{Impact of $\alpha$ Hyperparameter}
Figure \ref{fig:alpha_dev} shows the impact of the pragmatic speaker $S_1$'s $\alpha$ hyperparameter on meaning probabilities. Lowering the value of $\alpha$ improves probabilities on ironic scenarios but worsens then on literal scenarios. Lowering $\alpha$s flattens the distribution of $P_{L_0}$, which puts more emphasis on $P_{LLM}(u|c)$. \david{Need to think about how to explain this. It seems like the opposite of what I would expect.}
\david{Should we mention that the rate of change for ironic vs literal is reversed on the test set?}
\subsection{Qualitative Analysis}
We discuss an example in detail to demonstrate the effect of our method.
\david{Qualitative Analysis: Analyze a few examples and show how the probability distribution shifted from the LLM prob, to the adjusted listener to the pragmatic listener (a nice to have would be an example where one of the distractors is related semantically and thus has a higher probability for PL0adj, but this probability decreases when interpreted pragmatically using L1).}
\section{Conclusion}
Remind the reader of the two main contributions: the rhetorical-strategy-aware part of \rsatwospace and the adjustment ratio which establishes for the first time what to do when base listener probabilities are not well calibrated (I think calibrated here is exactly used in the correct way).
\section*{Limitations}
Limitation 1: Dataset set -> Say that will develop a larger figurative language understanding benchmark as a follow-up
Other limitations depend on the exact formulation we settle on for the adjustment ratios and the results we have
\bibliographystyle{acl_natbib}
\bibliography{custom}
\clearpage
\appendix
\section{Additional RSA and \rsatwospace Details}
\label{app:rsa_theory}
% Add a proof about 0 probability
% Add derivation of the intractability $2^{|\mathcal{U}| \times |\mathcal{M}|$
% This is correct because imagine grouping by M and then U and then considering whether
% each position is 0 or 1 : [0_m1 0_m2 0_m3 0_m4]_u1 ... [0_m1 0_m2 0_m3 0_m4]_un
\section{Weather-Related Utterances Experiment Additional Details}
\label{app:weather_related}
\section{Ironic Interpretation Experiment Additional Details}
\label{app:ironic_exp}
\section{Different Model Variations}
\label{app:model_variations}
\begin{table*}
\centering
\resizebox{\linewidth}{!}{
\begin{tabular}{lllrrrr}
\toprule
Dataset & rs & model & CorrectLiteral & IncorrectNonLiteral & IncorrectAssociate & IncorrectNonSequitur \\
\toprule
train & irony & Learned PL1(m|c,u) & 0.64 & 0.19 & 0.10 & 0.07 \\
train & irony & Learned PL0(m|c,u) & 0.88 & 0.08 & 0.03 & 0.01 \\
train & irony & PL1(m|c,u) & 0.50 & 0.33 & 0.10 & 0.07 \\
train & irony & PL0(m|c,u) & 0.76 & 0.19 & 0.04 & 0.01 \\
train & irony & PLLM(m|c,u) & 0.52 & 0.37 & 0.07 & 0.05 \\
train & irony & PMCUR PL1(m|c,u) & 0.44 & 0.46 & 0.07 & 0.04 \\
train & irony & Ironic PL0(m|c,u,r) & 0.78 & 0.19 & 0.03 & 0.01 \\
train & irony & Literal PL0(m|c,u,r) & 0.56 & 0.41 & 0.02 & 0.01 \\
train & irony & Ironic PLLM(m|c,u,r) & 0.63 & 0.23 & 0.08 & 0.06 \\
train & irony & Literal PLLM(m|c,u,r) & 0.49 & 0.38 & 0.08 & 0.05 \\
\midrule
test & irony & Learned PL1(m|c,u) & 0.55 & 0.23 & 0.13 & 0.08 \\
test & irony & Learned PL0(m|c,u) & 0.80 & 0.11 & 0.07 & 0.02 \\
test & irony & PL1(m|c,u) & 0.41 & 0.41 & 0.11 & 0.08 \\
test & irony & PL0(m|c,u) & 0.73 & 0.20 & 0.06 & 0.02 \\
test & irony & PLLM(m|c,u) & 0.45 & 0.41 & 0.10 & 0.05 \\
test & irony & PMCUR PL1(m|c,u) & 0.30 & 0.60 & 0.07 & 0.03 \\
test & irony & Ironic PL0(m|c,u,r) & 0.76 & 0.20 & 0.03 & 0.01 \\
test & irony & Literal PL0(m|c,u,r) & 0.44 & 0.50 & 0.05 & 0.01 \\
test & irony & Ironic PLLM(m|c,u,r) & 0.59 & 0.25 & 0.10 & 0.06 \\
test & irony & Literal PLLM(m|c,u,r) & 0.40 & 0.42 & 0.12 & 0.06 \\
\midrule
Dataset & rs & model & CorrectNonLiteral & IncorrectLiteral & IncorrectAssociate & IncorrectNonSequitur \\
\toprule
train & literal & Learned PL1(m|c,u) & 0.65 & 0.16 & 0.11 & 0.07 \\
train & literal & Learned PL0(m|c,u) & 0.80 & 0.15 & 0.04 & 0.02 \\
train & literal & PL1(m|c,u) & 0.72 & 0.12 & 0.10 & 0.06 \\
train & literal & PL0(m|c,u) & 0.85 & 0.11 & 0.04 & 0.01 \\
train & literal & PLLM(m|c,u) & 0.77 & 0.11 & 0.08 & 0.04 \\
train & literal & PMCUR PL1(m|c,u) & 0.76 & 0.14 & 0.07 & 0.03 \\
train & literal & Ironic PL0(m|c,u,r) & 0.74 & 0.23 & 0.02 & 0.01 \\
train & literal & Literal PL0(m|c,u,r) & 0.89 & 0.07 & 0.03 & 0.01 \\
train & literal & Ironic PLLM(m|c,u,r) & 0.59 & 0.27 & 0.08 & 0.06 \\
train & literal & Literal PLLM(m|c,u,r) & 0.76 & 0.12 & 0.08 & 0.05 \\
\midrule
test & literal & Learned PL1(m|c,u) & 0.52 & 0.25 & 0.15 & 0.08 \\
test & literal & Learned PL0(m|c,u) & 0.57 & 0.31 & 0.10 & 0.02 \\
test & literal & PL1(m|c,u) & 0.69 & 0.13 & 0.12 & 0.06 \\
test & literal & PL0(m|c,u) & 0.71 & 0.19 & 0.08 & 0.02 \\
test & literal & PLLM(m|c,u) & 0.68 & 0.15 & 0.12 & 0.04 \\
test & literal & PMCUR PL1(m|c,u) & 0.72 & 0.13 & 0.10 & 0.04 \\
test & literal & Ironic PL0(m|c,u,r) & 0.60 & 0.32 & 0.06 & 0.02 \\
test & literal & Literal PL0(m|c,u,r) & 0.80 & 0.11 & 0.08 & 0.01 \\
test & literal & Ironic PLLM(m|c,u,r) & 0.46 & 0.33 & 0.14 & 0.08 \\
test & literal & Literal PLLM(m|c,u,r) & 0.63 & 0.15 & 0.15 & 0.06 \\
\bottomrule
\end{tabular}
}
\caption{Average probabilities for each meaning type across all scenarios and datasets. Results are separated by dataset and between ironic and literal scenarios. These results are when using Llama-8B for generation and for probabilities.}
\label{tab:full_meaning_scores_ll}
\end{table*}
\begin{table*}
\centering
\resizebox{\linewidth}{!}{
\begin{tabular}{lllrrrr}
\toprule
Dataset & rs & model & CorrectLiteral & IncorrectNonLiteral & IncorrectAssociate & IncorrectNonSequitur \\
\toprule
train & irony & Learned PL1(m|c,u) & 0.85 & 0.09 & 0.05 & 0.01 \\
train & irony & Learned PL0(m|c,u) & 0.91 & 0.05 & 0.04 & 0.00 \\
train & irony & PL1(m|c,u) & 0.67 & 0.29 & 0.02 & 0.02 \\
train & irony & PL0(m|c,u) & 0.81 & 0.18 & 0.01 & 0.00 \\
train & irony & PLLM(m|c,u) & 0.55 & 0.40 & 0.03 & 0.03 \\
train & irony & PMCUR PL1(m|c,u) & 0.49 & 0.45 & 0.04 & 0.02 \\
train & irony & Ironic PL0(m|c,u,r) & 0.91 & 0.09 & 0.00 & 0.00 \\
train & irony & Literal PL0(m|c,u,r) & 0.54 & 0.45 & 0.00 & 0.00 \\
train & irony & Ironic PLLM(m|c,u,r) & 0.85 & 0.10 & 0.02 & 0.03 \\
train & irony & Literal PLLM(m|c,u,r) & 0.50 & 0.46 & 0.02 & 0.02 \\
\midrule
test & irony & Learned PL1(m|c,u) & 0.80 & 0.10 & 0.08 & 0.02 \\
test & irony & Learned PL0(m|c,u) & 0.84 & 0.07 & 0.08 & 0.00 \\
test & irony & PL1(m|c,u) & 0.70 & 0.23 & 0.05 & 0.01 \\
test & irony & PL0(m|c,u) & 0.81 & 0.15 & 0.05 & 0.00 \\
test & irony & PLLM(m|c,u) & 0.58 & 0.37 & 0.03 & 0.01 \\
test & irony & PMCUR PL1(m|c,u) & 0.43 & 0.55 & 0.02 & 0.01 \\
test & irony & Ironic PL0(m|c,u,r) & 0.95 & 0.04 & 0.00 & 0.00 \\
test & irony & Literal PL0(m|c,u,r) & 0.53 & 0.46 & 0.01 & 0.00 \\
test & irony & Ironic PLLM(m|c,u,r) & 0.86 & 0.08 & 0.03 & 0.02 \\
test & irony & Literal PLLM(m|c,u,r) & 0.49 & 0.46 & 0.04 & 0.01 \\
\midrule
Dataset & rs & model & CorrectNonLiteral & IncorrectLiteral & IncorrectAssociate & IncorrectNonSequitur \\
\toprule
train & literal & Learned PL1(m|c,u) & 0.91 & 0.05 & 0.03 & 0.01 \\
train & literal & Learned PL0(m|c,u) & 0.94 & 0.05 & 0.01 & 0.01 \\
train & literal & PL1(m|c,u) & 0.95 & 0.03 & 0.02 & 0.00 \\
train & literal & PL0(m|c,u) & 0.95 & 0.03 & 0.01 & 0.00 \\
train & literal & PLLM(m|c,u) & 0.94 & 0.02 & 0.03 & 0.01 \\
train & literal & PMCUR PL1(m|c,u) & 0.81 & 0.14 & 0.04 & 0.02 \\
train & literal & Ironic PL0(m|c,u,r) & 0.55 & 0.44 & 0.01 & 0.01 \\
train & literal & Literal PL0(m|c,u,r) & 0.94 & 0.02 & 0.03 & 0.01 \\
train & literal & Ironic PLLM(m|c,u,r) & 0.26 & 0.62 & 0.06 & 0.05 \\
train & literal & Literal PLLM(m|c,u,r) & 0.93 & 0.03 & 0.03 & 0.01 \\
\midrule
test & literal & Learned PL1(m|c,u) & 0.69 & 0.21 & 0.07 & 0.03 \\
test & literal & Learned PL0(m|c,u) & 0.68 & 0.25 & 0.05 & 0.01 \\
test & literal & PL1(m|c,u) & 0.82 & 0.14 & 0.03 & 0.01 \\
test & literal & PL0(m|c,u) & 0.81 & 0.16 & 0.02 & 0.01 \\
test & literal & PLLM(m|c,u) & 0.87 & 0.08 & 0.03 & 0.02 \\
test & literal & PMCUR PL1(m|c,u) & 0.83 & 0.13 & 0.03 & 0.01 \\
test & literal & Ironic PL0(m|c,u,r) & 0.39 & 0.57 & 0.04 & 0.01 \\
test & literal & Literal PL0(m|c,u,r) & 0.91 & 0.07 & 0.01 & 0.01 \\
test & literal & Ironic PLLM(m|c,u,r) & 0.23 & 0.64 & 0.08 & 0.05 \\
test & literal & Literal PLLM(m|c,u,r) & 0.88 & 0.07 & 0.03 & 0.02 \\
\bottomrule
\end{tabular}
}
\caption{Average probabilities for each meaning type across all scenarios and datasets. Results are separated by dataset and between ironic and literal scenarios. These results are when using Llama-8B for generation and Mistral-7B-Instruct for probabilities.}
\label{tab:full_meaning_scores_lm}
\end{table*}
\begin{table*}
\centering
\resizebox{\linewidth}{!}{
\begin{tabular}{lllrrrr}
\toprule
Dataset & rs & model & CorrectLiteral & IncorrectNonLiteral & IncorrectAssociate & IncorrectNonSequitur \\
\toprule
train & irony & Learned PL1(m|c,u) & 0.62 & 0.22 & 0.10 & 0.06 \\
train & irony & Learned PL0(m|c,u) & 0.87 & 0.08 & 0.04 & 0.01 \\
train & irony & PL1(m|c,u) & 0.45 & 0.39 & 0.10 & 0.06 \\
train & irony & PL0(m|c,u) & 0.73 & 0.21 & 0.05 & 0.01 \\
train & irony & PLLM(m|c,u) & 0.50 & 0.39 & 0.07 & 0.04 \\
train & irony & PMCUR PL1(m|c,u) & 0.41 & 0.50 & 0.06 & 0.04 \\
train & irony & Ironic PL0(m|c,u,r) & 0.80 & 0.17 & 0.02 & 0.01 \\
train & irony & Literal PL0(m|c,u,r) & 0.53 & 0.44 & 0.03 & 0.01 \\
train & irony & Ironic PLLM(m|c,u,r) & 0.65 & 0.21 & 0.08 & 0.06 \\
train & irony & Literal PLLM(m|c,u,r) & 0.46 & 0.40 & 0.09 & 0.06 \\
\midrule
test & irony & Learned PL1(m|c,u) & 0.55 & 0.24 & 0.13 & 0.08 \\
test & irony & Learned PL0(m|c,u) & 0.81 & 0.12 & 0.06 & 0.01 \\
test & irony & PL1(m|c,u) & 0.38 & 0.43 & 0.11 & 0.08 \\
test & irony & PL0(m|c,u) & 0.74 & 0.19 & 0.05 & 0.02 \\
test & irony & PLLM(m|c,u) & 0.44 & 0.41 & 0.10 & 0.05 \\
test & irony & PMCUR PL1(m|c,u) & 0.29 & 0.60 & 0.08 & 0.03 \\
test & irony & Ironic PL0(m|c,u,r) & 0.74 & 0.21 & 0.03 & 0.01 \\
test & irony & Literal PL0(m|c,u,r) & 0.46 & 0.48 & 0.05 & 0.01 \\
test & irony & Ironic PLLM(m|c,u,r) & 0.57 & 0.26 & 0.11 & 0.06 \\
test & irony & Literal PLLM(m|c,u,r) & 0.40 & 0.43 & 0.11 & 0.06 \\
\midrule
Dataset & rs & model & CorrectNonLiteral & IncorrectLiteral & IncorrectAssociate & IncorrectNonSequitur \\
\toprule
train & literal & Learned PL1(m|c,u) & 0.67 & 0.17 & 0.10 & 0.07 \\
train & literal & Learned PL0(m|c,u) & 0.80 & 0.15 & 0.03 & 0.01 \\
train & literal & PL1(m|c,u) & 0.73 & 0.12 & 0.09 & 0.05 \\
train & literal & PL0(m|c,u) & 0.84 & 0.11 & 0.04 & 0.01 \\
train & literal & PLLM(m|c,u) & 0.77 & 0.12 & 0.07 & 0.04 \\
train & literal & PMCUR PL1(m|c,u) & 0.74 & 0.14 & 0.07 & 0.04 \\
train & literal & Ironic PL0(m|c,u,r) & 0.75 & 0.22 & 0.02 & 0.01 \\
train & literal & Literal PL0(m|c,u,r) & 0.88 & 0.07 & 0.04 & 0.01 \\
train & literal & Ironic PLLM(m|c,u,r) & 0.58 & 0.27 & 0.08 & 0.07 \\
train & literal & Literal PLLM(m|c,u,r) & 0.76 & 0.12 & 0.08 & 0.05 \\
\midrule
test & literal & Learned PL1(m|c,u) & 0.49 & 0.27 & 0.15 & 0.09 \\
test & literal & Learned PL0(m|c,u) & 0.55 & 0.32 & 0.10 & 0.02 \\
test & literal & PL1(m|c,u) & 0.65 & 0.15 & 0.13 & 0.07 \\
test & literal & PL0(m|c,u) & 0.66 & 0.23 & 0.09 & 0.02 \\
test & literal & PLLM(m|c,u) & 0.66 & 0.16 & 0.14 & 0.05 \\
test & literal & PMCUR PL1(m|c,u) & 0.72 & 0.14 & 0.09 & 0.04 \\
test & literal & Ironic PL0(m|c,u,r) & 0.58 & 0.33 & 0.07 & 0.02 \\
test & literal & Literal PL0(m|c,u,r) & 0.80 & 0.12 & 0.07 & 0.01 \\
test & literal & Ironic PLLM(m|c,u,r) & 0.42 & 0.35 & 0.15 & 0.08 \\
test & literal & Literal PLLM(m|c,u,r) & 0.65 & 0.16 & 0.14 & 0.06 \\
\bottomrule
\end{tabular}
}
\caption{Average probabilities for each meaning type across all scenarios and datasets. Results are separated by dataset and between ironic and literal scenarios. These results are when using Mistral-7B-Instruct for generation and Llama-8B for probabilities.}
\label{tab:full_meaning_scores_ml}
\end{table*}
\begin{table*}
\centering
\resizebox{\linewidth}{!}{
\begin{tabular}{lllrrrr}
\toprule
Dataset & rs & model & CorrectLiteral & IncorrectNonLiteral & IncorrectAssociate & IncorrectNonSequitur \\
\toprule
train & irony & Learned PL1(m|c,u) & 0.84 & 0.10 & 0.05 & 0.01 \\
train & irony & Learned PL0(m|c,u) & 0.91 & 0.05 & 0.04 & 0.00 \\
train & irony & PL1(m|c,u) & 0.66 & 0.31 & 0.02 & 0.01 \\
train & irony & PL0(m|c,u) & 0.80 & 0.19 & 0.01 & 0.00 \\
train & irony & PLLM(m|c,u) & 0.55 & 0.40 & 0.03 & 0.02 \\
train & irony & PMCUR PL1(m|c,u) & 0.46 & 0.48 & 0.04 & 0.02 \\
train & irony & Ironic PL0(m|c,u,r) & 0.91 & 0.09 & 0.00 & 0.00 \\
train & irony & Literal PL0(m|c,u,r) & 0.53 & 0.47 & 0.01 & 0.00 \\
train & irony & Ironic PLLM(m|c,u,r) & 0.85 & 0.10 & 0.02 & 0.02 \\
train & irony & Literal PLLM(m|c,u,r) & 0.49 & 0.46 & 0.03 & 0.02 \\
\midrule
test & irony & Learned PL1(m|c,u) & 0.76 & 0.13 & 0.09 & 0.02 \\
test & irony & Learned PL0(m|c,u) & 0.83 & 0.08 & 0.09 & 0.00 \\
test & irony & PL1(m|c,u) & 0.68 & 0.26 & 0.04 & 0.02 \\
test & irony & PL0(m|c,u) & 0.80 & 0.16 & 0.04 & 0.00 \\
test & irony & PLLM(m|c,u) & 0.58 & 0.38 & 0.03 & 0.01 \\
test & irony & PMCUR PL1(m|c,u) & 0.39 & 0.58 & 0.02 & 0.01 \\
test & irony & Ironic PL0(m|c,u,r) & 0.95 & 0.05 & 0.00 & 0.00 \\
test & irony & Literal PL0(m|c,u,r) & 0.54 & 0.44 & 0.02 & 0.00 \\
test & irony & Ironic PLLM(m|c,u,r) & 0.86 & 0.09 & 0.03 & 0.02 \\
test & irony & Literal PLLM(m|c,u,r) & 0.47 & 0.46 & 0.05 & 0.02 \\
\midrule
Dataset & rs & model & CorrectNonLiteral & IncorrectLiteral & IncorrectAssociate & IncorrectNonSequitur \\
\toprule
train & literal & Learned PL1(m|c,u) & 0.89 & 0.06 & 0.03 & 0.01 \\
train & literal & Learned PL0(m|c,u) & 0.92 & 0.06 & 0.01 & 0.01 \\
train & literal & PL1(m|c,u) & 0.94 & 0.04 & 0.01 & 0.01 \\
train & literal & PL0(m|c,u) & 0.95 & 0.04 & 0.01 & 0.00 \\
train & literal & PLLM(m|c,u) & 0.94 & 0.03 & 0.02 & 0.01 \\
train & literal & PMCUR PL1(m|c,u) & 0.80 & 0.13 & 0.05 & 0.02 \\
train & literal & Ironic PL0(m|c,u,r) & 0.52 & 0.46 & 0.01 & 0.01 \\
train & literal & Literal PL0(m|c,u,r) & 0.94 & 0.01 & 0.04 & 0.01 \\
train & literal & Ironic PLLM(m|c,u,r) & 0.25 & 0.63 & 0.07 & 0.05 \\
train & literal & Literal PLLM(m|c,u,r) & 0.93 & 0.01 & 0.04 & 0.02 \\
\midrule
test & literal & Learned PL1(m|c,u) & 0.69 & 0.21 & 0.07 & 0.02 \\
test & literal & Learned PL0(m|c,u) & 0.69 & 0.23 & 0.07 & 0.01 \\
test & literal & PL1(m|c,u) & 0.83 & 0.13 & 0.03 & 0.01 \\
test & literal & PL0(m|c,u) & 0.82 & 0.15 & 0.02 & 0.01 \\
test & literal & PLLM(m|c,u) & 0.86 & 0.08 & 0.04 & 0.02 \\
test & literal & PMCUR PL1(m|c,u) & 0.84 & 0.13 & 0.02 & 0.01 \\
test & literal & Ironic PL0(m|c,u,r) & 0.39 & 0.57 & 0.03 & 0.01 \\
test & literal & Literal PL0(m|c,u,r) & 0.92 & 0.07 & 0.01 & 0.01 \\
test & literal & Ironic PLLM(m|c,u,r) & 0.24 & 0.62 & 0.09 & 0.05 \\
test & literal & Literal PLLM(m|c,u,r) & 0.89 & 0.06 & 0.03 & 0.02 \\
\bottomrule
\end{tabular}
}
\caption{Average probabilities for each meaning type across all scenarios and datasets. Results are separated by dataset and between ironic and literal scenarios. These results are when using Mistral-7B-Instruct for both generation and probabilities.}
\label{tab:full_meaning_scores_mm}
\end{table*}
\begin{figure*}[h]
\vspace{-1cm}
\centering
\includegraphics[width=\linewidth]{Images/rs_test_probs_v2.png}
\caption{Meaning probabilities on ironic and literal scenarios from the test set using \rsatwo. \rsatwospace $L_0$ performs well when given the correct rhetorical strategy, but struggles otherwise. \rsatwospace $L_1$ marginalized across both strategies performs poorly. These results are when using Llama-3.1-8B for generation and Mistral-7B-Instruct for probabilities, with $\alpha$ set to 1.}
\label{fig:rs_dev_latent_results}
\end{figure*}
\begin{figure*}[h]
\centering
\includegraphics[width=\linewidth]{Images/alpha_sweep_test_v3.png}
\caption{Meaning probabilities for different $\alpha$ values on ironic and literal scenarios from the test set. Increasing $\alpha$ improves performance on literal scenarios while worsening performance on ironic scenarios. These results are when using Llama-3.1-8B for $G$ and Mistral-7B-Instruct for $M$. Test set results can be found in Appendix \david{add ref}.}
\label{fig:alpha_dev}
\end{figure*}
\end{document}
% This must be in the first 5 lines to tell arXiv to use pdfLaTeX, which is strongly recommended.
\pdfoutput=1
% In particular, the hyperref package requires pdfLaTeX in order to break URLs across lines.
\documentclass[11pt]{article}
% Change "review" to "final" to generate the final (sometimes called camera-ready) version.
% Change to "preprint" to generate a non-anonymous version with page numbers.
\usepackage{acl}
% \usepackage{acl}
% Standard package includes
\usepackage{times}
\usepackage{latexsym}
\usepackage[T1]{fontenc}
\usepackage[utf8]{inputenc}
\usepackage{microtype}
\usepackage{inconsolata}
\usepackage{graphicx}
\usepackage{amsmath}
\usepackage{booktabs}
\usepackage{lipsum}
\usepackage[caption=false]{subfig}
\usepackage{float}
\usepackage{multirow}
\usepackage{amsmath}
\usepackage{changepage}
\usepackage{natbib}
\usepackage{amsmath}
\usepackage{stmaryrd}
\usepackage{dsfont}
\usepackage{geometry} % Optional, to ensure full-page width
\usepackage{changepage}
\usepackage{makecell}
\usepackage{listings} % Import the listings package
\usepackage{framed}
\lstset{
breaklines=true, % Enable line wrapping
basicstyle=\ttfamily, % Use monospace font
frame=single, % Add a frame around the code block
}
\DeclareMathOperator*{\argmax}{arg\,max}
\usepackage[]{todonotes} % use option "disable" to suppress notes
\makeatletter
\newcommand*\iftodonotes{\if@todonotes@disabled\expandafter\@secondoftwo\else\expandafter\@firstoftwo\fi}
\makeatother
\newcommand{\noindentaftertodo}{\iftodonotes{\noindent}{}\ignorespaces}
\newcommand{\david}[1]{{\color{purple}[David: #1]}}
\newcommand{\ces}[1]{{\color{red}[Ces: #1]}}
\newcommand{\rsatwo}{\textbf{RSA}\textsuperscript{2}}
\newcommand{\rsatwospace}{\rsatwo\,\,}
\newcommand{\response}[1]{\vspace{3pt}\hrule\vspace{3pt}\textbf{#1:}}
\interfootnotelinepenalty=10000
\usepackage[dvipsnames]{xcolor}
\definecolor{googlegreen}{RGB}{56, 118, 29}
\definecolor{googleblue}{RGB}{17, 85, 204}
\title{Are They Being Ironic or Literal? A Rhetorical-Strategy-Aware Rational Speech Act Framework for Irony Understanding}
\author{
Cesare Spinoso-Di Piano and David Austin\\
\texttt{\{cesare.spinoso-dipiano,david.e.austin\}@mail.mcgill.ca}\\
Mila – Quebec Artificial Intelligence Institute \\
McGill University \\
}
\begin{document}
\maketitle
\begin{abstract}
Ironic language is ubiquitous in human communication. Accordingly, it is essential that large language models (LLMs) are able to detect and interpret it when interacting with humans. However, previous work indicates that LLMs struggle with interpreting the intended meaning of ironic utterances. Prior research has applied the Rational Speech Act (RSA) framework to irony interpretation by assuming that affect is the sole motivator of irony and by crowdsourcing prior distributions. Our proposed method, \rsatwo, relaxes this assumption by modeling a speaker's rhetorical strategy. We also leverage LLMs in place of humans for prior estimation. While results appear mixed, we show that \rsatwospace can improve irony detection and interpretation in some circumstances. Beyond irony, our method has the potential of enabling LLMs to better handle different instances of figurative language.
\end{abstract}
%
\section{Introduction}
\label{sec:intro}
Figurative language is used pervasively in human communication \cite{roberts_why_figurative_1994}. For instance, figurative expressions like ``Can you pass the salt?'' or ``My car is a heap of junk.'' are frequently used to express non-literal meanings. As large language models (LLMs) are increasingly used to interpret human-generated natural language \cite{harrer_attention_2023, kasneci_chatgpt_2023}, it is imperative that they are able to understand figurative language to enable fluid human-AI interaction.
To address this need, we investigate if LLMs can understand a specific type of figurative language - irony. We define irony as when the intended meaning of an utterance is the opposite of its literal meaning. Figure \ref{fig:john-example} shows an example of an ironic utterance. The literal meaning of John's statement is that the student is smart. However, this meaning seems to contradict the fact that the student has done poorly on the exam. If we assume that the speaker is acting cooperatively \citep{grice1975logic}, then they must be communicating a non-literal meaning. As such, John's statement should be interpreted in context as saying that the student is not very clever. Using this definition of irony, we operationalize irony understanding through two tasks: irony detection and irony interpretation. Irony detection is indicating whether or not a statement is ironic. Irony interpretation is choosing the intended meaning of a possibly ironic statement given the context.
\begin{figure}[t]
\centering
\includegraphics[width=\linewidth]{Images/rsa_john_example.png}
\caption{A sample from the PragMega dataset showing probabilities of intended meanings as well as rhetorical strategies for a given scenario. The probabilities are computed using the Mistral 7B Instruct model.}
\label{fig:john-example}
\end{figure}
While LLMs appear to be good at detecting irony, \citet{huFinegrainedComparisonPragmatic2023} show that LLMs have difficulty with irony interpretation. The authors found that while LLMs perform on par with humans for some pragmatic phenomena, they struggle with tasks in which listener expectations are violated such as irony and humour. We hypothesize that LLMs struggle with these uses of language because violating listener expectations also violates next-token prediction. Next-token prediction does not explicitly model the meaning of the utterance in context, which allows for misinterpretation of ironic utterances.
A potential solution to the poor performance of LLMs on irony understanding is the Rational Speech Act (RSA) framework, a model of human communication designed to recover the intended meaning of an utterance, regardless of its form. The RSA framework has three components. First, a literal listener which maps between utterances and their literal meanings. Second, a pragmatic speaker which reasons about which utterance would best communicate a meaning to the literal listener. Third, a pragmatic listener which reasons about the intended meaning given the speaker's choice of utterance. As such, the RSA framework has the potential of mitigating LLM irony misinterpretation by forcing it to reason about the motivation of a speaker's utterance.
To enable the application of RSA in this context, we introduce a modification of the original RSA framework: \textbf{R}hetorical-\textbf{S}trategy-\textbf{A}ware \textbf{RSA} (\rsatwo). \rsatwospace addresses two current limitations of RSA. First, the original formulation of RSA is unable to derive non-literal meanings from an utterance. Previous work which has addressed this issue assumes that affect (e.g., joy, annoyance) is the sole factor responsible for non-literal utterances such as in irony. We relax this assumption by performing joint inference with a variable indicating the \emph{rhetorical strategy}, so that \rsatwospace can handle non-literal statements, such as irony, without having to assume a particular factor for its use. We define a rhetorical strategy as the way in which a speaker's utterance relates to their intended meaning. For example, speakers employing the \emph{ironic} rhetorical strategy will choose an utterance which literally means the opposite of their intended meaning. Second, we use LLMs to obtain prior estimates. Previously, these priors were surveyed from humans \citep{kao_number_words_2014,Kao2015Irony}. By using LLMs to obtain priors, we allow for more scalable and flexible usage.
We test our \rsatwospace on irony detection and interpretation on ironic and literal scenarios from the PragMega dataset \cite{huFinegrainedComparisonPragmatic2023}. Our findings are inconsistent, with explicit speaker models outperforming the LLM baseline on some tasks but performing worse on others. We identify the pragmatic speaker as a key failure point of our model. In particular, we find that our implementation of the pragmatic speaker causes probability mass to shift towards incorrect intended meanings, thus explaining the inconsistent behaviour of the pragmatic listener. We show that explicitly considering speaker intentions and rhetorical strategies has the potential to help LLMs better interpret verbal irony and potentially other types of figurative language, though further work is necessary.\footnote{Code (private repository shared with you) can be found at \url{https://github.com/cesare-spinoso/SarcRSA}}
\section{Related Work}
The goal of enabling language systems to understand figurative uses of language such as verbal irony has been a long-standing goal in cognitive science, computational linguistics and natural language processing. We first discuss how human uses of irony have been modeled in linguistics and cognitive science. We subsequently describe how the NLP community has approached the problem of irony understanding through tasks such as irony detection, interpretation and generation.
\subsection{Human Irony Prediction and Interpretation}
\label{sec:human_irony}
There has been extensive work to explain why human uses of irony occur in language. Early studies in pragmatics \citep{grice1975logic,horn1984towards} attempt to explain such uses of language by considering the communicative aspect of language. According to \citep{grice1975logic}, language users communicate with each other ``cooperatively'' by following certain ``conversational maxims'' (e.g., being informative and succinct, only stating what is true, etc.). As a result, because a listener expects a speaker to abide by these maxims, ironic statements which explicitly violate them can carry meaning beyond their literal interpretation. For instance, if Alice says to Bob ``It is so nice outside.'' when it is raining, then Bob will try to explain Alice’s ``uncooperativeness'' by supposing that she meant something more by her statement\footnote{Since rainy weather is not typically associated with being ``nice''.}, for instance that Alice is annoyed by the rain. Subsequent studies have furthered our understanding of human irony usage by showing that speakers use ironic statements to soften the tone of a critique or to be humorous \citep{robertsWhyPeopleUse1994,colstonSaltingWoundSugaring1997}, that ironic and other figurative uses of language can sometimes be easier to process cognitively than their literal counterparts \citep{ContextualEffectsUnderstanding1979} and that listeners understand ironic statements by explicitly interpreting the mental states of speakers through the ``Theory of Mind''-network part of the brain \citep{spotornoNeuralEvidenceThat2012}.
While the cooperative principle provides an intuitive explanation for why irony occurs, it cannot be used to predict its use and to recover its intended meaning. To address this gap, \citet{Kao2015Irony} adapt the RSA framework originally introduced by \citet{frankPredictingPragmaticReasoning2012}. In their work, the authors adapt the RSA framework to model ironic utterances about the weather by accounting for the affective attitudes of speakers. The study hired human participants to estimate probability priors (e.g., $P(\text{negative sentiment}|\text{rainy weather})$) which were then used with the RSA framework to predict the meaning of utterances of the form ``The weather is $X$.'' when the actual state of the weather was $Y$ (e.g., $X = \text{nice}, Y = \text{rainy}$). In this work, we aim to generalize this RSA adaptation by lifting its dependence on explicitly modeling speaker affect and by replacing human-derived probability estimates with probabilities induced by LLMs.
\subsection{Irony Modeling in NLP}
In NLP, the ability of language systems to understand irony has mainly been evaluated through irony detection and generation tasks \citep{van-hee-etal-2018-semeval,ZHANG20191633,balestrucci-etal-2024-im}. The most common approach to solving these tasks with language systems has been to collect irony-specific datasets and to fine-tune language classification and generation models on them \citep{van-hee-etal-2018-semeval}. However, as the focus in the field shifts towards LLMs and prompt-based approaches, there has been growing interest in evaluating pre-trained and instruction-tuned LLMs on their ability to recognize, interpret and generate figurative language such as irony and sarcasm \citep{gu-etal-2022-just,liu-etal-2022-testing,hu_fine-grained_2023}. Recent studies have shown that LLMs perform significantly worse than humans when it comes to interpreting figurative uses of language such as humour and irony. This failure of LLMs to correctly interpret ironic utterances is likely due to the fact that ironic utterances explicitly violate next-token prediction expectations. In this work, we aim to resolve this violation of expectation by augmenting an LLM's inference procedure with explicit models of listeners and speakers as prescribed by the RSA framework.
\subsection{RSA in NLP}
\citet{andreas_reasoning_2016} were the first to use neural listeners and speakers with RSA in the context of image captioning. \citet{friedUnifiedPragmaticModels2018} extended the use of RSA to instruction following and generation. \citet{shenPragmaticallyInformativeText2019} investigated RSA for abstractive summarization and \citet{cohn-gordonLostMachineTranslation2019} applied it to machine translation. In closer connection to our work on leveraging RSA for figurative language, \citet{careniniFullyInterpretableMore2024} combined the RSA framework with LLMs for the task of metaphor understanding. Our work aims to expand the application of RSA in this direction by adapting it to the detection and interpretation of ironic utterances.
\section{Irony Prediction and Interpretation via RSA}
We present our adaptation of the RSA framework to predict the use and intended meaning of ironic utterances. We begin by reviewing the RSA framework as originally introduced in \citet{goodman2013knowledge}. Subsequently, we show in more detail how \citet{kao_number_words_2014,Kao2015Irony} adapt this framework to model non-literal language. Finally, we discuss the limitations of this previous adaptation and present our alternative adaptation of the RSA framework - \textbf{R}hetorical-\textbf{S}trategy-\textbf{A}ware \textbf{RSA} (\rsatwo) - to account for both literal and ironic \emph{rhetorical strategies}. Our proposed method aims to force LLMs to explicitly reason about whether an utterance was generated ironically or literally before assigning an intended meaning to the utterance. We hypothesize that this additional reasoning step in the LLM's inference procedure may mitigate its bias towards favoring intended meanings with high utterance lexical overlap.
\paragraph{RSA} The RSA framework was originally introduced to model human communication and, more specifically, to explicitly account for listener and speaker expectations when predicting uses of scalar implicature \citep{goodman2013knowledge}. The goal of this framework is to derive a probability distribution over some fixed set of meanings $\mathcal{M}$ for some observed utterance $u$ within some context\footnote{The context $c$ can be linguistic (e.g., conversation history) as well as situational (e.g., the current state of the world).} $c$. To do so, this framework posits the existence of multiple speakers and listeners who reason about each other in a recursive fashion. This recursive procedure begins with a \emph{literal listener}, $L_0$, which reasons literally about the interpretation of utterances by defining the following conditional probability distribution over $\mathcal{M}$
%
\begin{align}
P_{L_0}(m|c,u) &\propto \mathds{1}_{m \in \llbracket u \rrbracket} \cdot P_{\mathcal{M}|\mathcal{C}}(m|c)
\end{align}
%
where $P_{\mathcal{M}|\mathcal{C}}$ is a prior conditional distribution over possible meanings given a context and where $\llbracket \cdot \rrbracket : \mathcal{U} \rightarrow \mathcal{P(M)}$ is a shared semantic understanding function\footnote{$\mathcal{P()}$ is the power set function.} which, given some utterance $u \in \mathcal{U}$, returns the set of possible intended meanings which are literally compatible with $u$.
With the base case of the reasoning procedure defined, the RSA framework attempts to provide a speaker-aware interpretation of the utterance $u$ by positing the existence of a \emph{pragmatic speaker}, $S_1$, which selects an utterance $u$ based on $L_0$'s expected information gain. That is,%
\begin{align}
P_{S_1}(u|c,m) &\propto P_{L_0}(m|c,u)
\end{align}%
Finally, to interpret a given utterance $u$ pragmatically, the \emph{pragmatic listener}, $L_1$, updates their prior belief about the distribution of possible intended meanings $P_{\mathcal{M}|\mathcal{C}}$ by using the likelihood that the utterance was generated by $S_1$ in the given context. In other words, the pragmatic listener's conditional probability distribution over $\mathcal{M}$ becomes the following posterior Bayesian update%
\begin{align}
P_{L_1}(m|c,u) \propto P_{S_1}(u|c,m) \cdot P_{\mathcal{M}}(m|c)
\end{align}%
By modeling communication in this manner, the RSA framework is able to explain the usage of certain utterances in natural language. For instance, in the context of modeling scalar implicature, a listener as modeled by $L_1$ would interpret the utterance $u_{\text{some}}\hspace{-0.5mm}=\hspace{-0.5mm}\text{``Bob ate some of the apples.''}$ to mean $m_{\text{only-some}}\hspace{-0.5mm}=\hspace{-0.5mm}\text{``Bob ate \emph{some but not all} of the apples.''}$ even though this utterance could \emph{literally} mean $m_{\text{all}}\hspace{-0.5mm}=\hspace{-0.5mm}\text{``Bob ate \emph{all} of the apples.''}$ $L_1$ assigns a much higher probability to $m_{\text{only-some}}$ than to $m_{\text{all}}$ since it knows that if $S_1$ had meant $m_{\text{all}}$ it would have used the maximally informative utterance $u_\text{all}\hspace{-0.5mm}=\hspace{-0.5mm}\text{``Bob ate all of the apples.''}$.
\paragraph{Affect-Aware RSA} While RSA has been shown to be effective at predicting the intended meaning of utterances in scalar implicature and in reference games \citep{frankPredictingPragmaticReasoning2012,goodman2013knowledge}, its standard formulation is ineffective at modeling phenomena like irony where the intended meaning $m$ of an utterance $u$ does not fall within its literal semantic interpretation $\llbracket u \rrbracket$. Indeed, it can be shown that any meaning $m$ which is not literally compatible with $u$ will receive zero probability mass in the standard RSA framework i.e., $\forall u \in \mathcal{U}, m \in \mathcal{M}, m \notin \llbracket u \rrbracket \implies P_{L_1}(m|c,u) = 0$.
As a result, previous work such as \citet{kao_number_words_2014,Kao2015Irony} account for this limitation of standard RSA by considering the intended \emph{affect} $a$ (joy, annoyance, etc.) of a given utterance. By designing $P_{L_0}(a|c,u)$ such that it is non-zero for non-literal utterances, the authors show that a pragmatic listener who jointly infers the intended meaning and affect will assign non-zero probability mass to meaning-affect pairs, $(m,a)$, even when $m \notin \llbracket u \rrbracket$. For instance, consider the non-literal utterance $u_{\text{million}}\hspace{-0.5mm}=\hspace{-0.5mm}\text{``I waited a million years to get a table.''}$. While standard RSA would only allow for the (clearly incorrect) literal interpretation of $u_{\text{million}}$, affect-aware RSA would place non-zero probability mass on non-literal interpretations of $u_{\text{million}}$ such as $(m = \text{waited 30 minutes},a = \text{annoyed})$.
\paragraph{\rsatwo} While the introduction of an affect variable allows non-zero mass to be placed on non-literal interpretations of utterances, it does not account for instances of irony where no clear affect is intended. For instance, in Figure~\ref{fig:john-example}, it is not clear what affect, if any, John is conveying by being ironic. In general, irony may be used to criticize \citep{colstonSaltingWoundSugaring1997}, to be humorous \citep{robertsWhyPeopleUse1994} or simply because it is the rhetorical strategy a listener most expects to hear in a given situation \citep{ContextualEffectsUnderstanding1979}.
As a result, in \rsatwo, we aim to jointly infer the intended meaning and the employed \emph{rhetorical strategy} (i.e., being literal vs. being ironic)\footnote{While there are several rhetorical strategies a speaker might employ (e.g., hyperbole, metaphor, etc.), in this work we restrict our focus to the literal and ironic rhetorical strategies.}. The key difference of \rsatwospace is that the semantic understanding function, $\llbracket \cdot \rrbracket_r : \mathcal{U} \rightarrow \mathcal{P}(\mathcal{M})$, now varies \emph{as a function of} the employed rhetorical strategy. Thus, a meaning which receives zero probability mass when an utterance is interpreted literally by $L_0$ may still receive non-zero probability mass when that same utterance is interpreted ironically.
The modified equations of \rsatwospace integrate a new rhetorical strategy variable $r \in \mathcal{R}$ as follows%
\begin{align}%
\hspace{-2cm}P_{L_0}(m,r|c,u) &\propto \mathds{1}_{m \in \llbracket u \rrbracket_r} \cdot P_{\mathcal{M} \times \mathcal{R}|\mathcal{C}}(m,r|c) \\%
\hspace{-2cm}P_{S_1}(u|c,m,r) &\propto P_{L_0} (m,r|c,u) \\%
\hspace{-2cm}P_{L_1}(m,r|c,u) &\propto P_{S_1}(u|c,m,r) \cdot P_{\mathcal{M} \times \mathcal{R}}(m,r|c)%
\end{align}%
% \begin{adjustwidth}{-4cm}{-4cm}%
% \vspace{-0.5cm}
% \begin{align}%
% P_{L_0}(m,r|c,u) &\propto \mathds{1}_{m \in \llbracket u \rrbracket_r} \cdot P_{\mathcal{M} \times \mathcal{R}|\mathcal{C}}(m,r|c) \\%
% P_{S_1}(u|c,m,r) &\propto P_{L_0} (m,r|c,u) \\%
% P_{L_1}(m,r|c,u) &\propto P_{S_1}(u|c,m,r) \cdot P_{\mathcal{M} \times \mathcal{R}}(m,r|c)%
% \end{align}%
% \end{adjustwidth}%
where $m \in \llbracket u \rrbracket_r$ may change as a function of $r$ and where the prior distribution, $P_{\mathcal{M} \times \mathcal{R} | \mathcal{C}}$, is now over both meanings and rhetorical strategies.
We illustrate the expected sequence of reasoning steps when \rsatwospace is presented with an ironic utterance in Figure~\ref{fig:rsa2_fig}. We use the running example from Figure~\ref{fig:john-example}, where the observed ironic utterance, $u_\text{obs}$, is ``This one is really sharp.'' As we see in $L_0$'s probability distribution, the interpretation of $u_\text{obs}$ is ambiguous as the speaker John could literally mean that the student is smart $r_{\text{literal}}, m_{\text{smart}}$ or could ironically mean that she is not very clever $r_{\text{ironic}}, m_{\neg\text{smart}}$. However, by reasoning about the speaker, $L_1$ concludes that if $S_1$ had meant $r_{\text{literal}}, m_{\text{smart}}$ then they would have used an alternative unambiguously literal utterance, $u_1$, such as ``Despite her poor performance, this one is usually really smart.'' As a result, in $L_1$'s posterior Bayesian update, the ironic interpretation gets upweighted and the literal one gets downweighted.%
\begin{figure}%
\hspace*{-2cm}%
\centering%
\includegraphics[width=1.3\linewidth]{Images/rsa2_fig.png}%
\caption{Illustration of the \rsatwospace framework. The observed utterance from Figure~\ref{fig:john-example}, ``This one is really sharp.'' is initially equally likely to mean $r_{\text{literal}}, m_{\text{smart}}$ and $r_{\text{ironic}}, m_{\neg\text{smart}}$. Once $L_1$ reasons about $S_1$, the former interpretation gets upweighted and the latter gets downweighted. }%
\label{fig:rsa2_fig}%
\end{figure}%
The advantage of \rsatwospace over previous adaptations of RSA is that it allows us to infer both the intended meaning as well as the rhetorical strategy of an utterance through the following marginalizations:
%
\begin{align}
P_{L_1}(m|c,u) &= \sum_{r'}P_{L_1}(m,r'|c,u) \label{eq:marginal_meaning} \\
P_{L_1}(r|c,u) &= \sum_{m'}P_{L_1}(m',r|c,u) \label{eq:marginal_rs}
\end{align}
%
Overall, we believe \rsatwospace to be a more principled approach to modeling non-literal language as it removes the need to model \emph{why} a particular rhetorical strategy was used. In addition, this framework can be easily generalized to account for rhetorical strategies beyond irony such as hyperbole and metaphor. Like other adaptations of RSA, \rsatwospace suffers from the fact that priors (e.g., $P_{\mathcal{M} \times \mathcal{R}|\mathcal{C}}$) and semantic understanding functions (e.g., $\llbracket u \rrbracket_r$) must be human derived. We address this limitation in Section~\ref{sec:experiments} where we experiment with integrating LLMs into this framework.
\section{Experiments}
\label{sec:experiments}
\subsection{Dataset}
We use the PragMega \citep{hu_fine-grained_2023} dataset to test the \rsatwospace framework on irony detection and irony interpretation. PragMega contains 25 instances of \emph{ironic scenarios} similar to the one shown in Figure~\ref{fig:john-example}. These scenarios all include some background context as well as an utterance generated by a speaker ironically. In addition, each scenario is accompanied by four possible intended meanings as they relate to the utterance: a literal interpretation of the utterance (\textsc{Literal Meaning}), an ironic interpretation of the utterance (\textsc{Ironic Meaning}), a distractor meaning designed to have high lexical overlap with the utterance (\textsc{Overlap Meaning}), and another distractor meaning which is designed to not logically follow from the context and the utterance (\textsc{Non-Sequitur Meaning}).
In addition, to validate our approach on both rhetorical strategies (\textsc{Literal} and \textsc{Ironic}), we manually modify the 25 scenarios from \citet{hu_fine-grained_2023} by making the context literally match what is said in the observed utterance. In this way, the correct interpretation of the utterance becomes the \textsc{Literal Meaning} rather than the \textsc{Ironic Meaning}. Examples of ironic and literal scenarios - which use the \textsc{Ironic} and \textsc{Literal} rhetorical strategy respectively - are shown in Table~\ref{tab:pragmega_examples}.
\begin{table*}
\centering
\resizebox{\linewidth}{!}{
\begin{tabular}{p{6cm}p{6cm}p{6cm}}
\toprule
Ironic Scenario & Literal Scenario & Intended Meanings \\
\midrule
John is a teacher at an elementary school. When talking with the principal about a new student, \textbf{who did poorly on her entrance examination}, John said, ``This one is really sharp.'' & John is a teacher at an elementary school. When talking with the principal about a new student, \textbf{who scored one of the highest scores}, John said, ``This one is really sharp.'' & 1. The student is smart. \newline 2. The student is not very clever. \newline 3. The entrance exam is unfair. \newline 4. The pencils need to be sharpened. \\\midrule
In a shop, Lara tries on a dress. \textbf{The dress is far too long for her.} Lara asks Simon: ``Does this dress fit me?'' Simon answers: ``Wow! That must be custom made! It's clearly the perfect size and length for you.'' & In a shop, Lara tries on a dress. \textbf{The dress fits her perfectly.} Lara asks Simon: ``Does this dress fit me?'' Simon answers: ``Wow! That must be custom made! It's clearly the perfect size and length for you.'' & 1. The dress is fitting well. \newline 2. Lara needs to get a dress of a shorter length. \newline 3. Simon does not like the color of this dress. \newline 4. Simon has to get back to work. \\
\bottomrule
\end{tabular}
}
\caption{Examples of the original ironic scenarios found in the PragMega dataset as well as corresponding modified literal scenarios. The intended meanings are shared for both original and modified scenarios and are ordered in the following way: 1. \textsc{Ironic Meaning} 2. \textsc{Literal Meaning} 3. \textsc{Overlap Meaning} 4. \textsc{Non-Sequitur Meaning}. The context of the original scenarios is changed so that it matches with the literal intended meaning.}
\label{tab:pragmega_examples}
\end{table*}
\subsection{Experimental Setup}
Previous approaches which implement and experimentally test the RSA framework on a given phenomenon have used human annotators to approximate probability priors and semantic understanding functions \citep{Goodman2013-GOOKAI,Kao2014FormalizingTP,Kao2014NonliteralUO,kao2015let}. In this work, we provide a more scalable approach to test \rsatwospace on irony detection and interpretation by leveraging LLMs in a similar way as \citet{careniniFullyInterpretableMore2024}. In particular, we leverage the Mistral 7B Instruct LLM \citep{jiang2023mistral7b} via HuggingFace \citep{wolf2020huggingfacestransformersstateoftheartnatural}, $M$, to estimate key components of the \rsatwospace framework including the literal listener's probability distribution as well as the space of alternative utterances $\mathcal{U}$. Using the implementations described in Sections~\ref{sec:litlisten} and \ref{sec:alt}, we derive irony detection and interpretation probabilities for the pragmatic listener, $L_1$, following the marginalizations described in Equations~\ref{eq:marginal_meaning} and \ref{eq:marginal_rs}.
\subsubsection{Literal Listener Estimation}
\label{sec:litlisten}
To approximate the literal listener's probability distribution, we leverage the Mistral LLM $M$ to estimate both the indicator semantic understanding function $\mathds{1}_{m \in \llbracket u \rrbracket_r}$ as well as the conditional prior probability $P_{\mathcal{M} \times \mathcal{R} | \mathcal{C}}$. This estimation is computed by passing a prompt to $M$ containing a general task instruction, the scenario and all possible intended meaning-rhetorical strategy pairs and by extracting $M$'s next-token probability for each pair. The utterance is only included in the prompt in the case of the estimation of $\mathds{1}_{m \in \llbracket u \rrbracket_r}$.
To extract the probability which $M$ associates with each intended meaning-rhetorical strategy pair, we number the 8 possible pairs in a multiple-choice question (MCQ) format and extract the next-token logit of the number corresponding to each pair\footnote{There are $4$ intended meanings per scenario and $2$ tested rhetorical strategies for a total of 8 pairs.}. These logits are re-normalized so that the sum of the probabilities equals one. In addition, to avoid positional bias, we randomly shuffle the order of the options in the prompt 5 times and compute average probabilities. The prompts used are included in Appendix~\ref{sec:prompts}.
With this implementation, $L_0$'s probability distribution becomes%
\begin{align*}%
P_{L_0}(m,r|c,u) \propto P_M(m,r|c,u) \cdot P_M(m,r|c) \\
= \frac{P_M(m,r|c,u) \cdot P_M(m,r|c)}{\sum_{m',r'} P_M(m',r'|c,u) \cdot P_M(m',r'|c)}
\end{align*}%
where $P_M(m,r|c,u)$ and $P_M(m,r|c)$ are the averaged and re-normalized next-token probabilities for each $(m,r) \in \mathcal{M} \times \mathcal{R}$.
\subsubsection{Alternative Utterance Generation}
\label{sec:alt}
To implement the \rsatwospace pragmatic speaker, we must compute a marginalization constant over the set of all possible utterances $\mathcal{U}$. Since $\mathcal{U}$ is in principle infinite, we decide to approximate it by generating $5$ alternative utterances for each triple $(m,r,c) \in \mathcal{M}\times\mathcal{R}\times\mathcal{C}$. These alternative utterances are generated by prompting $M$ (Appendix~\ref{sec:prompts}).
With this implementation, $S_1$'s probability distribution becomes
\begin{align*}
P_{S_1}(u|c,m,r) &\propto P_{L_0}(m,r|c,u) \\
&= \frac{P_{L_0}(m,r|c,u)}{\sum_{u' \in \mathcal{U}_c}P_{L_0}(m,r|c,u')}
\end{align*}
where $\mathcal{U}_c$ is the set of all alternative utterances generated with context $c$.
\subsubsection{Baselines}
We compare the $L_1$-derived meaning and rhetorical strategy probabilities with purely LLM-derived probabilities as well as $L_0$-derived probabilities.
\paragraph{LLM-derived} For irony interpretation, we prompt the Mistral LLM $M$ in the same MCQ-like fashion but using only the intended meanings as options. We derive intended meaning probabilities by averaging over $5$ random MCQ orders. For irony detection, we use the rhetorical strategies as the MCQ options and average over the $2$ possible MCQ orders. All prompt templates can be found in Appendix~\ref{sec:prompts}.
\paragraph{$L_0$-derived} We apply the same marginalizations from Equations~\ref{eq:marginal_meaning} and \ref{eq:marginal_rs} to the literal listener probabilities to obtain probability distributions for the irony interpretation and detection tasks.
\section{Results}
\begin{figure*}[h]
\subfloat[Irony detection probabilities in ironic scenarios]{%
\includegraphics[width=.49\linewidth]{Images/Rhetorical Strategy Probabilities (Ironic Scenarios).png}%
\label{subfig:ironic_rhetorical_probs}%
}\hfill
\subfloat[Irony detection probabilities in literal scenarios]{%
\includegraphics[width=.49\linewidth]{Images/Rhetorical Strategy Probabilities (Literal Scenarios).png}%
\label{subfig:literal_rhetorical_probs}%
}
\caption{Rhetorical strategy probabilities averaged across the original ironic PragMega dataset and our literal modification.}%
\label{fig:rhetorical_probs}
\end{figure*}
\begin{figure*}[h]
\subfloat[Meaning probabilities in ironic scenarios]{%
\includegraphics[width=.49\linewidth]{Images/Intended Meaning Probabilities (Ironic Scenarios).png}%
\label{subfig:ironic_meaning_probs}%
}\hfill
\subfloat[Meaning probabilities in literal scenarios]{%
\includegraphics[width=.49\linewidth]{Images/Intended Meaning Probabilities (Literal Scenarios).png}%
\label{subfig:literal_meaning_probs}%
}
\caption{Intended meaning probabilities averaged across the original ironic PragMega dataset and our literal modification.}%
\label{fig:meaning_probs}
\end{figure*}
Figure \ref{fig:rhetorical_probs} shows irony detection probabilities on both the original ironic PragMega dataset and our literal modification. We average the probabilities separately for the ironic and literal scenarios. For the ironic scenarios, the base LLM performs best, followed by the literal listener and then the pragmatic listener. However, on the pragmatic scenarios the LLM performs worse than the literal and pragmatic listeners.
Figure \ref{fig:meaning_probs} shows irony interpretation probabilities on both the original ironic PragMega dataset and our literal modification. Again, we average the probabilities separately for the ironic and literal scenarios. We use the human probabilities from \cite{huFinegrainedComparisonPragmatic2023} in Figure \ref{subfig:ironic_meaning_probs}. For ironic scenarios, $L_{0}$ performs much better than $L_{1}$, though both outperform the base LLM. In contrast, the base LLM performs best on the literal scenarios.
\section{Discussions}
\subsection{Detection-Interpretation Inconsistency}
Our results display a surprising incongruence in terms of how we expect the tasks of irony detection and interpretation to be related. We see that the base LLM performs the best at irony detection in ironic scenarios - i.e., when the utterance is truly ironic - and irony interpretation in literal scenarios - i.e., when the utterance is truly literal (Figures~\ref{subfig:ironic_rhetorical_probs},\ref{subfig:literal_meaning_probs}). However, it performs the worst at irony detection in literal scenarios and irony interpretation in ironic scenarios (Figures~\ref{subfig:literal_rhetorical_probs},\ref{subfig:ironic_meaning_probs}). We would expect performance on irony detection and interpretation to be correlated: being able to detect irony should allow the model to interpret it correctly. Instead, we see an inversion of performance in all cases. This indicates that language models may use different processes for classifying ironic statements and discerning their meaning.
\subsection{Literal-Pragmatic Inconsistency}
\begin{figure*}[h]
\subfloat[Probability comparisons for ironic scenarios.]{%
\includegraphics[width=.49\linewidth]{Images/prob_breakdown_ironic.png}%
\label{subfig:prob_breakdown_ironic}%
}\hfill
\subfloat[Probability comparisons for literal scenarios.]{%
\includegraphics[width=.49\linewidth]{Images/prob_breakdown_literal.png}%
\label{subfig:prob_breakdown_literal}%
}
\caption{Comparison of probabilities assigned to meaning and rhetorical strategy pairs for the observed utterance.}%
\label{fig:prob_breakdown}
\end{figure*}
In addition to the detection-interpretation incongruence, we also notice that the literal listener outperforms the pragmatic listener on both detection and interpretation tasks in the ironic scenarios (Figures~\ref{subfig:ironic_rhetorical_probs},\ref{subfig:ironic_meaning_probs}). This result is also surprising since we would expect additional reasoning layers to move probability mass in the correct direction. To explain this result, we note that $L_{0}$ is a product of two LLM-derived probabilities: $P_{M}(m,r|c)$ and $P_{M}(m,r|c,u)$. $L_{1}$ replaces $P_{M}(m,r|c,u)$ with the likelihood that a speaker generates $u$ given $(m,r,c)$. We compare these probabilities in Figure~\ref{fig:prob_breakdown}. We find that $P_{M}(m,r|c)$ and $P_{M}(m,r|c,u)$ are generally well-calibrated: they place most of the probability mass on the correct $(m, r)$. In contrast, the likelihood scores assigned to the observed utterances by the pragmatic speaker are relatively low and, importantly, are sometimes lower for the correct meaning-rhetorical strategy pair than incorrect pairs. As a result, while the marginalized probabilities for the correct meaning and rhetorical strategies are relatively high for $L_0$, they fall precipitously for $L_1$.
\begin{figure*}[h]
\subfloat[Speaker utterance probabilities for ironic scenarios.]{%
\includegraphics[width=.49\linewidth]{Images/ps1_ironic.png}%
\label{subfig:ironic_ps1}%
}\hfill
\subfloat[Speaker utterance probabilities for literal scenarios.]{%
\includegraphics[width=.49\linewidth]{Images/ps1_literal.png}%
\label{subfig:literal_ps1}%
}
\caption{Comparison of speaker probabilities assigned to the original utterance and all alternative utterances for (\textsc{Ironic Meaning}, \textsc{Ironic}) and (\textsc{Literal Meaning}, \textsc{Literal}) meaning and rhetorical strategy pairs. Note that the values for alternative utterances are the sum of speaker probabilities across the 5 alternative utterances generated for that meaning and rhetorical strategy pair.}%
\label{fig:ps1}
\end{figure*}
% We consider in more Figure \ref{fig:prob_breakdown} shows that the speaker probabilities are extremely low for the observed original utterance in all cases. We would expect $S_1$ to place more probability mass on alternative utterances for ``distractor'' meanings and more on the original utterance for the correct meaning. This would reduce ambiguity and improve performance in $L_1$. However, Figure \ref{fig:ps1} shows that the speaker places very low probability mass on the original utterance. We see that even within $S_1$, the probabilities are not well assigned.
To explain the low likelihood scores given by $S_1$ in Figure~\ref{fig:prob_breakdown}, we examine $S_1$'s probability distribution over all utterances (original and alternative) averaged across all scenarios. The probability distribution over all utterances in the ironic and literal scenarios are shown in Figure~\ref{fig:ps1} where utterances are grouped by the meaning-rhetorical strategy pair used to generate them. We find that the speaker places very little probability mass on the original utterance, while placing too much mass on incorrect alternative utterances (e.g., those generated to have high lexical overlap or to be non sequitur). We believe that part of the reason for the speaker's poor performance is that the alternative utterances are generated and evaluated by the same LLM. Thus, the LLM may naturally prefer LLM-generated alternative utterances over the original utterances found in the PragMega dataset. This finding suggests that more work needs to be done to calibrate the $S_1$ probability distribution such that it allows $L_1$ to correctly identify which meaning-rhetorical strategy pair best explains the observed utterance.
% We note also that the speaker probabilities also partially explain the incongruous performance of $L_1$. The speaker is misallocating probability mass to a meaning-rhetorical strategy pair which is only partially correct, inflating $L_0$'s performance on one task while reducing it on the other. In ironic scenarios, the speaker assigns a relatively high probability to the ironic rhetorical strategy and literal meaning pair. Accordingly, Figures \ref{subfig:ironic_rhetorical_probs} and \ref{subfig:ironic_meaning_probs} show $L_1$'s strong irony detection but poor irony interpretation in ironic scenarios. In contrast, the highest speaker probability in literal scenarios is assigned to literal meaning with literal rhetorical strategy and then to literal meaning with ironic rhetorical strategy. Again, Figures \ref{subfig:literal_rhetorical_probs} and \ref{subfig:literal_meaning_probs} show that $L_1$ displays strong rhetorical strategy detection but poor irony interpretation in literal scenarios.
\section{Conclusion}
In conclusion, we tackle the problem of irony understanding by focusing on irony detection and interpretation. To do so, we introduce a rhetorical strategy-aware version of the RSA framework, in which a speaker's rhetorical strategy is explicitly modeled. Additionally, we leverage LLMs as estimates of prior probabilities and semantic understanding functions. Our results indicate that \rsatwospace improves over an LLM baseline in some cases (e.g. detection under literal scenarios and interpretation under ironic scenarios) but performs worse in others. We find that the inconsistent performance is a result of the miscalibration of the pragmatic speaker model, where alternative utterances incorrectly receive more probability mass than the observed utterance. We hope that, with further work, this approach can provide insights on the best way for improving LLMs' understanding of irony as well as other figurative uses of language.
% \david{+ Ces : at the end}
% \ces{Some thoughts about significance of the work I've had while writing: Usually whether a system understands some linguistic concept we use recognition of that linguistic concept, but maybe there's a more general incongruence between recognizing the use of a certain linguistic concept vs actual interpreting it vs maybe also using it in text. Might be an interesting thing to test.}
\section*{Limitations}
We note several limitations to our work. First, we only evaluated a single language model, Mistral 7B, due to time constraints. A more thorough investigation of our method would evaluate several models of different sizes. By using larger models, we could investigate how our model functions in the context of scaling laws. The difference in performance could be immense between a 7B model and models like Llama-3 400B. It is possible that base LLM performance increases immensely with scale or that the speaker probabilities could become better calibrated.
Second, our speaker model was poorly calibrated. Our method was not able to provide a good measure of the speaker probabilities $P_{S_1}(u|m,r,c)$. As such, we feel that these results do not accurately capture the performance of an ideal RSA model for irony as portrayed in Figure~\ref{fig:rsa2_fig}. We propose two possible approaches for future work. First, more in-depth prompt engineering could better align model behaviour with our actual intention. Second, a cost function to penalize overly verbose alternative utterances or alternative utterances which look ``significantly different'' from the original utterances. Since we believe that the model prefers its own utterances, this may improve the relative performance on the original utterance.
\bibliographystyle{acl_natbib}
\bibliography{custom}
\clearpage
\appendix
\onecolumn
\section{LLM Prompts}
\label{sec:prompts}
\vspace{-6cm}
We provide the prompt templates used to run all our experiments as well as one specific example for each template using our running example from Figure~\ref{fig:john-example}. All templates are similar to those of \citet{hu_fine-grained_2023}.
% \subsection{\rsatwospace Prompts}
% \subsubsection{Literal Listener Estimation}
\begin{figure}[H]
\vspace{-5cm}
\begin{framed}
\begin{quote}
Task: You will read short stories that describe everyday situations. Each story will be followed by a multiple-choice question. Read each story and choose the best answer. Your task is to decide what meaning the character in the story will convey and what rhetorical strategy they will use to convey it (i.e., are they more likely to be literal or ironic). The answer options are 1, 2, 3, 4, 5, 6, 7, or 8. \newline\newline[scenario] What meaning will $X$ convey and what rhetorical strategy will $X$ use to convey it?\newline\newline[options]\newline\newline Answer:
\end{quote}
\end{framed}
\begin{framed}
\begin{quote}
Task: You will read short stories that describe everyday situations. Each story will be followed by a multiple-choice question. Read each story and choose the best answer. Your task is to decide what meaning the character in the story will convey and what rhetorical strategy they will use to convey it (i.e., are they more likely to be literal or ironic). The answer options are 1, 2, 3, 4, 5, 6, 7, or 8. \newline\newline John is a teacher at an elementary school. When talking with the principal about a new student, who did poorly on her entrance examination, what meaning will John convey and what rhetorical strategy will John use to convey it?\newline\newline1. Intended meaning: The student is smart., Rhetorical strategy: John will likely be ironic.\newline2. Intended meaning: The student is not very clever., Rhetorical strategy: John will likely be ironic.\newline3. Intended meaning: The student is smart., Rhetorical strategy: John will likely be literal.\newline4. Intended meaning: The pencils need to be sharpened., Rhetorical strategy: John will likely be literal.\newline5. Intended meaning: The entrance exam is unfair. , Rhetorical strategy: John will likely be literal.\newline6. Intended meaning: The pencils need to be sharpened., Rhetorical strategy: John will likely be ironic.\newline7. Intended meaning: The entrance exam is unfair. , Rhetorical strategy: John will likely be ironic.\newline8. Intended meaning: The student is not very clever., Rhetorical strategy: John will likely be literal.\newline\newline Answer:
\end{quote}
\end{framed}
\caption{Prompt template for $P_{M}(m,r|c)$ where [scenario] is replaced with one of the (ironic or literal) scenarios from the PragMega dataset \textbf{without the utterance} and [options] are the 8 possible intended meaning-rhetorical strategy pairs for this particular scenario. $X$ is replaced with the name of the speaker in the scenario which is extracted using a regular expression. An example is given using the running example from Figure~\ref{fig:john-example}.}
\end{figure}
\begin{figure}[H]
\begin{framed}
\begin{quote}
Task: You will read short stories that describe everyday situations. Each story will be followed by a multiple-choice question. Read each story and choose the best answer. Your task is to decide what meaning the character in the story is trying to convey and what rhetorical strategy they are using to convey it (i.e., are they being literal or ironic). The answer options are 1, 2, 3, 4, 5, 6, 7, or 8. \newline\newline[scenario] What meaning is $X$ trying to convey and what rhetorical strategy is $X$ using?\newline\newline[options]\newline\newline Answer:
\end{quote}
\end{framed}
\begin{framed}
\begin{quote}
Task: You will read short stories that describe everyday situations. Each story will be followed by a multiple-choice question. Read each story and choose the best answer. Your task is to decide what meaning the character in the story will convey and what rhetorical strategy they will use to convey it (i.e., are they more likely to be literal or ironic). The answer options are 1, 2, 3, 4, 5, 6, 7, or 8. \newline\newline John is a teacher at an elementary school. When talking with the principal about a new student, who did poorly on her entrance examination, John said, ``This one is really sharp.'' What meaning is John trying to convey and what rhetorical strategy is John using?\newline\newline1. Intended meaning: The entrance exam is unfair. , Rhetorical strategy: John is being ironic.\newline2. Intended meaning: The entrance exam is unfair. , Rhetorical strategy: John is being literal.\newline3. Intended meaning: The student is not very clever., Rhetorical strategy: John is being literal.\newline4. Intended meaning: The student is not very clever., Rhetorical strategy: John is being ironic.\newline5. Intended meaning: The student is smart., Rhetorical strategy: John is being ironic.\newline6. Intended meaning: The student is smart., Rhetorical strategy: John is being literal.\newline7. Intended meaning: The pencils need to be sharpened., Rhetorical strategy: John is being literal.\newline8. Intended meaning: The pencils need to be sharpened., Rhetorical strategy: John is being ironic.\newline\newline Answer:
\end{quote}
\end{framed}
\caption{Prompt template for $P_{M}(m,r|c,u)$ where [scenario] is replaced with one of the (ironic or literal) scenarios from the PragMega dataset \textbf{with the utterance} and [options] are the 8 possible intended meaning-rhetorical strategy pairs for this particular scenario. $X$ is replaced with the name of the speaker in the scenario which is extracted using a regular expression. An example is given using the running example from Figure~\ref{fig:john-example}.}
\end{figure}
\newpage
% \subsubsection{Alternative Utterance Generation}
\begin{figure}[t]
\begin{framed}
\begin{quote}
Task: You will read a scenario in which a speaker wants to convey some meaning to some listeners. Your task is to generate the speaker's utterance. The utterance should be within <u></u> tags. Assume that the speaker's utterance will be literal. That is, the speaker must say exactly what they want to convey by being literal. Generate 5 alternative utterances. They should all be different from each other. They should each be a single sentence. The utterances should be short, concise and literally true.\newline Scenario: [context] [intended meaning] \newline Utterances:
\end{quote}
\end{framed}
\begin{framed}
\begin{quote}
Task: You will read a scenario in which a speaker wants to convey some meaning to some listeners. Your task is to generate the speaker's utterance. The utterance should be within <u></u> tags. Assume that the speaker's utterance will be literal. That is, the speaker must say exactly what they want to convey by being literal. Generate 5 alternative utterances. They should all be different from each other. They should each be a single sentence. The utterances should be short, concise and literally true.\newline Scenario: John is a teacher at an elementary school. When talking with the principal about a new student, who did poorly on her entrance examination, John wants to convey that the student is smart.\newline Utterances:
\end{quote}
\end{framed}
\caption{Prompt template for generating the alternative utterances for $(m,r,c) \in \mathcal{M}\times\mathcal{R}\times\mathcal{C}$ when the rhetorical stategy is \textsc{Literal}. The [context] is replaced with the context $c$ from one of the (ironic or literal) scenarios from the PragMega dataset (does not include the utterance). The [intended meaning] is replaced with the meaning corresponding to $r$ in the scenario. An example is given using the running example from Figure~\ref{fig:john-example}.}
\end{figure}
\begin{figure}[t]
\begin{framed}
\begin{quote}
Task: You will read a scenario in which a speaker wants to convey some meaning to some listeners. Your task is to generate the speaker's utterance. The utterance should be within <u></u> tags. Assume that the speaker's utterance will be ironic. That is, the speaker must say the opposite of what they want to convey by using verbal irony. Generate 5 alternative utterances. They should all be different from each other. They should each be a single sentence. The utterances should be short, concise and ironic. \newline Scenario: John is a teacher at an elementary school. When talking with the principal about a new student, who did poorly on her entrance examination, John wants to convey that the student is smart.\newline Utterances:
\end{quote}
\end{framed}
\begin{framed}
\begin{quote}
Task: You will read a scenario in which a speaker wants to convey some meaning to some listeners. Your task is to generate the speaker's utterance. The utterance should be within <u></u> tags. Assume that the speaker's utterance will be ironic. That is, the speaker must say the opposite of what they want to convey by using verbal irony. Generate 5 alternative utterances. They should all be different from each other. They should each be a single sentence. The utterances should be short, concise and ironic. \newline Scenario: John is a teacher at an elementary school. When talking with the principal about a new student, who did poorly on her entrance examination, John wants to convey that the student is smart.\newline Utterances:
\end{quote}
\end{framed}
\caption{Prompt template for generating the alternative utterances for $(m,r,c) \in \mathcal{M}\times\mathcal{R}\times\mathcal{C}$ when the rhetorical stategy is \textsc{Ironic}. The [context] is replaced with the context $c$ from one of the (ironic or literal) scenarios from the PragMega dataset (does not include the utterance). The [intended meaning] is replaced with the meaning corresponding to $r$ in the scenario. An example is given using the running example from Figure~\ref{fig:john-example}.}
\end{figure}
\newpage
% \subsection{Baseline Prompts}
\begin{figure}[t]
\begin{framed}
\begin{quote}
Task: You will read short stories that describe everyday situations. Each story will be followed by a multiple-choice question. Read each story and choose the best answer. Your task is to decide what the character in the story is trying to convey. The answer options are 1, 2, 3, or 4.\newline\newline [scenario] What did $X$ want to convey?\newline\newline [options] \newline\newline Answer:
\end{quote}
\end{framed}
\begin{framed}
\begin{quote}
Task: You will read short stories that describe everyday situations. Each story will be followed by a multiple-choice question. Read each story and choose the best answer. Your task is to decide what the character in the story is trying to convey. The answer options are 1, 2, 3, or 4.\newline\newline John is a teacher at an elementary school. When talking with the principal about a new student, who did poorly on her entrance examination, John said, ``This one is really sharp.'' What did John want to convey?\newline\newline1. The pencils need to be sharpened.\newline2. The student is smart.\newline3. The student is not very clever.\newline4. The entrance exam is unfair. \newline\newline Answer:
\end{quote}
\end{framed}
\caption{Prompt template for $P_{M}(m|c,u)$ where [scenario] is replaced with one of the (ironic or literal) scenarios from the PragMega dataset \textbf{with the utterance} and [options] are the 4 possible intended meanings for this particular scenario. $X$ is replaced with the name of the speaker in the scenario which is extracted using a regular expression.}
\end{figure}
\begin{figure}[t]
\begin{framed}
\begin{quote}
Task: You will read short stories that describe everyday situations. Each story will be followed by a multiple-choice question. Read each story and choose the best answer. Your task is to decide whether the character is being literal or ironic. The answer options are 1 or 2.
\newline\newline
[scenario] Was $X$ being literal or ironic?
\newline\newline
[options]
\newline\newline
Answer:
\end{quote}
\end{framed}
\begin{framed}
\begin{quote}
Task: You will read short stories that describe everyday situations. Each story will be followed by a multiple-choice question. Read each story and choose the best answer. Your task is to decide whether the character is being literal or ironic. The answer options are 1 or 2.
\newline\newline John is a teacher at an elementary school. When talking with the principal about a new student, who did poorly on her entrance examination, John said, ``This one is really sharp.'' Was John being literal or ironic?\newline\newline1. Literal\newline2. Ironic\newline\newline Answer:
\end{quote}
\end{framed}
\caption{Prompt template for $P_{M}(r|c,u)$ where [scenario] is replaced with one of the (ironic or literal) scenarios from the PragMega dataset \textbf{with the utterance} and [options] are the 2 possible rhetorical strategies (ironic or literal). $X$ is replaced with the name of the speaker in the scenario which is extracted using a regular expression.}
\end{figure}
\end{document}
% This must be in the first 5 lines to tell arXiv to use pdfLaTeX, which is strongly recommended.
\pdfoutput=1
% In particular, the hyperref package requires pdfLaTeX in order to break URLs across lines.
\documentclass[11pt]{article}
% Change "review" to "final" to generate the final (sometimes called camera-ready) version.
% Change to "preprint" to generate a non-anonymous version with page numbers.
\usepackage[review]{acl}
% \usepackage{acl}
% Standard package includes
\usepackage{times}
\usepackage{latexsym}
\usepackage[T1]{fontenc}
\usepackage[utf8]{inputenc}
\usepackage{microtype}
\usepackage{inconsolata}
\usepackage{graphicx}
\usepackage{amsmath}
\usepackage{booktabs}
\usepackage{lipsum}
\usepackage[caption=false]{subfig}
\usepackage{float}
\usepackage{multirow}
\usepackage{amsmath}
\usepackage{changepage}
\usepackage{natbib}
\usepackage{amsmath}
\usepackage{stmaryrd}
\usepackage{dsfont}
\usepackage{geometry} % Optional, to ensure full-page width
\usepackage{changepage}
\usepackage{makecell}
\usepackage{listings} % Import the listings package
\usepackage{framed}
\lstset{
breaklines=true, % Enable line wrapping
basicstyle=\ttfamily, % Use monospace font
frame=single, % Add a frame around the code block
}
\DeclareMathOperator*{\argmax}{arg\,max}
\usepackage[]{todonotes} % use option "disable" to suppress notes
\makeatletter
\newcommand*\iftodonotes{\if@todonotes@disabled\expandafter\@secondoftwo\else\expandafter\@firstoftwo\fi}
\makeatother
\newcommand{\noindentaftertodo}{\iftodonotes{\noindent}{}\ignorespaces}
\newcommand{\david}[1]{{\color{purple}[David: #1]}}
\newcommand{\ces}[1]{{\color{red}[Ces: #1]}}
\newcommand{\rsatwo}{\textbf{(RSA)}\textsuperscript{2}}
\newcommand{\rsatwospace}{\rsatwo\,\,}
\newcommand{\response}[1]{\vspace{3pt}\hrule\vspace{3pt}\textbf{#1:}}
\interfootnotelinepenalty=10000
\usepackage[dvipsnames]{xcolor}
\definecolor{googlegreen}{RGB}{56, 118, 29}
\definecolor{googleblue}{RGB}{17, 85, 204}
\title{\rsatwo: A Rhetorical-Strategy-Aware Rational Speech Act Framework for Figurative Language Understanding}
\author{
Cesare Spinoso-Di Piano and David Austin\\
\texttt{\{cesare.spinoso-dipiano,david.e.austin\}@mail.mcgill.ca}\\
Mila – Quebec Artificial Intelligence Institute \\
McGill University \\
}
\begin{document}
\maketitle
% \section{Abstract}
\label{sec:abstract}
% This is the version meant for the SCiL workshop TODO
\begin{figure*}[t]
\centering
\includegraphics[width=\linewidth]{Images/meaning_probabilities.png}
\caption{Meaning probabilities for``The weather is terrible.'' under both the precipitation and sunny scenarios.}
\label{fig:weather}
\end{figure*}
% Paragraph 1: Figurative uses of language are ubiquitous/critical part of human communication. Provide some examples - use a weather example to prepare the reader for the running example.
Figurative language, in which expressions are meant to be interpreted non-literally, is pervasive in human communication \cite{roberts_why_figurative_1994}. For example, speakers will often generate utterances which are literally incompatible with the world and with their listeners expectations but which still convey meaning. While the utterance ``It is so nice outside.'' may appear inappropriate when uttered during a snow blizzard\footnote{Snow blizzards are not typically referred to as ``nice''.}, it may still carry meaning if interpreted ironically. The speaker may be subverting the listener's expectations for comedic effect or to convey their annoyance at the weather. As a result, it is critical to be able to account for figurative uses of language - predicting its use and deciphering its intended meaning - when designing computational models of human language and communication.
% Paragraph 2: Since it's so ubiquitous, figurative use of language should be modeled - Most common approach to doing so is RSA, but it cannot account for non-literal language
One of the most popular computational approaches to modeling human communication is the Rational Speech Acts (RSA) framework. In RSA, an utterance's meaning is interpreted probabilistically by positing the existence of several communicative agents which reason about each other recursively. The framework consists of three communicative agents. First, the \emph{literal listener}, denoted $P_{L_0}(m|c,u)$, which reasons about the literal meaning, $m$. of utterance $u$ produced in a context $c$. Second, the \emph{pragmatic speaker}, $P_{S_1}(u|c,m)$, which computes an utterance's expected utility based on its ability to convey the desired meaning to the literal listener. Third, the \emph{pragmatic listener}, $P_{L_1}(m|c,u)$, which interprets the intended meaning of an utterance by reasoning about the pragmatic speaker's likelihood of generating it. Altogether, these three layers of recursive reasoning have been shown to be effective at reducing a listener's uncertainty regarding the interpretation of literally ambiguous utterances such as ``some'' used in scalar implicatures.
% Paragraph 2.5: Continuation in more detail of paragraph 2
While RSA offers a way to refine the interpretation of literally ambiguous utterances, this framework cannot handle non-literal interpretations of utterances. In particular, it can be shown that interpretations of utterances which are not compatible with the literal meaning of an utterance will be assigned zero probability mass by the pragmatic listener. Thus, under the original RSA framework formulation, the utterance ``It is so nice outside.'' could only be interpreted as meaning that the speaker literally believes the weather to be nice.
% Paragraph 3: Well if RSA can't do it, how else are we supposed to model figurative language? Describe affect-aware RSA.
As a result, efforts to model human use of figurative language have sought to address this limitation by incorporating the affect of an utterance within the RSA framework. In Kao et al, the authors propose an affect-aware RSA formulation where an utterance's communicative goal is extended to include both the literal information it conveys as well as the affect it might be communicating (e.g., joy, annoyance, etc.). By explicitly modeling the connection between meaning and affect, the authors are able to assign non-zero probability mass to non-literal meanings because of the association between those meanings and a modeled affect. For instance, in our earlier example, a pragmatic listener may interpret the utterance ``It is so nice outside.'' in the context of a snowstorm as meaning that the weather is bad because this meaning is typically associated with a negative affect (e.g., annoyance or frustration).
% Paragraph 4: This approach may not be suitable for some theoretical reasons listed here: Don't always consider that a speaker is using figurative language to communicate affect
While affect-aware RSA is successful in enabling the RSA framework to obtain non-literal interpretations of utterances, it results in a framework which we believe too tightly couples the notion of non-literal meaning and affect. As a result of this coupling, the framework is unable to account for figurative uses of language which are not affect-driven. Most notably, certain uses of non-literal rhetorical strategies such as irony or hyperbole may have become conventionalized thus loosing any inferrable affect. For instance, the phrase ``The weather here is always terrible'' might be meant to describe how often bad weather occurs, rather than to express the speaker's feelings about it.
% Paragraph 5: (RSA)^2 introduction, and main inner workings.
As a result, we introduce a rhetorical-strategy-aware RSA framework, \rsatwo, where a speaker's rhetorical strategy is explicitly modeled and used to interpret, potentially non-literally, an utterance's intended meaning. We define a rhetorical strategy as the way in which an utterance relates to its intended meaning. For example, the \emph{ironic} rhetorical strategy interprets an utterance using the opposite of its literal meaning, while the \emph{hyperbole} rhetorical strategy interprets an utterance using meanings which are higher on a gradable scale than its literal meaning. Crucially, \rsatwospace enables a pragmatic listener to infer non-literal interpretations of utterances \emph{without} having to know an utterance's intended affect.
\rsatwospace generalizes the literal listener's semantic interpretation function $\llbracket u \rrbracket$ which maps an utterance to the set of all meanings literally compatible with $u$ by making it a function of the rhetorical strategy variable $r$, $\llbracket u \rrbracket^r$. The pragmatic listener induces a probability distribution over all possible intended meanings by marginalizing over all possible rhetorical strategies. The modified RSA equations are presented below.%
\begin{align}%
\hspace{-1mm}P_{L_0}(m|c,u,r) &\propto \mathds{1}_{m \in \llbracket u \rrbracket^r} \cdot P(m|c) \\
\hspace{-1mm}P_{S_1}(u|m,c,r) &\propto P_{L_0}(m|c,u,r) \cdot P(u|c) \\
\hspace{-1mm}P_{L_1}(m|c,u,r) &\propto P_{S_1}(u|m,c,r) \cdot P(m|c) \\
\hspace{-5mm}P_{L_1}(m|c,u) = \sum_{r'} &P_{L_1}(m|c,u,r') \cdot P(r'|c,u)
\end{align}%
%
% Paragraph 6: Discuss the weather setting and contrast your approach with Kao
We test our formulation on the same setting as Kao et al which studied the use of utterances to describe the weather. In this setting, the possible intended meanings included describing the weather as \texttt{terrible}, \texttt{bad}, \texttt{ok}, \texttt{good} or \texttt{amazing}. The set of possible utterances included all utterances of the form ``The weather is $x$'' where $x$ was one of the aforementioned intended meanings. Finally, these utterances and meanings were communicated under two contexts: one where the weather is sunny with clear skies and another with heavy precipitations (e.g., rain or snow) and heavy winds.
Furthermore, we use the same meaning priors $P(m|c)$ as elicited by Kao et al in their original paper and set the utterance and rhetorical strategy priors $P(u|c), P(r|c)$ to be uniform. We consider four rhetorical strategies: literal, ironic, hyperbole and understatement. For each of the rhetorical strategies, we manually set the values of $\llbracket u \rrbracket^r$ based on the prototypical interpretations of $m, u$ and $r$. For instance, $\texttt{terrible} \in \llbracket \text{``The weather is amazing.''} \rrbracket^\text{irony}$ and $\texttt{amazing} \in \llbracket \text{``The weather is good.''} \rrbracket^\text{understatement}$. Figure 1 shows the meaning interpretation of the utterance ``The weather is terrible.'' under both sunny and precipitation scenarios using the original, affect-aware and rhetorical-strategy-aware formulation of RSA.
We observe that, as expected, the pragmatic listener implemented with the original RSA formulation assigns all of the intended meaning probability mass to the meaning that is literally compatible with the utterance "The weather is terrible." i.e. \texttt{terrible}. In contrast, \rsatwospace provides a non-literal interpretation of the utterance that appears to align with the affect-aware RSA formulation. For instance, the pragmatic listener adopts a hyperbolic interpretation in the precipitation scenario, assigning more probability mass to \texttt{bad} (0.37) than \texttt{terrible} (0.26). Additionally, the \rsatwospace pragmatic listener adopts an ironic interpretation in the sunny weather scenario, assigning more probability mass to \texttt{amazing} (0.3) than \texttt{terrible} (0.27). A few discrepancies arise, such as \texttt{amazing} receiving significantly more mass under \rsatwospace (0.26) than affect-aware RSA (0.08). We believe this difference is due to the uniform prior, which could be further refined through human elicitation.
% Conclusion plus future work (3 sentences)
In conclusion, in this study we propose a rhetorical-strategy-aware formulation of RSA which enables it to provide non-literal interpretations to utterances without assuming any particular affect. We show that explicitly considering rhetorical strategies is sufficient to enable RSA to achieve non-literal interpretations of utterances. Future work in this direction could involve refining the rhetorical strategy prior and integrating the formulation of \rsatwospace in the generation procedure of existing language systems such as large language models for figurative language understanding tasks.
\bibliographystyle{acl_natbib}
\bibliography{custom}
\end{document}
% For instance, under this framework, the interpretation of the literally ambiguous utterance ``Bob ate some of the apples.'' ``narrows'' is refined to mean ``Bob ate some, but not all, of the apples.''
% [list the listeners here]
% Applying the RSA equations and marginalizing over all possible rhetorical strategies using a uniform prior we obtain a pragmatic listener which assigns non-zero probability mass to non-literal interpretations of weather-related utterances.
% To compare our approach with that of Kao et al, we consider the interpretation of the utterance ``the weather is terrible'' under two contexts: one where the weather is truly amazing (\texttt{amazing weather}) and another where the weather is truly bad (\texttt{bad weather}). We consider the same alternative utterances from the previous study: ``The weather is terrible'', ``The weather is bad.'', ``The weather is ok.'', ``The weather is good.'', ``The weather is amazing.'' as well as the same \emph{intended} meanings a speaker wishes to convey: \texttt{terrible}, \texttt{bad}, \texttt{ok}, \texttt{good}, \texttt{amazing}. Our results are displayed in Figure 1. We see that our modified RSA version is able to assign non-literal interpretations to utterances by only reasoning about the employed rhetorical strategy.
% Paragraph 7: Describe the Figure as the "main result"
% Paragraph 7: List contributions + extra things that are typically done in an abstract submission (ask Jackie)
\documentclass[11pt]{article}
% Change "review" to "final" to generate the final (sometimes called camera-ready) version.
% Change to "preprint" to generate a non-anonymous version with page numbers.
% \usepackage{acl}
% Standard package includes
\usepackage{times}
\usepackage{latexsym}
\usepackage[T1]{fontenc}
\usepackage[utf8]{inputenc}
\usepackage{microtype}
\usepackage{inconsolata}
\usepackage{graphicx}
\usepackage{amsmath}
\usepackage{booktabs}
\usepackage{lipsum}
\usepackage[caption=false]{subfig}
\usepackage{float}
\usepackage{multirow}
\usepackage{amsmath}
\usepackage{changepage}
\usepackage{natbib}
\usepackage{amsmath}
\usepackage{stmaryrd}
\usepackage{dsfont}
\usepackage{geometry} % Optional, to ensure full-page width
\usepackage{changepage}
\usepackage{makecell}
\usepackage{listings} % Import the listings package
\usepackage{framed}
\usepackage{amsfonts}
\usepackage{algorithm}
\usepackage{algorithmicx}
\usepackage{algpseudocode}
\newcommand{\rsatwo}{\textbf{(RSA)}\textsuperscript{2}}
\newcommand{\rsatwospace}{\rsatwo\,\,}
\title{Equivalence Between (RSA)\textsuperscript{2} and QUD-RSA}
\begin{document}
\maketitle
In this note, we wish to show equivalency between \rsatwospace and QUD-RSA. In fact, we will show that $(RSA)^2$ is \emph{strictly more expressive} than QUD-RSA under fairly general conditions.
Recall the \rsatwospace equations are defined as follows where $\mathcal{M}, \mathcal{C}, \mathcal{U}$ are finite sets and $f_r: \mathcal{M} \times \mathcal{C} \times \mathcal{U} \rightarrow [0,1]$ (we will see that our equivalency claims can be proved by assuming the image of $f_r$ is $\mathbb{R}^{\geq0}$):
%
\begin{align}
P_{L_0}(m \mid c, u, r) &\propto f_r(m, c, u) \cdot P(m \mid c) \\
P_{S_1}(u \mid c, m, r) &\propto P_{L_0}(m \mid c, u, r) \cdot P(u \mid c) \\
P_{L_1}(m \mid c, u, r) &\propto P_{S_1}(u \mid c, m, r) \cdot P(m \mid c)
\end{align}
%
The QUD-RSA equations are as follows:
%
\begin{align}
P_{L_0}(m \mid u, c) &\propto \mathds{1}_{m \in \llbracket u \rrbracket} \cdot P(m \mid c) \\
P_{L_0}(m \mid c, u, q) &\propto \sum_{m' \in \mathcal{M}} \mathds{1}_{q(m') = q(m)} \cdot P_{L_0}(m' \mid u, c) \\
P_{S_1}(u \mid c, m, q) &\propto P_{L_0}(m \mid c, u, q) \cdot P(u \mid c) \label{quds1}\\
P_{L_1}(m \mid c, u, q) &\propto P_{S_1}(u \mid c, m, q) \cdot P(m \mid c) \label{qudl1}
\end{align}
%
A few notes about QUD-RSA:
\begin{enumerate}
\item In Equation~\ref{quds1}, the authors actually use $P_{L_0}(q(m)|c,u)$ but that didn't look correct so I used $P_{L_0}(m|c,u,q)$ instead. I believe this might be equivalent to what they intended to say where all of the meanings outside of the projection $q$ receive probability 0.
\item In Equation~\ref{qudl1}, the authors actually use $P_{L_1}(m \mid c, u, q) \propto P_{S_1}(u \mid c, m, q) \cdot P(m \mid c) \cdot P(q|c)$, but this also doesn't look right. You would only add $P(q|c)$ if you were marginalizing over $q$ I believe. In the original non-literal numbers paper, the authors don't have this term.
\end{enumerate}
\textbf{Claim 1.} QUD-RSA can be simulated by $(\text{RSA})^2$ if $f_r : \mathcal{M} \times \mathcal{C} \times \mathcal{U} \rightarrow \mathbb{R}^{\geq 0}$ and $P(m \mid c) > 0$ for all $m, c$.
\textbf{Proof.} Given an instance of QUD-RSA $\mathcal{E}_{\text{QUD}}$, we can build an instance of $(\text{RSA})^2$—call it $\mathcal{E}_{(\text{RSA})^2}$—which simulates it.
We do this by setting $\mathcal{C}, \mathcal{M}, \mathcal{U}$ to be the same, and letting $\mathcal{R} := \mathcal{Q}$, and defining:
%
\begin{equation}
f_q(m, c, u) = \frac{ \sum_{m' \in \mathcal{M}} \mathds{1}_{q(m') = q(m)} \cdot P_{L_0}(m' \mid c, u) }{ P(m \mid c) }
\end{equation}
%
where $P_{L_0}(m' \mid c, u)$ is defined as in the $\mathcal{E}_{\text{QUD}}$ instance.
\textbf{Claim 2.} Not all instances of $(\text{RSA})^2$ can be simulated by QUD-RSA (again, assuming $f_r : \mathcal{M} \times \mathcal{C} \times \mathcal{U} \rightarrow \mathbb{R}^{\geq 0}$ and $P(m \mid c) > 0$ for all $m, c$ for simplicity).
\textbf{Proof.} Consider the literal listener distribution $P_{L_0}(m \mid c, u)$ from QUD-RSA. We normalize to create a proper distribution which leads to the following
%
\begin{equation}
P_{L_0}(m \mid c, u) =
\frac{\mathds{1}_{m \in \llbracket u \rrbracket} \cdot P(m \mid c) }
{ \sum\limits_{m''} \mathds{1}_{m'' \in \llbracket u \rrbracket} \cdot P(m'' \mid c) }
\end{equation}
%
Setting the denominator as a constant and replacing in the equation for $P_{L_0}(m \mid c,u,q)$, we get the following:
\[
\mathcal{K} = \sum_{m''} \mathds{1}_{m'' \in \llbracket u \rrbracket} \cdot P(m'' \mid c),
\]
\begin{align}
P_{L_0}(m \mid c, u, q)
&\propto \frac{
\sum\limits_{m'} \mathds{1}_{q(m') = q(m)} \cdot \mathds{1}_{m' \in \llbracket u \rrbracket} \cdot P(m' \mid c)
}{
\mathcal{K}
} \\
&\propto \sum_{m'} \mathds{1}_{q(m') = q(m)} \cdot \mathds{1}_{m' \in \llbracket u \rrbracket} \cdot P(m' \mid c)
\end{align}
Then we need only define, for each $q$, a function $f_r$ such that
$P_{L_0}(m \mid c, u, r)$ cannot be represented as a binary combination of $P(m \mid c)$.
We can design such an $f_r$ by considering all possible binary combinations of $P(m \mid c)$ (these are finite for a finite $\mathcal{M}$). That is,
%
\[
N = \{P(m_1 \mid c), P(m_2 \mid c),\dots,P(m_n \mid c), P(m_1 \mid c) + P(m_2 \mid c), \dots, P(m_1 \mid c) + \cdots + P(m_n \mid c) = 1\}
\]
%
From the coefficients in $N$, $|\mathcal{M}|$ of these will be used to induce a distribution $P_{L_0}(m|c,u,q)$. We can list all such distributions as probability vectors $\mathbb{P} = \{\vec{p}_1, \ldots, \vec{p}_T\}$.
We can create an $f_r$ which induces $P_{L_0}(m \mid c, u, r)$ outside of $\mathbb{P}$ by picking $q$ such that:
%
\begin{align}
p_{\min} &= \min_{\vec{p} \in \mathbb{P}, i \in \{1, \ldots, |\vec{p}|\}} \vec{p}[i] \\
p'_{\min} &= \min_{\vec{p} \in \mathbb{P}, i \in \{1, \ldots, |\vec{p}|\}, \vec{p}[i] \neq p_{\min}} \vec{p}[i] \\
q &= \frac{p_{\min} + p_{\min'}}{2}
\end{align}
We define $f_r$ such that $P_{L_0}(m \mid c, u, r) = q$ for some $m \in \mathcal{M}$.
Let
\[
f_r(m, c, u) =
\begin{cases}
\frac{q}{P(m_1 \mid c)} & \text{if } m = m_1 \\
\frac{1 - q}{P(m_2 \mid c)} & \text{if } m = m_2 \\
0 & \text{otherwise}
\end{cases}
\]
Then we have:
\begin{align}
P_{L_0}(m_1 \mid c, u, r)
&= \frac{f_r(m_1, c, u) \cdot P(m_1 \mid c)}{\sum_{m'} f_r(m', c, u) \cdot P(m' \mid c)} \\
&= \frac{ \frac{q}{P(m_1 \mid c)} \cdot P(m_1 \mid c) }
{ \frac{q}{P(m_1 \mid c)} \cdot P(m_1 \mid c) + \frac{1 - q}{P(m_2 \mid c)} \cdot P(m_2 \mid c) } \\
&= \frac{q}{q + 1 - q} = q \quad \text{as desired}.
\end{align}
But clearly this probability vector is not in $\mathbb{P}$ because one of the probability values is not in
$\left\{ \vec{p}[i] : \vec{p} \in \mathbb{P},\ i \in \{1, \ldots, |\vec{p}|\} \right\}$.
\end{document}
% This must be in the first 5 lines to tell arXiv to use pdfLaTeX, which is strongly recommended.
\pdfoutput=1
% In particular, the hyperref package requires pdfLaTeX in order to break URLs across lines.
\documentclass[11pt]{article}
% Change "review" to "final" to generate the final (sometimes called camera-ready) version.
% Change to "preprint" to generate a non-anonymous version with page numbers.
\usepackage{acl}
% \usepackage{acl}
% Standard package includes
\usepackage{times}
\usepackage{latexsym}
\usepackage[T1]{fontenc}
\usepackage[utf8]{inputenc}
\usepackage{microtype}
\usepackage{inconsolata}
\usepackage{graphicx}
\usepackage{amsmath}
\usepackage{booktabs}
\usepackage{lipsum}
\usepackage[caption=false]{subfig}
\usepackage{float}
\usepackage{multirow}
\usepackage{amsmath}
\usepackage{changepage}
\usepackage{natbib}
\usepackage{amsmath}
\usepackage{stmaryrd}
\usepackage{dsfont}
\usepackage{geometry} % Optional, to ensure full-page width
\usepackage{changepage}
\usepackage{makecell}
\usepackage{listings} % Import the listings package
\usepackage{framed}
\usepackage{amsfonts}
\usepackage{algorithm}
\usepackage{algorithmicx}
\usepackage{algpseudocode}
\usepackage{relsize}
\usepackage{tcolorbox}
\lstset{
breaklines=true, % Enable line wrapping
basicstyle=\ttfamily, % Use monospace font
frame=single, % Add a frame around the code block
}
\DeclareMathOperator*{\argmax}{arg\,max}
\usepackage[]{todonotes} % use option "disable" to suppress notes
\makeatletter
\newcommand*\iftodonotes{\if@todonotes@disabled\expandafter\@secondoftwo\else\expandafter\@firstoftwo\fi}
\makeatother
\newcommand{\noindentaftertodo}{\iftodonotes{\noindent}{}\ignorespaces}
\newcommand{\pab}[1]{{\color{red}#1}}
\newcommand{\david}[1]{{\color{purple}#1}}
\newcommand{\ces}[1]{{\color{blue}#1}}
\newcommand{\rsatwo}{\textbf{(RSA)}\textsuperscript{2}}
\newcommand{\rsatwospace}{\rsatwo\,\,}
\newcommand{\rsatwolearned}{\textbf{RSC--RSA}}
\newcommand{\rsatwolearnedspace}{\rsatwolearned\,\,}
\newcommand{\response}[1]{\vspace{3pt}\hrule\vspace{3pt}\textbf{#1:}}
\algnewcommand{\Hypothesis}[1]{\textbf{Hypothesis:} #1}
\interfootnotelinepenalty=10000
% \definecolor{googlegreen}{RGB}{56, 118, 29}
% \definecolor{googleblue}{RGB}{17, 85, 204}
\title{\rsatwo: A Rhetorical-Strategy-Aware Rational Speech Act Framework for Figurative Language Understanding}
\author{%
\bf \textsuperscript{*}Cesare Spinoso-Di Piano\textsuperscript{1,2} \quad\textsuperscript{*}David Austin\textsuperscript{1,2} \\
\bf \textbf{Pablo Piantanida}\textsuperscript{2,3,4} \quad \textbf{Jackie Chi Kit Cheung}\textsuperscript{1,2,5} \vspace{1mm}\\
\textsuperscript{1}McGill University, \quad \textsuperscript{2}Mila - Quebec AI Institute \\
\textsuperscript{3}International Laboratory on Learning Systems (ILLS)\\ %\quad \textsuperscript{4}Université Paris-Saclay, \quad \textsuperscript{5}\'Ecole de technologie sup\'erieure (ETS) \\
\textsuperscript{4}CNRS, CentraleSup\'elec - Université Paris-Saclay, \quad \textsuperscript{5}Canada CIFAR AI Chair, Mila \vspace{1mm}\\
\texttt{\{cesare.spinoso,david.austin,pablo.piantanida,cheungja\}@mila.quebec}\\
}
\begin{document}
\maketitle
\begin{abstract}
Figurative language (e.g., irony, hyperbole, understatement) is ubiquitous in human communication, resulting in utterances where the literal and the intended meanings do not match. The Rational Speech Act (RSA) framework, which explicitly models speaker intentions, is the most widespread theory of probabilistic pragmatics, but existing implementations are either unable to account for figurative expressions or require modeling the implicit motivations for using figurative language (e.g., to express joy or annoyance) in a setting-specific way. In this paper, we introduce the \textbf{R}hetorical-\textbf{S}trategy-\textbf{A}ware \textbf{RSA} \rsatwospace framework which models figurative language use by considering a speaker's employed rhetorical strategy. We show that \rsatwospace enables human-compatible interpretations of non-literal utterances without modeling a speaker's motivations for being non-literal. Combined with LLMs, it achieves state-of-the-art performance on the ironic split of PragMega+, a new irony interpretation dataset introduced in this study.\footnote{Code and data available at \url{https://github.com/cesare-spinoso/rsa2}.}
\end{abstract}
\def\thefootnote{*}\footnotetext{Equal contribution.}\def\thefootnote{\arabic{footnote}}
\section{Introduction}
\label{sec:intro}
Figurative uses of language --- where a speaker does not literally say what they mean --- are ubiquitous in human communication. For example, when faced with a blizzard with heavy winds, a speaker may say ``It's a little chilly out.'' to indicate that it is very cold outside. Similarly, as illustrated in Fig.~\ref{fig:john-example}, a teacher might refer to a student as ``really sharp'' when they intend to convey that they are in fact not very clever. As a result, it is imperative for language technologies and computational models of language to account for this linguistic phenomenon.
However, recent studies have shown that large language models (LLMs) struggle with figurative interpretations of language. For instance, \citet{hu-etal-2023-fine} show that LLMs often misinterpret utterances which subvert listener expectations such as in irony and humour. In Fig.~\ref{fig:john-example}, we show an example where the Mistral-7B-Instruct (V3) LLM overwhelmingly favours an incorrect literal interpretation of the utterance (The student is smart.) over the correct non-literal one (The student is not very clever.). Consequently, this behaviour suggests that LLMs do not correctly or fully model the underlying \emph{communicative goal} (i.e., intention) of utterances needed to interpret them, to the extent that this can be shown through behavioural analysis.
\begin{figure}[t]
\centering
\includegraphics[width=\linewidth]{Images/rsa_john_example.png}
\caption{A sample from the PragMega dataset \citep{hu-etal-2023-fine} showing probabilities of intended meanings for a given scenario. The probabilities are computed using the Mistral-7B-Instruct model (averaged across 10 different orders of presenting the meaning options).}
\label{fig:john-example}
\end{figure}
A potential solution to the poor performance of LLMs on figurative understanding is the Rational Speech Act (RSA) framework from probabilistic pragmatics in which the modeling of communicative intents is central. In RSA, an utterance's meaning is interpreted probabilistically by a \emph{pragmatic listener} which reasons about a posited \emph{pragmatic speaker}'s likelihood of generating the utterance under different intended meanings. However, while RSA was designed to interpret utterances based on their communicative goal, its original formulation does not allow non-literal meaning interpretations of utterances. Previous attempts to address this limitation require modeling a speaker's motivations for being non-literal (e.g., to express their affect of annoyance or joy) in a setting-specific way \citep{kao_number_words_2014,Kao2015Irony}.
The key insight that we leverage is that non-literal language usage follows systematic patterns, which can be grouped into \emph{rhetorical strategies} \citep{bertin2016linguistic}. For example, irony can be characterized by an intended meaning being the opposite of its literal meaning, whereas hyperbole involves overstating the intended meaning. Thus, we argue that pragmatic models should reify the rhetorical strategy as a mediating mechanism between language form and communicative intent. To this end, we introduce a novel formulation of the RSA framework: \textbf{R}hetorical-\textbf{S}trategy-\textbf{A}ware \textbf{RSA} \rsatwo. In \rsatwo, a speaker's rhetorical strategy is explicitly modeled as a latent variable and used by the pragmatic listener to interpret an utterance's potentially non-literal intended meaning.
We show experimentally that \rsatwospace enables pragmatic listeners to infer non-literal interpretations of utterances without needing to model a speaker’s motivations for being non-literal. Using datasets of non-literal number expressions \citep{kao_number_words_2014} and of ironic weather utterances \citep{Kao2015Irony}, we show that \rsatwospace provides non-literal interpretations of utterances which closely match those of humans and which often outperform those made by existing \emph{affect-aware RSA} methods. In addition, we couple \rsatwospace with LLMs and prompt engineering to achieve state-of-the-art performance on the ironic split of an expanded utterance interpretation dataset, PragMega+, which we design for this study. This latter result suggests that probabilistic pragmatics methods may help mitigate LLMs' bias towards literal interpretations and lexical overlap as exhibited in Fig.~\ref{fig:john-example}.
To summarize, in this work, we design \rsatwo, a communicative-goal-centered procedure of figurative language interpretation. By explicitly considering the rhetorical strategies a speaker might use, this framework provides pragmatic non-literal interpretations of utterances which are aligned with human interpretations. In addition, we show that \rsatwospace can be used as a tool to help align LLM interpretations of figurative utterances with their intended meaning. Beyond figurative language understanding, our work re-establishes the importance of computational models of language which focus on modeling the communicative goals of speakers and listeners.
\section{Related Work}
\label{sec:rw}
\label{sec:human_irony}
\paragraph{Modeling Human Uses of Figurative Language} There has been extensive work to explain why humans use figurative language. Early studies in pragmatics \citep{grice1975logic,horn1984towards} attempt to explain such uses of language by positing that language users communicate with each other ``cooperatively'' by following certain ``conversational maxims''. In this account, figurative statements can result from reconciling violations of conversational maxims with the cooperative principle. Subsequent studies have shown that speakers use figurative expressions to soften the tone of a critique or to be humorous \citep{robertsWhyPeopleUse1994,colstonSaltingWoundSugaring1997}, that figurative uses of language can sometimes be easier to process cognitively than their literal counterparts \citep{ContextualEffectsUnderstanding1979} and that listeners understand figurative statements by explicitly interpreting the mental states of speakers through the ``Theory of Mind''-network part of the brain \citep{spotornoNeuralEvidenceThat2012}.%
\citet{kao_number_words_2014,kao2014formalizing,Kao2015Irony} adapt the RSA framework originally introduced by \citet{frank_ref_game_2012} to enable non-literal interpretations of figurative utterances (e.g., hyperbolic utterances about prices, ironic utterances about the weather). To enable non-literal interpretations, the authors assume that figurative language use is motivated by affect (e.g., to convey joy, annoyance, etc.) and perform joint inference over both intended meaning and affect to achieve non-literal interpretations. In this work, we seek to computationally model figurative language \emph{without} having to explicitly model its motivation which may be difficult to determine in general. For instance, when John ironically says ``This one is really sharp.'' (Fig.~\ref{fig:john-example}), he may be motivated by one (or more) of several reasons: expressing some kind of affect (e.g., humour, frustration), gauging the principal's attention, using a conventionalized way of referring to students, etc. In contrast, \rsatwospace lifts its dependence on explicitly modeling a speaker's motivation to use figurative language and instead directly accounts for the possible rhetorical strategies being used to produce non-literal interpretations of utterances.%
\paragraph{Figurative Language Modeling in NLP} In NLP, the ability of language systems to understand figurative language has mainly been evaluated through detection and generation \citep{li-sporleder-2010-using,tsvetkov-etal-2014-metaphor,van-hee-etal-2018-semeval,balestrucci-etal-2024-im,lai-nissim-2022-multi,lai-etal-2023-multilingual}. A common approach to solving these tasks has been to collect figurative-language-specific datasets and to fine-tune language classification and generation models on them \citep{van-hee-etal-2018-semeval}. With the shift towards LLMs, there has been growing interest in evaluating pre-trained and instruction-tuned LLMs on their ability to recognize, interpret and generate figurative language such as irony and sarcasm \citep{gu-etal-2022-just,liu-etal-2022-testing,hu-etal-2023-fine}. LLMs have been shown to perform significantly worse than humans in interpreting figurative language, such as humour and irony. In this work, we propose a computational model of figurative language that explicitly models rhetorical strategies and which we show can be coupled with LLMs to better align their interpretations with speaker intentions.%
\paragraph{RSA in NLP} \citet{andreas_reasoning_2016} were the first to use neural listeners and speakers with RSA in the context of image captioning. \citet{friedUnifiedPragmaticModels2018} extended the use of RSA to instruction following and generation. \citet{shenPragmaticallyInformativeText2019} investigated RSA for abstractive summarization and \citet{cohn-gordonLostMachineTranslation2019} applied it to machine translation. In closer connection to our work, \citet{careniniFullyInterpretableMore2024} combined the RSA framework with LLMs for the task of metaphor understanding. \citet{tsvilodub2025nonliteralunderstandingnumberwords} leverage affect-aware RSA to enable LLMs to provide non-literal meaning interpretations of utterances with number words. In contrast, we develop a novel RSA framework, \rsatwo, and apply it to several non-literal utterance interpretation tasks.%
\section{Rhetorical-Strategy-Aware RSA}
\label{sec:rsa_two}
In this section, we introduce our novel formulation of the RSA framework --- \textbf{R}hetorical-\textbf{S}trategy-\textbf{A}ware \textbf{RSA} \rsatwo --- which explicitly incorporates the rhetorical strategy into the RSA framework. We first review the inner workings of the RSA framework and then present the novel \rsatwospace framework. Throughout this presentation, we will use the running example of a speaker ironically saying ``The weather is amazing.'' during a blizzard, inspired by \citet{Kao2015Irony}.
\subsection{The RSA Framework}
\label{sec:standard_rsa}
The RSA framework was introduced to model human communication by accounting for listener and speaker expectations, originally in the context of scalar implicature \citep{frank_ref_game_2012}. The goal of this framework is to derive a probability distribution over some fixed (finite) set of meanings $\mathcal{M}$ for some observed utterance $u \in \mathcal{U}$ generated in some context\footnote{The context $c$ can be linguistic (e.g., conversation history) as well as situational (e.g., the current state of the world).} $c \in \mathcal{C}$. To do so, this framework posits the existence of multiple speakers and listeners who reason about each other in a recursive fashion. This recursive procedure begins with a \emph{literal listener} $L_0$, which reasons literally about the interpretation of utterances. Its conditional distribution over $\mathcal{M}$ is defined as
%
\begin{align}
P_{L_0}(m|c,u) &\propto \mathds{1}_{m \in \llbracket u \rrbracket} \cdot P_{{M}|{C}}(m|c),
\end{align}
%
where $P_{{M}|{C}}$ is a prior conditional distribution over all meanings in $\mathcal{M}$ given a context $c$; and $\llbracket \cdot \rrbracket : \mathcal{U} \rightarrow \mathcal{P(M)}$ denotes a shared semantic understanding function\footnote{$\mathcal{P()}$ is the power set function.} which, given some utterance $u \in \mathcal{U}$, returns the set of possible intended meanings which are \emph{literally} compatible with $u$.
For instance, given our running example ``The weather is amazing.'' and a discrete set of weather states which a speaker might want to convey, $\mathcal{M} = \{\texttt{terrible}, \texttt{bad}, \texttt{ok}, \texttt{good}, \texttt{amazing}\}$, the semantic understanding function would return the following set:%
\[
\llbracket \text{``The weather is amazing.''} \rrbracket = \{\texttt{amazing}\}.
\]%
With the base case of the reasoning procedure defined, the RSA framework provides a speaker-aware interpretation of the utterance $u$ by positing the existence of a \emph{pragmatic speaker}, $S_1$, which selects an utterance $u$ based on its prior probability, $P_{{U}|C}$, and $L_0$'s expected information gain.\footnote{While Equation~\ref{eq:rsa_pragmatic_speaker} may appear different from its common expected-utility formulation \citep{goodman2013knowledge}, we show the equivalence of these two formulations in Appendix~\ref{app:equivalence_rsa_formulations}.} That is,
\begin{align}
\label{eq:rsa_pragmatic_speaker}
P_{S_1}(u|c,m) &\propto P_{L_0}(m|c,u)^\alpha P_{{U}|{C}}(u|c),
\end{align}%
where $\alpha$ is a rationality parameter that controls how much the pragmatic speaker will favor selecting the most informative utterance according to the literal listener. Note that the pragmatic speaker is normalized across a set of utterances which are either predefined or generated based on $c$ \citep{andreas_reasoning_2016}.
According to Equation~\ref{eq:rsa_pragmatic_speaker}, a speaker intending to convey meaning $m$ will distribute its probability mass proportional to the probability that $L_0$ infers $m$ from $u$. In our running example, assuming a discrete set of utterances of the form $\mathcal{U} = \{\text{``The weather is $m$.''} : m \in \mathcal{M}\}$, a speaker wishing to convey $m = \texttt{amazing}$ will place \emph{all} of its probability mass on the utterance ``The weather is amazing.'' since no other utterance enables $L_0$ to recover this intended meaning.
Finally, to interpret a given utterance $u$ pragmatically, the \emph{pragmatic listener}, $L_1$, updates their prior belief about the distribution of possible intended meanings $P_{{M}|{C}}$ by using the likelihood that the utterance was generated by $S_1$ in the given context. In other words, the pragmatic listener's conditional probability distribution over $\mathcal{M}$ becomes the following posterior Bayesian update:
\begin{align}
P_{L_1}(m|c,u) \propto P_{S_1}(u|c,m) P_{{M}|{C}}(m|c).
\end{align}%
In the case of our ironically uttered statement ``The weather is amazing.'', the pragmatic listener will reason through $S_1$'s generation process and will conclude that they intended to say that the weather is \texttt{amazing} \emph{despite} the blizzard. Thus, the pragmatic listener is unable to escape the literal interpretation of the utterance determined by $L_0$.%
In general, it can be shown that, under the RSA framework, interpretations of utterances which are not compatible with the literal meaning of an utterance will be assigned zero probability by the pragmatic listener (Proof in Appendix~\ref{app:standard_rsa_non_literal}). Existing \emph{affect-aware RSA} approaches discussed in Section~\ref{sec:human_irony} have attempted to mitigate this limitation by expanding the model of the speaker to include a random variable which accounts for their motivation to be figurative (e.g., for affect-related reasons). In contrast, our \rsatwospace approach does not require modeling a speaker's implicit motivations for being non-literal. Rather, the pragmatic listener in \rsatwospace reasons directly and explicitly about the possible rhetorical strategies being employed to achieve non-literal interpretations of utterances.
\subsection{The Rhetorical-Strategy-Aware RSA Framework}
\label{sec:rsatwooriginal}
We introduce a rhetorical-strategy-aware RSA framework, \rsatwo, which defines a rhetorical strategy variable $r \in \mathcal{R}$ and a rhetorical function for each value of $r$, $f_{r}: \mathcal{C} \times \mathcal{M} \times \mathcal{U} \rightarrow [0,1]$. This function generalizes the literal semantic understanding indicator function $\mathds{1}_{m \in \llbracket u \rrbracket}$ to an arbitrary function over $[0,1]$. The rhetorical function enables non-literal interpretations of utterances to arise based on the employed rhetorical strategy. For instance, using our running blizzard example, the ironic and hyperbolic rhetorical strategies might return the following, where $c = \texttt{blizzard}$ is the context variable:%
\noindent % To prevent potential indentation issues with the resizebox
\resizebox{0.95\linewidth}{!}{%
\begin{minipage}{\linewidth} % Or a fixed width if you know it
\begin{align*}
f_\textit{irony}(c, \texttt{terrible}, \text{``The weather is amazing.''}) &= 1, \\
f_\textit{irony}(c, \texttt{amazing}, \text{``The weather is amazing.''}) &= 0, \\
f_\textit{hyperbole}(c, \texttt{good}, \text{``The weather is amazing.''}) &= 1, \\
f_\textit{hyperbole}(c, \texttt{amazing}, \text{``The weather is amazing.''}) &= 0.
\end{align*}
\end{minipage}
}
\newline\newline
We replace the semantic understanding function with our generalized rhetorical function within the $P_{L_0}$ equation as follows:
\begin{align}
P_{L_0}(m|c,u,r) &\propto f_r(c,m,u) P_{M|C}(m|c). \label{eqn:rsatwol0}
\end{align}%
Thus, a meaning which receives zero probability mass when an utterance is interpreted literally by $L_0$ may still receive non-zero probability mass when that same utterance is interpreted using irony, hyperbole or some other non-literal rhetorical strategy. For example, using $f_\textit{irony}$ and $f_\textit{hyperbole}$ in our running example would enable the probability distribution of $L_0$ to shift towards non-literal interpretations of the utterance such as \texttt{terrible} and \texttt{good} respectively. Previous extensions of the RSA framework have modified the semantic understanding indicator function to account for lexical uncertainty \citep{bergen2016pragmatic}. To the best of our knowledge, we are the first to propose this generalization to model figurative language.
We define the pragmatic speaker and listener like in standard RSA, with our rhetorical strategy variable as an additional conditional factor:
\begin{align}%
P_{S_1}(u|m,c,r) &\propto P_{L_0}(m|c,u,r)^\alpha P_{U|C}(u|c), \label{eq:rsa_two_pragmatic_speaker} \\
P_{L_1}(m|c,u,r) &\propto P_{S_1}(u|m,c,r) P_{M|C}(m|c).
\end{align}%
At inference time, the rhetorical strategy being used is an unobserved latent variable which must be marginalized out while accounting for its probability, $P_{R|CU}(r|c,u)$ on $\mathcal{R}$, as follows:%
\begin{equation}
P_{L_1}(m|c,u) = \sum_{r'} P_{L_1}(m|c,u,r') P_{R|CU}(r'|c,u).\label{eq:rsa_marginalization}
\end{equation}%
Returning to our running example, let us consider a discrete set of rhetorical strategies $\mathcal{R} = \{\textit{literal}, \textit{irony}, \textit{hyperbole}\}$. We observe that the pragmatic listener conditioned with the \textit{literal} strategy would produce the same distribution as the standard-RSA-derived pragmatic listener. However, the utterance would also be interpreted by $P_{L_1}(m|c,u,\textit{irony})$ with most probability mass concentrated around \texttt{terrible}, the opposite of the utterance's literal meaning, and by $P_{L_1}(m|c,u,\textit{hyperbole})$ with most mass concentrated around \texttt{good}, the utterance's scaled down literal meaning. By marginalizing these listener distributions using $P_{R|CU}$,\footnote{$P_{R|CU}$ may be uniform, human-derived or computed experimentally.} we would obtain a distribution which assigns mass to meanings beyond the utterance's literal interpretation. In this way, \rsatwospace allows modeling figurative interpretations of language \emph{without} needing to explicitly model the motivations behind its use. In Appendix~\ref{app:qud_rsa_special_case}, we prove that this latter approach to non-literal language is a special case of \rsatwospace wherein the rhetorical strategy function is repurposed to model the motivations of non-literal utterances (e.g., affect).%
\section{Non-Literal Interpretations of Figurative Language with \rsatwo}
\label{sec:study1}
In this first experimental study, we aim to show that \rsatwospace can be used to derive non-literal meaning interpretations which match human interpretations as well as by existing affect-aware RSA methods. We implement \rsatwospace in two settings: non-literal number price expressions (e.g., ``This kettle costs 10000\$.'') and ironic weather utterances (e.g., ``The weather is amazing.'' during a winter blizzard). We show that \rsatwospace produces utterance interpretations that are on par with or better than affect-aware RSA interpretations.%
\subsection{Datasets}
\paragraph{Non-literal number expressions.} We use the dataset from \citet{kao_number_words_2014} which contains a collection of literal and non-literal number expressions related to the price of objects. The context space $\mathcal{C}$ is the set of objects being described, the meaning space $\mathcal{M}$ is the price of the object being described and the utterance space $\mathcal{U}$ is a verbalization of the object's price. These three sets are listed below:%
\begin{align*}
\mathcal{C} &= \{\text{electric kettle}, \text{laptop}, \text{watch}\}, \\
\mathcal{M} &= \{50, 51, 500, 501, 1000, 1001, \\
& \quad 5000, 5001, 10000, 10001\}, \\
\mathcal{U} &= \{\text{``The $c$ costs $m$ dollars.''} : c \in \mathcal{C}, m \in \mathcal{M}\}.
\end{align*}%
The authors collected meaning and affect priors $P(m|c), P(a|c)$ from 30 human participants as well as meaning and affect posteriors $P(m|c,u), P(a|c,u)$ from 120 human participants via a Likert-scale probability elicitation technique.%
To enable \rsatwo-based non-literal interpretations, we define the space of rhetorical strategies $\mathcal{R}$ as consisting of four types: \emph{literal} (describing the price exactly), \emph{hyperbole} (overstating the price), \emph{understatement} (understating the price) and \emph{halo} ( providing a round figure rather than the exact one).%
\paragraph{Ironic weather utterances.} We use the dataset from \citet{Kao2015Irony} which contains a collection of utterances about the weather similar to the one from our running example in Section~\ref{sec:rsa_two}. In this case, the context space $\mathcal{C}$ is represented visually through images depicting different weather conditions (see Appendix~\ref{app:weather_images} for the images), the utterance space $\mathcal{U}$ consists of statements about the weather (e.g., ``The weather is amazing.''), and the meaning space $\mathcal{U}$ corresponds to the true weather state being communicated by the speaker. These three sets are listed below:%
\begin{align*}
\mathcal{C} &= \{c_i : i \in [1 \dots 9]\}, \\
\mathcal{M} &= \{\texttt{terrible},\texttt{bad},\texttt{ok},\texttt{good},\texttt{amazing}\}, \\
\mathcal{U} &= \{\text{``The weather is $m$''} : m \in \mathcal{M}\}.
\end{align*}%
The context space contains three types of contexts: those where the weather is visibly good ($\{c_1,c_2,c_3\}$), those where the weather is visibly bad ($\{c_7,c_8,c_9\}$) and those where the weather is neither visibly good nor bad ($\{c_4,c_5,c_6\}$). The original authors of the paper collected meaning and affect priors $P(m|c), P(a|c)$ from 49 human participants as well as meaning, affect and irony posteriors $P(m|c,u), P(a|c,u), P(r|c,u)$ from 120 human participants using both normalized counts and a Likert-scale probability elicitation technique similar to \citet{kao_number_words_2014}.%
To enable \rsatwo-based non-literal interpretations, the space of rhetorical strategies $\mathcal{R}$ is defined using both the \textit{literal} (describing the weather as it is perceived) and \textit{irony} (describing the weather as the opposite of how it is perceived) rhetorical strategies.
\subsection{RSA Model Experiments}
\subsubsection{Baseline}
\paragraph{Affect-aware RSA.} For the non-literal number expressions dataset, we use the pragmatic listener as computed by \citet{kao_number_words_2014}. For the weather utterance dataset, we re-implement both the affect-aware literal and pragmatic listeners (see Appendix~\ref{app:reimplementation_details} for implementation details).
\subsubsection{\rsatwospace Implementations}
\paragraph{Non-literal number expressions.} We manually define $f_r$ based on the intuitive definitions of each of the rhetorical strategies in $\mathcal{R}$. To simplify notation, we only consider the integer parts of the utterance i.e., $u = \text{``The $c$ costs $y$ dollars.''}$ becomes $u = y$. We define $f_r$ as:%
\noindent % To prevent potential indentation issues with the resizebox
\resizebox{0.8\linewidth}{!}{%
\begin{minipage}{\linewidth} % Or a fixed width if you know it
\begin{align*}
f_r(c, m, u) =
\begin{cases}
1 & \text{if } r = \text{literal}, u = m \\
1 & \text{if } r = \text{hyperbole}, \\
& \quad u - m > 10 \\ % Ensured alignment for multi-line conditions
1 & \text{if } r = \text{understatement}, \\
& \quad m - u > 10 \\
1 & \text{if } r = \text{halo}, |u - m| = 1, \\
& \quad u \text{ ends in 0} \\
0.001 & \text{otherwise}
\end{cases}
\end{align*}
\end{minipage}
}
\newline\newline
Thus, for instance, if a kettle's price is $m = 50$, but the utterance used is $u = 500$, then this will trigger the \emph{hyperbole} rhetorical strategy. Similarly, if the kettle's price is $m = 501$ and the utterance is $u = 500$, then this will activate the \emph{halo} rhetorical strategy. We ignore the context $c$ in $f_r$ for simplicity, although in principle the effect of a rhetorical strategy like hyperbole could vary across objects. We substitute $f_r$ in $L_0$'s Equation~\ref{eqn:rsatwol0} and use the meaning prior $P(m|c)$ derived from \citet{kao_number_words_2014} and a uniform rhetorical strategy prior, $P(r|c,u) = \frac{1}{4}$, for the marginalization of Equation~\ref{eq:rsa_marginalization}. We set the rationality parameter $\alpha = 1$.
\paragraph{Ironic weather utterances.} To highlight the flexibility of \rsatwo, we approximate the rhetorical function $f_r$ across all 450 $(c,m,u,r)$ quadruples\footnote{$9\text{ contexts }\times5\text{ meanings }\times5\text{ utterances }\times2\text{ rhetorical strategies}$} using a neural network. Specifically, we split the dataset into training, validation and test sets ($60\%/20\%/20\%$) and train a 2-layer neural network ($16 \times 16 \times 5$) with sigmoid activations. We encode the contexts, meanings, utterances, and rhetorical strategies using a one-hot encoding and use a cross-entropy loss between $P_{L_1}(m|c,u)$ and the meaning selected by human participants given the context $c$ and the utterance $u$. The rationality parameter is set to $\alpha = 1$. We use the human-elicited probability $P(r|c,u)$ for the marginalization across the two rhetorical strategies in Equation~\ref{eq:rsa_marginalization}. Additional details can be found in Appendix~\ref{app:ironic_weather_utterances_exp_details}.
\subsection{Results}
We plot the meaning distribution of human posteriors and of affect-aware and \rsatwospace listeners for the hyperbolic number expression ``The electric kettle costs 1001 dollars.'' in Fig.~\ref{fig:the_kettle} and for the ironic expression ``The weather is amazing.'' uttered in the context of a blizzard in Fig.~\ref{fig:the_weather_is_amazing}. In both cases, we observe that the \rsatwospace listeners, especially the pragmatic listeners, induce human-like meaning distributions. Figures for all other context–meaning–utterance triples can be found in Appendices~\ref{app:non_literal_number_expressions} and~\ref{app:ironic_weather_utterances_add_results}.
\begin{figure}[h]
\centering
\includegraphics[width=\linewidth]{Images/screenshot_electric_kettle.png}
\caption{Meaning distribution of human posteriors and of affect-aware and \rsatwospace listeners for the utterance ``The electric kettle costs 1001 dollars''.}
\label{fig:the_kettle}
\end{figure}
\begin{figure}[h]
\centering
\includegraphics[width=\linewidth]{Images/screenshot_the_weather_is_amazing.png}
\caption{Meaning distribution of human posteriors and of affect-aware and \rsatwospace listeners for the utterance ``The weather is amazing.'' in the context of a blizzard (Image $c_8$ in Appendix~\ref{app:weather_images}).}
\label{fig:the_weather_is_amazing}
\end{figure}
We evaluate our model's predictive power by computing the mean absolute difference (MAD) between the human-derived meaning probability distributions and those generated by the listeners in both affect-aware RSA and \rsatwospace (Table~\ref{tab:mad_numbers}). We note that \rsatwospace is competitive with affect-aware RSA, surpassing it on the ironic weather utterances dataset. While the affect-aware listener outperforms our \rsatwospace listeners on non-literal number expressions, we believe this is due to the poorly calibrated uniform rhetorical strategy posterior we use, which sometimes causes probability mass to be distributed to the tail ends of the meaning space. For instance, while the expression ``The kettle costs 51 dollars.'' is most likely to be uttered with the \textit{literal} rhetorical strategy, a significant portion of the meaning mass is distributed across the rest of meaning space due to the non-zero posterior probability of the \textit{understatement} rhetorical strategy. Overall, we believe this initial study demonstrates that \rsatwospace can induce listener meaning distributions for figurative language that are at least as compatible with human interpretations as those produced by existing affect-aware RSA models.
\begin{table}[h]
\centering
\resizebox{\linewidth}{!}{
\begin{tabular}{clcc}
\toprule
Model & $L_i$ & Non-literal Numbers \(\downarrow\) & Weather Utterances \(\downarrow\) \\
\midrule
\multirowcell{2}{Affect-Aware \\ RSA} & $L_0$ & - & 0.2377 \\
& $L_1$ & \textbf{0.0436} & 0.1278 \\
\cmidrule{2-4}
\multirowcell{2}{\rsatwo} & $L_0$ & 0.0438 & 0.1647 \\
& $L_1$ & 0.0467 & \textbf{0.1229} \\
\bottomrule
\end{tabular}
}
\caption{Mean absolute differences between listener meaning distributions, $P_{L_i}(m|c,u), i = 0,1$, and the human posterior, $P(m|c,u)$, for both affect-aware and \rsatwospace on both the non-literal number expressions and ironic weather utterances datasets.}
\label{tab:mad_numbers}
\end{table}
\section{LLM Irony Interpretation with \rsatwo}
% \begin{table*}[t]
% \centering
% \resizebox{\linewidth}{!}{
% \begin{tabular}{p{6cm}p{6cm}p{6cm}}
% \toprule
% Intended Meaning is Non-Literal & Intended Meaning is Literal & Intended Meanings \\
% \midrule {\color{blue}John is a teacher at an elementary school. When talking with the principal about a new student, \textbf{who did poorly on her entrance examination}}, John said, {\color{purple}``This one is really sharp.''} & {\color{blue}John is a teacher at an elementary school. When talking with the principal about a new student, \textbf{who scored one of the highest scores}}, John said, {\color{purple}``This one is really sharp.''} & 1. The student is smart. (\texttt{LM}) \newline 2. The student is not very clever. (\texttt{NLM}) \newline 3. The entrance exam is unfair. (\texttt{OM}) \newline 4. The pencils need to be sharpened. (\texttt{NSM}) \\
% \bottomrule
% \end{tabular}
% }
% \caption{Example of a scenario where the intended meaning is non-literal as well as the corresponding modified scenario where the intended meaning is literal in the PragMega+ dataset as well as the shared intended meanings: 1. \texttt{Literal Meaning} (\texttt{LM}) 2. \texttt{Non-Literal Meaning} (\texttt{NLM}) 3. \texttt{Overlap Meaning} (\texttt{OM}) 4. \texttt{Non-Sequitur Meaning} (\texttt{NSM}). We bold the difference between the two scenarios and we color the different parts of the scenarios in {\color{blue}blue} and {\color{purple}purple} to identify the context and the utterance of that particular scenario respectively.}
% \label{tab:pragmega_examples}
% \end{table*}
\label{sec:experiment2}
In this section, we apply \rsatwospace to LLMs to generate non-literal interpretations of utterances that better align with human interpretations of figurative language. In particular, we focus on the task of interpreting ironic utterances in situational contexts beyond the weather examples from Section~\ref{sec:study1}. Our results demonstrate that integrating LLMs into \rsatwospace can, to a certain extent, mitigate their biases toward lexical-overlap and literal interpretations.
\subsection{The PragMega+ Dataset}
\label{sec:dataset}
We expand the PragMega dataset from \citet{hu-etal-2023-fine} to evaluate the \rsatwospace framework on ironic utterance interpretation. The original PragMega dataset contains 25 ironic scenarios where the intended meaning of an utterance is non-literal, such as in Fig.~\ref{fig:john-example}. Each scenario includes a background context $c$ and an utterance $u$ produced ironically by a speaker. In addition, each scenario is accompanied by four candidate intended meanings, which act as our meaning space$\mathcal{M}$: the literal meaning of $u$ (\texttt{Literal Meaning}), the intended meaning of $u$ (\texttt{Non-Literal Meaning}), a lexical overlap distractor meaning (\texttt{Overlap Meaning}), and a non-sequitur distractor meaning (\texttt{Non-Sequitur Meaning}).
Our expanded dataset, PragMega+, adds 25 ironic scenarios in the same format as part of our test set. In addition, we manually modify each of the 50 scenarios to create 50 \emph{literal} scenarios where the intended meaning is the \texttt{Literal Meaning}. Examples of scenarios with \texttt{Non-Literal} and \texttt{Literal} intended meanings from PragMega+ can be found in Appendix~\ref{app:pragmega_more_examples}.
\subsection{Experimental Setup}
\label{sec:experimental_setup}
\subsubsection{LLM Probability Estimation}
\label{sec:llms}
To integrate LLMs within the \rsatwospace framework, we use an LLM $N$ to estimate all conditional probabilities of the form $P_{N}(y|x)$. To do so, we use a prompt template similar to \citet{hu-etal-2023-fine} which, given $x$, lists all possible values of $y$ in a multiple-choice question (MCQ) format. After passing the prompt to the LLM $N$, we extract the next-token logit of the number corresponding to each option. These logits are re-normalized so that the sum of their corresponding probabilities equals one. In addition, to avoid positional bias, we randomly shuffle the order of the options in the prompt $10$ times\footnote{If the total number of permutations is less than $10$, we use that total instead.} and average across shuffles to obtain $P_{N}(y|x)$. We use this approach to estimate the meaning prior $P_{N}(m|c)$, the rhetorical strategy posterior $P_{N}(r|c,u)$, the meaning posterior $P_{N}(m|c,u)$, and the meaning posterior \emph{conditioned on} the rhetorical strategies, $P_{N}(m|c,u,r)$, where $\mathcal{R} = \{\textit{literal},\textit{irony}\}$. We experiment with two instruction-tuned models: Mistral-7B-Instruct (V3) LLM \citep{jiang2023mistral7b} and Llama-8B-Instruct (V3.1) \citet{llama3modelcard} via HuggingFace \citep{wolf2020huggingfacestransformersstateoftheartnatural}. We performed prompt engineering on the validation set. All prompts used on the test set are presented in Appendix~\ref{app:prompts}.
\subsubsection{Alternative Utterance Generation}
\label{sec:alt_utt_gen}
We use an LLM $G$ conditioned on a scenario's context $c$ to generate the alternative utterances needed to compute the pragmatic speaker's distribution. To do so, we condition $G$'s next-token generation on the full PragMega+ scenarios up to the original utterance's first quotation mark (e.g., \emph{... John said,``} in Fig.~\ref{fig:john-example}), and continue generation until another quotation mark is produced. We use a decoding temperature of $1.0$ and generate $50$ alternative utterances for each context. We also use the LLM’s generation likelihood, $P_G(u \mid c)$, as the utterance prior in our model. We experiment with the base versions of Mistral-7B (V3) \citep{jiang2023mistral7b} and Llama-8B (V3.1) \citep{llama3modelcard}.
\subsubsection{LLM Listeners}
\paragraph{LLM RSA} We implement an LLM RSA baseline by setting $P_{L_0}(m|c,u)$ equal to the raw LLM-derived probabilities, $P_{N}(m|c,u)$. This baseline follows prior neural listener work \citep{andreas_reasoning_2016, monroe-etal-2017-colors} which approximate $P_{L_0}(m|c,u)$ using trained neural networks. We set $\alpha = 1$.
% \ces{Standard RSA baseline with $\mathds{1}_{m \in \llbracket u \rrbracket}$ has been removed for space/because it doesn't add much}
% \paragraph{Standard RSA} As our second baseline, we implement an idealized version of the original RSA framework presented in Section~\ref{sec:rsatwooriginal} where we assume access to an oracle which can implement $\mathds{1}_{m \in \llbracket u \rrbracket}$ by returning $1$ when $m$ literally matches $u$ and $0$ otherwise.
\paragraph{LLM \rsatwo} We implement our LLM \rsatwospace listeners by setting $P_{L_0}(m|c,u,r)$ equal to the LLM-derived probabilities $P_{N}(m|c,u,r)$. For the rhetorical strategy posterior we use both the raw LLM-derived probabilities, $P_{N}(r|c,u)$, and an indicator posterior, $I(r|c,u) = \mathds{1}_{r = \arg\max_{r'} P_N(r'|c,u)}$. We set $\alpha = 1$.
\paragraph{Where is $f_r(c,m,u)$ in LLM \rsatwo?} Based on the LLM \rsatwospace literal listener, the rhetorical strategy function is implicitly set as:%
\begin{equation}
f_r(c,m,u) = \frac{P_{N}(m|c,u,r)}{k \cdot P_{N}(m|c)},\label{eq:what_is_frcmu}
\end{equation}%
where $k = \max_{m'} f_r(c,m',u)$ ensures that $f_r(c,m,u) \in [0,1]$.
\paragraph{Why is affect-aware RSA not included as a baseline?} While it is in principle possible to implement an LLM-based Affect-Aware RSA baseline \citep{tsvilodub2025nonliteralunderstandingnumberwords}, we do not pursue this direction, as it would require defining a suitable affect space for each situational context.
\subsection{Results}
Table~\ref{tab:meaning_scores} presents the average listener meaning probabilities for the correct, incorrect and distractor meanings (aggregated) across all 50 scenarios in the test set. We use the LLM pair $G = \text{Llama-8B (V3.1)}$ and $N = \text{Mistral-7B-Instruct (V3)}$, since this pair performed best on the validation set. Listener probabilities conditioned on each rhetorical strategy are shown in Table~\ref{tab:p_r_c_u}. Overall, we see that the LLM \rsatwospace listeners outperform the baseline LLM RSA listeners. In particular, the LLM \rsatwospace literal listener marginalized using the indicator rhetorical strategy posterior, $I(r|c,u)$, performs the best while the baseline LLM RSA literal listener performs the worst.
% Using $I(r|c,u)$ in the marginalization improves the $L_0$ and $L_1$ distributions because the raw $P_N(r|c,u)$ probabilities are typically spread out between the two rhetorical strategies causing a non-trivial proportion of the probability mass from the incorrect meaning to ``fold'' into the marginalization.
\begin{table}[h]
\centering
\resizebox{\linewidth}{!}{
\begin{tabular}{llccc}
\toprule
Model & $L_i$ & Correct \(\uparrow\) & Incorrect \(\downarrow\) & Distractor \(\downarrow\) \\
\midrule
LLM RSA & $L_0$ & 0.73 & 0.24 & 0.02 \\
& $L_1$ & 0.76 & 0.22 & 0.01 \\
\cmidrule{2-5}
\makecell{LLM \rsatwo\\with $P_N(r|c,u)$} & $L_0$ & 0.74 & 0.23 & 0.02 \\
& $L_1$ & 0.80 & 0.16 & 0.02 \\
\cmidrule{2-5}
\makecell{LLM \rsatwo\\with $I(r|c,u)$} & $L_0$ & \textbf{0.85} & \textbf{0.13} & 0.01 \\
& $L_1$ & 0.84 & 0.13 & \textbf{0.01} \\
\bottomrule
\end{tabular}
}
\caption{Average listener probabilities, $P_{L_i}(m|c,u), i = 0,1$ for the correct, incorrect and distractor intended meanings on the test set averaged across all 50 scenarios.}
\label{tab:meaning_scores}
\end{table}
\begin{figure*}[ht]
\centering
\includegraphics[width=\linewidth]{Images/screenshot_nonliteral_literal.png}
\caption{Average listener probabilities on the test set, split between the first 25 scenarios where the intended meaning is non-literal and the last 25 scenarios where the intended meaning is literal.}
\label{fig:pragmega_nonliteral_literal}
\end{figure*}
We study these results more carefully by plotting the listener probability distributions split between ironic and literal scenarios in Fig.~\ref{fig:pragmega_nonliteral_literal}. On ironic scenarios, the LLM \rsatwospace listener probabilities for the correct non-literal interpretation are all above $0.8$. In contrast, the LLM RSA literal listener assigns a probability of less than $0.5$ ($0.48$) to the correct non-literal intended meaning. However, on literal scenarios, the trend is reversed. The LLM RSA literal listener assigns most of its probability mass ($0.95$) to the correct literal meaning while the best performing of the LLM \rsatwospace listeners -- $L_0$ marginalized with $I(r|c,u)$ -- assigns $0.77$ of its mass to the correct literal intended meaning.
While the strong performance of the LLM RSA listeners on the literal scenarios is expected -- the correct intended meaning is consistent with the utterance and the context -- the drop in LLM \rsatwospace performance in these cases is surprising. Further analysis reveals that this result stems from the listener marginalization, specifically the rhetorical strategy posterior $P_N(r \mid c, u)$. This posterior performs poorly in the literal scenarios where the utterance is intended to be interpreted literally (Table~\ref{tab:p_r_c_u}). While for ironic scenarios the average probability of $P_N(r=\textit{irony}|c,u) = 0.88$ is reasonable, it is far worse on the sincere scenarios with $P_N(r=\textit{literal}|c,u) = 0.55$. This asymmetry in rhetorical strategy posteriors also explains why using $I(r|c,u)$ in the marginalization helps the LLM \rsatwospace listeners. The indicator function blocks more of the probability mass associated with the incorrect meaning from being incorporated into the marginalization.
\subsection{Probability Ablation Study}
\label{sec:analysis_pragmega_plus}
To study the impact of the RSA reasoning process on LLMs, we run an ablation study in which we set the probabilities $P(m \mid c)$ and $P(u \mid c)$ to uniform in both LLM RSA and LLM \rsatwo. Our analysis reveals that ablating $P(u|c)$ does not hinder performance and may slightly improve it. Surprisingly, ablating $P(m|c)$ reveals that most of the performance gain obtained by $L_1$ comes from the meaning prior \emph{and not} from the RSA reasoning process itself. We attribute this result to the way in which we generate alternative utterances (Section~\ref{sec:alt_utt_gen}) which tend to produce literal paraphrases of the intended meaning. For instance, Llama-8B (V3.1) generates alternatives to ``This one is really sharp.'' from Fig.~\ref{fig:john-example} such as ``Her scores were well below average.'' Thus, the pragmatic speaker conditioned on the correct intended meaning distributes probability mass across many compatible alternative utterances, leading to a Bayesian update that \emph{spreads}, rather than \emph{narrows}, the distribution over meanings. Future work on coupling LLMs with RSA may want to ensure diversity in the alternatives generated, for instance by conditioning the generation on individual meanings rather than the context.
\begin{table}[h]
\centering
\resizebox{\linewidth}{!}{
\begin{tabular}{llcc}
\toprule
Model & $L_i$ & w/o $P(m|c)$ & w/o $P(u|c)$ \\
\midrule
LLM RSA & $L_1$ & 0.44 (-42.7\%) & 0.78 (+1.8\%) \\
\cmidrule{2-4}
\makecell{LLM \rsatwo\\with $P_N(r|c,u)$} & $L_1$ & 0.44 (-44.8\%) & 0.80 (+0.3\%) \\
\cmidrule{2-4}
\makecell{LLM \rsatwo\\with $I(r|c,u)$} & $L_1$ & 0.51 (-39.4\%) & 0.84 (+0.2\%) \\
\bottomrule
\end{tabular}
}
\caption{Pragmatic listener probability ablations. We report the average listener posterior probabilities of the correct meaning on the test set (across all 50 scenarios) and the relative change with respect to the unablated model.}
\label{tab:probability_ablations}
\end{table}
Overall, despite RSA reasoning offering limited improvements, the results obtained with $L_0$ still suggest that explicitly modeling rhetorical strategies may help mitigate lexical-overlap and literal interpretation biases in LLMs.
\section{Conclusion}
In conclusion, we presented \rsatwo, a computational model of figurative language based on the RSA framework that explicitly incorporates rhetorical strategy to support non-literal interpretation. We show that \rsatwospace enables human-compatible interpretations of figurative utterances—including non-literal number expressions and ironic weather statements—\emph{without} explicitly modeling the speaker’s motivations behind using figurative language. We further demonstrated that combining \rsatwospace with LLMs yields state-of-the-art performance on the ironic split of the PragMega+ dataset. We hope this work inspires future research to incorporate pragmatic reasoning techniques within language technologies.
\section*{Limitations}
\paragraph{Limited dataset size and diversity.} While our experimental and theoretical results (Appendix~\ref{app:qud_rsa_special_case}) support the usefulness and applicability of \rsatwo, we believe that future work in computational pragmatics should include broader examples across languages and cultural contexts. Work such as \citet{park-etal-2024-multiprageval,sravanthi-etal-2024-pub} has begun to address these issues. Regarding \rsatwospace specifically, while we have demonstrated its applicability to irony, hyperbole, understatement, and pragmatic halo, evaluating our framework on additional pragmatic phenomena such as metaphor and politeness are promising directions for future research.
\paragraph{Alternative utterance generation.} As discussed in Section~\ref{sec:analysis_pragmega_plus}, pragmatic reasoning does not yield the typical probability mass narrowing associated with RSA due to the distribution of mass across multiple alternative utterances that effectively ``mean'' the same thing within this framework. We believe that this poses an interesting research problem for the field of computational pragmatics in particular (e.g. How do we generate relevant and non-overlapping alternative utterances?), and for natural language processing (e.g., LLM decoding) more broadly.
\paragraph{What if the rhetorical strategies are not known in advance?} While \rsatwospace addresses the challenge of explicitly modeling affect in affect-aware RSA, it introduces a new question: how to determine which rhetorical strategies are appropriate in a given context? We take a first step toward this in Appendix~\ref{app:rsc_rsa}, where we propose and test a clustering-based algorithm that automatically induces \emph{rhetorical strategy clusters}. While our algorithm underperforms LLM \rsatwo, we believe that developing a more principled approach to inducing rhetorical strategies is an important direction for future work.
\section*{Ethics Statement}
We propose \rsatwospace as a computational model of figurative language that aims to better capture human interpretations of non-literal utterances and to help LLMs interpret such utterances in a way that aligns more closely with their intended meaning. We recognize, however, that figurative language differs significantly across languages and cultures. Our model has not yet been validated on languages beyond English, or on figurative phenomena that may be less common in English. We encourage future work—similar to \citet{park-etal-2024-multiprageval}—to explore figurative language in a range of linguistic and cultural contexts beyond those commonly associated with English.
\section*{Acknowledgments}
The authors would like to thank the reviewers for their valuable comments. We would also like to thank Gaurav Kamath, Arkil Patel and Siva Reddy for their insightful comments on an earlier version of this paper. We are also extremely grateful to Justine Kao and Polina Tsvilodub for sharing the non-literal number expressions and ironic weather utterances datasets and for their helpful feedback. This work was supported by the Fonds de Recherche du Québec – Nature et Technologies (FRQNT) and the Natural Sciences and Engineering Research Council of Canada (NSERC). Jackie Chi Kit Cheung is supported by Canada CIFAR AI Chair program. We acknowledge material support from NVIDIA Corporation in the form of computational resources provided to Mila.
\bibliographystyle{acl_natbib}
\bibliography{custom}
\clearpage
\appendix
\onecolumn
\section{Additional Theoretical Results and Discussion}
\label{app:rsa_theory}
\subsection{Equivalence of RSA Formulations}
\label{app:equivalence_rsa_formulations}
We demonstrate that the pragmatic speaker formula introduced in Section~\ref{sec:standard_rsa} is equivalent to the formulation originally presented in \citet{frank_ref_game_2012}, which is given by
\begin{align}
P_{S_1}(u|m,c) \propto \exp(\alpha\cdot U(m,u,c)),
\label{eq:og_pragmatic_speaker}
\end{align}
where $U(m,u,c)$ is the utility function defined via information theory as
\begin{align}
U(m,u,c) = \log P_{L_0}(m|c,u) - \kappa(u),
\end{align}
where $\kappa: \mathcal{U} \rightarrow \mathbb{R}_{\geq 0}$ is some cost function.
Replacing $U(m,u,c)$ in Equation~\ref{eq:og_pragmatic_speaker}, this is equivalent to
\begin{equation}
P_{S_1}(u|m,c) \propto \exp(-\alpha\cdot\kappa(u)) \cdot P_{L_0}(m|c,u)^\alpha.
\label{eq:og_simplified}
\end{equation}
Firstly, we show that there exists a particular setting of $\kappa(u)$ in Equation~\ref{eq:og_simplified} for which we can recover Equation~\ref{eq:rsa_pragmatic_speaker} presented in Section~\ref{sec:standard_rsa}:
\begin{align*}
P_{S_1}(u|m,c) &\propto \exp(-\alpha\cdot\kappa(u)) \cdot P_{L_0}(m|c,u)^\alpha \\
&= \exp(\log (P(u|c)) \cdot P_{L_0}(m|c,u)^\alpha & \text{Setting }\kappa(u) = \frac{\log P(u|c)}{-\alpha}\\
&= P_{L_0}(m|c,u)^\alpha \cdot P(u|c)
\end{align*}
Secondly, we show that there exists a particular setting of $P(u|c)$ for which we can recover Equation~\ref{eq:og_simplified}:
\begin{align*}
P_{S_1}(u|m,c) &\propto P_{L_0}(m|c,u)^\alpha \cdot P(u|c) \\
&= P_{L_0}(m|c,u)^\alpha \cdot \texttt{softmax}(-\alpha\cdot\kappa(u)) & \text{Setting }P(u|c) = \texttt{softmax}(-\alpha\cdot\kappa(u)) \\
&\propto P_{L_0}(m|c,u)^\alpha \cdot \exp(-\alpha\cdot\kappa(u))
\end{align*}
This proves the equivalence.
\subsection{Standard RSA Provides Zero Probability Mass for Non-Literal Interpretations}
\label{app:standard_rsa_non_literal}
We show that in the standard RSA framework, if $\alpha > 0$, then for all $m \in \mathcal{M}$ and $u \in \mathcal{U}$, the condition $m \notin \llbracket u \rrbracket$ implies that $P_{L_1}(m|u) = 0$.
Let $\alpha > 0$ and consider $m \in \mathcal{M}$ and $u \in \mathcal{U}$ such that $m \notin \llbracket u \rrbracket$. Then, we have:
\begin{align*}
P_{L_0}(m|c,u) &\propto \mathds{1}_{m \in \llbracket u \rrbracket} \cdot P(m|c) \\
&= 0 \cdot P(m|c) \\
&= 0, \\
P_{S_1}(u|c,m) &\propto P_{L_0}(m|c,u)^\alpha \cdot P(u|c) \\
&= 0 \cdot P(u|c) \\
&= 0, \\
P_{L_1}(m|c,u) &\propto P_{S_1}(u|c,m) P(m|c) \\
&= 0 \cdot P(m|c) \\
&= 0.
\end{align*}
This proves the above statement.
\subsection{QUD-RSA is a Special Case of \rsatwo}
\label{app:qud_rsa_special_case}
We show that affect-aware RSA is a special case of \rsatwo. To do so, we use the more general formulation of affect-aware RSA, Question Under Discussion RSA (QUD-RSA) \citep{scontras2025rsa}.
In QUD-RSA, the meaning space $\mathcal{M}$ is projected to meaning subspaces $\mathcal{X}$ using projection functions $q : \mathcal{M} \rightarrow \mathcal{X}$. Typically, this is done when the meaning space has been augmented to a vector space in order to include additional meaning dimensions. For instance, in affect-aware RSA, the meaning vector space $\mathcal{M} = \mathcal{S} \times \mathcal{A}$ is broken into the state of the world being conveyed, $\mathcal{S}$, and the affect $\mathcal{A}$. In this setting, an affect projection $q_\texttt{affect} : \mathcal{S} \times \mathcal{A} \rightarrow \mathcal{A}$ might look like the following:
\begin{equation*}
q_\texttt{affect}(s,a) = a.
\end{equation*}
We first define the equations of QUD-RSA and of \rsatwospace and then show that any instance of the QUD-RSA equations can be re-written with the \rsatwospace equations, but not vice-versa.
\subsubsection{Definitions}
\textbf{Definition 1 (QUD-RSA).} Given a meaning space $\mathcal{M}$, an utterance space $\mathcal{U}$, a context space $\mathcal{C}$ and a set of $n$ projection functions $Q = \{q_i : i \in [1\dots n] \And q_i \text{ is a projection function from }\mathcal{M}\text{ to some subspace }\mathcal{X}\}$, the QUD-RSA equations $\mathcal{E}_\texttt{QUD-RSA}$, are as follows:\footnote{We overload notation here and use $q$ to represent both the latent variable which indexes its corresponding projection \emph{as well as} the projection function itself.}
%
\begin{align}
P_{L_0}(m \mid u, c) &\propto \mathds{1}_{m \in \llbracket u \rrbracket} \cdot P(m \mid c), \\
P_{L_0}(m \mid c, u, q) &\propto \sum_{m' \in \mathcal{M}} \mathds{1}_{q(m') = q(m)} \cdot P_{L_0}(m' \mid u, c), \label{eq:qud_rsa_literal}\\
P_{S_1}(u \mid c, m, q) &\propto P_{L_0}(m \mid c, u, q) \cdot P(u \mid c), \label{quds1}\\
P_{L_1}(m \mid c, u, q) &\propto P_{S_1}(u \mid c, m, q) \cdot P(m \mid c). \label{qudl1}
\end{align}
%
\vspace{0.5cm}
\noindent\textbf{Definition 2 (\rsatwo).} Given a meaning space $\mathcal{M}$, an utterance space $\mathcal{U}$, a context space $\mathcal{C}$, a rhetorical strategy space $\mathcal{R}$ and a set of rhetorical strategy functions $F_r = \{f_r: r \in \mathcal{R} \And f_r: \mathcal{M} \times \mathcal{C} \times \mathcal{U} \rightarrow [0,1]\}$, the \rsatwospace equations, $\mathcal{E}_{\rsatwo}$ are as follows:
%
\begin{align}
P_{L_0}(m \mid c, u, r) &\propto f_r(c, m, u) \cdot P(m \mid c), \label{eq:gen_rsa_pl0}\\
P_{S_1}(u \mid c, m, r) &\propto P_{L_0}(m \mid c, u, r) \cdot P(u \mid c), \\
P_{L_1}(m \mid c, u, r) &\propto P_{S_1}(u \mid c, m, r) \cdot P(m \mid c).
\end{align}
%
\subsubsection{QUD-RSA Can Be Simulated by \rsatwo}
\noindent\textbf{Lemma 1.} If $P(m \mid c) > 0$ for all $m \in \mathcal{M}, c \in \mathcal{C}$, any instance of the QUD-RSA equations can be represented as an instance of the \rsatwospace equations.
\textbf{Proof.} Given an instance of the QUD-RSA equations, $\mathcal{E}_\texttt{QUD-RSA}$, we can build an equivalent instance of the \rsatwospace equations, $\mathcal{E}_{\rsatwo}$.
We do this by setting the spaces $\mathcal{C}, \mathcal{M} \And \mathcal{U}$ in $\mathcal{E}_{\rsatwo}$ to be the same as the ones from $\mathcal{E}_\texttt{QUD-RSA}$. In addition, we define a rhetorical strategy variable $r$ and a corresponding rhetorical function $f_r$ for each projection variable and corresponding projection function $q$ of $\mathcal{E}_\texttt{QUD-RSA}$ as follows:
%
\begin{equation}
f_r(m, c, u) = \frac{ \sum_{m' \in \mathcal{M}} \mathds{1}_{q(m') = q(m)} \cdot P_{L_0}(m' \mid c, u) }{ k \cdot P(m \mid c) },
\end{equation}
%
where the division by $k = \max_{m'}f_r(c,m',u)$ is needed such that $f_r(c,m,u) \in [0,1]$.
Replacing $f_r$ in $\mathcal{E}_{\rsatwo}$'s Equation~\ref{eq:gen_rsa_pl0} enables us to recover the instance of the QUD-RSA equations, $\mathcal{E}_\texttt{QUD-RSA}$.
\vspace{0.5cm}
\subsubsection{Not All Instances of \rsatwospace Can Be Simulated by QUD-RSA}
\noindent\textbf{Lemma 2.} Given context space $\mathcal{C}$, meaning space $\mathcal{M}$, and utterance space $\mathcal{U}$, there exists an instance of the \rsatwospace equations which cannot be simulated by any instance of the QUD-RSA equations.
\textbf{Proof Idea.} The crux of this proof lies in the observation that the QUD-RSA equations can only induce literal listener distributions which are \emph{binary combinations} of meaning priors $P(m|c)$. Thus, one can pick some probability vector which is not such a combination and compute it using the rhetorical strategy function $f_r$.
\textbf{Proof.} We first demonstrate that the literal listener of any instance of the QUD-RSA equations is a binary combination of meaning priors.
Consider the normalized literal listener distribution $P_{L_0}(m \mid c, u)$ from the QUD-RSA equations:
%
\begin{equation}
P_{L_0}(m \mid c, u) =
\frac{\mathds{1}_{m \in \llbracket u \rrbracket} \cdot P(m \mid c) }
{ \sum\limits_{m''} \mathds{1}_{m'' \in \llbracket u \rrbracket} \cdot P(m'' \mid c) }.
\end{equation}
%
Replacing it in Equation~\ref{eq:qud_rsa_literal}, we get the following:
\begin{align}
P_{L_0}(m \mid c, u, q) &\propto \sum_{m'} \mathds{1}_{q(m') = q(m)} \cdot \Bigl( \frac{\mathds{1}_{m \in \llbracket u \rrbracket} \cdot P(m \mid c) }{ \sum\limits_{m''} \mathds{1}_{m'' \in \llbracket u \rrbracket} \cdot P(m'' \mid c) } \Bigr) \\
&\propto \sum_{m'} \mathds{1}_{q(m') = q(m)} \cdot \mathds{1}_{m \in \llbracket u \rrbracket} \cdot P(m \mid c) \\
&\propto \sum_{m'} \mathds{1}_{q(m') = q(m) \wedge m \in \llbracket u \rrbracket} \cdot P(m \mid c)
\end{align}
Thus, the QUD-RSA literal listeners are probability distributions induced by taking binary combinations of the meaning priors, $P(m|c)$. Since the meaning space $\mathcal{M}$ is finite, this implies that there is a finite set of distributions which the QUD-RSA literal listener can be set to. We define this set as follows:
\begin{align}
\mathcal{P_{L}}_0 = \{ p_{l_0} \in \mathbb{R}^{|\mathcal{M}|} : p_{l_0}\text{ is a QUD-RSA literal listener probability vector}\}.
\end{align}
Then, to complete this proof, we need only define a function $f_r$ such that $P_{L_0}(m \mid c, u, r) \notin \mathcal{P_{L}}_0$.
We create an $f_r$ which induces $P_{L_0}(m \mid c, u, r)$ outside of $\mathcal{P_{L}}_0$ by picking a real number $k$ such that it lies between $0$ and the minimum non-zero probability value in $\mathcal{P_{L}}_0$. That is,
%
\begin{align}
k = \frac{p_\text{min}}{2} \text{ such that } p_\text{min} = \min_{\vec{p} \in \mathcal{P_{L}}_0, \, i \in 1 \dots |\mathcal{M}|, \,\vec{p}[i] > 0} \vec{p}[i]
\end{align}
We define $f_r$ such that $P_{L_0}(m \mid c, u, r) = k$ for some $m \in \mathcal{M}$. Let $m_1, m_2 \in \mathcal{M}, m_1 \neq m_2$ and $f_r$ be defined as follows:
\[
f_r(m, c, u) =
\begin{cases}
\frac{k}{P(m_1 \mid c)} & \text{if } m = m_1 \\
\frac{1 - k}{P(m_2 \mid c)} & \text{if } m = m_2 \\
0 & \text{otherwise}
\end{cases}
\]
With this function $f_r$\footnote{$f_r$ should be divided by its maximum value to ensure it respects its $[0,1]$ co-domain constraint. We omit this step for compactness.}, we can show that $P_{L_0}(m_1 \mid c, u, r) = k$ which we have just shown is not a probability value in any of the probability vectors found in $\mathcal{P_{L}}_0$. This can be seen as follows:
\begin{align}
P_{L_0}(m_1 \mid c, u, r)
&= \frac{f_r(m_1, c, u) \cdot P(m_1 \mid c)}{\sum_{m'} f_r(m', c, u) \cdot P(m' \mid c)}, \\
&= \frac{ \frac{k}{P(m_1 \mid c)} \cdot P(m_1 \mid c) }
{ \frac{k}{P(m_1 \mid c)} \cdot P(m_1 \mid c) + \frac{1 - k}{P(m_2 \mid c)} \cdot P(m_2 \mid c) }, \\
&= \frac{k}{k + 1 - k} = k \quad \text{as desired}.
\end{align}
This shows that there exists an instance of the \rsatwospace equations which cannot be represented with QUD-RSA.
\section{Deriving Non-Literal Interpretations of Figurative Language with \rsatwo}
\label{app:affect_aware_rsa_exp}
We provide additional details regarding the ironic weather utterances experiment along with additional figures for both our non-literal number expression and ironic weather utterances experiments.
\subsection{Ironic Weather Utterances Experimental Details}
\subsubsection{Weather Contexts Images}
\label{app:weather_images}
We include the images associated with each of the 9 weather contexts in the ironic weather utterances dataset. These images were shown to human participants to elicit both prior and posterior probabilities. These can also be found in the original work by \citet{Kao2015Irony}.
\begin{figure}[H]
\centering
\includegraphics[width=\linewidth]{Images/kao_weather_images.png}
\caption{Images associated with the 9 weather contexts in the ironic weather utterances dataset.}
\label{fig:weather_images}
\end{figure}
\subsubsection{Re-Implementation Details of Affect-Aware RSA}
\label{app:reimplementation_details}
Using the ironic weather utterances dataset from \citet{Kao2015Irony}, we re-implement their affect-aware RSA model using the following equations where $s \in \mathcal{S}$ represents the conveyed state of the world (which we typically call $m$ in our notation), $a, v \in \mathcal{A}, \mathcal{V}$ represent the arousal and valence dimensions of affect respectively and where the three QUD projections used are $q_\texttt{literal}(s,a,v) = s, q_\texttt{arousal}(s,a,v) = a, q_\texttt{valence}(s,a,v) = v$:
\begin{align}
P(s,a,v|c) &= P(s|c) \cdot P(a|c) \cdot P(v|c) \\
P_{L_0}(s,a,v|c,u) &\propto P(s,a,v|c) \cdot \mathds{1}_{s \in \llbracket u \rrbracket}, \\
P_{L_0}(s,a,v|c,u,q) &\propto \sum_{(s',a',v') \in \mathcal{S} \times \mathcal{A} \times \mathcal{V}} \mathds{1}_{q(s',a',v') = q(s,a,v)} \cdot P_{L_0}(s',a',v'|c,u), \\
P_{S_1}(u|c,s,a,v,q) &\propto P_{L_0}(s,a,v|c,u,q)^\alpha \\
L_1(s,a,v|c,u,q) &= P(s,a,v|c) \cdot P_{S_1}(u|c,s,a,v,q) \label{eq:L1_def}
\end{align}
The QUD variables can be marginalized out from either listener with the following equations:
\begin{align}
P_{L_0}(s,a,v|c,u) &= \sum_{q \in Q} P_{L_0}(s,a,v|c,u,q) \cdot P(q|c) \label{eq:PL0_marginalized}, \\
P_{L_1}(s,a,v|c,u) &\propto \sum_{q \in Q} L_1(s,a,v|c,u,q) \cdot P(q|c) \label{eq:PL1_marginalized},
\end{align}
where, as described in \citet{Kao2015Irony}, we use $P(q_\texttt{literal}) = 0.3, P(q_\texttt{arousal}) = 0.4, P(q_\texttt{valence}) = 0.3$.
Finally, we can marginalize out the affect variables $a$ and $v$ from the $P_{L_i}(s,a,v|c,u)$ to get $P_{L_i}(s|c,u)$:
\begin{align}
P_{L_i}(s|c,u) = \sum_{a \in \mathcal{A}} \sum_{v \in \mathcal{V}} P_{L_i}(s,a,v|c,u) \label{eq:PLi_affect_marginalized}
\end{align}
To verify the correctness of our re-implementation, we reproduce Fig.~5 from the \citet{Kao2015Irony} paper and verify qualitatively that each context-utterance-meaning triple matches. Our reproduction of this figure can be found in Fig.~\ref{fig:reproduction_kao}.
\begin{figure}
\centering
\includegraphics[width=\linewidth]{Images/affect_aware_rsa_l1.png}
\caption{Meaning distributions, $P(m|c,u)$, for humans and for the affect-aware RSA method. This figure was originally generated by \citet{Kao2015Irony}. We reproduce it here with our re-implementation to verify the correctness of the re-implementation.}
\label{fig:reproduction_kao}
\end{figure}
\subsubsection{\rsatwospace Training Details}
To learn the rhetorical strategy function $f_r$, we trained a neural network with an architecture of $16 \times 16 \times 5$, employing sigmoid activation functions throughout. The input to the network was a $16 \times 1$ one-hot encoding vector, where the first 9 entries were reserved for the context indicator ($c_1$ to $c_9$), the next 5 entries were reserved for the utterance indicator, and the final 2 entries were reserved for the rhetorical strategy indicator. The network's output was a $5 \times 1$ vector, representing the values of $f_r$ for each of the five meanings. The use of sigmoid activations ensures that the output values are constrained within the interval $[0,1]$, thereby respecting the defined image of $f_r$. For the training process, the entire dataset was utilized as a single batch, and training proceeded for 500 epochs. We employed a strategy of saving the model that achieved the best performance, as evaluated by the validation loss. The Adam optimizer was chosen for optimization, configured with a learning rate of $0.001$ and a weight decay of $0.001$. The network was trained using a cross-entropy loss function with $P_{L_1}(m|c,u)$.
\label{app:ironic_weather_utterances_exp_details}
\subsection{Non-Literal Number Expressions}
\label{app:non_literal_number_expressions}
We show the listener meaning distributions for all context-meaning-utterances triples from the non-literal number expressions experiment. Figures \ref{fig:electric_kettle_split_0} and \ref{fig:electric_kettle_split_1} present these distributions for the electric kettle, Figures \ref{fig:laptop_split_0} and \ref{fig:laptop_split_1} present the distributions for the laptop and Figures \ref{fig:watch_split_0} and \ref{fig:watch_split_1} do the same for the watch.
\begin{figure}
\centering
\includegraphics[width=0.6\linewidth]{Images/state_probabilities_context=electric_kettle_splitidx=0.png}
\caption{Meaning probability distributions by humans along with the affect-aware pragmatic listener and the \rsatwospace literal and pragmatic listeners on utterances about the price of an electric kettle.}
\label{fig:electric_kettle_split_0}
\end{figure}
\begin{figure}
\centering
\includegraphics[width=0.6\linewidth]{Images/state_probabilities_context=electric_kettle_splitidx=1.png}
\caption{Meaning probability distributions by humans along with the affect-aware pragmatic listener and the \rsatwospace literal and pragmatic listeners on utterances about the price of an electric kettle (Continued).}
\label{fig:electric_kettle_split_1}
\end{figure}
\begin{figure}
\centering
\includegraphics[width=0.6\linewidth]{Images/state_probabilities_context=laptop_splitidx=0.png}
\caption{Meaning probability distributions by humans along with the affect-aware pragmatic listener and the \rsatwospace literal and pragmatic listeners on utterances about the price of a laptop.}
\label{fig:laptop_split_0}
\end{figure}
\begin{figure}
\centering
\includegraphics[width=0.6\linewidth]{Images/state_probabilities_context=laptop_splitidx=1.png}
\caption{Meaning probability distributions by humans along with the affect-aware pragmatic listener and the \rsatwospace literal and pragmatic listeners on utterances about the price of a laptop (Continued).}
\label{fig:laptop_split_1}
\end{figure}
\begin{figure}
\centering
\includegraphics[width=0.6\linewidth]{Images/state_probabilities_context=watch_splitidx=0.png}
\caption{Meaning probability distributions by humans along with the affect-aware pragmatic listener and the \rsatwospace literal and pragmatic listeners on utterances about the price of a watch.}
\label{fig:watch_split_0}
\end{figure}
\begin{figure}
\centering
\includegraphics[width=0.6\linewidth]{Images/state_probabilities_context=watch_splitidx=1.png}
\caption{Meaning probability distributions by humans along with the affect-aware pragmatic listener and the \rsatwospace literal and pragmatic listeners on utterances about the price of a watch (Continued).}
\label{fig:watch_split_1}
\end{figure}
\subsection{Ironic Weather Utterances Additional Results}
\label{app:ironic_weather_utterances_add_results}
Meaning probability distributions by humans, alongside the listener and pragmatic listeners of both affect-aware RSA and \rsatwospace models, are presented for each of the nine weather contexts (which are associated with Fig.~\ref{fig:weather_images}). Figures \ref{fig:Weather_context_1}, \ref{fig:Weather_context_2}, and \ref{fig:Weather_context_3} illustrate these distributions for weather contexts 1, 2, and 3 where the weather is visibly good. Figures \ref{fig:Weather_context_4}, \ref{fig:Weather_context_5}, and \ref{fig:Weather_context_6} show the distributions for weather contexts 4, 5, and 6 where the weather is neither visibly good nor bad. Finally, distributions for weather contexts 7, 8, and 9 where the weather is visibly bad are shown in Figures \ref{fig:Weather_context_7}, \ref{fig:Weather_context_8}, and \ref{fig:Weather_context_9}.
% For weather context 1 (derived from bar_plot_amazing_3.png)
\begin{figure}
\centering
\includegraphics[width=0.6\linewidth]{Images/bar_plot_amazing_3.png}
\caption{Meaning probability distributions by humans along with the listener and pragmatic listeners of both affect-aware RSA and \rsatwospace for weather context 1 from Fig.~\ref{fig:weather_images}.}
\label{fig:Weather_context_1}
\end{figure}
% For weather context 2 (derived from bar_plot_amazing_2.png)
\begin{figure}
\centering
\includegraphics[width=0.6\linewidth]{Images/bar_plot_amazing_2.png}
\caption{Meaning probability distributions by humans along with the listener and pragmatic listeners of both affect-aware RSA and \rsatwospace for weather context 2 from Fig.~\ref{fig:weather_images}.}
\label{fig:Weather_context_2}
\end{figure}
% For weather context 3 (derived from bar_plot_amazing_1.png)
\begin{figure}
\centering
\includegraphics[width=0.6\linewidth]{Images/bar_plot_amazing_1.png}
\caption{Meaning probability distributions by humans along with the listener and pragmatic listeners of both affect-aware RSA and \rsatwospace for weather context 3 from Fig.~\ref{fig:weather_images}.}
\label{fig:Weather_context_3}
\end{figure}
% For weather context 4 (derived from bar_plot_ok_6.png)
\begin{figure}
\centering
\includegraphics[width=0.6\linewidth]{Images/bar_plot_ok_6.png}
\caption{Meaning probability distributions by humans along with the listener and pragmatic listeners of both affect-aware RSA and \rsatwospace for weather context 4 from Fig.~\ref{fig:weather_images}.}
\label{fig:Weather_context_4}
\end{figure}
% For weather context 5 (derived from bar_plot_ok_5.png)
\begin{figure}
\centering
\includegraphics[width=0.6\linewidth]{Images/bar_plot_ok_5.png}
\caption{Meaning probability distributions by humans along with the listener and pragmatic listeners of both affect-aware RSA and \rsatwospace for weather context 5 from Fig.~\ref{fig:weather_images}.}
\label{fig:Weather_context_5}
\end{figure}
% For weather context 6 (derived from bar_plot_ok_4.png)
\begin{figure}
\centering
\includegraphics[width=0.6\linewidth]{Images/bar_plot_ok_4.png}
\caption{Meaning probability distributions by humans along with the listener and pragmatic listeners of both affect-aware RSA and \rsatwospace for weather context 6 from Fig.~\ref{fig:weather_images}.}
\label{fig:Weather_context_6}
\end{figure}
% For weather context 7 (derived from bar_plot_terrible_8.png)
\begin{figure}
\centering
\includegraphics[width=0.6\linewidth]{Images/bar_plot_terrible_8.png}
\caption{Meaning probability distributions by humans along with the listener and pragmatic listeners of both affect-aware RSA and \rsatwospace for weather context 7 from Fig.~\ref{fig:weather_images}.}
\label{fig:Weather_context_7}
\end{figure}
% For weather context 8 (derived from bar_plot_terrible_7.png)
\begin{figure}
\centering
\includegraphics[width=0.6\linewidth]{Images/bar_plot_terrible_7.png}
\caption{Meaning probability distributions by humans along with the listener and pragmatic listeners of both affect-aware RSA and \rsatwospace for weather context 8 from Fig.~\ref{fig:weather_images}.}
\label{fig:Weather_context_8}
\end{figure}
% For weather context 9 (derived from bar_plot_terrible_9.png)
\begin{figure}
\centering
\includegraphics[width=0.6\linewidth]{Images/bar_plot_terrible_9.png}
\caption{Meaning probability distributions by humans along with the listener and pragmatic listeners of both affect-aware RSA and \rsatwospace for weather context 9 from Fig.~\ref{fig:weather_images}.}
\label{fig:Weather_context_9}
\end{figure}
\newpage
\section{LLM Irony Interpretation with \rsatwo}
\label{app:pragmega_plus_exp}
\subsection{PragMega+ Dataset Examples}
\label{app:pragmega_more_examples}
\begin{table*}[ht]
\centering
\resizebox{\linewidth}{!}{
\begin{tabular}{p{6cm}p{6cm}p{6cm}}
\toprule
Non-literal Intended Meaning & Literal Intended Meaning & Intended Meanings \\
\midrule
In a shop, Lara tries on a dress. \textbf{The dress is far too long for her.} Lara asks Simon: ``Does this dress fit me?'' Simon answers: ``Wow! That must be custom made! It's clearly the perfect size and length for you.'' & In a shop, Lara tries on a dress. \textbf{The dress fits her perfectly.} Lara asks Simon: ``Does this dress fit me?'' Simon answers: ``Wow! That must be custom made! It's clearly the perfect size and length for you.'' & 1. The dress is fitting well. (\texttt{LM}) \newline 2. Lara needs to get a dress of a shorter length. (\texttt{NLM}) \newline 3. Simon does not like the color of this dress. (\texttt{OM}) \newline 4. Simon has to get back to work. (\texttt{NSM}) \\
\midrule
While Tom and a new acquaintance from work, Sara, were chatting at a party, they noticed a colleague across the room. \textbf{She was standing alone holding a drink and a CD.} Tom points at the girl and comments: ``The life of the party, right there.'' & While Tom and a new acquaintance from work, Sara, were chatting at a party, they noticed a colleague across the room. \textbf{She was standing at the center of a group of colleagues, holding a drink and and telling another one of her classic stories.} Tom points at the girl and comments: ``The life of the party, right there.'' & 1. Their colleague is very sociable. (\texttt{LM}) \newline 2. Their colleague is quite unsociable. (\texttt{NLM}) \newline 3. Their colleague has good taste in music. (\texttt{OM}) \newline 4. Their colleague has good taste in fashion. (\texttt{NSM}) \\
\midrule
After a long day, Bruce returns home and notices \textbf{that his kids have not cleaned their room.} Bruce says ``I love how clean your room is.'' What did Bruce want to convey? & After a long day, Bruce returns home and notices \textbf{that his kids have cleaned their room.} Bruce says ``I love how clean your room is.'' What did Bruce want to convey? & 1. Bruce is happy with how clean his kids' room is. (\texttt{LM}) \newline 2. Bruce is annoyed that his kids have not cleaned their room. (\texttt{NLM}) \newline 3. It's important to keep one's room clean. (\texttt{OM}) \newline 4. Bruce forgot to make dinner. (\texttt{NSM}) \\
\midrule
After asking his parents several times, Edward has finally received a new gaming console for his birthday. \textbf{However, even though Edward promised that he would not spend more than two hours a day gaming, his parents quickly realize that Edward has no intention of keeping that promise.} Edward's mom tells he, ``I see you are quite good at keeping your promises.'' & After asking his parents several times, Edward has finally received a new gaming console for his birthday. \textbf{Edward promised that he would not spend more than two hours a day gaming.} To their surprise, Edward's parents realize that Edward has every intention of keeping that promise. Edward's mom tells he, ``I see you are quite good at keeping your promises.'' & 1. She is disappointed that Edward has not kept he promise. (\texttt{LM}) \newline 2. Edward has kept his promise. (\texttt{NLM}) \newline 3. Spending too much time gaming is bad for one's health. (\texttt{OM}) \newline 4. She needs to buy a new remote control. (\texttt{NSM}) \\
\bottomrule
\end{tabular}
}
\caption{Examples of a scenario with a non-literal intended meaning as well as its corresponding modified scenario with a literal intended meaning in the PragMega+ dataset as well as the shared intended meanings: 1. \texttt{Literal Meaning} (\texttt{LM}) 2. \texttt{Non-Literal Meaning} (\texttt{NLM}) 3. \texttt{Overlap Meaning} (\texttt{OM}) 4. \texttt{Non-Sequitur Meaning} (\texttt{NSM}). The first two scenarios are from the original PragMega dataset while the last two were manually generated by the authors as part of the test set.}
\label{tab:pragmega_examples}
\end{table*}
\subsection{Prompt Templates}
\label{app:prompts}
We designed four prompt templates to elicit different conditional meaning distributions from language models. Each prompt targets a specific probability of interest within our framework. Figure~\ref{fig:prompt_p_mcu} elicits $P(m \mid c, u)$, Figure~\ref{fig:prompt_p_rcu} elicits $P(r \mid c, u)$, Figure~\ref{fig:prompt_p_mcur} elicits for $P(m \mid c, u, r)$, and Figure~\ref{fig:prompt_p_mc} elicits $P(m \mid c)$.
\begin{figure}[htbp]
\centering
\begin{tcolorbox}[
title={Prompt Template for $P(m|c,u)$},
colback=gray!10!white, % Light gray background
colframe=gray!50!black, % Darker gray frame
coltitle=white, % White title text
fonttitle=\bfseries, % Bold title font
boxsep=5pt, % Space between the frame and the content
arc=4pt, % Rounded corners
boxrule=0.8pt, % Thickness of the frame
left=10pt, % Left padding
right=10pt, % Right padding
top=5pt, % Top padding
bottom=5pt % Bottom padding
]
Template:
\begin{verbatim}
Task: You will read short stories that describe everyday situations.
Each story will be followed by a multiple-choice question.
Read each story and choose the best answer.
Your task is to decide what the character in the story is trying to convey.
The answer options are 1, 2, 3, or 4.
[scenario]
What meaning is X likely conveying?
[options]
Answer:
\end{verbatim}
Sample:
\begin{verbatim}
Task: You will read short stories that describe everyday situations.
Each story will be followed by a multiple-choice question.
Read each story and choose the best answer.
Your task is to decide what the character in the story is trying to convey.
The answer options are 1, 2, 3, or 4.
John is a teacher at an elementary school. When talking with the
principal about a new student, who did poorly on her entrance
examination, John said, ``This one is really sharp.''
What meaning is John trying to convey?
1. The pencils need to be sharpened.
2. The student is smart.
3. The student is not very clever.
4. The entrance exam is unfair.
Answer:
\end{verbatim}
\end{tcolorbox}
\caption{Prompt template for $P(m|c,u)$ where [scenario] is replaced with one of the (ironic or literal) scenarios from the PragMega dataset with the utterance and [options] are the 4 possible intended meanings for this particular scenario. $X$ is replaced with the name of the speaker in the scenario which is extracted using a regular expression.}
\label{fig:prompt_p_mcu}
\end{figure}
\begin{figure}[htbp]
\centering
\begin{tcolorbox}[
title={Prompt Template for $P(r|c,u)$},
colback=gray!10!white, % Light gray background
colframe=gray!50!black, % Darker gray frame
coltitle=white, % White title text
fonttitle=\bfseries, % Bold title font
boxsep=5pt, % Space between the frame and the content
arc=4pt, % Rounded corners
boxrule=0.8pt, % Thickness of the frame
left=10pt, % Left padding
right=10pt, % Right padding
top=5pt, % Top padding
bottom=5pt % Bottom padding
]
Template:
\begin{verbatim}
Task: You will read short stories that describe everyday situations.
Each story will be followed by a multiple-choice question.
Read each story and choose the best answer.
Your task is to decide, given the situation and what the character has said,
whether the character in the story is being "Sincere" or "Not Sincere".
A character is "Sincere" if what they are saying is consistent with the
context of the story. For instance, if the weather outside is
sunny then a "Sincere" utterance might be "The weather
outside is amazing." Likewise, if the weather outside is rainy then
a "Sincere" utterance might be "The weather is terrible."
In contrast, a character is "Not Sincere" if they are saying something
that contradicts the context of the story. For instance, if the weather
outside is rainy then a "Not Sincere" utterance might be "The weather
outside is amazing." Similarly, if the weather outside is sunny then
a "Not Sincere" utterance might be "The weather outside is terrible."
The answer options are 1 or 2.
[scenario]
Was X being sincere or not sincere?
[options]
Answer:
\end{verbatim}
Sample:
\begin{verbatim}
Task: You will read short stories that describe everyday situations.
Each story will be followed by a multiple-choice question.
Read each story and choose the best answer.
Your task is to ... [see above].
The answer options are 1 or 2.
John is a teacher at an elementary school. When talking with the
principal about a new student, who did poorly on her entrance
examination, John said, ``This one is really sharp.''
Was John being sincere or not sincere?
1. Sincere
2. Not Sincere
Answer:
\end{verbatim}
\end{tcolorbox}
\caption{Prompt template for $P(r|c,u)$ where [scenario] is replaced with one of the (ironic or literal) scenarios from the PragMega dataset with the utterance and [options] are either sincere or not sincere. $X$ is replaced with the name of the speaker in the scenario which is extracted using a regular expression. We used the ``Sincere''/``Not sincere'' terminology because we found that the term ``Irony'' would hurt performance.}
\label{fig:prompt_p_rcu}
\end{figure}
% \begin{figure}[H]
% \begin{framed}
% \begin{quote}
% Task: You will read short stories that describe everyday situations. Each story will be followed by a multiple-choice question. Read each story and choose the best answer. Your task is to decide whether the character is being literal or ironic. The answer options are 1 or 2.
% \newline\newline
% [scenario] Was $X$ being literal or ironic?
% \newline\newline
% [options]
% \newline\newline
% Answer:
% \end{quote}
% \end{framed}
% \end{figure}
\begin{figure}[htbp]
\centering
\begin{tcolorbox}[
title={Prompt Template for $P(m|c,u,r)$},
colback=gray!10!white,
colframe=gray!50!black,
coltitle=white,
fonttitle=\bfseries,
boxsep=5pt,
arc=4pt,
boxrule=0.8pt,
left=10pt,
right=10pt,
top=5pt,
bottom=5pt
]
Template:
\begin{verbatim}
Task: You will read short stories that describe everyday situations and which
finish with a character saying something. Your task is to decide, given the
situation and what the character has said, what meaning the character is trying
to convey. Each story will be followed by 4 possible meaning interpretations
listed from 1 to 4. Read each story and choose the number corresponding to the
best meaning interpretation. You can only answer with 1, 2, 3, or 4.
\end{verbatim}
If sincere:
\begin{verbatim}
Assume that the character is saying exactly what they want to convey literally
when choosing the character's true intended meaning **even if** it contradicts
the context. For instance, if it's a sunny day and the character says "The
weather is terrible." then they do actually mean that the weather is terrible.
Similarly, if it's a rainy day and the character says "The weather is amazing."
then they do actually mean that the weather is amazing.
\end{verbatim}
If not sincere:
\begin{verbatim}
In addition, when choosing the intended meaning, assume that the character is
saying the opposite of what they want to convey. For instance, if they say "The
weather is terrible." then they actually mean that the weather is amazing.
Similarly, if they say "The weather is amazing." then they actually mean that
the weather is terrible.
\end{verbatim}
\begin{verbatim}
[scenario]
What meaning is X trying to convey?
[options]
Answer:
\end{verbatim}
\end{tcolorbox}
\caption{Prompt template for $P(m|c,u,r)$ where [scenario] is replaced
with one of the (ironic or literal) scenarios from the PragMega dataset
with the utterance and [options] are the 4 possible intended
meanings for this particular scenario. $X$ is replaced with the name of
the speaker in the scenario which is extracted using a regular expression.}
\label{fig:prompt_p_mcur}
\end{figure}
\begin{figure}[htbp]
\centering
\begin{tcolorbox}[
title={Prompt Template for $P(m|c)$},
colback=gray!10!white, % Light gray background
colframe=gray!50!black, % Darker gray frame
coltitle=white, % White title text
fonttitle=\bfseries, % Bold title font
boxsep=5pt, % Space between the frame and the content
arc=4pt, % Rounded corners
boxrule=0.8pt, % Thickness of the frame
left=10pt, % Left padding
right=10pt, % Right padding
top=5pt, % Top padding
bottom=5pt % Bottom padding
]
Template:
\begin{verbatim}
Task: You will read short stories that describe everyday situations
and which finish with a character saying something. Your task is to
decide, given the situation, what meaning the character is most
likely conveying. Each story will be followed by 4 possible meaning
interpretations listed from 1 to 4. Read each story and choose the
number corresponding to the most likely meaning. You can only answer
with 1, 2, 3, or 4.
[scenario]
What meaning is X likely conveying?
[options]
Answer:
\end{verbatim}
Sample:
\begin{verbatim}
Task: You will read short stories that describe everyday situations
and which finish with a character saying something. Your task is ... [see above]
John is a teacher at an elementary school. When talking with the
principal about a new student, who did poorly on her entrance
examination, John said something.
What meaning is John likely conveying?
1. The pencils need to be sharpened.
2. The student is smart.
3. The student is not very clever.
4. The entrance exam is unfair.
Answer:
\end{verbatim}
\end{tcolorbox}
\caption{Prompt template for $P(m|c)$ where [scenario] is replaced with one of the (ironic or literal) scenarios from the PragMega dataset without the utterance and [options] are the 4 possible intended meanings for this particular scenario. $X$ is replaced with the name of the speaker in the scenario which is extracted using a regular expression.}
\label{fig:prompt_p_mc}
\end{figure}
\newpage
\subsection{Additional Results}
\begin{table}[h]
\centering
\resizebox{\linewidth}{!}{
\begin{tabular}{p{0.22\textwidth} % First column for descriptive text
>{\centering\arraybackslash}p{0.18\textwidth} % Centered paragraph for math headers
>{\centering\arraybackslash}p{0.18\textwidth}
>{\centering\arraybackslash}p{0.18\textwidth}
>{\centering\arraybackslash}p{0.18\textwidth}}
\toprule
& \makecell[c]{$P_{L_0}(m = \text{\texttt{NLM}} \mid$ \\ $c,u,r=\text{\textit{irony}})$}
& \makecell[c]{$P_{L_1}(m = \text{\texttt{NLM}} \mid$ \\ $c,u,r=\text{\textit{irony}})$}
& \makecell[c]{$P_{L_0}(m = \text{\texttt{LM}} \mid$ \\ $c,u,r=\text{\textit{literal}})$}
& \makecell[c]{$P_{L_1}(m = \text{\texttt{LM}} \mid$ \\ $c,u,r=\text{\textit{literal}})$} \\
\midrule
\makecell[c]{Scenarios where the\\intended meaning is\\non-literal} & 0.92 & 0.94 & 0.47 & 0.62 \\
\makecell[c]{Scenarios where the\\intended meaning is\\literal} & 0.16 & 0.59 & 0.95 & 0.88 \\
\bottomrule
\end{tabular}
}
\caption{Average listener probability distributions \emph{conditioned on} the \textit{irony} and \textit{literal} rhetorical strategies on both the ironic split (i.e., the split in which the intended meaning is non-literal) and the literal split (i.e., the split in which the intended meaning is literal).}
\label{tab:listeners_by_rs}
\end{table}
\begin{table}[h]
\centering
\begin{tabular}{lcc}
\toprule
& $P_N(r = \textit{irony} \mid c, u)$
& $P_N(r = \textit{literal} \mid c, u)$ \\
\midrule
\makecell[c]{Scenarios where the\\intended meaning is\\non-literal} & 0.88 & 0.12 \\
\makecell[c]{Scenarios where the\\intended meaning is\\literal} & 0.45 & 0.55 \\
\bottomrule
\end{tabular}
\caption{Rhetorical strategy posteriors $P_N(r|c,u)$ on both the ironic split (i.e., the split in which the intended meaning is non-literal) and the literal split (i.e., the split in which the intended meaning is literal).}
\label{tab:p_r_c_u}
\end{table}
\newpage
\twocolumn
\section{\rsatwolearnedspace: A Rhetorical Strategy Clustering Algorithm}
\label{app:rsc_rsa}
\begin{algorithm}
\caption{\rsatwolearned}\label{alg:learned_rsa}
\begin{algorithmic}[1]
\Ensure Context $c$, Meaning $m$, Utterance $u$
\Require Set of induced rhetorical function values $F(c, m, u) = \{f_1(c, m, u), \dots, f_n(c, m, u)\}$
\Comment{Initialize the following objects.}
\State LLMs $M$ and $G$
\State Embedding function $\texttt{embed}: \mathcal{U} \to \mathbb{R}^n$
\State K-means clustering function $\texttt{k-means}: \mathcal{P}(\mathbb{R}^n) \to \mathcal{P}(\mathbb{R}^n)$
\State Cosine similarity $\texttt{cosine-sim}: \mathbb{R}^n \times \mathbb{R}^n \to [0,1]$
\Comment{Generate, embed and cluster alternative utterances.}
\State $\mathcal{U}_\text{alt} \leftarrow \{ u \} \cup \{ u_i \sim P_G(u|c) \}$
\State $E \leftarrow \{ \texttt{embed}(u') : u' \in \mathcal{U}_\text{alt} \}$
\State $\mathcal{X} \leftarrow \texttt{k-means}(E)$
\Comment{Compute rhetorical function value for each cluster.}
\State $F(c, m, u) \leftarrow \emptyset$
\For{$r \in \mathcal{X}$ }
\State $\mathcal{U}_r \leftarrow \{u' \in \mathcal{X}_\text{alt} : u' \in \text{ cluster }r\}$
\Comment{Compute cluster meaning probability and weight it by the utterance distance to the centroid.}
\State $p_{mc}\! \leftarrow \! \frac{\sum_{u' \in \mathcal{U}_r}P_{M}(m|c,u')}{\sum_{m' \in \mathcal{M}}\sum_{u' \in \mathcal{U}_r}P_{M}(m'|c,u')}$
\State $f_r(c, m, u)\! \leftarrow \! \frac{p_{mc}}{P_M(m|c,u)}$
\State Append $f_r(c, m,u)$ to $F(c, m, u)$
\EndFor
\State \Return $F(c, m, u)$
\end{algorithmic}
\end{algorithm}
We present our clustering algorithm, \textbf{R}hetorical \textbf{S}trategy \textbf{C}luster RSA \rsatwolearned, which attempts to automatically \emph{induce} the most salient rhetorical strategies and their corresponding rhetorical functions. To do so, we rely on the intuition that utterances which are generated with the same intended meaning and rhetorical strategy are likely to be semantically similar. For example, the utterances ``The weather is amazing.'' and ``Gosh, this weather is so great!'', uttered in the context of bad weather, are likely to both employ the same rhetorical strategy (irony) to convey the same intended meaning (that the weather is in fact terrible). The algorithm which computes the set of induced rhetorical function values, $F(c, m, u) = \{f_1(c, m, u), \dots, f_n(c, m, u)\}$, generates alternative utterances using a base LLM, embeds them and clusters them with k-means to induce \emph{rhetorical strategy clusters} which act as proxies of prototypical rhetorical strategies. The full procedure is presented in Algorithm~\ref{alg:learned_rsa}.
In brief, the \rsatwolearnedspace algorithm generates, embeds and clusters a set of alternative utterances which could have occurred in context $c$ to create the rhetorical strategy clusters $\mathcal{X}$. Using the generated clusters $\mathcal{X}$, \rsatwolearnedspace computes the corresponding rhetorical function values, $f_r(c, m, u)$, for each cluster $r$ for a given $c,m,u$ triple. To do so, \rsatwolearnedspace uses a formula which averages the meaning probabilities of all the utterances in that cluster and divides them by $P_M(m|c,u)$. In this way, the induced rhetorical strategy values parallel those from Equation~\ref{eq:what_is_frcmu} in that they return a ratio of two probabilities. The equations for computing $f_r$ are as follows:%
\begin{align}
p_{mc} &= \frac{\sum_{u' \in \mathcal{U}_r}P_{M}(m|c,u')}{\sum_{m' \in \mathcal{M}}\sum_{u' \in \mathcal{U}_r}P_{M}(m'|c,u')}, \\
f_r(c,m,u) &= \frac{p_{mc}}{P_M(m|c,u)}.
\end{align}%
These values are then used to compute $P_{L_0}(m|c,u,r)$ as defined in Equation~\ref{eqn:rsatwol0}. To marginalize across rhetorical strategy clusters, we use the relative cluster size, i.e. $P_{R|CU}(r|c,u) = \frac{|\mathcal{U}_r|}{|\mathcal{U}|}$.
If the alternative utterances for $u = \text{``The weather is amazing.''}$ include $u_1 = \text{``Gosh, this weather is so great!''}$, $u_2 = \text{``The weather is terrible.''}$ and $u_3 = \text{``The weather is so bad.''}$, then we expect for the embeddings of $u$ and $u_1$ to be in one cluster and for the embeddings of $u_2$ and $u_3$ to be in another cluster. These two rhetorical strategy clusters would then resemble the \emph{ironic} and \emph{literal} rhetorical strategies from Section~\ref{sec:rsatwooriginal} and their \rsatwolearned-derived rhetorical function values would approximate those of $f_\textit{irony}$ for $u$ and $u_1$ and of $f_\textit{literal}$ for $u_2$ and $u_3$.
\begin{table*}[ht]
\centering
\resizebox{\linewidth}{!}{
\begin{tabular}{>{\raggedright\arraybackslash}p{0.25\textwidth}
>{\centering\arraybackslash}p{0.22\textwidth}
>{\centering\arraybackslash}p{0.22\textwidth}
>{\centering\arraybackslash}p{0.22\textwidth}}
\toprule
Listener &
\makecell[c]{Average $P(m|c,u)$ \\ across all scenarios} &
\makecell[c]{Average $P(m|c,u)$ \\ across ironic scenarios} &
\makecell[c]{Average $P(m|c,u)$ \\ across literal scenarios} \\
\midrule
RSA-RSC – $L_0$ with $P(m|c)$ & 0.59 & 0.91 & 0.28 \\
RSA-RSC – $L_1$ with $P(m|c)$ & 0.66 & 0.91 & 0.41 \\
RSA-RSC – $L_0$ without $P(m|c)$ & 0.27 & 0.50 & 0.034 \\
RSA-RSC – $L_1$ without $P(m|c)$ & 0.28 & 0.52 & 0.036 \\
\bottomrule
\end{tabular}
}
\caption{Average $P(m|c,u)$ for different RSA-RSC listener models.}
\label{tab:rsa_rsc_clustering_performance}
\end{table*}
\subsection{\rsatwolearnedspace Implementation Details}
To implement the embedding and clustering procedures of \rsatwolearned, we utilized the Sentence-BERT architecture \citep{reimers-gurevych-2019-sentence} and the k-means clustering algorithm from scikit-learn \citep{sklearn_api}. The k-means algorithm was executed with 10 initializations, using default settings from scikit-learn.
\subsection{Clustering Results}
We experimented with 2, 4, and 8 clusters yielding no significant difference. As a result, we report results of the \rsatwolearnedspace clustering method with 4 clusters with and without the meaning prior, $P(m|c)$, in Table~\ref{tab:rsa_rsc_clustering_performance}. We see that across all listeners and scenario types, the performance is largely driven by the meaning prior $P(m|c)$ with the performance on the literal scenarios being worse than random. We encourage future work to improve upon our initial attempt at automatically uncovering the most salient rhetorical strategies in a given context and leveraging them within the \rsatwospace framework.
\end{document}
% This must be in the first 5 lines to tell arXiv to use pdfLaTeX, which is strongly recommended.
\pdfoutput=1
% In particular, the hyperref package requires pdfLaTeX in order to break URLs across lines.
\documentclass[11pt]{article}
% Change "review" to "final" to generate the final (sometimes called camera-ready) version.
% Change to "preprint" to generate a non-anonymous version with page numbers.
\usepackage{acl}
% \usepackage{acl}
% Standard package includes
\usepackage{times}
\usepackage{latexsym}
\usepackage[T1]{fontenc}
\usepackage[utf8]{inputenc}
\usepackage{microtype}
\usepackage{inconsolata}
\usepackage{graphicx}
\usepackage{amsmath}
\usepackage{booktabs}
\usepackage{lipsum}
\usepackage[caption=false]{subfig}
\usepackage{float}
\usepackage{multirow}
\usepackage{amsmath}
\usepackage{changepage}
\usepackage{natbib}
\usepackage{amsmath}
\usepackage{stmaryrd}
\usepackage{dsfont}
\usepackage{geometry} % Optional, to ensure full-page width
\usepackage{changepage}
\usepackage{makecell}
\usepackage{listings} % Import the listings package
\usepackage{framed}
\usepackage{caption}
\usepackage{subcaption}
\lstset{
breaklines=true, % Enable line wrapping
basicstyle=\ttfamily, % Use monospace font
frame=single, % Add a frame around the code block
}
\DeclareMathOperator*{\argmax}{arg\,max}
\usepackage[]{todonotes} % use option "disable" to suppress notes
\makeatletter
\newcommand*\iftodonotes{\if@todonotes@disabled\expandafter\@secondoftwo\else\expandafter\@firstoftwo\fi}
\makeatother
\newcommand{\noindentaftertodo}{\iftodonotes{\noindent}{}\ignorespaces}
\newcommand{\david}[1]{{\color{purple}[David: #1]}}
\newcommand{\ces}[1]{{\color{red}[Ces: #1]}}
\newcommand{\rsatwo}{\textbf{(RSA)}\textsuperscript{2}}
\newcommand{\rsatwospace}{\rsatwo\,\,}
\newcommand{\response}[1]{\vspace{3pt}\hrule\vspace{3pt}\textbf{#1:}}
\interfootnotelinepenalty=10000
\usepackage[dvipsnames]{xcolor}
\definecolor{googlegreen}{RGB}{56, 118, 29}
\definecolor{googleblue}{RGB}{17, 85, 204}
\title{\rsatwo: A Rhetorical-Strategy-Aware Rational Speech Act Framework for Figurative Language Understanding}
\author{%
\bf Cesare Spinoso-Di Piano\textsuperscript{1,2} \quad David Austin\textsuperscript{1,2} \\
\bf \textbf{Pablo Piantanida}\textsuperscript{2,3,4} \quad \textbf{Jackie Chi Kit Cheung}\textsuperscript{1,2,5} \vspace{1mm}\\
\textsuperscript{1}McGill University, \quad \textsuperscript{2}Mila - Quebec AI Institute \\
\textsuperscript{3}International Laboratory on Learning Systems (ILLS)\\ %\quad \textsuperscript{4}Université Paris-Saclay, \quad \textsuperscript{5}\'Ecole de technologie sup\'erieure (ETS) \\
\textsuperscript{4}CNRS, CentraleSup\'elec - Université Paris-Saclay, \quad \textsuperscript{5}Canada CIFAR AI Chair, Mila \vspace{1mm}\\
\texttt{\{cesare.spinoso,david.austin,pablo.piantanida,cheungja\}@mila.quebec}\\
}
% \author{
% Cesare Spinoso-Di Piano and David Austin\\
% \texttt{\{cesare.spinoso-dipiano,david.e.austin\}@mail.mcgill.ca}\\
% Mila – Quebec Artificial Intelligence Institute \\
% McGill University \\
% }
\begin{document}
\maketitle
%%%%%%%%%%% OUTLINE %%%%%%%%%%%%%%%
% Outline of revisions based on reviewer feedback
% 1. Integrate work from QUD-RSA (R1,R2) and discuss mathematical connections/differences as requested by the reviewers (R1,R2,R3)
% 2. Include experimental results (R2)
% 3. Mention the connection of our idea (i.e., the rhetorical strategy function) with the lexical uncertainty function (R3)
%%%%%%%%%%%%%%%%%%%%%%%%%%
\section{Introduction}
\begin{figure*}[t]
\centering
\begin{minipage}[t]{0.48\textwidth}
\centering
\includegraphics[width=\linewidth]{Images/screenshot_electric_kettle.png}
\captionof{figure}{%
Meaning probability distributions from human judgments along with the listeners of both affect-aware RSA and \protect\rsatwospace for the utterance ``The electric kettle costs 1001 dollars''.
}
\label{fig:the_kettle}
\end{minipage}
\hfill
\begin{minipage}[t]{0.48\textwidth}
\centering
\includegraphics[width=\linewidth]{Images/screenshot_the_weather_is_amazing.png}
\captionof{figure}{%
Meaning probability distributions from human judgments along with the listeners of both affect-aware RSA and \protect\rsatwospace for the utterance ``The weather is amazing.'' in the context of a blizzard.
}
\label{fig:the_weather_is_amazing}
\end{minipage}
\end{figure*}
% Paragraph 1: Figurative uses of language are ubiquitous/critical part of human communication. Provide some examples - use a weather example to prepare the reader for the running example.
Figurative language, in which expressions are meant to be interpreted non-literally, is ubiquitous in human communication. For example, speakers will often generate utterances which are literally incompatible with the world but which still convey meaning. While the utterance ``It is so nice outside.'' may appear infelicitous when uttered during a blizzard\footnote{Blizzards are not typically referred to as being ``nice''.}, it may still carry meaning if interpreted ironically. As a result, it is critical to be able to account for figurative uses of language --- predicting its use and deciphering its intended meaning --- when designing computational models of human language and communication.
% Paragraph 2: Since it's so ubiquitous, figurative use of language should be modeled - Most common approach to doing so is RSA, but it cannot account for non-literal language
One of the most popular computational approaches to modeling human communication is the Rational Speech Acts (RSA) framework \citep{frank_ref_game_2012}. In RSA, an utterance's meaning is interpreted probabilistically by positing the existence of three communicative agents which reason about each other recursively. First, the \emph{literal listener}, denoted $P_{L_0}(m|c,u)$, which reasons about the literal meaning, $m$, of an utterance $u$ produced in a context $c$. Second, the \emph{pragmatic speaker}, $P_{S_1}(u|c,m)$, which computes an utterance's expected utility based on its ability to convey the desired meaning to the literal listener. Third, the \emph{pragmatic listener}, $P_{L_1}(m|c,u)$, which interprets the intended meaning of an utterance by reasoning about the pragmatic speaker's likelihood of generating it. Altogether, these three layers of recursive reasoning have been shown to be effective at reducing a listener's uncertainty regarding the interpretation of literally ambiguous utterances such as ``some'' in scalar implicatures.
% Paragraph 2.5: Continuation in more detail of paragraph 2
While RSA offers a way to refine the interpretation of literally ambiguous utterances, this framework cannot handle non-literal interpretations of utterances. In particular, it can be shown that interpretations of utterances which are not compatible with the literal meaning of an utterance will be assigned zero probability by the pragmatic listener.
% Paragraph 3: Well if RSA can't do it, how else are we supposed to model figurative language? Describe affect-aware RSA.
As a result, efforts to model human use of figurative language have sought to address this limitation by incorporating the affect of an utterance within the RSA framework. In \citet{kao2014nonliteral,kao2015let}, the authors propose an affect-aware RSA formulation where an utterance's communicative goal is extended to include both the literal information it conveys as well as the affect it might be communicating (e.g., joy, annoyance, etc.). By explicitly modeling the connection between meaning and affect, the authors are able to assign non-zero probability mass to non-literal meanings because of the association between those meanings and a modeled affect.
\section{The Rhetorical-Strategy-Aware RSA Framework}
% Paragraph 4: This approach may not be suitable for some theoretical reasons listed here: Don't always consider that a speaker is using figurative language to communicate affect
While affect-aware RSA is successful in enabling the RSA framework to obtain non-literal interpretations of utterances, it results in a framework which we believe too tightly couples the notion of non-literal meaning and affect. As a result, we introduce a rhetorical-strategy-aware RSA framework, \rsatwo, where a speaker's rhetorical strategy is explicitly modeled and used to interpret an utterance's potentially non-literal intended meaning \citep{spinoso2025rsatwo}. To do so, we define a rhetorical strategy variable $r$ along with a corresponding rhetorical function $f_r(c,m,u)$ which generalizes the RSA indicator function $\mathds{1}_{m \in \llbracket u \rrbracket}$. Thus, for example, the \emph{ironic} rhetorical strategy function, $f_r(c,m,u)$, might return $1$ if the literal meaning of $u$ is the opposite of $m$. Crucially, \rsatwospace enables a pragmatic listener to infer non-literal interpretations of utterances \emph{without} having to model an utterance's intended affect. The \rsatwospace equations are the following:%
\begin{align}%
\hspace{-1mm}P_{L_0}(m|c,u,r) &\propto f_r(c,m,u) \cdot P(m|c), \\\label{eqn:rsatwol0}
\hspace{-1mm}P_{S_1}(u|m,c,r) &\propto P_{L_0}(m|c,u,r) \cdot P(u|c), \\
\hspace{-1mm}P_{L_1}(m|c,u,r) &\propto P_{S_1}(u|m,c,r) \cdot P(m|c), \\
\hspace{-5mm}P_{L_i}(m|c,u) = \sum_{r'} &P_{L_i}(m|c,u,r') \cdot P(r'|c,u).
\end{align}%
%
\section{Experimental Results}
% Paragraph 6: Discuss the weather setting and contrast your approach with Kao
We evaluate our formulation on two previously studied settings: non-literal number price expressions (e.g., ``This kettle costs 1001\$.'') \citep{kao2014nonliteral} and ironic weather utterances (e.g., ``The weather is amazing.'' during a winter blizzard) \citep{kao2015let}. We provide plots for the human, affect-aware RSA and \rsatwospace meaning distributions on both settings in Figures~\ref{fig:the_kettle}~and~\ref{fig:the_weather_is_amazing}. In addition, we evaluate predictive accuracy by computing the mean absolute difference (MAD) between the human-derived meaning probability distributions and those generated by the listeners in both affect-aware RSA and \rsatwospace (Table~\ref{tab:mad_numbers}). We note that \rsatwospace is competitive with affect-aware RSA, surpassing it on the ironic weather utterances dataset. Overall, we believe these results demonstrate that \rsatwospace can induce listener meaning distributions for figurative language that are at least as compatible with human interpretations as those produced by existing affect-aware RSA models.
\begin{table}[h]
\centering
\resizebox{\linewidth}{!}{
\begin{tabular}{clcc}
\toprule
Model & $L_i$ & Non-literal Numbers \(\downarrow\) & Weather Utterances \(\downarrow\) \\
\midrule
\multirowcell{2}{Affect-Aware \\ RSA} & $L_0$ & - & 0.2377 \\
& $L_1$ & \textbf{0.0436} & 0.1278 \\
\cmidrule{2-4}
\multirowcell{2}{\rsatwo} & $L_0$ & 0.0438 & 0.1647 \\
& $L_1$ & 0.0467 & \textbf{0.1229} \\
\bottomrule
\end{tabular}
}
\caption{Mean absolute differences between listener meaning distributions, $P_{L_i}(m|c,u), i = 0,1$, and the human posterior, $P(m|c,u)$, for both affect-aware and \rsatwospace on both the non-literal numbers and ironic weather utterances datasets.}
\label{tab:mad_numbers}
\end{table}
\section{Conclusion}
% Conclusion plus future work (3 sentences)
In conclusion, we presented a rhetorical-strategy-aware formulation of the RSA framework and showed that it is able to achieve human-compatible non-literal interpretations of utterances \emph{without} the need to explicitly motivate its use through affect. Future work in this direction will involve expanding the applicability of \rsatwospace to other figurative language phenomena and integrating the formulation of \rsatwospace in modern language technologies such as large language models which we begin to do in \citet{spinoso2025rsatwo}.
\vspace{-0.5cm}
\bibliographystyle{acl_natbib}
\bibliography{custom}
\end{document}
% For instance, under this framework, the interpretation of the literally ambiguous utterance ``Bob ate some of the apples.'' ``narrows'' is refined to mean ``Bob ate some, but not all, of the apples.''
% [list the listeners here]
% Applying the RSA equations and marginalizing over all possible rhetorical strategies using a uniform prior we obtain a pragmatic listener which assigns non-zero probability mass to non-literal interpretations of weather-related utterances.
% To compare our approach with that of Kao et al, we consider the interpretation of the utterance ``the weather is terrible'' under two contexts: one where the weather is truly amazing (\texttt{amazing weather}) and another where the weather is truly bad (\texttt{bad weather}). We consider the same alternative utterances from the previous study: ``The weather is terrible'', ``The weather is bad.'', ``The weather is ok.'', ``The weather is good.'', ``The weather is amazing.'' as well as the same \emph{intended} meanings a speaker wishes to convey: \texttt{terrible}, \texttt{bad}, \texttt{ok}, \texttt{good}, \texttt{amazing}. Our results are displayed in Figure 1. We see that our modified RSA version is able to assign non-literal interpretations to utterances by only reasoning about the employed rhetorical strategy.
% Paragraph 7: Describe the Figure as the "main result"
% Paragraph 7: List contributions + extra things that are typically done in an abstract submission (ask Jackie)
\begin{thebibliography}{37}
\expandafter\ifx\csname natexlab\endcsname\relax\def\natexlab#1{#1}\fi
\bibitem[{AI@Meta(2024)}]{llama3modelcard}
AI@Meta. 2024.
\newblock \href {https://github.com/meta-llama/llama3/blob/main/MODEL_CARD.md} {Llama 3 model card}.
\bibitem[{Andreas and Klein(2016)}]{andreas_reasoning_2016}
Jacob Andreas and Dan Klein. 2016.
\newblock \href {https://doi.org/10.18653/v1/D16-1125} {Reasoning about {Pragmatics} with {Neural} {Listeners} and {Speakers}}.
\newblock In \emph{Proceedings of the 2016 {Conference} on {Empirical} {Methods} in {Natural} {Language} {Processing}}, pages 1173--1182, Austin, Texas. Association for Computational Linguistics.
\bibitem[{Balestrucci et~al.(2024)Balestrucci, Casola, Lo, Basile, and Mazzei}]{balestrucci-etal-2024-im}
Pier~Felice Balestrucci, Silvia Casola, Soda~Marem Lo, Valerio Basile, and Alessandro Mazzei. 2024.
\newblock \href {https://doi.org/10.18653/v1/2024.findings-emnlp.847} {{I}{'}m sure you{'}re a real scholar yourself: Exploring ironic content generation by large language models}.
\newblock In \emph{Findings of the Association for Computational Linguistics: EMNLP 2024}, pages 14480--14494, Miami, Florida, USA. Association for Computational Linguistics.
\bibitem[{Bergen et~al.(2016)Bergen, Levy, and Goodman}]{bergen2016pragmatic}
Leon Bergen, Roger Levy, and Noah Goodman. 2016.
\newblock Pragmatic reasoning through semantic inference.
\newblock \emph{Semantics and Pragmatics}, 9:20--1.
\bibitem[{Bertin et~al.(2016)Bertin, Atanassova, Sugimoto, and Lariviere}]{bertin2016linguistic}
Marc Bertin, Iana Atanassova, Cassidy~R Sugimoto, and Vincent Lariviere. 2016.
\newblock The linguistic patterns and rhetorical structure of citation context: an approach using n-grams.
\newblock \emph{Scientometrics}, 109:1417--1434.
\bibitem[{Buitinck et~al.(2013)Buitinck, Louppe, Blondel, Pedregosa, Mueller, Grisel, Niculae, Prettenhofer, Gramfort, Grobler, Layton, VanderPlas, Joly, Holt, and Varoquaux}]{sklearn_api}
Lars Buitinck, Gilles Louppe, Mathieu Blondel, Fabian Pedregosa, Andreas Mueller, Olivier Grisel, Vlad Niculae, Peter Prettenhofer, Alexandre Gramfort, Jaques Grobler, Robert Layton, Jake VanderPlas, Arnaud Joly, Brian Holt, and Ga{\"{e}}l Varoquaux. 2013.
\newblock {API} design for machine learning software: experiences from the scikit-learn project.
\newblock In \emph{ECML PKDD Workshop: Languages for Data Mining and Machine Learning}, pages 108--122.
\bibitem[{Carenini et~al.(2024)Carenini, Bischetti, Schaeken, and Bambini}]{careniniFullyInterpretableMore2024}
Gaia Carenini, Luca Bischetti, Walter Schaeken, and Valentina Bambini. 2024.
\newblock \href {https://doi.org/10.48550/ARXIV.2404.02983} {Towards a {Fully} {Interpretable} and {More} {Scalable} {RSA} {Model} for {Metaphor} {Understanding}}.
\newblock Publisher: arXiv Version Number: 1.
\bibitem[{Cohn-Gordon and Goodman(2019)}]{cohn-gordonLostMachineTranslation2019}
Reuben Cohn-Gordon and Noah Goodman. 2019.
\newblock \href {http://arxiv.org/abs/1902.09514} {Lost in {Machine} {Translation}: {A} {Method} to {Reduce} {Meaning} {Loss}}.
\newblock ArXiv:1902.09514 [cs].
\bibitem[{Colston(1997)}]{colstonSaltingWoundSugaring1997}
Herbert~L. Colston. 1997.
\newblock \href {https://doi.org/10.1080/01638539709544980} {Salting a wound or sugaring a pill: {The} pragmatic functions of ironic criticism}.
\newblock \emph{Discourse Processes}, 23(1):25--45.
\newblock Publisher: Routledge \_eprint: https://doi.org/10.1080/01638539709544980.
\bibitem[{Frank and Goodman(2012)}]{frank_ref_game_2012}
Michael~C. Frank and Noah~D. Goodman. 2012.
\newblock \href {https://doi.org/10.1126/science.1218633} {Predicting {Pragmatic} {Reasoning} in {Language} {Games}}.
\newblock \emph{Science}, 336(6084):998--998.
\newblock Publisher: American Association for the Advancement of Science.
\bibitem[{Fried et~al.(2018)Fried, Andreas, and Klein}]{friedUnifiedPragmaticModels2018}
Daniel Fried, Jacob Andreas, and Dan Klein. 2018.
\newblock \href {https://doi.org/10.18653/v1/N18-1177} {Unified {Pragmatic} {Models} for {Generating} and {Following} {Instructions}}.
\newblock In \emph{Proceedings of the 2018 {Conference} of the {North} {American} {Chapter} of the {Association} for {Computational} {Linguistics}: {Human} {Language} {Technologies}, {Volume} 1 ({Long} {Papers})}, pages 1951--1963, New Orleans, Louisiana. Association for Computational Linguistics.
\bibitem[{Gibbs~Jr.(1979)}]{ContextualEffectsUnderstanding1979}
Raymond~W. Gibbs~Jr. 1979.
\newblock \href {https://doi.org/10.1080/01638537909544450} {Contextual effects in understanding indirect requests}.
\newblock \emph{Discourse Processes}, 2(1):1--10.
\newblock Publisher: Routledge \_eprint: https://doi.org/10.1080/01638537909544450.
\bibitem[{Goodman and Stuhlmüller(2013)}]{goodman2013knowledge}
Noah~D Goodman and Andreas Stuhlmüller. 2013.
\newblock Knowledge and implicature: Modeling language understanding as social cognition.
\newblock \emph{Topics in cognitive science}, 5(1):173--184.
\bibitem[{Grice(1975)}]{grice1975logic}
HP~Grice. 1975.
\newblock Logic and conversation.
\newblock \emph{Syntax and Semantics}, 3:43--58.
\bibitem[{Gu et~al.(2022)Gu, Fu, Pyatkin, Magnusson, Dalvi~Mishra, and Clark}]{gu-etal-2022-just}
Yuling Gu, Yao Fu, Valentina Pyatkin, Ian Magnusson, Bhavana Dalvi~Mishra, and Peter Clark. 2022.
\newblock \href {https://doi.org/10.18653/v1/2022.flp-1.12} {Just-{DREAM}-about-it: Figurative language understanding with {DREAM}-{FLUTE}}.
\newblock In \emph{Proceedings of the 3rd Workshop on Figurative Language Processing (FLP)}, pages 84--93, Abu Dhabi, United Arab Emirates (Hybrid). Association for Computational Linguistics.
\bibitem[{Horn(1984)}]{horn1984towards}
Laurence Horn. 1984.
\newblock Towards a new taxonomy for pragmatic inference: Q-and r-based implicature.
\newblock \emph{Meaning, form and use in context}.
\bibitem[{Hu et~al.(2023)Hu, Floyd, Jouravlev, Fedorenko, and Gibson}]{hu-etal-2023-fine}
Jennifer Hu, Sammy Floyd, Olessia Jouravlev, Evelina Fedorenko, and Edward Gibson. 2023.
\newblock \href {https://doi.org/10.18653/v1/2023.acl-long.230} {A fine-grained comparison of pragmatic language understanding in humans and language models}.
\newblock In \emph{Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)}, pages 4194--4213, Toronto, Canada. Association for Computational Linguistics.
\bibitem[{Jiang et~al.(2023)Jiang, Sablayrolles, Mensch, Bamford, Chaplot, de~las Casas, Bressand, Lengyel, Lample, Saulnier, Lavaud, Lachaux, Stock, Scao, Lavril, Wang, Lacroix, and Sayed}]{jiang2023mistral7b}
Albert~Q. Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra~Singh Chaplot, Diego de~las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, Lélio~Renard Lavaud, Marie-Anne Lachaux, Pierre Stock, Teven~Le Scao, Thibaut Lavril, Thomas Wang, Timothée Lacroix, and William~El Sayed. 2023.
\newblock \href {http://arxiv.org/abs/2310.06825} {Mistral 7b}.
\bibitem[{Kao et~al.(2014{\natexlab{a}})Kao, Bergen, and Goodman}]{kao2014formalizing}
Justine Kao, Leon Bergen, and Noah Goodman. 2014{\natexlab{a}}.
\newblock Formalizing the pragmatics of metaphor understanding.
\newblock In \emph{Proceedings of the annual meeting of the Cognitive Science Society}, volume~36.
\bibitem[{Kao and Goodman(2015)}]{Kao2015Irony}
Justine~T. Kao and Noah~D. Goodman. 2015.
\newblock \href {https://api.semanticscholar.org/CorpusID:4693345} {Let's talk (ironically) about the weather: Modeling verbal irony}.
\newblock \emph{Cognitive Science}.
\bibitem[{Kao et~al.(2014{\natexlab{b}})Kao, Wu, Bergen, and Goodman}]{kao_number_words_2014}
Justine~T. Kao, Jean~Y. Wu, Leon Bergen, and Noah~D. Goodman. 2014{\natexlab{b}}.
\newblock \href {https://doi.org/10.1073/pnas.1407479111} {Nonliteral understanding of number words}.
\newblock \emph{Proceedings of the National Academy of Sciences}, 111(33):12002--12007.
\bibitem[{Lai and Nissim(2022)}]{lai-nissim-2022-multi}
Huiyuan Lai and Malvina Nissim. 2022.
\newblock \href {https://aclanthology.org/2022.coling-1.519/} {Multi-figurative language generation}.
\newblock In \emph{Proceedings of the 29th International Conference on Computational Linguistics}, pages 5939--5954, Gyeongju, Republic of Korea. International Committee on Computational Linguistics.
\bibitem[{Lai et~al.(2023)Lai, Toral, and Nissim}]{lai-etal-2023-multilingual}
Huiyuan Lai, Antonio Toral, and Malvina Nissim. 2023.
\newblock \href {https://doi.org/10.18653/v1/2023.findings-acl.589} {Multilingual multi-figurative language detection}.
\newblock In \emph{Findings of the Association for Computational Linguistics: ACL 2023}, pages 9254--9267, Toronto, Canada. Association for Computational Linguistics.
\bibitem[{Li and Sporleder(2010)}]{li-sporleder-2010-using}
Linlin Li and Caroline Sporleder. 2010.
\newblock \href {https://aclanthology.org/N10-1039/} {Using {G}aussian mixture models to detect figurative language in context}.
\newblock In \emph{Human Language Technologies: The 2010 Annual Conference of the North {A}merican Chapter of the Association for Computational Linguistics}, pages 297--300, Los Angeles, California. Association for Computational Linguistics.
\bibitem[{Liu et~al.(2022)Liu, Cui, Zheng, and Neubig}]{liu-etal-2022-testing}
Emmy Liu, Chenxuan Cui, Kenneth Zheng, and Graham Neubig. 2022.
\newblock \href {https://doi.org/10.18653/v1/2022.naacl-main.330} {Testing the ability of language models to interpret figurative language}.
\newblock In \emph{Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies}, pages 4437--4452, Seattle, United States. Association for Computational Linguistics.
\bibitem[{Monroe et~al.(2017)Monroe, Hawkins, Goodman, and Potts}]{monroe-etal-2017-colors}
Will Monroe, Robert~X.D. Hawkins, Noah~D. Goodman, and Christopher Potts. 2017.
\newblock \href {https://doi.org/10.1162/tacl_a_00064} {Colors in context: A pragmatic neural model for grounded language understanding}.
\newblock \emph{Transactions of the Association for Computational Linguistics}, 5:325--338.
\bibitem[{Park et~al.(2024)Park, Lee, Park, Jeong, Koo, Hwang, Park, and Lee}]{park-etal-2024-multiprageval}
Dojun Park, Jiwoo Lee, Seohyun Park, Hyeyun Jeong, Youngeun Koo, Soonha Hwang, Seonwoo Park, and Sungeun Lee. 2024.
\newblock \href {https://doi.org/10.18653/v1/2024.genbench-1.7} {{M}ulti{P}rag{E}val: Multilingual pragmatic evaluation of large language models}.
\newblock In \emph{Proceedings of the 2nd GenBench Workshop on Generalisation (Benchmarking) in NLP}, pages 96--119, Miami, Florida, USA. Association for Computational Linguistics.
\bibitem[{Reimers and Gurevych(2019)}]{reimers-gurevych-2019-sentence}
Nils Reimers and Iryna Gurevych. 2019.
\newblock \href {https://doi.org/10.18653/v1/D19-1410} {Sentence-{BERT}: Sentence embeddings using {S}iamese {BERT}-networks}.
\newblock In \emph{Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)}, pages 3982--3992, Hong Kong, China. Association for Computational Linguistics.
\bibitem[{Roberts and Kreuz(1994)}]{robertsWhyPeopleUse1994}
Richard~M. Roberts and Roger~J. Kreuz. 1994.
\newblock \href {https://www.jstor.org/stable/40063089} {Why {Do} {People} {Use} {Figurative} {Language}?}
\newblock \emph{Psychological Science}, 5(3):159--163.
\newblock Publisher: [Association for Psychological Science, Sage Publications, Inc.].
\bibitem[{Scontras et~al.(2025)Scontras, Tessler, and Franke}]{scontras2025rsa}
Gregory Scontras, Michael~Henry Tessler, and Michael Franke. 2025.
\newblock Probabilistic language understanding: An introduction to the rational speech act framework.
\newblock \url{https://www.problang.org}.
\bibitem[{Shen et~al.(2019)Shen, Fried, Andreas, and Klein}]{shenPragmaticallyInformativeText2019}
Sheng Shen, Daniel Fried, Jacob Andreas, and Dan Klein. 2019.
\newblock \href {http://arxiv.org/abs/1904.01301} {Pragmatically {Informative} {Text} {Generation}}.
\newblock ArXiv:1904.01301 [cs].
\bibitem[{Spotorno et~al.(2012)Spotorno, Koun, Prado, Van Der~Henst, and Noveck}]{spotornoNeuralEvidenceThat2012}
Nicola Spotorno, Eric Koun, Jérôme Prado, Jean-Baptiste Van Der~Henst, and Ira~A. Noveck. 2012.
\newblock \href {https://doi.org/10.1016/j.neuroimage.2012.06.046} {Neural evidence that utterance-processing entails mentalizing: {The} case of irony}.
\newblock \emph{NeuroImage}, 63(1):25--39.
\bibitem[{Sravanthi et~al.(2024)Sravanthi, Doshi, Tankala, Murthy, Dabre, and Bhattacharyya}]{sravanthi-etal-2024-pub}
Settaluri Sravanthi, Meet Doshi, Pavan Tankala, Rudra Murthy, Raj Dabre, and Pushpak Bhattacharyya. 2024.
\newblock \href {https://doi.org/10.18653/v1/2024.findings-acl.719} {{PUB}: A pragmatics understanding benchmark for assessing {LLM}s' pragmatics capabilities}.
\newblock In \emph{Findings of the Association for Computational Linguistics: ACL 2024}, pages 12075--12097, Bangkok, Thailand. Association for Computational Linguistics.
\bibitem[{Tsvetkov et~al.(2014)Tsvetkov, Boytsov, Gershman, Nyberg, and Dyer}]{tsvetkov-etal-2014-metaphor}
Yulia Tsvetkov, Leonid Boytsov, Anatole Gershman, Eric Nyberg, and Chris Dyer. 2014.
\newblock \href {https://doi.org/10.3115/v1/P14-1024} {Metaphor detection with cross-lingual model transfer}.
\newblock In \emph{Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)}, pages 248--258, Baltimore, Maryland. Association for Computational Linguistics.
\bibitem[{Tsvilodub et~al.(2025)Tsvilodub, Gandhi, Zhao, Fränken, Franke, and Goodman}]{tsvilodub2025nonliteralunderstandingnumberwords}
Polina Tsvilodub, Kanishk Gandhi, Haoran Zhao, Jan-Philipp Fränken, Michael Franke, and Noah~D. Goodman. 2025.
\newblock \href {http://arxiv.org/abs/2502.06204} {Non-literal understanding of number words by language models}.
\bibitem[{Van~Hee et~al.(2018)Van~Hee, Lefever, and Hoste}]{van-hee-etal-2018-semeval}
Cynthia Van~Hee, Els Lefever, and V{\'e}ronique Hoste. 2018.
\newblock \href {https://doi.org/10.18653/v1/S18-1005} {{S}em{E}val-2018 task 3: Irony detection in {E}nglish tweets}.
\newblock In \emph{Proceedings of the 12th International Workshop on Semantic Evaluation}, pages 39--50, New Orleans, Louisiana. Association for Computational Linguistics.
\bibitem[{Wolf et~al.(2020)Wolf, Debut, Sanh, Chaumond, Delangue, Moi, Cistac, Rault, Louf, Funtowicz, Davison, Shleifer, von Platen, Ma, Jernite, Plu, Xu, Scao, Gugger, Drame, Lhoest, and Rush}]{wolf2020huggingfacestransformersstateoftheartnatural}
Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Rémi Louf, Morgan Funtowicz, Joe Davison, Sam Shleifer, Patrick von Platen, Clara Ma, Yacine Jernite, Julien Plu, Canwen Xu, Teven~Le Scao, Sylvain Gugger, Mariama Drame, Quentin Lhoest, and Alexander~M. Rush. 2020.
\newblock \href {http://arxiv.org/abs/1910.03771} {Huggingface's transformers: State-of-the-art natural language processing}.
\end{thebibliography}