DF-RAG: Query-Aware Diversity for Retrieval-Augmented Generation
Extracted source text · arXiv 2601.17212
% This must be in the first 5 lines to tell arXiv to use pdfLaTeX, which is strongly recommended.
\pdfoutput=1
% In particular, the hyperref package requires pdfLaTeX in order to break URLs across lines.
\documentclass[11pt]{article}
% Change "review" to "final" to generate the final (sometimes called camera-ready) version.
% Change to "preprint" to generate a non-anonymous version with page numbers.
\usepackage[preprint]{acl}
% Standard package includes
\usepackage{times}
\usepackage{latexsym}
\usepackage{booktabs}
\usepackage{booktabs,tabularx,makecell}
\usepackage{graphicx}
\usepackage{amssymb}
\usepackage{hyperref}
\usepackage{float}
\usepackage{arydshln} % put in your preamble
\usepackage{graphicx}
\usepackage{subcaption}
\usepackage{multirow}
\usepackage{graphicx, array, booktabs}
\newcolumntype{C}[1]{>{\centering\arraybackslash}m{#1}} % centered, vertical middle
\usepackage{lipsum} % For placeholder text, you can remove this
\usepackage{xltabular}
%sssss
\usepackage[most]{tcolorbox}
\usepackage{xcolor}
\usepackage{soul}
\usepackage{amsmath}
% For proper rendering and hyphenation of words containing Latin characters (including in bib files)
\usepackage[T1]{fontenc}
% For Vietnamese characters
% \usepackage[T5]{fontenc}
% See https://www.latex-project.org/help/documentation/encguide.pdf for other character sets
% This assumes your files are encoded as UTF8
\usepackage[utf8]{inputenc}
% This is not strictly necessary, and may be commented out,
% but it will improve the layout of the manuscript,
% and will typically save some space.
\usepackage{microtype}
% This is also not strictly necessary, and may be commented out.
% However, it will improve the aesthetics of text in
% the typewriter font.
\usepackage{inconsolata}
%Including images in your LaTeX document requires adding
%additional package(s)
\usepackage{graphicx}
\usepackage{xcolor}
\newcommand{\TODO}{\textcolor{red}{MODIFY HERE}}
% If the title and author information does not fit in the area allocated, uncomment the following
%
%\setlength\titlebox{<dim>}
%
% and set <dim> to something 5cm or larger.
\title{DF-RAG: Query-Aware Diversity for Retrieval-Augmented Generation}
%Enhancing RAG for Multi-Hop Question Answering by Balancing Relevance and Diversity of Retrieved Chunks
% Author information can be set in various styles:
% For several authors from the same institution:
% \author{Author 1 \and ... \and Author n \\
% Address line \\ ... \\ Address line}
% if the names do not fit well on one line use
% Author 1 \\ {\bf Author 2} \\ ... \\ {\bf Author n} \\
% For authors from different institutions:
% \author{Author 1 \\ Address line \\ ... \\ Address line
% \And ... \And
% Author n \\ Address line \\ ... \\ Address line}
% To start a separate ``row'' of authors use \AND, as in
% \author{Author 1 \\ Address line \\ ... \\ Address line
% \AND
% Author 2 \\ Address line \\ ... \\ Address line \And
% Author 3 \\ Address line \\ ... \\ Address line}
%\author{Saadat Hasan Khan \\
% Affiliation / Address line 1 \\
% Affiliation / Address line 2 \\
% Affiliation / Address line 3 \\
% \texttt{email@domain} \\\And
% Second Author \\
% Affiliation / Address line 1 \\
% Affiliation / Address line 2 \\
% Affiliation / Address line 3 \\
% \texttt{email@domain} \\}
\author{
\textbf{Saadat Hasan Khan\textsuperscript{ 1
\thanks{Majority of work done during internship at Capital One}}}
\textbf{Spencer Hong\textsuperscript{ 2}}
\textbf{Jingyu Wu\textsuperscript{ 2}}
\textbf{Kevin Lybarger\textsuperscript{ 1}} \\
\textbf{Youbing Yin\textsuperscript{ 2}}
\textbf{Erin Babinsky\textsuperscript{ 2}}
\textbf{Daben Liu\textsuperscript{ 2}}
\\
% \textbf{Fifth Author\textsuperscript{1,2}},
% \textbf{Sixth Author\textsuperscript{1}},
% \textbf{Seventh Author\textsuperscript{1}},
% \textbf{Eighth Author \textsuperscript{1,2,3,4}},
%\\
% \textbf{Ninth Author\textsuperscript{1}},
% \textbf{Tenth Author\textsuperscript{1}},
% \textbf{Eleventh E. Author\textsuperscript{1,2,3,4,5}},
% \textbf{Twelfth Author\textsuperscript{1}},
%\\
% \textbf{Thirteenth Author\textsuperscript{3}},
% \textbf{Fourteenth F. Author\textsuperscript{2,4}},
% \textbf{Fifteenth Author\textsuperscript{1}},
% \textbf{Sixteenth Author\textsuperscript{1}},
%\\
% \textbf{Seventeenth S. Author\textsuperscript{4,5}},
% \textbf{Eighteenth Author\textsuperscript{3,4}},
% \textbf{Nineteenth N. Author\textsuperscript{2,5}},
% \textbf{Twentieth Author\textsuperscript{1}}
%\\
%\\
George Mason University\textsuperscript{1}
Capital One\textsuperscript{2} \\
\texttt{\{skhan225, klybarge\}@gmu.edu} \\
\texttt{\{spencer.hong, jingyu.wu, youbing.yin, erin.babinksy, daben.liu\}@capitalone.com} \\
% \textsuperscript{3}Affiliation 3,
% \textsuperscript{4}Affiliation 4,
% \textsuperscript{5}Affiliation 5
%\\
% \small{
% \textbf{Correspondence:} \href{mailto:email@domain}{email@domain}
% }
\\
}
\begin{document}
\maketitle
\begin{abstract}
%Retrieval-augmented generation (RAG) enhances large language models (LLMs) by grounding generation in external knowledge. Yet, its effectiveness in multi-hop question answering (QA) remains limited, as retrieval based on relevance only introduces redundancy, which reduces information recall. In this work, we first show that incorporating diversity into retrieval at the dataset level improves RAG performance across challenging multi-hop QA benchmarks. We then design an Oracle that dynamically balances diversity per query, revealing an upper bound of up to 18\% gains in $F_{1}$ over vanilla RAG just by selecting the right chunks. Motivated by this gap, we propose DF-RAG, a pipeline that adapts the level of diversity for each query at test time without requiring prior information. DF-RAG leverages a diversity focused retrieval, built on maximal marginal relevance (MMR), guided by LLM-driven planning. Experiments on five QA benchmarks demonstrate that DF-RAG consistently outperforms strong baselines, achieving 4–10\% $F_{1}$ improvements over vanilla RAG and recovering up to 91.3\% of the Oracle-estimated performance over vanilla RAG.
Retrieval-augmented generation (RAG) is a common technique for grounding language model outputs in domain-specific information. However, RAG is often challenged by reasoning-intensive question-answering (QA), since common retrieval methods like cosine similarity maximize relevance at the cost of introducing redundant content, which can reduce information recall. To address this, we introduce \textbf{Diversity-Focused Retrieval-Augmented Generation (DF-RAG)} that systematically incorporates diversity into the retrieval step to improve performance on complex, reasoning-intensive QA benchmarks. DF-RAG builds upon the Maximal Marginal Relevance framework to select information chunks that are both relevant to the query and maximally dissimilar from each other. A key innovation of DF-RAG is its ability to optimize the level of diversity for each query dynamically at test time without requiring any additional fine-tuning or prior information. We show that DF-RAG improves $F_{1}$ performance on reasoning-intensive QA benchmarks by 4–10\% over vanilla RAG using cosine similarity and also outperforms other established baselines. Furthermore, we estimate an Oracle ceiling of up to 18\% absolute $F_{1}$ gains over vanilla RAG, of which DF-RAG captures up to 91.3\%. %Our code is publicly available at \url{https://github.com/SaadatKhan/DF-RAG}.
\end{abstract}
\section{Introduction}
%
Retrieval-Augmented Generation (RAG) \cite{lewis2020rag, pmlr-v119-guu20a} is a common technique for grounding Large Language Models' (LLMs) outputs in external knowledge due to its lightweight and composable architecture. However, RAG is often challenged by complex, reasoning-intensive question answering (QA) tasks as these questions involve retrieving and integrating relevant information scattered across multiple contexts \cite{li2025structrag, liu2024raghelpreasoningllm}. One central limitation of common RAG implementations is its reliance on cosine similarity, which frequently over-selects semantically redundant content \cite{carbonell1998mmr, liu-etal-2025-pointwise} and excludes complementary evidence that is crucial for performing complex reasoning \cite{asai2024selfrag, xiong2021answering}.
Fundamentally, for such complex, reasoning-intensive QA, a RAG system should only need to retrieve the few critical pieces of information required to bridge reasoning gaps to generate the right answer \cite{khattab}. We are thus motivated to ask our first research question: \\ \\
\textit{Can we improve our retrieval strategy to incorporate complementary evidence that current RAG approaches ignore such that it bridges reasoning gaps in complex QA tasks?}
\\
%Multi-hop Question Answering (QA) poses a significant challenge for Large Language Models (LLMs), as it requires reasoning over evidence scattered across multiple contexts \cite{yang-etal-2018-hotpotqa,xanh2020_2wikimultihop}. Recent advances in long-context LLMs \cite{grattafiori2024llama3herdmodels,qwen2025qwen25technicalreport} have enabled models to process the extended contexts at once , thereby bridging information gaps in a single forward pass to answer such questions. Despite their notable achievements, relying on full-context processing does not always yield the best result and also not feasible due to several well-documented limitations, including the lost-in-the-middle effect \cite{liu2024lost}, the substantial computational cost of long-context inference \cite{xu2024retrieval}, and most importantly, the constraint that input contexts can frequently exceed the model’s token limit. Retrieval-Augmented Generation (RAG) \cite{lewis2020rag} has emerged as a practical alternative that mitigates these issues by retrieving only a subset of relevant passages and has shown impressive achievements in various downstream tasks \cite{caciularu-etal-2022-long, ram2023incontext, shi2023replug, gao2022rarr}.
%Various recent works such as LongRAG \cite{zhao-etal-2024-longrag}, RAPTOR \cite{sarthi2024raptor}, and EfficientRAG \cite{zhuang-etal-2023-open} have achieved notable gains on multi-hop QA by improving information recall within retrieved context. However, these methods often incur substantial computational overhead due to task-specific fine-tuning, iterative agentic coordination, and processing large context as much as to the token limits. These beat the core goal of RAG, which is to retrieve the most useful chunks for a given query to facilitate answer generation. Fundamentally for multi-hop QA, a RAG system only needs to retrieve the few critical pieces of information required to bridge reasoning gaps and support answer generation \cite{khattab}, without the need to resort to computationally heavy processing methods. This observation motivates us to ask the following research questions: 1) \textit{Can we reliably improve information recall for RAG to support multi-hop QA under strict context length budgets?} 2) \textit{Can such a RAG system be designed through lightweight strategies that avoid computationally heavy methods like full-context processing, multi-agent pipelines, or task-specific fine-tuning?}
To answer this, we investigate the role of retrieval diversity in RAG pipelines for reasoning-intensive QA. We build upon Maximal Marginal Relevance (MMR) \cite{carbonell1998mmr}, a technique that balances query relevance with dissimilarity from selected information via a diversity parameter, $\lambda$. We hypothesize that strategically controlling retrieval diversity can bridge critical reasoning gaps required in complex reasoning QA and outperform current RAG baselines.
We first test this hypothesis by replacing cosine similarity with MMR using a fixed $\lambda$ for all queries in a dataset. We find that the optimal $\lambda$ varies across datasets, which suggests that individual queries within a dataset may benefit from tailored $\lambda$ values. To estimate the upper bound of performance of dynamically selecting $\lambda$ at the query-level, we design an Oracle that selects the optimal $\lambda$ for each query using the ground truth. The performance of our Oracle helps us
address our first research question: Oracle surpasses vanilla RAG by $\sim$18\% and also outperforms full-context baselines while utilizing $\sim$15\% of the context. However, estimating the right $\lambda$ at test time without any task-specific tuning is challenging. This leads to our second research question: \\ \\
\textit{How can we design a RAG system capable of incorporating the optimal level of diversity at test time?}
\\
\\
\indent To this end, we introduce \textbf{Diversity-Focused Retrieval-Augmented Generation (DF-RAG)}, a RAG pipeline designed to approximate the Oracle's performance without access to the ground truth. DF-RAG adaptively determines the optimal $\lambda$ for each query at test time through a multi-step process: (1) decomposing each query into individual steps with a planner LLM, (2) retrieving multiple candidate sets of chunks, each generated at predefined $\lambda$ levels; and (3) selecting the chunk set with the highest potential to address the decomposed steps with an Evaluator LLM. DF-RAG does not require fine-tuning, utilizes a plug-and-play retrieval framework, and directly addresses our second question. By benchmarking DF-RAG across five QA datasets, we demonstrate that leveraging DF-RAG to find the optimal $\lambda$ at test time can close the performance gap to the Oracle. \\
\indent To empirically validate our systems for such complex reasoning-intensive QA tasks, we benchmark them on: 1) multi-hop QA datasets as they require reasoning over evidence scattered across multiple contexts and 2) non-multi-hop but challenging long context QA datasets as they still require reasoning over context to generate answers \cite{yang-etal-2018-hotpotqa,xanh2020_2wikimultihop}. Our empirical evaluation across $\sim$180 experiments shows that DF-RAG routinely outperforms strong baselines on multi-hop QA, achieving up to 9.5\% gains over vanilla RAG across context lengths and closing as much as up to 91.3\% of the gap to the Oracle. On non-multi-hop QA, DF-RAG achieves up to a 4\% improvement over vanilla RAG and other baselines.
To our knowledge, this is the first work to leverage query-level diversity to estimate Oracle performance bounds and to propose a RAG framework that narrows this gap by dynamically balancing diversity and relevance at test time.
\section{Related Work}
\subsection{Multi-hop Question Answering}
\begin{figure*}[h]
\centering
\includegraphics[width=\textwidth]{ system_architecture.png}
\caption{Architecture of DF-RAG: A training-free framework that dynamically selects the right level of diversity to generate answers for reasoning-intensive QA tasks. DF-RAG adaptively adjusts diversity for each query by utilizing the Planner, Evaluator and the Diversity-focused Retrieval. We illustrate this with an example query from HotpotQA \cite{yang-etal-2018-hotpotqa}.}
\label{fig:fullwidth_diagram}
\end{figure*}
Early work on multi-hop QA established foundational benchmarks like HotpotQA \cite{yang-etal-2018-hotpotqa} and MuSiQue \cite{trivedi-etal-2022-MuSiQue}, which require finding and reasoning over multiple supporting documents. Many methods perform the task by conducting inference over static or dynamic graphs to find reasoning paths \cite{fang-etal-2020-hierarchical, ding-etal-2019-cognitive, 10.1145/3404835.3462853}. Neural-based approaches have also attempted to decompose multi-hop questions into single-hop questions or generate follow-up questions based on retrieved information \cite{min-etal-2019-multi, cao-etal-2019-bag, malon2020generatingfollowupquestionsinterpretable}. Recent advances in multi-hop QA include LLM-based approaches that leverage strong natural language understanding capabilities \cite{xu-etal-2021-exploiting-reasoning} through reasoning chain generation \cite{reasoning_chain}, question decomposition \cite{wu2024gendecrobustgenerativequestiondecomposition}, and graph construction \cite{li-du-2023-leveraging}. %Despite the advent of LLMs, notable limitations still impede their performance in multi-hop QA, including hallucination and knowledge gaps \cite{feng-etal-2024-dont}.
%Early work on multi-hop QA established foundational benchmarks like HotpotQA \cite{yang-etal-2018-hotpotqa} and MuSiQue \cite{trivedi-etal-2022-MuSiQue}, which require finding and reasoning over multiple supporting documents. Various methodological approaches have emerged to tackle this challenge. Many methods performs the task as performing inference over static
%or dynamic graphs to find the reasoning paths \cite{fang-etal-2020-hierarchical, ding-etal-2019-cognitive, 10.1145/3404835.3462853}. A number of neural based works have also attempted to decompose the multi-hop questions into single hop questions or generate follow-up questions based on the retrieved information \cite{min-etal-2019-multi, cao-etal-2019-bag, malon2020generatingfollowupquestionsinterpretable}. Recent advances in multi-hop QA include LLM based approaches that utilize LLMs strong natural language understanding capabilites \cite{xu-etal-2021-exploiting-reasoning} and have used LLMs to answer such queries through approaches like reasoning chain generation \cite{reasoning_chain}, question-decomposition \cite{wu2024gendecrobustgenerativequestiondecomposition} and graph construction \cite{li-du-2023-leveraging}. Despite the advent of LLMs, there are commonly notable limitations that impede their performance in multi-hop QA like hallucination and the problem of knowledge gaps \cite{feng-etal-2024-dont}.
%LCQA requires models to process and reason over inputs that often span tens of thousands of tokens. However, the effective context length of LLMs is bounded by their training sequence length, leading to performance degradation on such tasks \cite{zhao2023survey, Jin}. To mitigate this limitation, two primary directions have emerged: (i) scaling the context length of existing models through training or fine-tuning with longer sequences, exemplified by methods such as RMT \cite{NEURIPS2022_47e28862}, YaRN \cite{peng2024yarn}, and Active Beacon \cite{zhang2024activation}, and (ii) context compression techniques \cite{jiang-etal-2024-longllmlingua, li-etal-2023-compressing} that reduce input length while preserving salient information. Complementing these strategies, advances in computational resources \cite{xu2024retrieval} have enabled the release of long-context LLMs such as Llama 3, which supports up to 128k tokens \cite{grattafiori2024llama3herdmodels}. While these approaches facilitate the handling of longer inputs, LCQA remains challenging due to the “lost-in-the-middle” problem \cite{liu2024lost}, where critical information located mid-context is often overlooked by LLMs.
\subsection{Retrieval Augmented Generation}
Despite the superior language understanding and generation capabilities of LLMs, their performance remains limited on various downstream tasks due to challenges, such as outdated and long-tail knowledge \cite{he2022rethinking, pmlr-v202-kandpal23a}, hallucination \cite{10.1145/3624918.3625329} and the lack of domain-specific knowledge \cite{shen2023chatgpt}. RAG \cite{pmlr-v119-guu20a,lewis2020rag} has become the most widely adopted approach to address these issues by grounding LLM outputs in external knowledge for domain-specific downstream tasks. RAG pipelines operate by retrieving the most relevant document segments for a given query, and their effectiveness is therefore strongly influenced by the chunking strategies employed during retrieval. Recent advances in RAG include methods for reducing noise from chunks \cite{yan2024crag, zhuang-etal-2023-open}, chunk-free strategies to mitigate semantic loss \cite{qian2024cfic}, hierarchical arrangements of chunks via summarization techniques to capture global context \cite{sarthi2024raptor}, adaptive chunk-length strategies \cite{li-etal-2024-retrieval}, and approaches that extract global cues to generate supporting questions for improved answers \cite{zhao-etal-2024-longrag}.
\subsection{Diversity in Information Retrieval}
Traditionally, information retrieval has primarily emphasized relevance. Early work introduced Maximal Marginal Relevance (MMR) \cite{carbonell1998mmr}, which balances relevance and novelty. This inspired intent-aware ranking models aimed at improving subtopic coverage and evaluation \citep{agrawal2009diversifying, clarke2008novelty}. Some approaches have incorporated explicit diversification objectives into learning-to-rank frameworks, jointly optimizing for relevance and subtopic coverage \citep{santos2015search}.
Recent efforts have also explored diversity in RAG-based pipeline to reduce redundant evidence for downstream taks \cite{zhang-etal-2025-lexical, wang2025diversityenhancesllmsperformance}. While these RAG pipelines highlight the importance of diversity, they either depend on additional fine-tuning or lack mechanisms to adaptively determine the optimal diversity, limiting their generalizability and effectiveness in practice.
\section{Methodology}
\subsection{Diversity-focused Retrieval}
\label{sec:scoring function}
To reduce content redundancy, we design a diversity-focused retrieval function % while boost the diversity of relevant chunks, we propose DF-RAG - a framework built on the
built on MMR \cite{carbonell1998mmr}. Given a query $q$, a candidate chunk $c \in \mathcal{C}$, and a set of previously
selected chunks $S = \{s_1, s_2, \dots\}$, the retrieval function assigns a geometric MMR score (gMMR) to $c$ as follows:
%\begin{equation}
%\label{eq:mmr_final}
%\text{Score}(c) = \lambda \cdot \text{sim}(\vec{q}, \vec{c}) \;+\; (1-\lambda) \cdot \left\| \vec{c} - \frac{1}{|S|} \sum_{s %\in S} \vec{s} \right\|_2
%\end{equation}
\newcommand{\cS}{\vec{c}_{S}} % centroid of S, where \cS = (1/|S|)\sum_{s\in S}\vec{s}
\begin{equation}
\label{eq:mmr_final}
\begin{split}
\text{gMMR}(c) &= \lambda\,\operatorname{cos}(\vec{q},\vec{c}) \\
&\quad + (1-\lambda)\,\sqrt{\,2 - 2\,\operatorname{cos}\!\left(\vec{c}, \cS\right)}
\end{split}
\end{equation}
Where the hyperparameter $\lambda$ modulates the trade-off between relevance and diversity, and $\cS$ is the centroid of $S$ so far, calculated by:
\begin{equation}
\label{eq:set_vector}
\vec{c}_S \triangleq \frac{1}{|S|} \sum_{s \in S} \vec{s}
\end{equation}
To measure diversity, we replace standard cosine similarity with the Euclidean distance between the normalized embeddings, calculated as $\sqrt{\,2 - 2\cos\!\left(\vec{c}, \cS\right)}$ . This approach offers two advantages. First, it grounds diversity in a geometric framework enabling us to frame diversity as the spatial distance between a candidate chunk ($\vec{c}$) and the centroid of the already selected chunks ($\cS$). Second, adopting this distance-based variant of MMR yields improved empirical performance in our downstream tasks. A detailed comparison against classical MMR is provided in Appendix \ref{sec:eucvs}.
%Here, $\text{sim}(\vec{q}, \vec{c})$ denotes cosine similarity between $q$ and $c$, while $ \left\| \vec{c} - \frac{1}{|S|} %\sum_{s \in S} \vec{s} \right\|_2$ denotes the diversity as the Euclidean distance between $c$ and the centroid of our %selected chunks $S$. Assuming L2-normalized embeddings, the Euclidean distance between $c$ and the centroid of $S$ can be %computed as $\sqrt{2 - 2 \cdot \text{sim}(\vec{c}, \vec{{\frac{1}{|S|} \sum_{s \in S} \vec{s}}})}$, enabling efficient %implementation using cosine similarity operations alone. We employ Euclidean distance in the normalized embedding space %rather than the pairwise cosine term in classical MMR, as it not only provides a geometrically interpretable measure of %dissimilarity to the centroid of the selected set but also yielded empirical performance gains in our retrieval. A %performance comparison of our proposed system (highlighted in \ref{sec:df-rag architecture}) using the classical MMR and our %adopted variant of MMR is provided in the appendix.
%We greedily pick our chunk, $c^*$, iteratively at each step - $k$ by using the diversity-focused retrieval function under the conditions: %. For the first iteration, $S = \{ \phi \}$ and therefore the first chunk chosen is based solely on the chunk that maximizes the cosine similarity with $q$. For each subsequent step $k$ ($2 \leq k \leq K$), we pick the chunk, $c^*$ that maximizes our scoring function, i.e:
We greedily select the chunk $c^*$ at the $k$-th step using the following conditions:
\begin{equation}
\label{eq:chunk_selection_piecewise}
c^* =
\left\{
\begin{aligned}
& \arg\max_{\,c \in \mathcal{C}} \cos(\vec{q}, \vec{c})
&& \text{if } S = \varnothing, \\
& \arg\max_{\,c \in \mathcal{C}\setminus S} \text{gMMR}(c)
&& \text{otherwise.}
\end{aligned}
\right.
\end{equation}
%
%\begin{equation}
%\label{eq:chunk_selection_piecewise}
%c^* =
%\left\{
%\begin{aligned}
%& \arg\max_{\,c \in C} \; \cos(\vec{q}, \vec{c}), && \text{if } S=\varnothing, \\[0.6em]
%& \arg\max_{\,c \in \mathcal{C}\setminus S}
% \left[
% \lambda \cos(\vec{q}, \vec{c})
% \right. && \text{otherwise,} \\
%& \left. {}+ (1-\lambda)\sqrt{\,2 - 2\,\cos(\vec{c}, \vec{c}_S)} \right], &&
%\end{aligned}
%\right.
%\end{equation}
%\begin{equation}
%\label{eq:chunk_selection}
%\begin{split}
%c^* = \arg\max_{\,c \in \mathcal{C} \setminus S} \Big[
%&\lambda\,\operatorname{cos}(\vec{q},\vec{c}) \\
%&\quad + (1-\lambda)\,\sqrt{\,2 - 2\,\operatorname{cos}\!\left(\vec{c}, \vec{c}_S\right)}
%\Big]
%\end{split}
%\end{equation}
\subsection{Baseline: RAG with $\lambda$-Fixed gMMR}
\label{sec:RAGwMMR}
We implement a baseline using RAG with our gMMR function (Equation~\ref{eq:mmr_final}). In this approach, a constant $\lambda$ is applied uniformly to all queries in a given benchmark. We vary $\lambda \in [0,1]$ in increments of 0.1 to identify the value that yields the highest $F_1$ score for each dataset.
\subsection{Oracle: A Theoretical Upper Bound for Query-Adaptive gMMR}
We introduce an Oracle to represent the hypothetical upper bound for our query-adaptive approach. This Oracle is designed to determine the ideal gMMR diversity parameter $\lambda$ for each query by leveraging the ground truth. This is achieved by sampling values from 0 to 1 in increments of 0.1 and selecting the $\lambda$s whose corresponding chunk set yields the highest $F_1$ score with respect to the ground truth.
\subsection{The DF-RAG Architecture}
\label{sec:df-rag architecture}
The Oracle establishes a theoretical performance upper bound for our query-adaptive gMMR. We next introduce DF-RAG, an architecture designed to approximate Oracle at test time for each query. %without access to ground truth answers.
%Our Oracle's results (detailed in section \ref{sec:roacle_results}) reveal substantial gains attainable if the optimal $\lambda$ for each query were known. This performance gains provide the motivation for DF-RAG, whose main objective is to predict the optimal $\lambda$ at test time without any prior query-specific assumptions, aiming to close the gap between vanilla RAG and the Oracle.
As depicted in Figure~\ref{fig:fullwidth_diagram}, the architecture of DF-RAG integrates a plug-and-play retrieval component (highlighted in the purple dashed box) that uses an LLM-based Planner and an Evaluator for selecting the optimal $\lambda$ for each query. %Furthermore, DF-RAG’s retrieval component (highlighted in the purple dashed box in Figure~\ref{fig:fullwidth_diagram}) is a plug-and-play module that can be seamlessly integrated into modern RAG architectures to select optimally diverse chunks at test time. \
When a query, \textit{q}, is received, two processes are initiated in parallel: plan generation via the Planner as well as candidate chunk set retrieval using gMMR. Once the plan, \textit{P}, and the candidate chunk sets are available, the Evaluator evaluates and selects the optimal chunk set thereby determining the optimal $\lambda$. Finally, given the selected chunk set, the Generator synthesizes a final answer to the query. These components are explained in more details below.
\noindent\textbf{Planner for Step Decomposition:} Given a query, \(q\), the Planner generates an explicit sequence of steps \(\{p_1, p_2, \dots p_m\} \) that outline how the query should be solved, which we denote as a plan, \(P\). Our hypothesis is that any candidate chunk set \(S\) sufficient to answer \(q\) should also be sufficient to execute \(P\) with the collective information it holds. Therefore, we assess the quality of a given \(S\) based on how well it supports \(P\) to make an informed prediction of \(\lambda\) at test time. The Planner operates under a one-shot prompting setup, with the prompting details detailed in Appendix \ref{sec:systemprompts}.
% motivated by techniques from query expansion \cite{query_expansion, query_expansion2, zhou2023leasttomost, khot2023decomposed},
%Given a query \(q\), the Planner, motivated by techniques from query expansion \cite{query_expansion, query_expansion2, khot2023decomposed}, generates an explicit sequence of reasoning steps \(\{p_1, p_2, p_3, \dots\}\) that outline how the query should be solved, which we denote as a as a plan, $p$. Our hypothesis is that any candidate chunk set \(c\) able to answer \(q\) should be able to execute $p$ with the collective information it holds. This planner enables our system to assess retrieved chunks against a hypothesized set of steps which becomes very helpfulfor making informed \(\lambda\)-prediction at test time. The Planner operates under a one-shot prompting setup, where a single illustrative example specifies the expected plan format. The prompting details for the Planner are provided in Section \ref{sec:systemprompts}.
%Given a query \(q\), the Planner, motivated by techniques from query expansion \cite{query_expansion, query_expansion2, zhou2023leasttomost, khot2023decomposed}, generates an explicit sequence of reasoning steps \(\{p_1, p_2, p_3, \dots\}\) that outline how the query should be solved, which we denote as a as a plan, $p$. These steps constitute a reasoning plan that any candidate chunk set \(c\) is expected to collectively support in order to answer \(q\). By introducing this intermediate plan, the Planner enables our system to assess retrieved chunks against a hypothesized reasoning structure, thereby facilitating more informed \(\lambda\)-prediction at test time. To maintain consistency across queries, the Planner operates under a one-shot prompting setup, where a single illustrative example specifies the expected plan format. The prompting details for the Planner are provided in Section \ref{sec:systemprompts}.
\noindent\textbf{Candidate Chunk Sets Retrieval with gMMR:} We retrieve candidate chunk sets, $S = \{s_1,s_2,\dots,s_k\}$, from the document set, $\mathcal{C}$, using gMMR (Section \ref{sec:scoring function}). For our experiments, we sample chunks uniformly with $\lambda \in [0.1,1]$ with a step size of $0.1$. We exclude sampling a chunk set at $\lambda = 0$ because selecting chunks $2 \dots k$ at $\lambda = 0$ disregards query relevance and introduces noisy information. The computational complexity of DF-RAG depends directly on the sampling strategy and uniform sampling yields a $O(n)$ complexity. Additionally, we propose a simpler binary-search-based sampling strategy that achieves $O(\log(n))$ complexity with comparable performance. More details and corresponding results of binary-search based sampling strategy for DF-RAG is presented in Appendix \ref{sec:bs}.
\noindent\textbf{Evaluator for Optimal Diversity Selection:} Given a pool of candidate chunk sets retrieved using different $\lambda$s, and a plan by the Planner, \(P=\{p_i\}_{i=1}^m\), for the query, $q$, the Evaluator evaluates each candidate chunk set with respect to $P$. For each item $p_i \in P$, the Evaluator assigns a support score \(r(p_i)\in\{0,\dots,5\}\), where \(0\) indicates no supporting information is present and \(5\) indicates direct evidence satisfying that step in the candidate chunk set. The overall score, denoted by $Retreival\ Support$, of a chunk set retrieved under \(\lambda\) is then computed as:
\begin{equation}
Retreival\ Support(\lambda, P) = \sum_{i=1}^m r(p_i)
\end{equation}
The optimal query-specific $\lambda$, denoted $\lambda^*$, is then selected as follows:
\begin{equation}
\lambda^* = \arg\max_{\lambda \in \Lambda} Retreival \ Support(\lambda, P)
\end{equation}
%Note that the chunk scores computed by \(S(\lambda, P)\) are not normalized across queries; since \(S\) is a function of \(P\), all chunk sets for a given query are evaluated on the same scale, ensuring fair within-query comparison.
The Evaluator utilizes a few-shot prompting approach and the prompting details and the specific few-shot examples are discussed in Appendix \ref{sec:systemprompts}.
%It is to be noted that chunk scores assessed by the function, $S(\lambda, P(q))$, are not normalized across queries, as $S$ being a fucntion of $q$, allows selection is performed independently per query on the same fixed scale, ensuring fair within-query comparison.
\noindent\textbf{Tie-Breaking Gate:} DF-RAG's Evaluator's assessment may identify multiple \(\lambda\)s as equally optimal for a given query, but the system must select a single \(\lambda\) and its corresponding chunk set for answer generation due to the limit on context length we impose. To handle this, we introduce a Tie-Breaking Gate that selects one chunk set from the set of tied candidates. Our tie-breaking mechanism is deliberately lightweight and operates by selecting the median \(\lambda\) (upper median in case of even number of selected values) among the tied candidates. %(b) performing majority voting \textcolor{red}{[Still to come]} over the answers generated from each tied \(\lambda\)'s corresponding chunk set.
\noindent\textbf{Generator for Final Answer Synthesis:} Similar to standard RAG systems, the final component is an LLM-based Generator. It is tasked with producing an answer given the original query \textit{q} and the optimally-diverse chunk set $S^*$ retrieved using $\lambda^*$. We follow the prompt released in LongBench \cite{bai-etal-2024-longbench} for answer generation, which is further detailed in Appendix \ref{sec:systemprompts}.
\begin{table*}[!t]
\centering
\resizebox{\textwidth}{!}{%
\begin{tabular}{lcccccccccc}
\toprule
\multirow{2}{*}{\textbf{Experiment Type}} &
\multicolumn{5}{c}{\textbf{Llama 3.3 70B}} &
\multicolumn{5}{c}{\textbf{Qwen 2.5 72B}} \\
\cmidrule(lr){2-6} \cmidrule(lr){7-11}
& \textbf{HotpotQA} & \textbf{MuSiQue} & \textbf{2WikiMQA} & \textbf{MultifieldQA} & \textbf{En.QA}
& \textbf{HotpotQA} & \textbf{MuSiQue} & \textbf{2WikiMQA} & \textbf{MultifieldQA} & \textbf{En.QA} \\
\midrule
%Long Context & 63.8 & 42.7 & 67.2 & 54.4 &X & 65.4 & 38.0 & 63.4 & 52.5 & X \\
%RAPTOR & X & X & X &X &X & X & X & X & X & X \\
%Oracle at 1500 word context & 63.8 & 42.7 & 67.2 & 54.4 &X & 65.4 & 38.0 & 63.4 & 52.5 & X \\
%\midrule
\multicolumn{11}{c}{\textit{context length = 500 words }} \\
\midrule
Oracle & 69.5 & 46.2 & 65.3 & 54.9 & 41.3 & 67.7 & 43.8 & 64.8 & 57.2 & 40.1 \\
\hdashline
%\multicolumn{11}{c}{Baselines} \\
%\hdashline
%Vanilla RAG & 46.5 & 29.7 & 45.7 & 44.7 & 25.4 & X & X & X & X & X \\
Vanilla RAG & 51.1 & 29.9 & 49.8 & 47.7 & 26.3 & 51.2 & 24.8 & 47.7 & 47.5 & 26.1\\
%LongRAG & 55.1 & \textbf{35.6} & 53.2 & \textbf{50.7} & 20.4 & 54.3 & 35.3 & 53.1 & 49.1 & 21.1 \\
%\hdashline
%\multicolumn{11}{c}{DF-RAG with different tie breaking strategies} \\
%\hdashline
RAG with gMMR & 53.3 & 30.9 & 54.8 & \underline{48.2} & \underline{27.6} & 55.0 & 27.6 & 52.7 & 49.7 & \underline{27.7}\\
DF-RAG & \textbf{58.1} & \underline{33.3} & \textbf{58.1} & \textbf{48.3} & \textbf{29.3} & \underline{58.5} & \underline{31.3} & \underline{53.7} & \textbf{51.0} & \textbf{28.2} \\
DF-RAG + IC & \underline{56.8} & \textbf{34.0} & \underline{55.2} & 47.0 & 27.5 & \textbf{60.1} & \textbf{32.7} & \textbf{56.8} & \underline{48.8} & \underline{27.7} \\
%\% Recovered & 38 & 25 & 53.5 & 8.3 & X & 38 & 25 & 53.5 & 8.3 & X \\
\midrule
\multicolumn{11}{c}{\textit{context length = 1000 words}} \\
\midrule
Oracle & 68.6 & 49.7 & 68.6 & 56.2 & 45.2 & 67.5 & 48.7 & 67.2 & 56.7 & 44.1 \\
\hdashline
%\hdashline
%\multicolumn{11}{c}{Baselines} \\
%\hdashline
%Vanilla RAG & 52.9 & 31.9 & 56.7 & 50.8 & 26.3 & X & X & X & X & X \\
Vanilla RAG & 56.0 & 36.0 & 58.2 & 50.0 & 28.3 & 55.9 & \underline{31.9} & 55.0 & \underline{50.6} & 26.9 \\
%LongRAG & 56.1 & \underline{38.4} & 55.6 & \textbf{52.6} & 20.4 & 53.9 & \textbf{35.9} & 53.6 & 47.5 & 21.5 \\
%\hdashline
%\multicolumn{11}{c}{DF-RAG with different tie breaking strategies} \\
%\hdashline
RAG with gMMR & 56.0 & \underline{37.5} & 58.7 & \underline{50.9} & \underline{30.1} & 56.7 & \textbf{33.3} & 55.5 & \textbf{51.9} & \underline{30.3}\\
DF-RAG & \textbf{60.3} & 36.3 & \underline{58.9} & \textbf{51.6} & \textbf{31.5} & \underline{60.4} & 28.8 & \underline{57.4} & 50.4 & \textbf{31.1} \\
DF-RAG + IC & \underline{58.4} & \textbf{38.6} & \textbf{63.3} & 48.6 & 29.3 & \textbf{62.1} & 31.7 & \textbf{59.7} & 49.3 & 29.6 \\
%\% Recovered & 34.1 & 19.0 & 49.3 & 25.8 & X & 34.1 & 19.0 & 49.3 & 25.8 & X \\
\midrule
\multicolumn{11}{c}{\textit{context length = 1500 words}} \\
\midrule
Oracle & 69.5 & 51.9 & 74.9 & 56.1 & 47.8 & 68.7 & 48.6 & 69.9 & 55.9 & 47.4 \\
\hdashline
%\hdashline
%\multicolumn{11}{c}{Baselines} \\
%\hdashline
%Vanilla RAG & 53.6 & 32.9 & 58.7 & 48.3 & 29.2 & X & X & X & X & X \\
Vanilla RAG & 56.1 & 35.4 & 61.6 & 50.4 & \underline{33.1} & 57.9 & \underline{33.6} & 59.5 & 48.1 & 31.3 \\
%RAG w/ gMMR & 57.0 & 33.6 & 56.4 & \textbf{43.6} & X & X & X & X & X & X \\
%LongRAG & 56.2 & \textbf{38.4} & 58.4 & \textbf{51.0} & 20.1 & 53.6 & \textbf{35.1} & 55.1 & 47.5 & 21.1 \\
%\hdashline
%\multicolumn{11}{c}{DF-RAG with different tie breaking strategies} \\
%\hdashline
RAG with gMMR & 58.1 & 35.9 & 63.4 & \textbf{50.9} & \textbf{33.2} & 60.1 & \textbf{34.6} & 59.8 & \textbf{50.1} & 32.5\\
DF-RAG & \underline{59.6} & \underline{37.9} & \underline{63.9} & \underline{50.6} & \underline{33.1} & \textbf{61.8} & 32.2 & \underline{62.1} & \underline{50.0} & \textbf{34.0} \\
DF-RAG + IC & \textbf{61.8} & \textbf{38.0} & \textbf{68.6} & 46.5 & 31.8 & \underline{60.9} & 33.3 & \textbf{69.0} & 48.7 & \underline{33.2} \\
%\% Recovered & 42.5 & 15.7 & 52.6 & 3.5 & X & 42.5 & 15.7 & 52.6 & 3.5 & X \\
\midrule
\multicolumn{11}{c}{\textit{context length independent baselines}} \\
\midrule
LongRAG & 57.4 & 38.1 & 61.7 & \textbf{52.9} & 22.9 & 57.7 & 36.8 & 59.9 & 47.8 & 22.4 \\
RAPTOR & 54.5 & 34.3 & 58.9 & 48.3 & X & 53.0 & 29.6 & 58.4 & 49.6 & X \\
HippoRAG & 46.2& 33.5 & 57.1 & 48.3 & X & 47.0 & 32.8 & 53.2 & 48.9 & X \\
\bottomrule
\end{tabular}
}
\caption{$F_1$ scores of vanilla RAG and DF-RAG on five QA datasets across varying context lengths, evaluated using Llama 3.3 70B and Qwen 2.5 72B. Context length is varied by changing chunk size $w \in \{100,200,300\}$ words. \textbf{Bold} and \underline{underlined} results indicate first and second in $F_1$ score. Results marked with X could not be performed due to memory limit restrictions.}
\label{tab:performance_acl_dfrag_transposed}
\end{table*}
\subsection{Experimental Details}
\noindent\textbf{Preliminaries:} We perform comprehensive experiments using Qwen 2.5 72B \cite{qwen2025qwen25technicalreport} and Llama 3.3 70B \cite{grattafiori2024llama3herdmodels} as backbone LLMs. We use 16-bit floating precision and run our experiments on 2 NVIDIA A100 80GB GPUs. We perform retrieval based on sentence embeddings generated by `multi-qa-mpnet-base-cos-v1'. For our proposed experiments (other than established baselines), we always retrieve 5 chunks where chunks are generated at the word level. We vary context lengths by varying the number of words, \(w \in \{100, 200, 300\}\), corresponding to context lengths of approximately 500, 1000, and 1500 words. All our results are based on single-run experiments. \\ %Our system prompts (see Appendix \ref{sec:systemprompts}) can be utilized to reproduce our system and results.\\
\noindent\textbf{RAG with $\lambda$-Fixed gMMR:} We benchmark RAG with gMMR against vanilla RAG to highlight the impact of diversity-focused retrieval. Both models use the same generator and differ only in their chunk selection strategy. Vanilla RAG selects the top 5 chunks based on cosine similarity of their embeddings, whereas RAG with $\lambda$-Fixed gMMR selects chunks using gMMR at fixed $\lambda$ values across datasets.
\noindent\textbf{Oracle Upper Bound:} We benchmark our Oracle against vanilla RAG, RAG with $\lambda$-Fixed gMMR, and a long-context baseline. The long-context baseline (denoted as Long Context) is implemented by concatenating the entire context with the input query and feeding it to the LLM generator. This baseline is only applicable when the full context fits within the model’s token limit, which excludes En.QA from long-context evaluation.
\noindent\textbf{DF-RAG Baselines:}
We implement two DF-RAG variants: (1) DF-RAG , where median \(\lambda\) is the tie-breaking gate used and (2) DF-RAG + incremental context, denoted by IC. The IC serves as a condensed summary added to the Generator's context. The IC comprises the plan produced by the Planner and the short answer generated by the Evaluator when executing that plan. The motivation and analysis of ICs benefits are further detailed in Appendix \ref{sec:ic}. We compare our DF-RAG variants to the Oracle and five baselines. The five baselines are: RAG with $\lambda$-Fixed gMMR, Vanilla RAG, RAPTOR \cite{sarthi2024raptor}, LongRAG \cite{zhao-etal-2024-longrag} and HippoRAG \cite{hipporag}. To experiment with RAPTOR, LongRAG and HippoRAG, we replace the backbone LLM while keeping all other implementation details consistent with their published setups. For LongRAG, we employ both the filter and extractor modules, as this configuration yields the best performance in their reported results. For HippoRAG, we use both Llama 3.3 70B and Qwen 2.5 72B for answer generation to be consistent with other baselines, however, the entity extraction phase was carried out with a Llama 3.3 70B due to HippoRAG's reported issues on some models' compatibility at the time of writing this paper. We benchmark against LongRAG, RAPTOR and HippoRAG given their demonstrated success in prior work and their strategic designs for retrieving critical information in multi-hop queries. All three architectures excel at complex reasoning tasks, making them well-suited as baselines.
\noindent\textbf{Datasets:} We benchmark our baselines, Oracle and DF-RAG on multiple reasoning-intensive QA datasets. Following prior work \cite{zhao-etal-2024-longrag, li-etal-2024-retrieval}, we select datasets that (a) are in English, (b) contain queries that require reasoning, (c) are structured for QA evaluation. To this end, our benchmark suite includes three multi-hop QA datasets from LongBench \cite{bai-etal-2024-longbench} — HotpotQA, MuSiQue, and 2WikiMultihopQA — one non–multi-hop QA dataset from LongBench, MultifieldQA (English version), and one dataset from $\infty$Bench \cite{zhang-etal-2024-bench}, En.QA. Together, these datasets cover diverse domains, including Wikipedia articles, meetings, narratives, and research papers, making it challenging to reason over the context when answering the queries. Datasets' statistics are presented in Appendix \ref{dataset-stat}.
%We benchmark RAG with gMMR against vanilla RAG to show the benefit of our diversity-focused retrieval and then move to benchmark our Oracle with LLMs having access to full context to prove our hypothesis of doing better with lesser context. Following these experiments, we move to benchmarking DF-RAG against several strong baselines: RAPTOR \cite{sarthi2024raptor}, LongRAG \cite{zhao-etal-2024-longrag} with filter and extractor, vanilla RAG and RAG w/ gMMR (at best performing $\lambda$ for the corresponding benchmark). We benchmark against LongRAG and RAPTOR given their demonstrated success in prior work and their strategic designs for retrieving critical information beyond plain cosine similarity. Both methods are tailored for multi-hop reasoning, making them well-suited comparators for multi-hop QA tasks. We also assess how close DF-RAG gets to Oracle level performance at various context length settings. We show two DF-RAG variants: (1) DF-RAG w/ Median \(\lambda\), where median \(\lambda\) is the tie-breaking gate used and (2) DF-RAG w/ Median \(\lambda\) + incremental context, denoted by IC. The IC serves as a condensed summary added to the Generator's context. The IC comprises the plan produced by the Planner and the short answer generated by the Executor when executing that plan. Its inclusion is motivated by two factors: (i) it is already produced as a by-product of the chunk selection process, requiring no extra computation, and (ii) it provides a condensed explanation that guides the Generator, helping mitigate the lost-in-the-middle effect in longer-context settings.
%We evaluate our methods on multiple RAG benchmark pipelines using word-level chunking with $w=100$ words per chunk. The number of retrieved chunks is set to $K \in \{5, 10, 15\}$, corresponding to context lengths of approximately 500, 1000, and 1500 words. For evaluation, we follow the official metrics of LongBench and $\infty$Bench, namely the response-level F1 score against the reference answers. For backbone LLMs, we utilize Qwen 2.5 70B and Llama~3.3~70B. It is to be noted that DF-RAG utilizes only one LLM in its architecture supporting all its components.
\noindent\textbf{Evaluation Metrics:} We follow the official metrics of LongBench and \(\infty\)Bench for the selected benchmarks, reporting answer-level $F_1$ scores against the ground truth.
\section{Experimental Results}
%We begin by examining the performance of RAG w/ gMMR (section \ref{sec:RAGwMMRres}), showing how a fixed dataset-level \(\lambda\) improves performance and motivates our Oracle design. We then detail the Oracle results (section \ref{sec:roacle_results}) before turning to the evaluation of DF-RAG (section \ref{sec:dfr-res}). For comparison,
%We benchmark RAG w/ gMMR against vanilla RAG to show the benefit of our diversity-focused retrieval and move to benchmark our Oracle with LLMs having access to full context to prove our hypothesis of doing better with lesser context. Following these, we benchmark DF-RAG against several strong baselines: RAPTOR \cite{sarthi2024raptor}, LongRAG \cite{zhao-etal-2024-longrag} with filter and extractor, vanilla RAG and RAG w/ gMMR (at best performing $\lambda$ for the corresponding benchmark). We benchmark against LongRAG and RAPTOR given their demonstrated success in prior work and their strategic designs for retrieving critical information beyond plain cosine similarity. Both methods are tailored for multi-hop reasoning, making them well-suited comparators for multi-hop QA tasks.
%In our experiments, we assess two DF-RAG variants: (1) DF-RAG w/ Median \(\lambda\), where median \(\lambda\) is the tie-breaking gate used and (2) DF-RAG w/ Median \(\lambda\) + incremental context, denoted by IC. The IC serves as a condensed summary added to the Generator's context. The IC comprises the plan produced by the Planner and the short answer generated by the Executor when executing that plan. Its inclusion is motivated by two factors: (i) it is already produced as a by-product of the chunk selection process, requiring no extra computation, and (ii) it provides a condensed explanation that guides the Generator, helping mitigate the lost-in-the-middle effect in longer-context settings. We present our experimental results are presented in Table~\ref{tab:performance_acl_dfrag_transposed}.
%We choose LongRAG and RAPTOPR because firstly, its core method is to retrieve all possibly relevant context, which is complementary to DF-RAG's -- that is by design strategic about what to retrieve and secondly, is benchmarked against multi-hop QA tasks. We do not include any other baselines further due to our computational limitations of benchmarking these methods at various context settings.
% We do not include any other baselines further due to our computational limitations of benchmarking these methods at various context settings
%We choose LongRAG because its core method is to retrieve all possibly relevant context and place it directly into the language model's expanded context window. This approach is complementary to ours that is by design strategic about what to retrieve. Other methods like RAPTOR, GraphRAG or EfficientRAG can co-exist with DF-RAG (e.g. retrieving diverse yet relevant global summaries).
%In this section, we present a multi-dimensional comparison and analysis of the evaluated systems. We first analyze the performance of RAG w/ gMMR (section \ref{sec:RAGwMMRres}) and show how a constant dataset-level $\lambda$ can boost performance framing our motivation for the Oracle. We then dive into the details of our Oracle (section \ref{sec:roacle_results}) and finally to DF-RAG's results (section \ref{sec:dfr-res}). For experimental comparisons, we compare DF-RAG against several strong baselines: LCQA with full-context access (permitted by the model’s token limit), Vanilla RAG, RAG w/ gMMR (reported at the performance peak), and an external baseline: LongRAG \cite{zhao-etal-2024-longrag}. We also examine three variants of DF-RAG for comparison with the baselines:
%1. DF-RAG w/ Median $\lambda$,
%2. DF-RAG w/ Majority of LLM Answer, and
%3. DF-RAG w/ Median $\lambda$ + Plan and Answer.
%For the purposes of brevity, the comprehensive results is presented in the appendix in Table \ref{tab:performance_acl_dfrag_transposed}. However, we %summarize the main findings and present them in Table X and Figure \ref{fig:five-horizontal} in this section along the dimensions we assess our models.
%In this section, we present a multi-dimensional comparison and analysis of the evaluated systems. The comprehensive results are provided in Table \ref{tab:performance_acl_dfrag_transposed}, which reports performance across five benchmark datasets at three different context lengths. For experimental comparisons, we compare DF-RAG against several strong baselines: LCQA with full-context access (permitted by the model’s token limit), Vanilla RAG, RAG w/ gMMR (reported at the performance peak), and an external baseline: LongRAG \cite{zhao-etal-2024-longrag}. We choose LongRAG because it was introduced within the past two years, provide official reproducible code, and, like our approach, leverage a finite number of LLM calls to enhance RAG performance. We examine three variants of DF-RAG for comparison with the baselines:
%1. DF-RAG w/ Median $\lambda$, which resolves ties during chunk selection using the median strategy.
%2. DF-RAG w/ Majority of LLM Answer, which aggregates multiple generated answers via majority voting and selects the one with the highest vote.
%3. DF-RAG w/ Median $\lambda$ + Plan and Answer, which extends DF-RAG w/ Median $\lambda$ variant by incorporating both the planner’s output plan and %the associated rationale for each selected chunk.
%
%The inclusion of the plan and answer is motivated by two factors. First, the planner’s rationale is already produced as a by-product of the chunk selection mechanism, and using it as an additional input serves as a condensed explanation that guides the answer generation module, often mitigating the lost-in-the-middle phenomenon. Second, this strategy does not incur additional computational cost, since the rationale is generated as part of DF-RAG's mechanism. We present our findings and analysis for each of the component we designed in this work.
%\begin{figure*}[h]
% \centering
% % Row 1
% \begin{subfigure}[t]{0.33\linewidth}
% \includegraphics[width=\linewidth]{ images/context_graphs/Q_HotpotQA.png}
% \end{subfigure}\hfill
% \begin{subfigure}[t]{0.33\linewidth}
% \includegraphics[width=\linewidth]{ images/context_graphs/Q_MuSiQue.png}
% \end{subfigure}\hfill
% \begin{subfigure}[t]{0.33\linewidth}
% \includegraphics[width=\linewidth]{ images/context_graphs/Q_2WikiMQA.png}
% \end{subfigure}
%
% \vspace{0.1mm}
%
% % Row 2
% \begin{subfigure}[t]{0.33\linewidth}
% \includegraphics[width=\linewidth]{ images/context_graphs/L70B_HotpotQA.png}
% \end{subfigure}\hfill
% \begin{subfigure}[t]{0.33\linewidth}
% \includegraphics[width=\linewidth]{ images/context_graphs/L70B_MuSiQue.png}
% \end{subfigure}\hfill
% \begin{subfigure}[t]{0.33\linewidth}
% \includegraphics[width=\linewidth]{ images/context_graphs/L70B_2WikiMQA.png}
% \end{subfigure}
% \caption{Performance curves of Oracle, Vanilla RAG, and Best DF--RAG across context lengths on multi-hop %QA benchmarks. The first row shows performance variation using Qwen 2.5 72B as the backbone LLM whereas %the second row utilizes Llama 3.3 70B.}
% \label{fig:two-by-three}
%\end{figure*}
%In this section, we present a multi-dimensional comparison and analysis of the systems evaluated in this paper. Our comprehensive results are presented in Table \ref{tab:performance_acl_dfrag_transposed} which includes performance comparison of various baselines and variants of DF-RAG across the 5 benchmark datasets at our above-mentioned 3 different context lengths. For DF-RAG's performance comparison, we compare LCQA performance with an LLM that has access to the full context (permitted by their token limit), Vanilla RAG, RAG w/ gMMR (performance recorded where peak occurs), and Long-RAG \cite{zhao-etal-2024-longrag}. The three variants of DF-RAG are DF-RAG w/ Median $\lambda$ which includes DF-RAG with choosing the median as the tie-breaking strategy, DF-RAG with Majority of LLM answer which performs a majority voting of the generated answer and selects the one with the highest vote, and DF-RAG w/ Median $\lambda$ + Plan and Answer which us the same as DF-RAG w/ Median $\lambda$ but includes the plan from the planner and the corresponding output rationale for each respective chunk against the plan. The motivation of including the plan and answer is of two folds, 1) it has been already produced as a bi-product in our mechanism as during the chunk selection strategy so including this acts like a condensed rationale which can guide the answer generation model, often eliminating the lost in the middle phenomenon that could arise in the selected chunks and 2) adding this does not increase computational complexity as it is already produced in the mechanism.
\subsection{RAG with $\lambda$-Fixed gMMR}
\label{sec:RAGwMMRres}
Our results revealed that RAG with $\lambda$-Fixed gMMR outperforms vanilla RAG (see Table \ref{tab:performance_acl_dfrag_transposed} and Figure \ref{fig:oracvslc}). We also found that numerous distinct dataset-level $\lambda$ values outperformed vanilla RAG (see Figure \ref{fig:RAGwMMR}). This helped to confirm that diversity-focused methods are beneficial, motivating our investigation into whether an optimal $\lambda$ could be determined at the query-level. Details of the performance trends for each of these benchmarks across $\lambda$ is further detailed in Appendix \ref{sec:RAGWMMRcontd}.
\subsection{Oracle: Upper Bound for Query-Adaptive gMMR}
Our Oracle consistently outperforms both vanilla RAG and RAG with $\lambda$-Fixed gMMR across all benchmarks (see Figure \ref{fig:oracvslc}). When compared to vanilla RAG, we observe consistent gains of $\sim$7--19\% $F_1$ points with the largest gains observed for multi-hop QA benchmarks. A similar behavior is also observed as the Oracle beats RAG with $\lambda$-Fixed gMMR by $\sim6- 16\%$. Furthermore, the Oracle also outperforms Long Context by up to 9.2\%, even when using up to $\sim 70-87$\% less context (see Figure \ref{fig:oracvslc}). % and Appendix Table \ref{tab:performance_acl_dfrag_transposed} ).These results hold across different context lengths with results for the shorter lengths found in Appendix Table \ref{tab:performance_acl_dfrag_transposed}.
This demonstrates that query-aware, optimally-tuned retrieval diversity can be more effective than processing the entire context, provided we find the right $\lambda$. As the Oracle achieves the highest performance among all baselines, it serves as DF-RAG’s performance target. Accordingly, a dashed line follows Oracle’s score in Table \ref{tab:performance_acl_dfrag_transposed}.
\begin{figure}[!t]
\centering
\includegraphics[width=0.9\linewidth]{ images/context_graphs/ovslcvsvan.png}
\caption{$F_1$ scores of vanilla RAG, RAG with gMMR, Long Context, and Oracle across LongBench benchmarks. Vanilla RAG, RAG with gMMR and Oracle operate at 1500 word context length. Results on En.QA are not reported as its context length exceeds our context limit. }
\label{fig:oracvslc}
\end{figure}
%with the largest improvements (15.5--18.4\%) at 500-word contexts. At 1000-word and 1500-word settings, the Oracle still outperforms vanilla RAG by 10.4--13.7\% and 13.3--16.5\%, respectively. These results highlight that optimal chunk selection via a well-chosen $\lambda$ yields performance that vanilla RAG cannot match, even when given up to three times more context. This effectively establishes an upper bound on the gains achievable through retrieval alone, underscoring the importance of diversity-aware selection strategies.
\label{sec:roacle_results}
%\noindent\textbf{Oracle beats Long Context:}
%Figure \ref{fig:oracvslc} also compares answer generation with full-context processing (Long Context) against our Oracle and Vanilla RAG. The Oracle consistently outperforms the Long Context baseline across all benchmarks by up to 9.2\% while using only a 1500-word context, saving up to 90\% of tokens relative to full context. This demonstrates that retrieving the right chunk is more effective than processing the entire context and that strong performance can be achieved at substantially shorter context lengths. As the best-performing and most efficient setting, the Oracle thus becomes our practical upper bound for our experimented baselines.
%Figure \ref{fig:oracvslc} also compares answer generation using full context - denoted by Long Context with our Oracle and Vanilla RAG. we show that our Oracle consistently outperforms the Long Context baseline across all benchmarks while using only a 1500-word context, saving up to 90\% tokens compared to full context in our benchmarks. This shows that retrieving the right chunk is more effective than processing the entire context, which can be achieved at very short context lengths. With the Oracle outperforming Long Context, this successfully becomes the upper-bound of performance achievable for our experimented baselines as it not only gets the highest performance but is also computationally very efficient.
%Our Oracle consistently outperforms the Long Context baseline across all benchmarks while using only a 500-word context, saving up to 90\% tokens compared to full context in our multi-hop QA datasets. This demonstrates the effectiveness of retrieving the right chunk rather than relying on exhaustive context processing. By presenting only the most relevant evidence, the Oracle avoids the pitfalls of long context processing. Moreover, these results also help us attain an estimation of upper bound for settings such as En.QA, where long-context experiments are not even possible due to context length exceeding model capacity and prohibitive compute requirements.
\begin{figure*}[h]
\centering
% Row 1 (Qwen 2.5 72B)
\begin{subfigure}[t]{0.19\linewidth}
\includegraphics[width=\linewidth]{ images/context_graphs/Q_HotpotQA.png}
\end{subfigure}\hfill
\begin{subfigure}[t]{0.19\linewidth}
\includegraphics[width=\linewidth]{ images/context_graphs/Q_MuSiQue.png}
\end{subfigure}\hfill
\begin{subfigure}[t]{0.19\linewidth}
\includegraphics[width=\linewidth]{ images/context_graphs/Q_2WikiMQA.png}
\end{subfigure}\hfill
\begin{subfigure}[t]{0.19\linewidth}
\includegraphics[width=\linewidth]{ images/context_graphs/Q_MultifieldQA.png}
\end{subfigure}\hfill
\begin{subfigure}[t]{0.19\linewidth}
\includegraphics[width=\linewidth]{ images/context_graphs/Q_En.QA.png}
\end{subfigure}
\vspace{0.1mm}
% Row 2 (Llama 3.3 70B)
\begin{subfigure}[t]{0.19\linewidth}
\includegraphics[width=\linewidth]{ images/context_graphs/L70B_HotpotQA.png}
\end{subfigure}\hfill
\begin{subfigure}[t]{0.19\linewidth}
\includegraphics[width=\linewidth]{ images/context_graphs/L70B_MuSiQue.png}
\end{subfigure}\hfill
\begin{subfigure}[t]{0.19\linewidth}
\includegraphics[width=\linewidth]{ images/context_graphs/L70B_2WikiMQA.png}
\end{subfigure}\hfill
\begin{subfigure}[t]{0.19\linewidth}
\includegraphics[width=\linewidth]{ images/context_graphs/L70B_MultifieldQA.png}
\end{subfigure}\hfill
\begin{subfigure}[t]{0.19\linewidth}
\includegraphics[width=\linewidth]{ images/context_graphs/L70B_En.QA.png}
\end{subfigure}
\caption{Performance ($F_1$ score) curves, of Oracle, vanilla RAG, and best DF--RAG across context lengths on all five QA benchmarks. The first row shows Qwen 2.5 72B as the backbone LLM whereas the second row shows Llama 3.3 70B.}
\label{fig:df-ragvsvanilla}
\end{figure*}
\subsection{DF-RAG}
\label{sec:dfr-res}
\noindent\textbf{DF-RAG beats RAG with $\lambda$-Fixed gMMR. } We observe that DF-RAG beats RAG with $\lambda$-Fixed gMMR consistently in various benchmarks across context lengths. Specifically, DF-RAG outperforms RAG with $\lambda$-Fixed gMMR with gains ranging from $\sim2-10 \%$ across context lengths and models (see Figure Table \ref{tab:performance_acl_dfrag_transposed}).
We generally observe larger gains with DF-RAG over RAG with $\lambda$-Fixed gMMR at longer context length settings (1500 words), suggesting that DF-RAG's ability of successfully picking the right $\lambda$ for each query helps longer context settings, as it gives more flexibility at retrieving information. However, we also observe that DF-RAG is outperformed by RAG with $\lambda$-Fixed gMMR in a small number of cases (MuSiQue and MultifieldQA datasets), but the magnitude of drop in performance is very minimal. All together, the performance shows DF-RAG's success and importance of its ability to accurately predict optimal $\lambda$ at test time.%, which is a key limitation of RAG with $\lambda$-Fixed gMMR.
%On multi-hop QA benchmarks DF-RAG beats RAG with gMMR at various margins ranging from $\sim 2-10 \%$ across context lengths and models. This helps us support the advantage of query aware diversity in cases with no prior information. DF-RAG also beats RAG with gMMR in non-multi-hop scenarios but the gains are less pronounced.
%The performance details of DF-RAG is provided in Table \ref{tab:performance_acl_dfrag_transposed}. Below, we analyze DF-RAG's performance from the results of Table \ref{tab:performance_acl_dfrag_transposed} across a few dimensions described below.\\
\noindent\textbf{DF-RAG outperforms various RAG baselines.}
Across benchmarks, \textsc{DF-RAG} consistently outperforms vanilla RAG, achieving up to +8.9\% absolute improvement at the 500-word context length, and up to +6.2\% and +9.5\% at the 1000- and 1500-word settings, respectively. \textsc{DF-RAG} maintains a clear advantage over vanilla RAG across all benchmarks, with more pronounced gains on HotpotQA and 2WikiMQA (see Figure~\ref{fig:df-ragvsvanilla}). Beyond vanilla RAG, \textsc{DF-RAG} also outperforms LongRAG and RAPTOR, two context length--independent baselines (LongRAG leverages global cues, while RAPTOR recursively summarizes the full context). Their results are shown in the last two rows of Table~\ref{tab:performance_acl_dfrag_transposed}. Notably, at the 1500-word context length, \textsc{DF-RAG} surpasses RAPTOR across all benchmarks and model families, and similarly outperforms LongRAG on every benchmark except MultifieldQA with Llama~3.3 70B and MuSiQue with Qwen~2.5 72B.
\noindent\textbf{DF-RAG approaches Oracle performance.} Beyond outperforming baselines across benchmarks, DF-RAG narrows the gap to the Oracle, revealing its potential across datasets, model families, and context lengths. To measure its effectiveness, we calculated the portion of the performance gap between vanilla RAG and the Oracle that DF-RAG closes. On multi-hop QA benchmarks, DF-RAG closes an average of 41.5\%, 32.1\%, and 47.7\% of this gap at 500, 1000, and 1500-word context lengths, respectively (see Figure \ref{fig:df-ragvsvanilla} and Table \ref{tab:performance_acl_dfrag_transposed}). For non–multi-hop QA benchmarks, DF-RAG covers on average $\sim$12-20\% of the gap. These results highlight two key insights: (1) DF-RAG effectively bridges reasoning gaps in multi-hop QA by adaptively balancing relevance and diversity through its retrieval component as it approaches Oracle performance (analyzed in Appendix \ref{sec:perf}). In one case, DF-RAG covers up to 91.3\% of the gap on a particular benchmark (2WikiMQA for Qwen 2.5 72B at 1500-word context length). (2) DF-RAG also maintains strong performance on complex non–multi-hop QA tasks, too (just below around 5\% of the Oracle for MultifieldQA). %This demonstrates that its reasoning ability extends beyond simply bridging information—it can also systematically select optimal diversity to retrieve complementary evidence for complex reasoning when required.
Together, the performance comparison of DF-RAG with the Oracle showcases DF-RAG's ability to adaptively predict optimal $\lambda$ at test time. Further analysis of DF-RAG's success and failure cases are presented in Appendix \ref{sec:ea}. %without performance degradation. % where simpler queries can be effectively handled by standard cosine similarity, showcasing its ability to flexibly balance relevance and diversity, when needed without performance degradation.\\
\section{Discussion and Analysis}
\subsection{Sensitivity of $\lambda$ in selecting chunks}
To justify the difference in $\lambda$ sweeps in our experiments, we assess the sensitivity of $\lambda$ in selecting chunks within a given chunk-set. To illustrate this, we measure the Jaccard similarity index of the selected chunk-sets at each distinct $\lambda$, which provides an intuitive idea of the uniqueness of chunk-sets at neighboring $\lambda$ values. The Jaccard index (J) between two chunk sets $S_{\lambda_i}$ and $S_{\lambda_j}$ (sampled at $\lambda = i$ and $\lambda = j$) is measured as:
\[
\mathrm{J}(S_{\lambda_i}, S_{\lambda_j})
= \frac{\left| S_{\lambda_i} \cap S_{\lambda_j} \right|}
{\left| S_{\lambda_i} \cup S_{\lambda_j} \right|}.
\]
\begin{figure*}[h]
\centering
% Row 1 (Qwen 2.5 72B)
\begin{subfigure}[t]{0.19\linewidth}
\includegraphics[width=\linewidth]{ images/lambda_sens/hotpotqa.png}
\end{subfigure}\hfill
\begin{subfigure}[t]{0.19\linewidth}
\includegraphics[width=\linewidth]{ images/lambda_sens/musique.png}
\end{subfigure}\hfill
\begin{subfigure}[t]{0.19\linewidth}
\includegraphics[width=\linewidth]{ images/lambda_sens/2wikimqa.png}
\end{subfigure}\hfill
\begin{subfigure}[t]{0.19\linewidth}
\includegraphics[width=\linewidth]{ images/lambda_sens/multifieldqa_en.png}
\end{subfigure}\hfill
\begin{subfigure}[t]{0.19\linewidth}
\includegraphics[width=\linewidth]{ images/lambda_sens/enqa.png}
\end{subfigure}
\caption{Heatmap showing Jaccard index values between candidate chunk-sets sampled at different $\lambda$. Darker color represents more common chunks between the corresponding chunk-sets.}
\label{fig:lambda-sens}
\end{figure*}
Figure~\ref{fig:lambda-sens} shows the Jaccard index values between each pair of sampled chunk-sets. From the figure, we observe that even a small $\lambda$ increment of 0.1 introduces on average of about 40\% unique information.
%We observe an average chunk overlap of 61.3\% between two selected chunk-sets with a $+$0.1 magnitude sweep of $\lambda$.
\subsection{Latency Analysis}
We assess the latency of our DF-RAG by measuring its wall-clock runtime. For evaluation on latency, we use 20 randomly selected samples from MultifieldQA, which features reasonably long contexts, utilizing 300 word-chunks. While DF-RAG utilizes multiple LLM calls and its most significant time consuming component is the Evaluator which evaluates each chunk-set independently. However, due to the Evaluator's independent nature, its process can be parallelized which can significantly reduce the latency. The figure below shows how the runtime per sample varies with presence of parallelizable nodes. With more executable worker nodes, the Evaluator process can be parallelized, which reduces the latency.
\begin{figure}[H]
\centering
\includegraphics[width=0.9\linewidth]{ images/runtime.png}
\caption{Runtime comparison of DF-RAG per sample with parallelizable worker nodes.}
\label{fig:oracvslc}
\end{figure}
Furthermore, DF-RAG utilizes a mechanism that uses constant amount of token-processing with LLMs, regardless of the length of the context whereas, most advanced RAG systems' (as reported in our baselines) tokens process scale with the length of context supplied, due to extracting global cues, building graphs, etc. Therefore on very long contexts, DF-RAG becomes computationally efficient and a feasible approach, which is outlined by results obtained using our En.QA dataset, a dataset which HippoRAG and RAPTOR could not be evaluated on due to their demanding nature of time.
\subsection{Utilization of gMMR in retrieving from large corpus}
Although, like most advanced RAG systems, DF-RAG’s overall time complexity is primarily governed by the number of LLM calls required to generate an answer, retrieval complexity can become a bottleneck when operating over very large corpora or databases. In such settings, gMMR can be applied strategically as a re-ranking module on top of an initial Approximate Nearest Neighbor (ANN) retrieval stage. This design significantly reduces the computational overhead introduced by gMMR’s iterative selection process.
\subsection{Effectiveness of the Planner}
\label{sec:pln}
Since the Planner is responsible for breaking complex reasoning-intensive questions into intermediate steps, we evaluate its effectiveness based on the number of steps it generates per query. We hypothesize that an effective planner should avoid overplanning while breaking queries into a reasonable number of steps. Therefore, a good planner should produce steps close to the number of hops for multi-hop QA benchmarks. The Planner produces an average of \textbf{2.88} steps for 2WikiMQA, \textbf{2.71} for HotpotQA, and \textbf{2.81} for MuSiQue. These values closely align with the original number of hops in these datasets, supporting our claim that the Planner generates well-calibrated plans.
\subsection{Effectiveness of the Evaluator}
\label{sec:exc}
\begin{figure*}[h]
\centering
% Row 1
\begin{subfigure}[t]{0.3\linewidth}
\includegraphics[width=\linewidth]{ images/executor_val/dist_500.png}
\caption{500 word context length}
\end{subfigure}\hfill
\begin{subfigure}[t]{0.3\linewidth}
\includegraphics[width=\linewidth]{ images/executor_val/dist_1000.png}
\caption{1000 word context length}
\end{subfigure}\hfill
\begin{subfigure}[t]{0.3\linewidth}
\includegraphics[width=\linewidth]{ images/executor_val/dist_1000.png}
\caption{1500 word context length}
\end{subfigure}
\vspace{0.1mm}
\caption{Distribution of $\lambda$ values from the Evaluator of DF-RAG (green) and the Oracle (gray). Both exhibit smooth distributions, though the Oracle’s is slightly smoother, reflecting the source of DF-RAG’s performance gap.}
\label{fig:two-by-three}
\end{figure*}
We evaluate the effectiveness of the Evaluator in our DF-RAG pipeline by examining the distribution of its predicted $\lambda$ values from the Evaluator across datasets. An effective Evaluator should assign a diverse range of $\lambda$ values, reflecting its ability to adapt retrieval diversity to the unique requirements of each query. Such behavior would generate a smooth and broad distribution of $\lambda$s. In contrast, a poorly performing Evaluator would produce a narrow, concentrated distribution -- indicating that it defaults to similar $\lambda$ values for most queries, failing to capture query-specific diversity needs and effectively outputting non-informative predictions. Figure \ref{fig:two-by-three} shows the distribution of $\lambda$s from Evaluator and from our Oracle.
\subsection{Effectiveness of appending the Incremental Context}
\label{sec:ic}
Our results show that the DF-RAG variant outperforms the DF-RAG + IC variant in several settings -- for example, on HotpotQA, 2WikiMQA and MultifieldQA and En.QA at the 500-word context length with Llama 3.3 70B; on HotpotQA, MultifieldQA and En.QA at 1500 words with Qwen 2.5 72B. The IC augmentation was intended to provide a concise rationale, which—following findings from Chain-of-Thought (CoT) \cite{10.5555/3600270.3602070}—should have enhanced performance. However, our experiments reveal that in some cases (notably always on MultifieldQA and En.QA), appending the IC leads to slightly lower performance. We attribute this finding to two factors: 1) For non-multi-hop reasoning, extended reasoning chains are not only computationally inefficient \cite{chen2024not} but also potentially detrimental to performance \cite{hassid2025don}. Drawing a parallel to this ``overthinking'' problem, the IC could make less complicated queries seemingly complicated through excessive planning steps - deviating LLMs reasoning line of bridging information that hurts performance. Consequently, mitigation strategies developed to curb model overthinking may also be applicable to DF-RAG \cite{fang2025thinkless}. 2) Our experiments use small contexts (500-1500 words) where the information required to reach a multi-hop answer is already highly condensed, leaving little room for condensed reasoning to provide additional benefit.
%To assess the computational efficiency of DF-RAG, we compare it against our reported RAG baselines along two key dimensions: tokens processed and wall-clock latency (see Table \ref{tab:latency_comparison}). We evaluate all methods on 20 randomly selected samples from MultifieldQA, which features reasonably long contexts. Both DF-RAG and vanilla RAG operate with 300-word chunks. While DF-RAG processes the largest number of tokens among the baselines on this dataset, DF-RAG's token usage has a constant upper bound, unlike other methods whose token consumption scales with context length and structure. DF-RAG’s latency is significantly lower than RAPTOR and LongRAG (see Table \ref{tab:latency_comparison}), as its most time-consuming component -- the Executor -- can parallelize its independent operations. To visualize the benefits of parallelization in our DF-RAG experiments, we reduce the number of Executor LLM calls to one, as parallelized calls incur the same latency as a single call.
%\begin{table}[t]
%\centering
%\begin{tabular}{lcc}
%\toprule
%\textbf{Method} & \textbf{Tokens Processed} & \textbf{Latency} \\
%\midrule
%%Long Context & $\sim 7000$ & $O(X)$ \\
%Vanilla RAG & $\sim 1,500$ & 5s \\
%RAPTOR & $\sim 10,000$& 326s \\
%LongRAG & $\sim 14,000$& 127s \\
%DF-RAG & $\sim 15,000$& 27s \\
%\bottomrule
%\end{tabular}
%\caption{Computational characteristics of different RAG methods, %comparing total tokens processed and wall-clock latency averaged across %20 samples in MultifieldQA.}
%\label{tab:latency_comparison}
%\end{table}
%\noindent\textbf{DF-RAG approaches Oracle performance:} Apart from just beating baselines across benchmarks, DF-RAG makes progress in reaching its ultimate potential - the Oracle and understand the potential across benchmarks and model families and the different context length settings. We observe that a larger proportion of the gap between the Oracle and vanilla RAG is covered by DF-RAG on multi-hop QA benchmarks. On average, DF-RAG covers X,Y,Z percents of the vanilla RAG-Oracle gap at 500, 1000, 1500 word context lengths respectively. However for non-multi-hop QA benchmarks, DF-RAG covers on average X,Y,Z percent of the gap. This shows two things: 1) DF-RAG successfully is able to bridge the reasoning gaps for multi-hop QA by retrieving the right information through diversity based selection as opposed to vanilla RAG and showing that it heads in the right direction. This statement is further supported by the fact that DF-RAG covers as high as 91.3\% of the gap for a particular benchmark. 2) DF-RAG's does not drop performance for non-multi-hop QA benchmarks, even though it covers less gap it is still better performing as it can adaptively balance relevance and diversity when queries are relatively simple. %the DF-RAG covers up to up to 54\% of the Oracle performance at 500 words, 49.3\% at 1000 words, and 91.3\% at 1500 words context length settings, showing that DF-RAG successfully gets better to its potential with context length.
%At 500-word context length DF-RAG beats vanilla RAG by 7\%, 4.1\%, 8.3\%, 0.6\% and 3.1\% for HotpotQA, MuSiQue, 2WikiMQA and En.QA respectively. At 1000-word context, DF-RAG beats by 4.3\%, 2.8\%, 5.1\%, 1.6\% and 5.5\%. At 1500 word context, DF-RAG beats by 5.7\%, 2.6\%, 7\%, 0.2\% and 4.3\% \\
%\noindent\textbf{DF-RAG follows the law of diminishing returns}. DF-RAG's performance increases with context length but the utility of the system drops compared to the baselines as we increase the context length. This effect is precisely shown in Figure Z where each plot shows the performance of DF-RAG and vanilla RAG at different context lengths for a specific benchmark dataset. For each of these plots, the best performing variant of DF-RAG for the corresponding benchmark dataset is chosen. The plots reveal that the area between DF-RAG's and vanilla RAG's performance decreases over context length increments.
%\noindent\textbf{DF-RAG outperforms baselines utilizing 33\% of the context length} We observe that DF-RAG's performance scales with context length. However, in general across our 5 benchmarks, DF-RAG also reaches performance utilizing 500-word context length that different baselines do not achieve with with 1500-word context length. DF-RAG at 500-word context length successfully beats all baselines at 1500-word context length for HotpotQA and En.QA. For 2WikiMQA and MultifieldQA, DF-RAG at 500-word achieves very close to performances of the baselines (less than 3\%). DF-RAG at 1000 word context length beats all baselines at 1500 word-context length. Thes findings bolster our hypothesis of picking diverse but relevant chunks to improves performance as the most important information are selected even at smaller context settings which effectively bumps the performance
%\noindent\textbf{DF-RAG outperforms baselines using only 33\% of the context length.} We observe that DF-RAG's performance scales positively with context length. However, in general across our 5 benchmarks, DF-RAG at a 500-word context length surpasses or gets close performance to baselines using 1500 words. This is evidenced on HotpotQA and En.QA, where DF-RAG at 500-word surpasses baselines at 1500-word context length, and on 2WikiMQA and MultifieldQA, where it comes within 3\% of their performance. Furthermore, DF-RAG at 1000 words consistently outperforms all baselines at 1500-word context length. These findings support our hypothesis that selecting diverse yet relevant chunks enables a RAG pipeline to capture the most critical information early, thereby achieving strong performance even under shorter context budgets.
%\section{Discussion}
%
%
%
%\noindent\textbf{DF-RAG outperforms vanilla RAG with shorter context:}
%%\noindent\textbf{DF-RAG outperforms Baselines with shorter context:}
%Beyond overall gains, we find DF-RAG outperforms baselines with shorter context. %Specifically, we find DF-RAG variants at 500-word context outperform vanilla RAG at %1000- and 1500-word context across multiple multi-hop QA benchmarks and model families. %With Qwen 2.5-72B, the best DF-RAG variant at 500 words outperforms vanilla RAG at 1000 %words, while for both Llama 3.3-70B and Qwen 2.5-72B, DF-RAG at 1000 words exceeds %vanilla RAG at 1500 words (see Table \ref{tab:performance_acl_dfrag_transposed}). These %findings highlight that DF-RAG maintains its advantage while using only 50–66\% of the %context required by vanilla RAG, thereby showcasing DF-RAG's ability to capture the %most critical information early through its retrieval mechanism.
%
%
%
%\noindent\textbf{Assessing the performance trends of Oracle and DF-RAG at different %context length settings:}
%Our Oracle improves monotonically with longer contexts, as larger windows help capture %complementary information that bridges reasoning gaps. DF-RAG closely tracks the %Oracle’s trend, improving also at longer context settings. Furthermore, we also %observed DF-RAG covers more of the vanilla RAG - Oracle gap at longer context settings %(average at 47.1\% for multi-hop QA; see HotpotQA and 2WikiMQA in Fig. \ref{fig:df-%ragvsvanilla}). These observations help us establish that whenever additional headroom %for improvement exists through the inclusion of more diverse and relevant evidence, DF-%RAG is able to effectively capitalize on it, which acts a self-validation aspect of DF-%RAG's behavior.
%
%%\textbf{Assessment of Oracle's performance}.
%Our Oracle’s performance consistently improves with increasing context length on the three multi-hop QA benchmarks, whereas this trend is less pronounced on the general long-context QA benchmarks (En.QA and MultifieldQA). This discrepancy stems from the inherent nature of multi-hop queries, which require integrating information scattered across multiple chunks to arrive at a correct answer. Providing a longer context allows more room to accommodate additional information, increasing the likelihood that the Oracle retrieves all the complementary evidence needed to bridge these reasoning gaps and boost performance. In contrast, non-multi-hop queries often hinge on localized information, limiting the benefits of additional context. We also notice a similar pattern with DF-RAG's performance as context length increases. These results together highlight the advantage of diversity-focused retrieval for multi-hop QA tasks.
%\textbf{Assessment of DF-RAG's performance:}
%DF-RAG’s performance closely tracks Oracle's performance, improving in settings where the Oracle shows higher gains. This indicates that whenever there is headroom for improvement through the inclusion of more diverse and relevant evidence, DF-RAG effectively captures it. This trend is particularly evident in our three multi-hop QA benchmarks, where both the Oracle and DF-RAG consistently improve with increasing context length, serving as a form of self-validation for the system’s design.
%\textbf{Appending the Plan and Answer to Selected Chunks Enhances DF-RAG Performance}
%DF-RAG generally benefits from appending the plan generated by the planner and the answer produced by the executor to the selected optimal chunk. This effect is particularly evident on the three multi-hop QA benchmarks, especially at the 1500-word context length, where notable performance gains are observed. The appended plan and answer serve as a condensed rationale that surfaces the key pieces of information required for multi-hop reasoning, helping them stand out from surrounding noisy content that often causes important evidence to be lost in the middle. However, the utility of appending the plan and answers is marginal under shorter context length settings. We attribute this to the fact that shorter contexts (100-word chunks) are already densely packed with salient information, making them less prone to such overshadowing of key information.
%\section{Discussion}
%\textbf{{Understanding performance trends across benchmarks}
%\textbf{Multi-hop QA benchmarks} We observe that DF-RAG is able to close a substantial %portion of the performance gap between vanilla RAG and our Oracle on two of the three %multi-hop QA benchmarks—2WikiMQA and HotpotQA. However, the improvements are smaller on %MuSiQue for DF-RAG compared to vanilla RAG. One reason for this discrepancy is the %relative complexity of the benchmarks. HotpotQA predominantly contains 2-hop questions, %where the reasoning chain involves fewer bridges and is thus easier compared to questions %requiring more hops. For instance, in the question: \textit{Prior to playing for Michigan %State, Keith Nichol played football for a school located in what city?}, the reasoning %needs to identify: 1) \textit{Identify the school Keith Nichol played for before Michigan %State} and 2) \textit{Determine the city where that school is located}. Both of these %questions can be answered solely by relying on the query as the entities of interested %are directly placed there. By retrieving a sufficiently diverse set of information about %\textit{Keith Nichol}, we can find details about his school prior to Michigan State. %Similarly, by retrieving information about \textit{schools}, we can also identify the %city in which the school is located.
%
%In contrast, MuSiQue contains a greater number of hops per question. For example, the %question \textit{When was the last time Peter Till's sports team beat the winner of the %1894–95 FA Cup?} requires essentially three hops: 1) \textit{Identify Peter Till's sports %team}, 2) \textit{Identify the winner of the 1894–95 FA Cup}, and 3) \textit{Determine %the last time the team identified in the first step defeated the winner of the 1894–95 FA %Cup}. We can answer the first hop by relying on retrieving a sufficiently diverse %retrieval set on Peter Till. We can also perform the second hop by retriving information %about the winners of FA cup on 1894-1895. However the third hop needs connecting the %entities we find in the first and second hop, which is impossible to guess without %answering them and therefore retrieving information solely based on question provides %minimal support. As a result, unlike in HotpotQA, simply relying on the question often %provides limited guidance on what evidence needs to be retrieved to answer the final hop %and hence both DF-RAG and Vanilla RAG fails. Moreover, MuSiQue mentions that for each %question presented, it is compulsory to perform each hop as they filtered out questions %where later hops could be answered without knowing the earlier answers, making no %shortcut jumps possible and a difficult dataset to address at short context scenarios.
%
\section{Conclusion}
In this work, we addressed the challenges RAG pipelines face on complex, reasoning-intensive QA
tasks, identifying their reliance on cosine similarity as a key limitation that leads to redundant contexts. To overcome this, we introduced a retrieval function that balances diversity and relevance by a control parameter -- $\lambda$. We incorporated this retrieval
function at a dataset-level which showed performance improvements over vanilla RAG. With our given retrieval mechanism, we then designed an Oracle to understand the upper bound of performance improvement possible by controlling our $\lambda$ at a query level, which revealed up to 18\% absolute performance gains from vanilla RAG. Building on these insights, we proposed DF-RAG, which employs query-aware diversity by predicting the optimal $\lambda$ for each query at test time without prior task-specific tuning. To test our system's performance on complex reasoning-intensive QA tasks, we evaluated DF-RAG on multi-hop QA and long-context non–multi-hop QA datasets across various context lengths and 2 model families. DF-RAG routinely outperforms strong baselines across context lengths and recovers up to 91.3\% of the performance gap between vanilla RAG and the Oracle. Furthermore, DF-RAG uses constant token processing and can achieve faster execution through parallelization. Our results highlight the importance of diversity-focused retrieval for improving RAG pipelines on QA tasks and direct future directions for further innovation at closing the gap to Oracle performance.
\section*{Limitations}
While DF-RAG brings about performance improvements, there are limitations in our approach that merit discussion.
First, given the large number of experiments conducted (over 180 in total), we only benchmarked using two LLM families (Qwen and Llama). Second, while we benchmarked on 1,101 samples across five datasets from LongBench and $\infty$Bench, leveraging a wider range of datasets with more samples from such unified suites would enable stronger assessments of generalizability. Third, we deliberately focused on fine-tuning-free baselines while benchmarking DF-RAG's performance for fair comparison; investigating its performance and benefits over training-dependent methods such as Self-RAG \cite{asai2024selfrag} remains as a future work. Finally, a central component of DF-RAG is its Planner and Evaluator, which rely on LLMs. While we conduct systematic ablation studies on both these components to evaluate their performances, improved prompting and other test time strategies \cite{snell2024scaling} may improve the effectiveness and performance of DF-RAG.
\section*{Ethical Considerations}
Our research underwent a standard internal review. We used publicly available datasets and LLMs in accordance with their intended scope, and we refer readers to the original dataset papers for details on anonymization, preprocessing, and annotator information. All artifacts (existing baselines and benchmarks) used in this this paper are available under their respective open-source licenses (except for LongRAG; we only reproduce their code). Our experiments focus exclusively on the English language; however, the proposed methodology makes no assumptions about the language of operation and we encourage future work to explore the applicability of our approach in multilingual settings. Furthermore, we acknowledge the propagation of various biases through LLM-based systems and while our methods do not study the implications of biases in our system, we recommend that any system implementing our methodology undergo rigorous testing of (1) the documents retrieved to prevent the propagation of biases, and (2) the models used to generate answers to ensure they do not promote biases during generation. Such systems should also be subject to continuous monitoring. We also acknowledge the use of ChatGPT for fixing grammatical and spelling errors.
\newpage
%\begin{table*}[h!]
%\centering
%\caption{Oracle vs Top-K Performance vs DF-RAG (First Iteration) (ROUGE-L) }
%\label{tab:performance_acl}
%\begin{tabular}{lccc}
%\toprule
%\textbf{Dataset} & \textbf{Oracle Performance} & \textbf{Top-K Performance} & %\textbf{DF-RAG Performance}\\
%\midrule
%\multicolumn{3}{l}{\textit{Strategy: 100 word chunks, 10\% overlap}} \\
%\midrule
%HotpotQA & 0.6610 & 0.4767 & \textbf{0.5315} \\
%MuSiQue & 0.4458 & 0.29 & \textbf{0.30} \\
%NarrativeQA & 0.3474 & 0.208 & \textbf{0.25} \\
%2WikiMQA & 0.638 & 0.4943 & \textbf{0.5081} \\
%MultifieldQA & 0.497 & 0.4263 & \textbf{0.45} \\
%\midrule
%\multicolumn{3}{l}{\textit{Strategy: 500 word chunks, 10\% overlap}} \\
%\midrule
%HotpotQA & 0.68 & \textbf{0.5844} & 0.5765 \\
%MuSiQue & 0.44 & \textbf{0.32} & 0.30 \\
%NarrativeQA & 0.36 & 0.24 & \textbf{0.27} \\
%2WikiMQA & 0.65 & 0.54 & \textbf{0.57} \\
%MultifieldQA & 0.54 & 0.47 & \textbf{0.48} \\
%\bottomrule
%\end{tabular}
%\end{table*}
%\begin{table*}[h!]
%\centering
%\caption{DFRAG vs Vanilla RAG Performance (ROUGE-L) in C1 system for whole dataset}
%\label{tab:performance_Oracle}
%\begin{tabular}{lcc}
%\toprule
%\textbf{Dataset} & \textbf{DFRAG Performance} & \textbf{Vanilla RAG Performance} \\
%\midrule
%
%\multicolumn{3}{l}{\textit{Strategy: 100 word chunks, 10\% overlap}} \\
%\midrule
%HotpotQA & \textbf{0.54} & 0.49 \\
%MuSiQue & \textbf{0.29} & 0.29 \\
%NarrativeQA & \textbf{0.24} & 0.23 \\
%2WikiMQA & \textbf{0.56} & 0.49 \\
%
%MultifieldQA & \textbf{0.44} & 0.43 \\
%
%
%\midrule
%
%\multicolumn{3}{l}{\textit{Strategy: 200 word chunks, 10\% overlap}} \\
%\midrule
%HotpotQA & \textbf{0.60} & 0.53
%\\
%MuSiQue & \textbf{0.36} & 0.35 \\
%NarrativeQA & \textbf{0.22} & 0.25 \\
%2WikiMQA & \textbf{0.57} & 0.54 \\
%
%MultifieldQA & \textbf{0.45} & 0.45 \\
%\midrule
%
%\multicolumn{3}{l}{\textit{Strategy: 300 word chunks, 10\% overlap}} \\
%\midrule
%HotpotQA & 0.59 & 0.56
%\\
%MuSiQue & 0.39 & 0.33 \\
%NarrativeQA & 0.26 & 0.24 \\
%2WikiMQA & 0.61 & 0.56 \\
%
%MultifieldQA & 0.46 & 0.46 \\
%\bottomrule
%\end{tabular}
%\end{table*}
\bibliography{custom}
\newpage
\appendix
\onecolumn
\section{Methods}
\label{sec:appendix}
\subsection{Dataset Statistics}
\label{dataset-stat}
The dataset statistics are presented in Table \ref{tab:dataset_stats}.
\begin{table}[H]
\centering
\begin{tabular}{lcc}
\toprule
\textbf{Dataset} & \textbf{No. of Instances} & \textbf{Avg. Length} \\
\midrule
\multicolumn{3}{c}{\textit{LongBench} \cite{bai-etal-2024-longbench}} \\
\midrule
HotpotQA & 200 & 9,151 \\
2WikiMQA & 200 & 4,887 \\
MuSiQue & 200 & 11,214 \\
MultifieldQA & 150 & 4,559 \\
\midrule
\multicolumn{3}{c}{\textit{$\infty$Bench} \cite{zhang-etal-2024-bench}} \\
\midrule
EN.QA & 351 & 192,600 \\
\bottomrule
\end{tabular}
\caption{Total number of queries and average length of contexts in number of words for each QA dataset.}
\label{tab:dataset_stats}
\end{table}
\subsection{Design choice of selecting chunks}
Despite the many possible strategies for constructing a chunk set, we adopt our proposed approach of selecting a fixed-size set of \( n = 5 \) chunks from the document chunk collection \( \mathcal{C} \) at a single \( \lambda \) value. This design enables a one-time computation of gMMR scores over all candidate chunks in \( \mathcal{C} \) at a particular $\lambda$, followed by a single evaluation of the resulting chunk-set using the Evaluator. However, it is also possible that the chunk-set selection strategy can be more adaptive at the cost of more computation, like iteratively selecting the best chunk and incrementally building the set which would requiring up to \( n \) additional Evaluator calls per query.
\subsection{System Prompts and Reproducibility}
\label{sec:systemprompts}
We release all our system prompts starting from the Planner, Evaluator and the Generator. Figure \ref{fig:planner} shows the prompt used for the Planner in DF-RAG. Figure \ref{fig:exec} shows the prompt used by our Evaluator. The Evaluator utilizes few shot prompting and the few shot examples specifically utilized are further detailed in Figure \ref{fig:few-shot} and Figure \ref{fig:few-shot-contd}. Our system can be reproduced by following these system prompts and the architecture proposed in Figure \ref{fig:df-ragvsvanilla}. We use the vanilla Hugging Face Instruct versions for all backbone LLMs.
\section{Additional Experimental Results}
\subsection{RAG with $\lambda$-Fixed gMMR:}
\label{sec:RAGWMMRcontd}
Figure~\ref{fig:RAGwMMR} presents a detailed performance comparison between vanilla RAG and RAG with $\lambda$-Fixed gMMR across the five benchmarks at the different context lengths settings with Llama 3.3 70B as the backbone LLM. In vanilla RAG, performance is invariant to $\lambda$, producing a horizontal line, whereas RRAG with $\lambda$-Fixed gMMR's performance varies with $\lambda$, with each benchmark exhibiting a clear peak at a given context length. For each benchmark, we find at least one context length contains a $\lambda$ where RAG with $\lambda$-Fixed gMMR outperforms vanilla RAG. At the 500-word context length, the benefits of using RAG with $\lambda$-Fixed gMMR is more prominent. whereas for 1000 and 1500-word context length settings, $F_1$ improvements are less substantial.
\subsection{Performance comparison with classical MMR vs gMMR}
\label{sec:eucvs}
We compare DF-RAG using classical MMR with DF-RAG using our proposed gMMR retrieval function. The only difference in these two retreival functions is the distance metric, where traditional MMR uses cosine similarity and gMMR uses Euclidean distance in a normalized embedding space. Experiments are conducted with 500-word chunks and Llama 3.3 70B as the backbone LLM (see Table \ref{tab:computation}). DF-RAG with gMMR outperforms the classical MMR variant on three out of five benchmarks, and closely matches its performance on the remaining two. Overall, gMMR yields higher average gains compared to classical MMR.
\begin{table}[h]
\centering
\begin{tabular}{lcc}
\toprule
\textbf{Benchmarks} & \textbf{classical MMR} & \textbf{gMMR} \\
\midrule
HotpotQA & 56.7 & \textbf{58.1} \\
MuSiQue & \textbf{34.1} & 34.0 \\
2WikiMQA & 53.6 & \textbf{58.1} \\
MultifieldQA & \textbf{49.0} & 48.3\\
En.QA & 28.2 & \textbf{29.3} \\
\bottomrule
\end{tabular}%
\caption{$F_1$ scores of DF-RAG performing with classical MMR vs gMMR}
\label{tab:computation}
\end{table}
%
%\subsection{Assessing complexity of DF-RAG vs Baselines}
%\label{sec:latency}
%To better understand computational efficiency, we compare our approach against baseline methods %along two key dimensions: the number of tokens processed and wall-clock latency. Table %\ref{tab:latency_comparison} summarizes the latency comparison, contextualized by our earlier %discussion on DF-RAG’s latency considerations. To isolate computational complexity, we further %construct a small subset of long-context examples, allowing us to more clearly capture the trade-%offs in token processing and response time across our selected baselines.
%
%
%
%
%\begin{table}[h]
%\centering
%\begin{tabular}{lcc}
%\toprule
%\textbf{Method} & \textbf{Tokens Processed} & \textbf{Latency} \\
%\midrule
%Long Context & $O(X)$ & $O(X)$ \\
%RAPTOR & $O(X)$ & $O(X)$ \\
%LongRAG & $O(Z)$ & $O(Z)$ \\
%Vanilla RAG & $O(Y)$ & $O(Y)$ \\
%DF-RAG & $O(Y)$ & $O(Y)$ \\
%\bottomrule
%\end{tabular}
%\caption{Computational characteristics of different strategies, comparing total tokens processed %and wall-clock latency. DF-RAG benefits from parallelization, making its latency scale with the %cost of a single executor call rather than the sum of all tokens processed.}
%\label{tab:latency_comparison}
%\end{table}
\subsection{DF-RAG with Binary-Search sampling}
\label{sec:bs}
DF-RAG’s latency is primarily determined by the number of LLM calls made by the Evaluator, which is in turn contingent upon the sampling strategy used to sample chunk sets. To improve efficiency, we introduce a binary-search–based sampling strategy, motivated by its effectiveness in locating peaks in unimodal distributions. This approach reduces the sampling complexity from $n$ to $\log n$. Binary-search sampling achieves comparable performance to uniform sampling across models (see Table \ref{tab:results_halfpage}), with only minimal drops in accuracy.
\begin{table*}[h]
\centering
\scriptsize
\begin{tabular}{l cc cc}
\toprule
\multirow{2}{*}{\textbf{Dataset}}
& \multicolumn{2}{c}{\textbf{DF-RAG + IC (uniform sampling)}}
& \multicolumn{2}{c}{\textbf{DF-RAG + IC (Binary-Search)}} \\
\cmidrule(lr){2-3} \cmidrule(lr){4-5}
& \textbf{Qwen 2.5 72B} & \textbf{Llama 3.3 70B}
& \textbf{Qwen 2.5 72B} & \textbf{Llama 3.3 70B} \\
\midrule
HotPotQA & 60.1 & 56.8 & 56.0 & 54.1 \\
MuSiQue & 32.7 & 34.0 & 30.3 & 33.4 \\
2WikiMQA & 56.8 & 55.2 & 54.3 & 50.5 \\
%\midrule
%
%\multicolumn{5}{c}{\textit{context length = 1000 words}} \\
%\midrule
%
%HotPotQA & 62.1 & 60.3 & 55.0 & 55.9 \\
%MuSiQue & 31.7 & 38.6 & 29.6 & 36.2 \\
%2WikiMQA & 59.7 & 63.3 & 55.5 & 57.4 \\
\bottomrule
\end{tabular}
\caption{Comparison of DF-RAG + IC with uniform sampling and Binary-Search based sampling.}
\label{tab:results_halfpage}
\end{table*}
\section{Error Analysis}
\label{sec:ea}
We analyze certain cases where DF-RAG both succeeds and fails and try to draw intuition about why such cases arise. In these cases, we compare DF-RAG with vanilla RAG to better understand the point of success and failure.\\
\subsection{DF-RAG successful cases}
In most successful cases, we find that DF-RAG is able to retrieve information that vanilla RAG overlooks. This often occurs when DF-RAG captures chunks that align with each reasoning step required to answer a multi-hop query by fetching diverse and complementary information, whereas vanilla RAG often misses key bridging information because such chunks have low surface similarity to the query and are therefore excluded by cosine-similarity–based retrieval. An illustrative example is provided in Table \ref{tab:success}. For the input query \textit{What is the place of birth of the performer of the song Ruleta (Inna Song)?}, DF-RAG retrieves both complementary pieces of information: \textit{Ruleta is recorded by Romanian singer Inna} and \textit{Inna Elena Alexandra Apostoleanu was born in Mangalia} (highled in the example). In contrast, vanilla RAG only retrieves the first piece of information about \textit{Ruleta being recorded by Inna}, but not the birthplace information. Vanilla RAG retrieves many chunks that mostly discuss Ruleta because they are highly relevant to the query, but this displaces the other information needed to make the final reasoning step and generate the answer, i.e., \textit{Inna being born in Mangalia}.\\
%We analyze certain cases where DF-RAG both succeeds and fails and try to draw intuition why such cases arises. For such cases, we compare DF-RAG head to head with vanilla RAG. In most success cases we see that DF-RAG is able to retrieve information that vanilla RAG overlooks. This often happens when DF-RAG captures chunks that align with each reasoning step required to answer a multi-hop query by fetching diverse complementary information, whereas vanilla RAG often misses key bridging information because such chunks have low surface similarity to the query and are therefore excluded by cosine-similarity–based retrieval. An illustrative example is provided in Table \ref{tab:success}. For an input query \textit{What is the place of birth of the performer of song Ruleta (Inna Song)?} DF-RAG first fetches both the complementary information: \textit{Ruleta is recoreded by Romanian singer Inna} and \textit{Inna Elena Alexandra Apostoleanu is born in Mangalia}. Whereas vanilla RAG only gets the first information about Ruleta being recorded by Inna and not the birthplace information. If we see, vanilla RAG fetches a lot of chunk that mostly talk about Ruleta because they are highly relevant to the query but this displaces the other information needed to perform the last reasoning jump for generating the answer, i.e Inna being born in Mangalia.\\
\subsection{DF-RAG failure cases}
Our analysis to visualize certain cases where DF-RAG fails revealed that DF-RAG sometiems fails to retrieve information that is needed for reasoning over multi-hop QAs, and this is a limitation common to RAG that stems from the inherent nature of information loss in retrieval. However, there are also cases where key information is retrieved appropriately, but due to the structure of the chunks presented, the Generator finds it difficult to reason effectively to generate the answer (see Table \ref{tab:failure}). For the question \textit{What is the place of birth of the director of the film Ninamaninja Kalpadukal?}, it is important first to find \textit{who the director of Ninamaninja Kalpadukal is} and second \textit{where his birthplace is}. If we look at the chunks retrieved by both DF-RAG and vanilla RAG, we can see that both pieces of information are retrieved by the systems (highlighted in the examples). However, it is important how the information is presented to the generator. For the chunks retrieved by DF-RAG, the chunk that talks about the village \textit{Methala} appears somewhat abruptly, making it difficult to decipher that Methala is the birthplace of the identified director of Ninamaninja Kalpadukal. By contrast, in the chunks retrieved by vanilla RAG, the information required for performing the second reasoning step comes right after introducing the director of Ninamaninja Kalpadukal, making it sound like the place where the director spent his early life, and thus the generator can bridge the gap to generate \textit{Methala} as the answer.
%We also extend our analysis to visualize where DF-RAG fails. In some certain cases DF-RAG fails to retrieve information that is needed for reasoning multi-hop QAs and this is a limitation that is common in RAG and stems from the inherent nature of information loss in RAG. However, there are also cases where key information is retrieved appropriately but due to the structure of chunks presented, the Generator LLM finds it difficult to reason appropriately to generate the answer. Such an example is presented in Table \ref{tab:failure}. For a given question \textit{What is the place of birth of the director of film Ninamaninja Kalpadukal?}, it is important to find first \textit{who the director of Ninamaninja Kalpaduka is} and second \textit{Where his birthplace is.} If we look into the chunks retrieved by both DF-RAG and vanilla RAG, we can see both these information is retrieved by the systems. However, It is important in how the information is presented to the generator. For the chunks retrieved by DF-RAG, the chunk that talks about the village \textit{Methala} comes as somewhat abrupt, making it difficult to decipher where the birthplace of the identified director of is Ninamaninja Kalpadukal. However, as for the chunks retrieved by vanilla RAG, the information required for performing the second jump comes right after introducing the director of Ninamaninja Kalpadukal making it sound like his place where the director spent his early life and thus the generator can bridge the gap to generate \textit{Methala} as the answer.
\subsection{Understanding performance trends across benchmarks}
\label{sec:perf}
\textbf{Multi-hop QA benchmarks:} We observe that DF-RAG is able to close a substantial portion of the performance gap between vanilla RAG and our Oracle on two of the three multi-hop QA benchmarks—2WikiMQA and HotpotQA. However, the improvements are smaller on MuSiQue for DF-RAG compared to vanilla RAG. One reason for this discrepancy is the relative complexity of the benchmarks. HotpotQA predominantly contains 2-hop questions, where the reasoning chain involves fewer bridges and is thus easier compared to questions requiring more hops. For instance, in the question: \textit{Prior to playing for Michigan State, Keith Nichol played football for a school located in what city?}, the reasoning needs to identify: 1) \textit{Identify the school Keith Nichol played for before Michigan State} and 2) \textit{Determine the city where that school is located}. Both of these questions can be answered solely by relying on the query as the entities of interested are directly placed there. By retrieving a sufficiently diverse set of information about \textit{Keith Nichol}, we can find details about his school prior to Michigan State. Similarly, by retrieving information about \textit{schools}, we can also identify the city in which the school is located.
\noindent In contrast, MuSiQue contains a greater number of hops per question. For example, the question \textit{When was the last time Peter Till's sports team beat the winner of the 1894–95 FA Cup?} requires essentially three hops: 1) \textit{Identify Peter Till's sports team}, 2) \textit{Identify the winner of the 1894–95 FA Cup}, and 3) \textit{Determine the last time the team identified in the first step defeated the winner of the 1894–95 FA Cup}. We can answer the first hop by relying on retrieving a sufficiently diverse retrieval set on Peter Till. We can also perform the second hop by retriving information about the winners of FA cup on 1894-1895. However the third hop needs connecting the entities we find in the first and second hop, which is impossible to guess without answering them and therefore retrieving information solely based on question provides minimal support. As a result, unlike in HotpotQA, simply relying on the question often provides limited guidance on what evidence needs to be retrieved to answer the final hop and hence both DF-RAG and Vanilla RAG fails. Moreover, MuSiQue mentions that for each question presented, it is compulsory to perform each hop as they filtered out questions where later hops could be answered without knowing the earlier answers, making no shortcut jumps possible and a difficult dataset to address at short context scenarios. \\
\noindent\textbf{Non multi-hop QA benchmarks:} DF-RAG also demonstrates improvements on MultifieldQA and En.QA. Although these datasets are not inherently multi-hop, their contexts often contain distracting information that appears relevant but leads models to incorrect answers. DF-RAG mitigates this by retrieving not only relevant but also diverse evidence, increasing the likelihood of capturing the true answer amid such noise. For example, in MultifieldQA, the question \textit{What is the score achieved by the authors for Track-2?} often misleads vanilla RAG. The retriever selects a chunk stating \textit{All of our submissions were the top submissions for each track, which surpassed the next best competitors by a margin of 4.5\% and 5.6\% for Track-1 and Track-2 respectively}, causing the model to output 5.6\%, which is the margin of improvement rather than the score. In contrast, DF-RAG successfully retrieves the chunk \textit{We achieve a score of 58.54\% for Track-1 and 85.61\% for Track-2}, which, despite surrounded with noisy information, provides the correct answer.
\begin{table*}[h!]
\centering
\renewcommand{\arraystretch}{1.3} % row spacing
\begin{tabular}{p{\linewidth}}
\hline
\textbf{Question:} What is the place of birth of the performer of song Ruleta (Inna Song)? \\
\hline
\textbf{Selected chunks by DF-RAG}:
EDM and reggaeton-influenced track, containing touches of Indian and Caribbean music. When asked by a Direct Lyrics interviewer to describe the lyrics and sound of "Ruleta", Inna said, "It's summerish, it's cool, it's fun and I dare to say... it's catchy and addictive!" Reception Upon its release, "Ruleta" received positive reviews from music critics. Currinn from CelebMix wrote, "this new song has Inna proving that she's multilingual", and compared the style of "Ruleta" to her "Heaven" (2016). Kevin Apaza, writing for Direct Lyrics thought, "summer doesn't officially start until Romanian pop queen Inna releases her summer single", further calling it
Passage 1: \hl{Inna Elena Alexandra Apostoleanu (born 16 October 1986), known professionally as Inna (stylized in all caps), is a Romanian singer and songwriter. Born in Mangalia and raised in Neptun, she studied political science at Ovidius University before meeting the Romanian trio Play \& Win and pursuing a music career.} She adopted the stage name "Alessandra" and a pop-rock style in 2008; later that year, she changed her stage name to "Inna" and began releasing house music. "Hot" (2008), her debut single, was a commercial success worldwide and topped the Romanian and Billboard's Hot Dance Airplay chart, among others.
mtvU for his video that depicted a love affair between a woman and her television, with the TV containing the green-screened Decemberists video footage. Passage 3: \hl{Ruleta (Inna song) "Ruleta" (English: "Roulette") is a song recorded by Romanian singer Inna}, featuring collaborative vocals by Romanian performer Erik, for her fifth studio album, Nirvana (2017) ... \\
\hline
\textbf{Selected Chunks by Vanilla RAG:} ... An editor of Bihoreanul praised the "attractive" rhythms of "Ruleta" and noted its potential commercial success in Romania.Commercially, the song debuted at number 32 on Romania's Airplay 100 for the week ending 9 July 2017 as the highest new entry, marking one of Inna's highest debuts on the chart. In late August 2017, "Ruleta" reached its peak position at number three, becoming the singer's highest charting single since "Bop Bop" (2015), which peaked at number two. The track also opened at number 39 on the French Club 40 chart for the week ending 15 July 2017 and
rose up to number 26 on the following week. "Ruleta" further charted in other countries, most notably reaching number two in Bulgaria. Music video An accompanying music video for "Ruleta" was shot at the Domeniul Greaca resort in Romania by Barna Nemethi, with Marius Apopei serving as the director of photography. Inna recalled the experience, "Actually, I was super chill in that day, super relaxed, just going with the flow. Most of the times, I'm kind of stressed out, because I want everything to be perfect. Not too many funny stuff, just having fun."The clip was uploaded onto her official
mtvU for his video that depicted a love affair between a woman and her television, with the TV containing the green-screened Decemberists video footage. Passage 3: \hl{Ruleta (Inna song) "Ruleta" (English: "Roulette") is a song recorded by Romanian singer Inna, featuring collaborative vocals by Romanian performer Erik, for her fifth studio album, Nirvana (2017)}. It was made available for digital download and streaming on 21 June 2017 by Global Records as the album's second single. The track was written by Inna, Erik, Marius Dia and Breyan Isaac, while production was handled by Sebastian Barac, Marcel Botezan and David Ciente. \\
\hline
%2017 by Global Records as the album's second single. The track was written by Inna, Erik, Marius Dia and Breyan Isaac, while production was handled by Sebastian Barac, Marcel Botezan and David Ciente. "Ruleta"
%und Theater and as yoga teacher. North was the mother of two children, her daughter was born in 2001 and her son in 2006. Her sister Ondria North works as make-up artist and hair stylist in the German film industry. She died in June 2019 at the age of 45 years from pancreatic cancer. Discography with Cultured PearlsAlbums 1996: Sing Dela Sing (German chart position 92, 3 weeks) 1997: Space Age Honeymoon (German chart position 54, 6 weeks) 1999: Liquefied Days (German chart position 19, 9 weeks) 2002: Life on a Tuesday (German chart position 74, 1 week)Singles 1996: Tic
%features lyrics in English and Spanish. It is an EDM and reggaeton-influenced song, with touches of Indian and Caribbean music. Music critics were positive towards the recording, praising its summery style and noting its commercial appeal. An accompanying music video for "Ruleta" was shot by Barna Nemethi at the Domeniul Greaca resort in Romania. It mainly portrays Inna and other people dancing to the song at a tennis court. Uploaded onto the singer's YouTube channel simultaneously with the single's release, it became trending on the platform worldwide. Commercially, the song reached the top five in Romania, Bulgaria and Turkey.\\
\textbf{Answer of DF-RAG:} Mangalia \\
\textbf{Vanilla RAG Answer:} Romania \\
\textbf{Golden Answer:} Mangalia \\
\hline
\end{tabular}
\caption{Selected chunks of a successful case by DF-RAG and Vanilla RAG. DF-RAG successfully retrieves the information needed to perform the 2nd hop to answer the query.}
\label{tab:success}
\end{table*}
\begin{table*}[t]
\centering
\renewcommand{\arraystretch}{1.3} % row spacing
\begin{tabular}{p{\linewidth}}
\hline
\textbf{Question:} What is the place of birth of the director of film Ninamaninja Kalpadukal? \\
\hline
\textbf{Selected chunks by DF-RAG}:
an assistant director for Tamil - Telugu movies in Newton Studios, he later distinguished himself as a producer, director and scriptwriter. His directorial debut, Ninamaninja Kalpadukal, which portrays the trials the Indo-China war, won 4 awards including the President's silver medal for best regional film, and the award for The Best Director. He directed about 6 movies and produced 2 movies on his own. He wrote the screen play and directed 4 episodes of Aithihyamala for Doordarshan. He also directed a tele-film titled Kudajadri. Filmography Passage 4: Peter Levin Peter Levin is an American director of film, television and theatre.
\hl{into a family known as "Kallil" in Methala, Near Kalady in Kerala, he completed his schooling in Paravoor and then was graduated in economics from Serhampur}, Bengal. He was an avid reader always and appreciated the nuances of literature. His first short story was published in the weekly Prasanna Keralam from Kottayam when he was studying in high school. Since then, his stories have appeared in weeklies. Writing career One of his first of his short stories was published in the weekly Prasanna Keralam from Kottayam, when he was studying in high school.The editor of Kaumudi, K Balakrishnan, lead him
Norwegian Museum of Cultural History. In 2010 he was decorated with the Royal Norwegian Order of St. Olav. Passage 8: S. N. Mathur S.N. Mathur was the Director of the Indian Intelligence Bureau between September 1975 and February 1980. He was also the Director General of Police in Punjab. Passage 9: Ian Barry (director) Ian Barry is an Australian director of film and TV. Select credits Waiting for Lucas (1973) (short) Stone (1974) (editor only) The Chain Reaction (1980) Whose Baby? (1986) (mini-series) Minnamurra (1989) Bodysurfer (1989) (mini-series) Ring of Scorpio (1990) (mini-series) Crimebroker (1993) Inferno (1998) (TV movie) Miss
Passage 1: \hl{Ninamaninja Kalpadukal Ninamaninja Kalpadukal (Bloodstained Footprints) is a 1963 Malayalam language film, directed by N. N. Pisharody and produced by N.K. Karunakara Pillai and Shobhana Parameswaran Nair}. The lead role is played by Prem Nazir, with Ambika, Sheela and Madhu, who debuted with this film. The film is based on a novel by Parappurath and portrays the trials the Indo-China war. It won the National Film Award for Best Feature Film in Malayalam. It was a super hit movie.There are many evergreen songs in the film, including ... \\ %"Mamalakalkkappurathu" (by P. B. Sreenivas) and "Anuraga Natakathil" (by K. P.
%and an Israeli culture entrepreneur. Biography Dana Blankstein was born in Switzerland in 1981 to theatre director Dedi Baron and Professor Alexander Blankstein. She moved to Israel in 1983 and grew up in Tel Aviv. Blankstein graduated from the Sam Spiegel Film and Television School, Jerusalem in 2008 with high honors. During her studies she worked as a personal assistant to directors Savi Gabizon on his film Nina's Tragedies and to Renen Schorr on his film The Loners. She also directed and shot 'the making of' film on Gavison's film Lost and Found. Her debut film Camping competed at the... \\
\hline
\textbf{Selected Chunks by Vanilla RAG:} an assistant director for Tamil - Telugu movies in Newton Studios, he later distinguished himself as a producer, director and scriptwriter. His directorial debut, Ninamaninja Kalpadukal, which portrays the trials the Indo-China war, won 4 awards including the President's silver medal for best regional film, and the award for The Best Director. He directed about 6 movies and produced 2 movies on his own. He wrote the screen play and directed 4 episodes of Aithihyamala for Doordarshan. He also directed a tele-film titled Kudajadri. Filmography Passage 4: Peter Levin Peter Levin is an American director of film, television and theatre.
Passage 1: \hl{Ninamaninja Kalpadukal Ninamaninja Kalpadukal (Bloodstained Footprints) is a 1963 Malayalam language film, directed by N. N. Pisharody and produced by N.K. Karunakara Pillai and Shobhana Parameswaran Nair}. The lead role is played by Prem Nazir, with Ambika, Sheela and Madhu, who debuted with this film. The film is based on a novel by Parappurath and portrays the trials the Indo-China war. It won the National Film Award for Best Feature Film in Malayalam. It was a super hit movie.There are many evergreen songs in the film, including "Mamalakalkkappurathu" (by P. B. Sreenivas) and "Anuraga Natakathil" (by K. P.
\hl{into a family known as "Kallil" in Methala, Near Kalady in Kerala, he completed his schooling in Paravoor and then was graduated in economics from Serhampur}, Bengal. He was an avid reader always and appreciated the nuances of literature. His first short story was published in the weekly Prasanna Keralam from Kottayam when he was studying in high school. Since then, his stories have appeared in weeklies...\\ %Writing career One of his first of his short stories was published in the weekly Prasanna Keralam from Kottayam, when he was studying in high school.The editor of Kaumudi, K Balakrishnan, lead him ... \\
%Norwegian Museum of Cultural History. In 2010 he was decorated with the Royal Norwegian Order of St. Olav. Passage 8: S. N. Mathur S.N. Mathur was the Director of the Indian Intelligence Bureau between September 1975 and February 1980. He was also the Director General of Police in Punjab. Passage 9: Ian Barry (director) Ian Barry is an Australian director of film and TV. Select credits Waiting for Lucas (1973) (short) Stone (1974) (editor only) The Chain Reaction (1980) Whose Baby? (1986) (mini-series) Minnamurra (1989) Bodysurfer (1989) (mini-series) Ring of Scorpio (1990) (mini-series) Crimebroker (1993) Inferno (1998) (TV movie) Miss
%weekly - a story titled Aandal Puram Pogum Vazhi (En route to Aandal Puram). Shree Books, Aluwa has published this story as a book. He also wrote the screen play for about 40 dramatic plays for Aakashavani Thrissur. He wrote screenplays for the following too - Hiranya Garbham, Sarppa Sathram, Ivideyo Naalathe Sooryodayam, Vishathan Kaavilinnarattu - each of these plays were meant for various art groups. His famous novels are Virunnusala & Vellam.He was unmarried and a resident of Radha Niwas, Kanjoor. He died on 30 August 2008. Film career His association with film industry lasted 30 years. Starting as
\hline
\textbf{Answer of DF-RAG:} not specified in the provided passages. \\
\textbf{Vanilla RAG Answer:} methala, near kalady in kerala \\
\textbf{Golden Answer:} Methala \\
\hline
\end{tabular}
\caption{Selected chunks of a failure case by DF-RAG and Vanilla RAG. DF-RAG successfully retrieves the information needed but fails to stitch the reasoning gap to answer the query.}
\label{tab:failure}
\end{table*}
%\subsection{DF-RAG with other Baselines}
%\subsection{Performance variation with model families}
\begin{figure*}[h]
\centering
% --- Column Headers ---
\parbox[c][1em][c]{2em}{}% Ghost parbox for alignment
\begin{minipage}[c][1em][c]{\textwidth}
\centering
\begin{subfigure}[b]{0.3\textwidth}
\centering\textbf{context length = 500 words}
\end{subfigure}%
\hfill%
\begin{subfigure}[b]{0.3\textwidth}
\centering\textbf{context length = 1000 words}
\end{subfigure}%
\hfill%
\begin{subfigure}[b]{0.3\textwidth}
\centering\textbf{context length = 1500 words}
\end{subfigure}
\end{minipage}
\vspace{0.2cm}
% --- Row 1: Hotpot QA ---
\parbox[c][4cm][c]{2em}{\rotatebox{90}{\centering \textbf{Hotpot QA}}}%
\begin{minipage}[c][4cm][c]{0.9\textwidth}
\centering
\begin{subfigure}[b]{0.33\textwidth}
\centering
\includegraphics[width=\textwidth]{images/c500/hotpotqa500.png}
\label{fig:plot1}
\end{subfigure}%
\hfill%
\begin{subfigure}[b]{0.33\textwidth}
\centering
\includegraphics[width=\textwidth]{images/c1000/hotpotqa1000.png}
\label{fig:plot2}
\end{subfigure}%
\hfill%
\begin{subfigure}[b]{0.33\textwidth}
\centering
\includegraphics[width=\textwidth]{images/c1500/hotpotqa1500.png}
\label{fig:plot3}
\end{subfigure}
\end{minipage}
\vspace{0.5cm}
% --- Row 2: MuSiQue ---
\parbox[c][4cm][c]{2em}{\rotatebox{90}{\centering \textbf{MuSiQue}}}%
\begin{minipage}[c][4cm][c]{0.9\textwidth}
\centering
\begin{subfigure}[b]{0.33\textwidth}
\centering
\includegraphics[width=\textwidth]{ images/c500/musique500.png}
\label{fig:plot4}
\end{subfigure}%
\hfill%
\begin{subfigure}[b]{0.33\textwidth}
\centering
\includegraphics[width=\textwidth]{ images/c1000/musique1000.png}
\label{fig:plot5}
\end{subfigure}%
\hfill%
\begin{subfigure}[b]{0.33\textwidth}
\centering
\includegraphics[width=\textwidth]{ images/c1500/musique1500.png}
\label{fig:plot6}
\end{subfigure}
\end{minipage}
\vspace{0.5cm}
% --- Row 4: 2WikiMQA ---
\parbox[c][4cm][c]{2em}{\rotatebox{90}{\centering \textbf{2WikiMQA}}}%
\begin{minipage}[c][4cm][c]{0.9\textwidth}
\centering
\begin{subfigure}[b]{0.33\textwidth}
\centering
\includegraphics[width=\textwidth]{ images/c500/2wikimqa500.png}
\label{fig:plot10}
\end{subfigure}%
\hfill%
\begin{subfigure}[b]{0.33\textwidth}
\centering
\includegraphics[width=\textwidth]{ images/c1000/2wikimqa1000.png}
\label{fig:plot11}
\end{subfigure}%
\hfill%
\begin{subfigure}[b]{0.33\textwidth}
\centering
\includegraphics[width=\textwidth]{ images/c1500/2wikimqa1500.png}
\label{fig:plot12}
\end{subfigure}
\end{minipage}
\vspace{0.5cm}
% --- Row 5: MultiFieldQA ---
\parbox[c][4cm][c]{2em}{\rotatebox{90}{\centering \textbf{MultiFieldQA}}}%
\begin{minipage}[c][4cm][c]{0.9\textwidth}
\centering
\begin{subfigure}[b]{0.33\textwidth}
\centering
\includegraphics[width=\textwidth]{ images/c500/multifieldqa_en500.png}
\label{fig:plot13}
\end{subfigure}%
\hfill%
\begin{subfigure}[b]{0.33\textwidth}
\centering
\includegraphics[width=\textwidth]{ images/c1000/multifieldqa_en1000.png}
\label{fig:plot14}
\end{subfigure}%
\hfill%
\begin{subfigure}[b]{0.33\textwidth}
\centering
\includegraphics[width=\textwidth]{ images/c1500/multifieldqa_en1500.png}
\label{fig:plot15}
\end{subfigure}
\end{minipage}
\vspace{0.5cm}
% --- Row 3: Narrative QA ---
\parbox[c][4cm][c]{2em}{\rotatebox{90}{\centering \textbf{EN.QA}}}%
\begin{minipage}[c][4cm][c]{0.9\textwidth}
\centering
\begin{subfigure}[b]{0.33\textwidth}
\centering
\includegraphics[width=\textwidth]{ images/c500/longbook_qa_eng500.png}
\label{fig:plot7}
\end{subfigure}%
\hfill%
\begin{subfigure}[b]{0.33\textwidth}
\centering
\includegraphics[width=\textwidth]{ images/c1000/longbook_qa_eng1000.png}
\label{fig:plot8}
\end{subfigure}%
\hfill%
\begin{subfigure}[b]{0.33\textwidth}
\centering
\includegraphics[width=\textwidth]{ images/c1500/longbook_qa_eng1500.png}
\label{fig:plot9}
\end{subfigure}
\end{minipage}
\caption{$F_1$ Scores comparison of RAG with $\lambda$-Fixed gMMR vs vanilla RAG at different chunk sizes for our 5 benchmarks utilizing Llama 3.3 70B as the backbone LLM.}
\label{fig:RAGwMMR}
\end{figure*}
\begin{figure*}[t]
\begin{tcolorbox}[
colback=white,
colframe=gray,
boxrule=1pt,
arc=0pt,
width=\textwidth,
title={\textbf{Planner}},
fonttitle=\normalsize
]
Instruction: You are an expert question decomposer planner. \\[6pt]
Task: Break the user's \textbf{single question} into the minimal
ordered list of sub-questions that must all be answered
to fully answer the original question harnessing your zero-shot capabilites.
Follow the output format exactly as shown in the example below to generate a plan (a list of sub-questions).
\vspace{0.3cm}
Question: Where was the author of the book `Jacob' Born?
\vspace{0.3cm}
Output Planner :\\
1) Identify the author of Jacob\\
2) Identify where the author was born
\vspace{0.3cm}
Only produce the PLAN section — no extra commentary for the following question:
\vspace{0.3cm}
Question: \textcolor{blue}{<Question>}
\end{tcolorbox}
\caption{Prompt used by the Planner}
\label{fig:planner}
\end{figure*}
\begin{figure*}[t]
\begin{tcolorbox}[
colback=white,
colframe=gray,
boxrule=1pt,
arc=0pt,
width=\textwidth,
title={\textbf{Evaluator}},
fonttitle=\normalsize
]
Instruction: You are a strict judge evaluating text chunks. Only count information that is explicitly stated. \\[6pt]
Task: Score each question 0-5 based on how directly it can be answered:\\
5: EXPLICIT answer with specific facts/numbers stated directly\\
4: Clear and complete answer with direct relevant information present\\
3: Mostly complete answer, minor missing detail, but can be inferred\\
2: Partial information, key details missing\\
1: Very limited relevance, mostly vague \\
0: No relevant information or requires inference\\
STRICT RULES: \\
- Only count information DIRECTLY STATED in chunks \\
- Do NOT give points for logical leaps or inferences \\
- Do NOT give points for vague or tangential content \\
Your reasoning for the score values you provide should be within compact. **You MUST remember this.**
At the end you must answer in the format: Total Score: <Total Score> \\
Follow the examples for understanding the task: \\
\textcolor{red}{<Few Shot Examples>} \\
Now do the same for the following: \\
Plan: \textcolor{red}{<Plan>}
Chunks:
\textcolor{red}{<Candidate Chunk Set>} \\
\end{tcolorbox}
\caption{Prompt used by the Evaluator}
\label{fig:exec}
\end{figure*}
\begin{figure*}[t]
\begin{tcolorbox}[
colback=white,
colframe=gray,
boxrule=1pt,
arc=0pt,
width=\textwidth,
title={\textbf{Few Shot examples for Evaluator}},
fonttitle=\normalsize
]
--------------------------------------------------------------------------------------------------------------------
\\
Example 1
\\
--------------------------------------------------------------------------------------------------------------------
\\
Plan:\\
1) Find the protagonist in the movie 'Inception'\\
2) Find the protagonist's birthplace\\
3) Find the population of that city\\
Chunks:\\
"Inception is a 2010 science fiction film starring Leonardo DiCaprio as Dom Cobb, a professional thief who infiltrates people's dreams..."
"Leonardo DiCaprio was born in Los Angeles, California, United States on November 11, 1974..."
"Los Angeles is the most populous city in California with a population of approximately 4 million people as of 2023..."\\
For each question in the plan, judge if the information needed is clearly and explicitly present in the chunks. The chunk set should collectively cover all different types of information required by the steps.\\
1. Score: 5. Short Explanation: Leonardo DiCaprio is explicitly named as the protagonist.\\
2. Score: 5. Short Explanation: Born in Los Angeles, California is directly stated.\\
3. Score: 5. Short Explanation: Population of 4 million is explicitly mentioned.\\
Total Score: 15\\
--------------------------------------------------------------------------------------------------------------------
\\
Example 2
\\
--------------------------------------------------------------------------------------------------------------------
\\
Plan:\\
1) Find the director of the movie 'Titanic'\\
2) Find the director's net worth\\
3) Find the director's upcoming projects\\
Chunks:\\
"Titanic is a 1997 epic romance and disaster film. The movie was a massive box office success..."
"The film won 11 Academy Awards including Best Picture and Best Director..."
"Leonardo DiCaprio and Kate Winslet starred as the main characters in this epic love story..."\\
For each question in the plan, judge if the information needed is clearly and explicitly present in the chunks. The chunk set should collectively cover all different types of information required by the steps.\\
1. Score: 0. Short Explanation: Director name not mentioned, only that it won Best Director\\
2. Score: 0. Short Explanation: No financial information about anyone provided\\
3. Score: 0. Short Explanation: No information about future projects mentioned\\
Total Score: 0\\
\end{tcolorbox}
\caption{Few Shot Examples used by the Evaluator}
\label{fig:few-shot}
\end{figure*}
\begin{figure*}[t]
\begin{tcolorbox}[
colback=white,
colframe=gray,
boxrule=1pt,
arc=0pt,
width=\textwidth,
title={\textbf{Few Shot Examples contd.}},
fonttitle=\normalsize
]
-------------------------------------------------------------------------------------------------------------------
\\
Example 3
\\
--------------------------------------------------------------------------------------------------------------------
\\
Plan:\\
1) Find the author of 'Harry Potter'\\
2) Find the author's age\\
3) Find the number of books in the series\\
Chunks:
"Harry potter, the book written by the sister of James Franco is fantasy book series that became globally popular..."
" James Franco and along with all his siblings travelled to Europe..."
"In Europe, Franco and JK Rowling played Hokkey like all the siblings"
"The series consists of seven main novels plus several companion books..."
"Rowling has become one of the most successful authors in modern publishing..."\\
For each question in the plan, judge if the information needed is clearly and explicitly present in the chunks. The chunk set should collectively cover all different types of information required by the steps.
1. Score: 3. Short Explanation: J.K. Rowling is inferred named as the author from the relationships.\\
2. Score: 0. Short Explanation: Age not mentioned anywhere in the chunks.\\
3. Score: 5. Short Explanation: Seven main novels is directly stated.\\
Total Score: 8\\
\end{tcolorbox}
\caption{Few Shot Examples used by the Evaluator contd.}
\label{fig:few-shot-contd}
\end{figure*}
\begin{figure*}[t]
\begin{tcolorbox}[
colback=white,
colframe=gray,
boxrule=1pt,
arc=0pt,
width=\textwidth,
title={\textbf{Generator}},
fonttitle=\normalsize
]
Instruction: You are a question answering assistant. \\[6pt]
Task:
Answer the question based on
the given passages. Only give
me the answer and do not output
any other words. The following
are given passages.
\textcolor{red}{<Context>}
Answer the question based on
the given passages. Only give
me the answer and do not output
any other words.
Question: \textcolor{red}{<Query>}
Answer:
\end{tcolorbox}
\caption{Prompt used by the Generator}
\label{fig: gen-prompt}
\end{figure*}
%\begin{table*}[h!]
%\centering
%\caption{Pre-LLM Recall for each Dataset}
%\label{tab:your_label_here}
%\begin{tabular}{lcc}
%\toprule
%\textbf{Dataset} & \textbf{Top-K} & \textbf{DF-RAG} \\
%\midrule
%HotpotQA & 0.54 & 0.61 \\
%MuSiQue & 0.23 & \\
%NarrativeQA & NA & NA \\
%2WikiMQA & 0.64 & 0.68 \\
%MultifieldQA & 0.25 & 0.26 \\
%\bottomrule
%\end{tabular}
%\end{table*}
\end{document}
\begin{thebibliography}{47}
\providecommand{\natexlab}[1]{#1}
\bibitem[{Agrawal et~al.(2009)Agrawal, Gollapudi, Halverson, and Ieong}]{agrawal2009diversifying}
Rakesh Agrawal, Sreenivas Gollapudi, Alan Halverson, and Samuel Ieong. 2009.
\newblock \href {https://doi.org/10.1145/1498759.1498766} {Diversifying search results}.
\newblock In \emph{Proceedings of the Second ACM International Conference on Web Search and Data Mining (WSDM)}, pages 5--14. ACM.
\bibitem[{Asai et~al.(2024)Asai, Wu, Wang, Sil, and Hajishirzi}]{asai2024selfrag}
Akari Asai, Zeqiu Wu, Yizhong Wang, Avirup Sil, and Hannaneh Hajishirzi. 2024.
\newblock \href {https://openreview.net/forum?id=hSyW5go0v8} {Self-{RAG}: Learning to retrieve, generate, and critique through self-reflection}.
\newblock In \emph{The Twelfth International Conference on Learning Representations}.
\bibitem[{Bai et~al.(2024)Bai, Lv, Zhang, Lyu, Tang, Huang, Du, Liu, Zeng, Hou, Dong, Tang, and Li}]{bai-etal-2024-longbench}
Yushi Bai, Xin Lv, Jiajie Zhang, Hongchang Lyu, Jiankai Tang, Zhidian Huang, Zhengxiao Du, Xiao Liu, Aohan Zeng, Lei Hou, Yuxiao Dong, Jie Tang, and Juanzi Li. 2024.
\newblock \href {https://doi.org/10.18653/v1/2024.acl-long.172} {{L}ong{B}ench: A bilingual, multitask benchmark for long context understanding}.
\newblock In \emph{Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)}, pages 3119--3137, Bangkok, Thailand. Association for Computational Linguistics.
\bibitem[{Cao et~al.(2019)Cao, Fang, and Tao}]{cao-etal-2019-bag}
Yu~Cao, Meng Fang, and Dacheng Tao. 2019.
\newblock \href {https://doi.org/10.18653/v1/N19-1032} {{BAG}: Bi-directional attention entity graph convolutional network for multi-hop reasoning question answering}.
\newblock In \emph{Proceedings of the 2019 Conference of the North {A}merican Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)}, pages 357--362, Minneapolis, Minnesota. Association for Computational Linguistics.
\bibitem[{Carbonell and Goldstein(1998)}]{carbonell1998mmr}
Jaime Carbonell and Jade Goldstein. 1998.
\newblock The use of mmr, diversity-based reranking for reordering documents and producing summaries.
\newblock In \emph{Proceedings of the 21st Annual International ACM SIGIR Conference on Research and Development in Information Retrieval}, pages 335--336.
\bibitem[{Chen et~al.(2024)Chen, Xu, Liang, He, Pang, Yu, Song, Liu, Zhou, Zhang et~al.}]{chen2024not}
Xingyu Chen, Jiahao Xu, Tian Liang, Zhiwei He, Jianhui Pang, Dian Yu, Linfeng Song, Qiuzhi Liu, Mengfei Zhou, Zhuosheng Zhang, and 1 others. 2024.
\newblock Do not think that much for 2+ 3=? on the overthinking of o1-like llms.
\newblock \emph{arXiv preprint arXiv:2412.21187}.
\bibitem[{Clarke et~al.(2008)Clarke, Kolla, Cormack, Vechtomova, Ashkan, B{\"u}ttcher, and MacKinnon}]{clarke2008novelty}
Charles L.~A. Clarke, Mounia Kolla, Gordon~V. Cormack, Olga Vechtomova, Azin Ashkan, Stefan B{\"u}ttcher, and Ian MacKinnon. 2008.
\newblock \href {https://doi.org/10.1145/1390334.1390446} {Novelty and diversity in information retrieval evaluation}.
\newblock In \emph{Proceedings of the 31st Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR)}, pages 659--666. ACM.
\bibitem[{Ding et~al.(2019)Ding, Zhou, Chen, Yang, and Tang}]{ding-etal-2019-cognitive}
Ming Ding, Chang Zhou, Qibin Chen, Hongxia Yang, and Jie Tang. 2019.
\newblock Cognitive graph for multi-hop reading comprehension at scale.
\newblock In \emph{Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics}, pages 2694--2703, Florence, Italy. Association for Computational Linguistics.
\bibitem[{Fang et~al.(2025)Fang, Ma, and Wang}]{fang2025thinkless}
Gongfan Fang, Xinyin Ma, and Xinchao Wang. 2025.
\newblock Thinkless: Llm learns when to think.
\newblock \emph{arXiv preprint arXiv:2505.13379}.
\bibitem[{Fang et~al.(2020)Fang, Sun, Gan, Pillai, Wang, and Liu}]{fang-etal-2020-hierarchical}
Yuwei Fang, Siqi Sun, Zhe Gan, Rohit Pillai, Shuohang Wang, and Jingjing Liu. 2020.
\newblock Hierarchical graph network for multi-hop question answering.
\newblock In \emph{Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)}, pages 8823--8838, Online. Association for Computational Linguistics.
\bibitem[{Grattafiori et~al.(2024)Grattafiori, Dubey, Jauhri, Pandey, Kadian, Al-Dahle, Letman, Mathur, Schelten, Vaughan, Yang, Fan, Goyal, Hartshorn, Yang, Mitra, Sravankumar, Korenev, Hinsvark, Rao, Zhang, Rodriguez, Gregerson, Spataru, Roziere, Biron, Tang, Chern, Caucheteux, Nayak, Bi, Marra, McConnell, Keller, Touret, Wu, Wong, Ferrer, Nikolaidis, Allonsius, Song, Pintz, Livshits, Wyatt, Esiobu, Choudhary, Mahajan, Garcia-Olano, Perino, Hupkes, Lakomkin, AlBadawy, Lobanova, Dinan, Smith, Radenovic, Guzmán, Zhang, Synnaeve, Lee, Anderson, Thattai, Nail, Mialon, Pang, Cucurell, Nguyen, Korevaar, Xu, Touvron, Zarov, Ibarra, Kloumann, Misra, Evtimov, Zhang, Copet, Lee, Geffert, Vranes, Park, Mahadeokar, Shah, van~der Linde, Billock, Hong, Lee, Fu, Chi, Huang, Liu, Wang, Yu, Bitton, Spisak, Park, Rocca, Johnstun, Saxe, Jia, Alwala, Prasad, Upasani, Plawiak, Li, Heafield, Stone, El-Arini, Iyer, Malik, Chiu, Bhalla, Lakhotia, Rantala-Yeary, van~der Maaten, Chen, Tan, Jenkins, Martin, Madaan, Malo, Blecher,
Landzaat, de~Oliveira, Muzzi, Pasupuleti, Singh, Paluri, Kardas, Tsimpoukelli, Oldham, Rita, Pavlova, Kambadur, Lewis, Si, Singh, Hassan, Goyal, Torabi, Bashlykov, Bogoychev, Chatterji, Zhang, Duchenne, Çelebi, Alrassy, Zhang, Li, Vasic, Weng, Bhargava, Dubal, Krishnan, Koura, Xu, He, Dong, Srinivasan, Ganapathy, Calderer, Cabral, Stojnic, Raileanu, Maheswari, Girdhar, Patel, Sauvestre, Polidoro, Sumbaly, Taylor, Silva, Hou, Wang, Hosseini, Chennabasappa, Singh, Bell, Kim, Edunov, Nie, Narang, Raparthy, Shen, Wan, Bhosale, Zhang, Vandenhende, Batra, Whitman, Sootla, Collot, Gururangan, Borodinsky, Herman, Fowler, Sheasha, Georgiou, Scialom, Speckbacher, Mihaylov, Xiao, Karn, Goswami, Gupta, Ramanathan, Kerkez, Gonguet, Do, Vogeti, Albiero, Petrovic, Chu, Xiong, Fu, Meers, Martinet, Wang, Wang, Tan, Xia, Xie, Jia, Wang, Goldschlag, Gaur, Babaei, Wen, Song, Zhang, Li, Mao, Coudert, Yan, Chen, Papakipos, Singh, Srivastava, Jain, Kelsey, Shajnfeld, Gangidi, Victoria, Goldstand, Menon, Sharma, Boesenberg,
Baevski, Feinstein, Kallet, Sangani, Teo, Yunus, Lupu, Alvarado, Caples, Gu, Ho, Poulton, Ryan, Ramchandani, Dong, Franco, Goyal, Saraf, Chowdhury, Gabriel, Bharambe, Eisenman, Yazdan, James, Maurer, Leonhardi, Huang, Loyd, Paola, Paranjape, Liu, Wu, Ni, Hancock, Wasti, Spence, Stojkovic, Gamido, Montalvo, Parker, Burton, Mejia, Liu, Wang, Kim, Zhou, Hu, Chu, Cai, Tindal, Feichtenhofer, Gao, Civin, Beaty, Kreymer, Li, Adkins, Xu, Testuggine, David, Parikh, Liskovich, Foss, Wang, Le, Holland, Dowling, Jamil, Montgomery, Presani, Hahn, Wood, Le, Brinkman, Arcaute, Dunbar, Smothers, Sun, Kreuk, Tian, Kokkinos, Ozgenel, Caggioni, Kanayet, Seide, Florez, Schwarz, Badeer, Swee, Halpern, Herman, Sizov, Guangyi, Zhang, Lakshminarayanan, Inan, Shojanazeri, Zou, Wang, Zha, Habeeb, Rudolph, Suk, Aspegren, Goldman, Zhan, Damlaj, Molybog, Tufanov, Leontiadis, Veliche, Gat, Weissman, Geboski, Kohli, Lam, Asher, Gaya, Marcus, Tang, Chan, Zhen, Reizenstein, Teboul, Zhong, Jin, Yang, Cummings, Carvill, Shepard, McPhie,
Torres, Ginsburg, Wang, Wu, U, Saxena, Khandelwal, Zand, Matosich, Veeraraghavan, Michelena, Li, Jagadeesh, Huang, Chawla, Huang, Chen, Garg, A, Silva, Bell, Zhang, Guo, Yu, Moshkovich, Wehrstedt, Khabsa, Avalani, Bhatt, Mankus, Hasson, Lennie, Reso, Groshev, Naumov, Lathi, Keneally, Liu, Seltzer, Valko, Restrepo, Patel, Vyatskov, Samvelyan, Clark, Macey, Wang, Hermoso, Metanat, Rastegari, Bansal, Santhanam, Parks, White, Bawa, Singhal, Egebo, Usunier, Mehta, Laptev, Dong, Cheng, Chernoguz, Hart, Salpekar, Kalinli, Kent, Parekh, Saab, Balaji, Rittner, Bontrager, Roux, Dollar, Zvyagina, Ratanchandani, Yuvraj, Liang, Alao, Rodriguez, Ayub, Murthy, Nayani, Mitra, Parthasarathy, Li, Hogan, Battey, Wang, Howes, Rinott, Mehta, Siby, Bondu, Datta, Chugh, Hunt, Dhillon, Sidorov, Pan, Mahajan, Verma, Yamamoto, Ramaswamy, Lindsay, Lindsay, Feng, Lin, Zha, Patil, Shankar, Zhang, Zhang, Wang, Agarwal, Sajuyigbe, Chintala, Max, Chen, Kehoe, Satterfield, Govindaprasad, Gupta, Deng, Cho, Virk, Subramanian, Choudhury,
Goldman, Remez, Glaser, Best, Koehler, Robinson, Li, Zhang, Matthews, Chou, Shaked, Vontimitta, Ajayi, Montanez, Mohan, Kumar, Mangla, Ionescu, Poenaru, Mihailescu, Ivanov, Li, Wang, Jiang, Bouaziz, Constable, Tang, Wu, Wang, Wu, Gao, Kleinman, Chen, Hu, Jia, Qi, Li, Zhang, Zhang, Adi, Nam, Yu, Wang, Zhao, Hao, Qian, Li, He, Rait, DeVito, Rosnbrick, Wen, Yang, Zhao, and Ma}]{grattafiori2024llama3herdmodels}
Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, Amy Yang, Angela Fan, Anirudh Goyal, Anthony Hartshorn, Aobo Yang, Archi Mitra, Archie Sravankumar, Artem Korenev, Arthur Hinsvark, and 542 others. 2024.
\newblock \href {https://arxiv.org/abs/2407.21783} {The llama 3 herd of models}.
\newblock \emph{Preprint}, arXiv:2407.21783.
\bibitem[{Guti\'{e}rrez et~al.(2024)Guti\'{e}rrez, Shu, Gu, Yasunaga, and Su}]{hipporag}
Bernal~Jim\'{e}nez Guti\'{e}rrez, Yiheng Shu, Yu~Gu, Michihiro Yasunaga, and Yu~Su. 2024.
\newblock Hipporag: neurobiologically inspired long-term memory for large language models.
\newblock In \emph{Proceedings of the 38th International Conference on Neural Information Processing Systems}, NIPS '24, Red Hook, NY, USA. Curran Associates Inc.
\bibitem[{Guu et~al.(2020)Guu, Lee, Tung, Pasupat, and Chang}]{pmlr-v119-guu20a}
Kelvin Guu, Kenton Lee, Zora Tung, Panupong Pasupat, and Mingwei Chang. 2020.
\newblock \href {https://proceedings.mlr.press/v119/guu20a.html} {Retrieval augmented language model pre-training}.
\newblock In \emph{Proceedings of the 37th International Conference on Machine Learning}, volume 119 of \emph{Proceedings of Machine Learning Research}, pages 3929--3938. PMLR.
\bibitem[{Hassid et~al.(2025)Hassid, Synnaeve, Adi, and Schwartz}]{hassid2025don}
Michael Hassid, Gabriel Synnaeve, Yossi Adi, and Roy Schwartz. 2025.
\newblock Don't overthink it. preferring shorter thinking chains for improved llm reasoning.
\newblock \emph{arXiv preprint arXiv:2505.17813}.
\bibitem[{He et~al.(2022)He, Zhang, and Roth}]{he2022rethinking}
Hangfeng He, Hongming Zhang, and Dan Roth. 2022.
\newblock \href {https://arxiv.org/abs/2301.00303} {Rethinking with retrieval: Faithful large language model inference}.
\newblock \emph{arXiv preprint arXiv:2301.00303}.
\bibitem[{Ho et~al.(2020)Ho, Duong~Nguyen, Sugawara, and Aizawa}]{xanh2020_2wikimultihop}
Xanh Ho, Anh-Khoa Duong~Nguyen, Saku Sugawara, and Akiko Aizawa. 2020.
\newblock Constructing a multi-hop {QA} dataset for comprehensive evaluation of reasoning steps.
\newblock In \emph{Proceedings of the 28th International Conference on Computational Linguistics}, pages 6609--6625, Barcelona, Spain (Online).
\bibitem[{Kandpal et~al.(2023)Kandpal, Deng, Roberts, Wallace, and Raffel}]{pmlr-v202-kandpal23a}
Nikhil Kandpal, Haikang Deng, Adam Roberts, Eric Wallace, and Colin Raffel. 2023.
\newblock \href {https://proceedings.mlr.press/v202/kandpal23a.html} {Large language models struggle to learn long-tail knowledge}.
\newblock In \emph{Proceedings of the 40th International Conference on Machine Learning}, volume 202 of \emph{Proceedings of Machine Learning Research}, pages 15696--15707. PMLR.
\bibitem[{Khattab et~al.(2021)Khattab, Potts, and Zaharia}]{khattab}
Omar Khattab, Christopher Potts, and Matei Zaharia. 2021.
\newblock Baleen: robust multi-hop reasoning at scale via condensed retrieval.
\newblock In \emph{Proceedings of the 35th International Conference on Neural Information Processing Systems}, NIPS '21, Red Hook, NY, USA. Curran Associates Inc.
\bibitem[{Lewis et~al.(2020)Lewis, Perez, Piktus, Petroni, Karpukhin, Goyal, K{\"u}ttler, Lewis, Yih, Rockt{\"a}schel, Riedel, and Kiela}]{lewis2020rag}
Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich K{\"u}ttler, Mike Lewis, Wen-tau Yih, Tim Rockt{\"a}schel, Sebastian Riedel, and Douwe Kiela. 2020.
\newblock Retrieval-augmented generation for knowledge-intensive nlp tasks.
\newblock In \emph{Advances in Neural Information Processing Systems}.
\bibitem[{Li and Du(2023)}]{li-du-2023-leveraging}
Ruosen Li and Xinya Du. 2023.
\newblock \href {https://doi.org/10.18653/v1/2023.findings-emnlp.452} {Leveraging structured information for explainable multi-hop question answering and reasoning}.
\newblock In \emph{Findings of the Association for Computational Linguistics: EMNLP 2023}, pages 6779--6789, Singapore. Association for Computational Linguistics.
\bibitem[{Li et~al.(2025)Li, Chen, Yu, Lin, Lu, Tang, Huang, Han, Sun, and Li}]{li2025structrag}
Zhuoqun Li, Xuanang Chen, Haiyang Yu, Hongyu Lin, Yaojie Lu, Qiaoyu Tang, Fei Huang, Xianpei Han, Le~Sun, and Yongbin Li. 2025.
\newblock \href {https://openreview.net/forum?id=GhexuBLxbO} {Struct{RAG}: Boosting knowledge intensive reasoning of {LLM}s via inference-time hybrid information structurization}.
\newblock In \emph{The Thirteenth International Conference on Learning Representations}.
\bibitem[{Li et~al.(2024)Li, Li, Zhang, Mei, and Bendersky}]{li-etal-2024-retrieval}
Zhuowan Li, Cheng Li, Mingyang Zhang, Qiaozhu Mei, and Michael Bendersky. 2024.
\newblock \href {https://doi.org/10.18653/v1/2024.emnlp-industry.66} {Retrieval augmented generation or long-context {LLM}s? a comprehensive study and hybrid approach}.
\newblock In \emph{Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing: Industry Track}, pages 881--893, Miami, Florida, US. Association for Computational Linguistics.
\bibitem[{Liu et~al.(2024)Liu, Lin, and Liu}]{liu2024raghelpreasoningllm}
Jingyu Liu, Jiaen Lin, and Yong Liu. 2024.
\newblock \href {https://arxiv.org/abs/2410.02338} {How much can rag help the reasoning of llm?}
\newblock \emph{Preprint}, arXiv:2410.02338.
\bibitem[{Liu et~al.(2025)Liu, Qi, He, Bisazza, Sachan, and Cotterell}]{liu-etal-2025-pointwise}
Tianyu Liu, Jirui Qi, Paul He, Arianna Bisazza, Mrinmaya Sachan, and Ryan Cotterell. 2025.
\newblock \href {https://doi.org/10.18653/v1/2025.naacl-long.78} {Pointwise mutual information as a performance gauge for retrieval-augmented generation}.
\newblock In \emph{Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers)}, pages 1628--1647, Albuquerque, New Mexico. Association for Computational Linguistics.
\bibitem[{Malon and Bai(2020)}]{malon2020generatingfollowupquestionsinterpretable}
Christopher Malon and Bing Bai. 2020.
\newblock \href {https://arxiv.org/abs/2002.12344} {Generating followup questions for interpretable multi-hop question answering}.
\newblock \emph{Preprint}, arXiv:2002.12344.
\bibitem[{Min et~al.(2019)Min, Zhong, Zettlemoyer, and Hajishirzi}]{min-etal-2019-multi}
Sewon Min, Victor Zhong, Luke Zettlemoyer, and Hannaneh Hajishirzi. 2019.
\newblock Multi-hop reading comprehension through question decomposition and rescoring.
\newblock In \emph{Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics}, pages 6097--6109, Florence, Italy. Association for Computational Linguistics.
\bibitem[{Qian et~al.(2024)Qian, Liu, Mao, Zhou, and Dou}]{qian2024cfic}
Hongjin Qian, Zheng Liu, Kelong Mao, Yujia Zhou, and Zhicheng Dou. 2024.
\newblock \href {https://arxiv.org/abs/2402.09760} {Grounding language model with chunking-free in-context retrieval}.
\newblock \emph{arXiv preprint arXiv:2402.09760}.
\bibitem[{Santos et~al.(2015)Santos, Macdonald, and Ounis}]{santos2015search}
Rodrygo L.~T. Santos, Craig Macdonald, and Iadh Ounis. 2015.
\newblock \href {https://doi.org/10.1561/1500000040} {Search result diversification}.
\newblock \emph{Foundations and Trends® in Information Retrieval}, 9(1):1--90.
\bibitem[{Sarthi et~al.(2024)Sarthi, Abdullah, Tuli, Khanna, Goldie, and Manning}]{sarthi2024raptor}
Parth Sarthi, Salman Abdullah, Aditi Tuli, Shubh Khanna, Anna Goldie, and Christopher~D Manning. 2024.
\newblock \href {https://openreview.net/forum?id=GN921JHCRw} {{RAPTOR}: Recursive abstractive processing for tree-organized retrieval}.
\newblock In \emph{The Twelfth International Conference on Learning Representations}.
\bibitem[{Shen et~al.(2023)Shen, Chen, Backes, and Zhang}]{shen2023chatgpt}
Xinyue Shen, Zeyuan Chen, Michael Backes, and Yang Zhang. 2023.
\newblock \href {https://arxiv.org/abs/2304.08979} {In chatgpt we trust? measuring and characterizing the reliability of chatgpt}.
\newblock \emph{arXiv preprint arXiv:2304.08979}.
\bibitem[{Snell et~al.(2024)Snell, Lee, Xu, and Kumar}]{snell2024scaling}
Charlie Snell, Jaehoon Lee, Kelvin Xu, and Aviral Kumar. 2024.
\newblock Scaling llm test-time compute optimally can be more effective than scaling model parameters.
\newblock \emph{arXiv preprint arXiv:2408.03314}.
\bibitem[{Trivedi et~al.(2022)Trivedi, Balasubramanian, Khot, and Sabharwal}]{trivedi-etal-2022-MuSiQue}
Harsh Trivedi, Niranjan Balasubramanian, Tushar Khot, and Ashish Sabharwal. 2022.
\newblock \href {https://doi.org/10.1162/tacl_a_00475} {Musique: Multihop questions via single-hop question composition}.
\newblock \emph{Transactions of the Association for Computational Linguistics}, 10:539--554.
\bibitem[{Wang et~al.(2025)Wang, Bi, Luo, Asur, and Cheng}]{wang2025diversityenhancesllmsperformance}
Zhichao Wang, Bin Bi, Yanqi Luo, Sitaram Asur, and Claire~Na Cheng. 2025.
\newblock \href {https://arxiv.org/abs/2502.09017} {Diversity enhances an llm's performance in rag and long-context task}.
\newblock \emph{Preprint}, arXiv:2502.09017.
\bibitem[{Wei et~al.(2022)Wei, Wang, Schuurmans, Bosma, Ichter, Xia, Chi, Le, and Zhou}]{10.5555/3600270.3602070}
Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed~H. Chi, Quoc~V. Le, and Denny Zhou. 2022.
\newblock Chain-of-thought prompting elicits reasoning in large language models.
\newblock In \emph{Proceedings of the 36th International Conference on Neural Information Processing Systems}, NIPS '22, Red Hook, NY, USA. Curran Associates Inc.
\bibitem[{Wu et~al.(2024)Wu, Yang, Ji, Huang, Karlsson, and Okumura}]{wu2024gendecrobustgenerativequestiondecomposition}
Jian Wu, Linyi Yang, Yuliang Ji, Wenhao Huang, Börje~F. Karlsson, and Manabu Okumura. 2024.
\newblock \href {https://arxiv.org/abs/2402.11166} {Gendec: A robust generative question-decomposition method for multi-hop reasoning}.
\newblock \emph{Preprint}, arXiv:2402.11166.
\bibitem[{Xiong et~al.(2021)Xiong, Li, Iyer, Du, Lewis, Wang, Mehdad, Yih, Riedel, Kiela, and Oguz}]{xiong2021answering}
Wenhan Xiong, Xiang Li, Srini Iyer, Jingfei Du, Patrick Lewis, William~Yang Wang, Yashar Mehdad, Scott Yih, Sebastian Riedel, Douwe Kiela, and Barlas Oguz. 2021.
\newblock \href {https://openreview.net/forum?id=EMHoBG0avc1} {Answering complex open-domain questions with multi-hop dense retrieval}.
\newblock In \emph{International Conference on Learning Representations}.
\bibitem[{Xu et~al.(2021)Xu, Deng, Zhang, Cai, and Lam}]{xu-etal-2021-exploiting-reasoning}
Weiwen Xu, Yang Deng, Huihui Zhang, Deng Cai, and Wai Lam. 2021.
\newblock \href {https://doi.org/10.18653/v1/2021.findings-emnlp.99} {Exploiting reasoning chains for multi-hop science question answering}.
\newblock In \emph{Findings of the Association for Computational Linguistics: EMNLP 2021}, pages 1143--1156, Punta Cana, Dominican Republic. Association for Computational Linguistics.
\bibitem[{Yan et~al.(2024)Yan, Gu, Zhu, and Ling}]{yan2024crag}
Shi-Qi Yan, Jia-Chen Gu, Yun Zhu, and Zhen-Hua Ling. 2024.
\newblock \href {https://arxiv.org/abs/2401.15884} {Corrective retrieval augmented generation}.
\newblock \emph{arXiv preprint arXiv:2401.15884}.
\bibitem[{Yang et~al.(2025)Yang, Yang, Zhang, Hui, Zheng, Yu, Li, Liu, Huang, Wei, Lin, Yang, Tu, Zhang, Yang, Yang, Zhou, Lin, Dang, Lu, Bao, Yang, Yu, Li, Xue, Zhang, Zhu, Men, Lin, Li, Tang, Xia, Ren, Ren, Fan, Su, Zhang, Wan, Liu, Cui, Zhang, and Qiu}]{qwen2025qwen25technicalreport}
An~Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo~Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, Huan Lin, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Yang, Jiaxi Yang, Jingren Zhou, Junyang Lin, Kai Dang, and 23 others. 2025.
\newblock \href {https://arxiv.org/abs/2412.15115} {Qwen2.5 technical report}.
\newblock \emph{Preprint}, arXiv:2412.15115.
\bibitem[{Yang et~al.(2018)Yang, Qi, Zhang, Bengio, Cohen, Salakhutdinov, and Manning}]{yang-etal-2018-hotpotqa}
Zhilin Yang, Peng Qi, Saizheng Zhang, Yoshua Bengio, William Cohen, Ruslan Salakhutdinov, and Christopher~D. Manning. 2018.
\newblock \href {https://doi.org/10.18653/v1/D18-1259} {{H}otpot{QA}: A dataset for diverse, explainable multi-hop question answering}.
\newblock In \emph{Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing}, pages 2369--2380, Brussels, Belgium. Association for Computational Linguistics.
\bibitem[{Zelikman et~al.(2022)Zelikman, Wu, Mu, and Goodman}]{reasoning_chain}
Eric Zelikman, Yuhuai Wu, Jesse Mu, and Noah~D. Goodman. 2022.
\newblock Star: self-taught reasoner bootstrapping reasoning with reasoning.
\newblock In \emph{Proceedings of the 36th International Conference on Neural Information Processing Systems}, NIPS '22, Red Hook, NY, USA. Curran Associates Inc.
\bibitem[{Zhang et~al.(2024)Zhang, Chen, Hu, Xu, Chen, Hao, Han, Thai, Wang, Liu, and Sun}]{zhang-etal-2024-bench}
Xinrong Zhang, Yingfa Chen, Shengding Hu, Zihang Xu, Junhao Chen, Moo Hao, Xu~Han, Zhen Thai, Shuo Wang, Zhiyuan Liu, and Maosong Sun. 2024.
\newblock \href {https://doi.org/10.18653/v1/2024.acl-long.814} {$\infty${B}ench: Extending long context evaluation beyond 100{K} tokens}.
\newblock In \emph{Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)}, pages 15262--15277, Bangkok, Thailand. Association for Computational Linguistics.
\bibitem[{Zhang et~al.(2021)Zhang, Nie, Ramamurthy, and Song}]{10.1145/3404835.3462853}
Yuyu Zhang, Ping Nie, Arun Ramamurthy, and Le~Song. 2021.
\newblock \href {https://doi.org/10.1145/3404835.3462853} {Answering any-hop open-domain questions with iterative document reranking}.
\newblock In \emph{Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval}, SIGIR '21, page 481–490, New York, NY, USA. Association for Computing Machinery.
\bibitem[{Zhang et~al.(2025)Zhang, Ma, Wang, He, Wang, He, Wang, and Liu}]{zhang-etal-2025-lexical}
Zhange Zhang, Yuqing Ma, Yulong Wang, Shan He, Tianbo Wang, Siqi He, Jiakai Wang, and Xianglong Liu. 2025.
\newblock \href {https://doi.org/10.18653/v1/2025.acl-long.1346} {Lexical diversity-aware relevance assessment for retrieval-augmented generation}.
\newblock In \emph{Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)}, pages 27758--27781, Vienna, Austria. Association for Computational Linguistics.
\bibitem[{Zhao et~al.(2024)Zhao, Wang, Cen, Zha, Tan, Dong, and Tang}]{zhao-etal-2024-longrag}
Qingfei Zhao, Ruobing Wang, Yukuo Cen, Daren Zha, Shicheng Tan, Yuxiao Dong, and Jie Tang. 2024.
\newblock \href {https://doi.org/10.18653/v1/2024.emnlp-main.1259} {{L}ong{RAG}: A dual-perspective retrieval-augmented generation paradigm for long-context question answering}.
\newblock In \emph{Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing}, pages 22600--22632, Miami, Florida, USA. Association for Computational Linguistics.
\bibitem[{Zhuang et~al.(2023)Zhuang, Liu, Koopman, and Zuccon}]{zhuang-etal-2023-open}
Shengyao Zhuang, Bing Liu, Bevan Koopman, and Guido Zuccon. 2023.
\newblock \href {https://doi.org/10.18653/v1/2023.findings-emnlp.590} {Open-source large language models are strong zero-shot query likelihood models for document ranking}.
\newblock In \emph{Findings of the Association for Computational Linguistics: EMNLP 2023}, pages 8807--8817, Singapore. Association for Computational Linguistics.
\bibitem[{Zuccon et~al.(2023)Zuccon, Koopman, and Shaik}]{10.1145/3624918.3625329}
Guido Zuccon, Bevan Koopman, and Razia Shaik. 2023.
\newblock \href {https://doi.org/10.1145/3624918.3625329} {Chatgpt hallucinates when attributing answers}.
\newblock In \emph{Proceedings of the Annual International ACM SIGIR Conference on Research and Development in Information Retrieval in the Asia Pacific Region}, SIGIR-AP '23, page 46–51, New York, NY, USA. Association for Computing Machinery.
\end{thebibliography}