SciFig: Towards Automating Editable Figure Generation for Scientific Papers
Extracted source text · arXiv 2601.04390
% in % CVPR 2026 Paper Template; see https://github.com/cvpr-org/author-kit
\documentclass[10pt,twocolumn,letterpaper]{article}
%%%%%%%%% PAPER TYPE - PLEASE UPDATE FOR FINAL VERSION
\usepackage{cvpr} % To produce the CAMERA-READY version
% \usepackage[review]{cvpr} % To produce the REVIEW version
% \usepackage[pagenumbers]{cvpr} % To force page numbers, e.g. for an arXiv version
% Import additional packages in the preamble file, before hyperref
\input{preamble}
\usepackage{multirow}
\setlength{\textfloatsep}{12pt plus 2.0pt minus 2.0pt}
\setlength{\floatsep}{2pt plus 1.0pt minus 2.0pt}
\setlength{\abovecaptionskip}{3pt plus 2pt minus 2pt}
\usepackage[table]{xcolor}
\definecolor{Light}{HTML}{edf7f7}
% It is strongly recommended to use hyperref, especially for the review version.
% hyperref with option pagebackref eases the reviewers' job.
% Please disable hyperref *only* if you encounter grave issues,
% e.g. with the file validation for the camera-ready version.
%
% If you comment hyperref and then uncomment it, you should delete *.aux before re-running LaTeX.
% (Or just hit 'q' on the first LaTeX run, let it finish, and you should be clear).
\definecolor{cvprblue}{rgb}{0.21,0.49,0.74}
\usepackage[pagebackref,breaklinks,colorlinks,allcolors=cvprblue]{hyperref}
%%%%%%%%% PAPER ID - PLEASE UPDATE
\def\paperID{} % *** Enter the Paper ID here
\def\confName{CVPR}
\def\confYear{2026}
\newcommand{\snag}[1]{\textcolor{magenta}{[\textbf{SN:} #1]}}
%%%%%%%%% TITLE - PLEASE UPDATE
\title{SciFig: Towards Automating Scientific Figure Generation
}
%%%%%%%%% AUTHORS - PLEASE UPDATE
\author{Siyuan Huang$^{1\dagger*}$, Yutong Gao$^*$, Juyang Bai$^{1*}$, Yifan Zhou$^{2*}$, Zi Yin$^{3*}$, Xinxin Liu$^4$, \\
Rama Chellappa$^1$,
Chun Pong Lau$^5$, Sayan Nag$^6$, Cheng Peng$^{1\ddag}$, Shraman Pramanick$^{1\ddag}$\\
$^1$Johns Hopkins University, $^2$Zhejiang University, $^3$Tsinghua University, \\
$^4$University of Central Florida, $^5$City University of Hong Kong, $^6$Adobe\\
$^\dagger$Project Lead, $^*$Equal Contributions, $^\ddag$Co-Supvisers\\
{\tt\small shuan124@jhu.edu}
% For a paper whose authors are all at the same institution,
% omit the following lines up until the closing ``}''.
% Additional authors and addresses can be added with ``\and'',
% just like the second author.
% To save space, use either the email address or home page, not both
% \and
% Second Author\\
% Institution2\\
% First line of institution2 address\\
% {\tt\small secondauthor@i2.org}
}
\begin{document}
% \maketitle
\twocolumn[{%
\renewcommand\twocolumn[2][]{#1}
\maketitle
\vspace{-10mm}
\begin{center}
\centering \centering
% \iftoggle{cvprfinal}{\vspace{-0.7cm}}{\vspace{-1cm}}
\includegraphics[width=\linewidth]{agentfig-teaser.pdf}
% \vspace{-6mm}
\captionof{figure}{\textbf{SciFig: Automatically creates publication-ready figures.} Existing methods either require days of manual work \cite{leblanc2019scientific, jiang2019high} or produce poor AI outputs \cite{zala2023diagrammergpt,chen2024makes, shen2024sg, neau2023fine} (blurry images, wrong connections, flat layouts). SciFig is the first end-to-end system that automatically generates high-quality method figures through hierarchical layout generation, iterative improvement, and quality evaluation—reducing creation time from days to minutes while matching human-designed figures.}
\label{fig:teaser}
% \vspace{-1.5em}
\end{center}
}]
\input{sec/0_abstract}
\input{sec/1_intro}
\input{sec/2_related_work_shraman_compressed}
\input{sec/3_method_shraman_compressed}
\input{sec/4_experiments}
\input{sec/5_conclusion}
\newpage
{
\small
\bibliographystyle{ieeenat_fullname}
\bibliography{main}
}
\input{sec/X_suppl}
% WARNING: do not forget to delete the supplementary pages from your submission
% \input{sec/X_suppl}
\end{document}
%% This file contains a number of tweaks that are typically applied to the main document.
%% They are not enabled by default, but can be enabled by uncommenting the relevant lines.
%%
%% Inline annotations; for predefined colors, refer to "dvipsnames" in the xcolor package:
%% https://tinyurl.com/overleaf-colors
%%
\newcommand{\red}[1]{{\color{red}#1}}
\newcommand{\todo}[1]{{\color{red}#1}}
\newcommand{\TODO}[1]{\textbf{\color{red}[TODO: #1]}}
%%
%% disable for camera ready / submission by uncommenting these lines
%%
% \renewcommand{\TODO}[1]{}
% \renewcommand{\todo}[1]{#1}
%%
%% work harder in optimizing text layout. Typically shrinks text by 1/6 of page, enable
%% it at the very end of the writing process, when you are just above the page limit
%%
% \usepackage{microtype}
%%
%% fine-tune paragraph spacing
%%
% \renewcommand{\paragraph}[1]{\vspace{.5em}\noindent\textbf{#1.}}
%%
%% globally adjusts space between figure and caption
%%
% \setlength{\abovecaptionskip}{.5em}
%%
%% Allows "the use of \paper to refer to the project name"
%% with automatic management of space at the end of the word
%%
% \usepackage{xspace}
% \newcommand{\paper}{ProjectName\xspace}
%%
%% Commonly used math definitions
%%
% \DeclareMathOperator*{\argmin}{arg\,min}
% \DeclareMathOperator*{\argmax}{arg\,max}
%%
%% Tigthen underline
%%
% \usepackage{soul}
% \setuldepth{foobar}
\usepackage{tabularx}
\usepackage{mathrsfs}
\usepackage{algorithm}
\usepackage{algpseudocode}
\usepackage[most]{tcolorbox}
\usepackage{mdframed}
\usepackage{xcolor}
\usepackage{listings}
\newcommand{\SPcomment}[1]{\textcolor{blue}{SP: #1}}
\newcommand{\SP}[1]{\textcolor{red}{#1}}
\begin{abstract}
\vspace{-2mm}
Creating high-quality figures and visualizations for scientific papers is a time-consuming task that requires both deep domain knowledge and professional design skills. Despite over 2.5 million scientific papers published annually, the figure generation process remains largely manual. We introduce \textbf{SciFig}, an end-to-end AI agent system that generates publication-ready pipeline figures directly from research paper texts. SciFig uses a hierarchical layout generation strategy, which parses research descriptions to identify component relationships, groups related elements into functional modules, and generates inter-module connections to establish visual organization. Furthermore, an iterative chain-of-thought (CoT) feedback mechanism progressively improves layouts through multiple rounds of visual analysis and reasoning. We introduce a rubric-based evaluation framework that analyzes 2,219 real scientific figures to extract evaluation rubrics and automatically generates comprehensive evaluation criteria. SciFig demonstrates remarkable performance: achieving 70.1\% overall quality on dataset-level evaluation and 66.2\% on paper-specific evaluation, and consistently high scores across metrics such as visual clarity, structural organization, and scientific accuracy. SciFig figure generation pipeline and our evaluation benchmark will be open-sourced.
% The core challenge in automatic scientific figure generation lies in three critical aspects: (1) \textbf{layout design} – creating logically structured, hierarchically organized pipeline layout with proper inter-component connections and arrow flows, (2) \textbf{evaluation framework} – systematically evaluating figure quality, and (3) \textbf{system integration} – building a fully automatic pipeline that handles the entire generation process from text to editable figures. Previous methods either focus on isolated subtasks, lack structural reasoning capabilities, or fail to produce publication-quality outputs, making them inadequate for real-world scientific figure generation.
% SciFig solves these challenges through a new multi-agent architecture with three key novelties. First, we propose a \textbf{hierarchical layout generation strategy} that parses research descriptions to identify component relationships, groups related elements into functional modules, and generates inter-module connections to significantly improve logical clarity and visual organization. Second, we develop an \textbf{iterative Chain-of-Thought (CoT) feedback mechanism} that progressively improves layouts through multiple rounds of visual analysis and reasoning, ensuring structural correctness and aesthetic quality. Third, we introduce a \textbf{rubric-based evaluation framework} that analyzes 2K+ real pipeline figures to extract evaluation rubrics and automatically generates comprehensive evaluation criteria, enabling systematic quality benchmarking without human annotation.
% Our system demonstrates remarkable performance: achieving x\% reduction in figure creation time, x\% accuracy in matching human-created figures, and consistently high scores across metrics such as visual clarity, structural organization, and scientific accuracy. Experiments including human evaluation studies with 20+ participants confirm that SciFig produces figures comparable to or better than manually created ones, while eliminating the need for technical and design expertise.
\end{abstract}
\section{Introduction}
\label{sec:intro}
% \begin{figure*}
% \centering
% \includegraphics[width=\linewidth]{SciFig_teaser.png}
% \caption{\textbf{SciFig: Automatically creates publication-ready figures.} Existing methods either require days of manual work \cite{leblanc2019scientific, jiang2019high} or produce poor AI outputs \cite{zala2023diagrammergpt,chen2024makes, shen2024sg, neau2023fine} (blurry images, wrong connections, flat layouts). SciFig is the first end-to-end system that automatically generates high-quality method figures through hierarchical layout generation, iterative improvement, and quality evaluation—reducing creation time from days to minutes while matching human-designed figures.}
% \label{fig:teaser}
% \vspace{-1.5em}
% \end{figure*}
Scientific figures, particularly schematic diagrams describing detailed methodology, are essential visual communicators in research manuscripts. Good figures allow readers to quickly grasp complex workflows and system architectures.
% illustrating research methods, serve as the visual backbone of research papers, enabling readers to quickly grasp complex algorithmic workflows and system architectures.
% In fields ranging from Computer Vision to Computational Biology, a well-designed pipeline figure can show months of research effort in a single glance.
However, creating these figures is one of the most time-consuming aspects in scientific writing. Researchers not only need to have a deep understanding of their technical contributions but also require sophisticated tools and skills to produce high-quality visualizations.
% This dual expertise requirement creates a significant bottleneck:
With over 2.5 million scientific papers published annually, countless researcher hours are spent on creating figures.
% The challenge of automatic scientific figure generation has remained largely unsolved despite significant progress in AI-assisted content creation.
While recent progress in vision language models (VLMs) and diffusion models have shown text and image generation abilities~\cite{liu2023visual,li2023blip,zhang2023adding}, generating methodology figures from text descriptions presents unique challenges. General image generation models such as DALL-E~\cite{ramesh2021zero,ramesh2022hierarchical,betker2023improving}, Stable Diffusion~\cite{rombach2022high}, and Midjourney~\cite{jaruga2022artificial} are good at generating visually plausible natural images from text descriptions; however, they are often ridled with inconsistencies for scientific figures and is not directly interactable due to the pixel-level outputs. Diagram visualization tools like GraphViz~\cite{gansner2000open,ellson2001graphviz} and Mermaid~\cite{sveidqvist2021official} provide full editability, but require precise specifications and cannot automatically interpret natural language descriptions or make design decisions. LLM-based methods, e.g., layout generation systems designed for document design, user interfaces, or graphic layouts~\cite{wang2024docllm,feng2023layoutgpt,inoue2023layoutdm,yang2024posterllava,zheng2023layoutdiffusion,iwai2024layout} lie in the middle. While they are more editable than image generation models, they still fail to maintain logical coherence from manuscript descriptions. Current generated figures, as shown in \textbf{Fig. \ref{fig:teaser}}, often fail to satisfy such a complex task. This begs the question: how do we design figures and how can AI imitate this process?
% A pipeline illustrating a machine learning system, for example, must clearly distinguish between data preprocessing, feature extraction, and loss computation stages, with connections that reveal the computational dependencies and presentable aesthetics.
Humans naturally perceive and describe information in a hierarchical manner. In contrast, current generative models lack such structured representations, often producing disconnected modules that are difficult to interpret. This limitation makes it challenging to establish coherent logical relationships among components, frequently resulting in conflicting connections, misaligned elements, and inconsistent visual flow. Moreover, human design is inherently iterative—comprising cycles of ideation, sketching, refinement based on collaborator feedback, and meticulous proofreading—where both low-level details (e.g., word choice, visual style) and high-level design decisions are repeatedly revised. Existing models, however, do not incorporate such iterative, cross-granularity validation. Finally, there remains no systematic evaluation framework for assessing generated figures, hindering progress in benchmarking, and the assurance of consistent quality.
% Existing diagram generation tools either require extensive manual specification or produce visually inconsistent, structurally flawed outputs unsuitable for publication.
% \textbf{What makes paper-to-pipeline generation fundamentally challenging?} Unlike natural image generation or simple diagram creation, scientific methodology figures demand three critical properties that must be satisfied simultaneously. First, \textbf{structural correctness}: components must be organized hierarchically to reflect functional relationships, with arrows indicating precise information flow between modules rather than creating visual clutter through arbitrary connections. A pipeline illustrating a machine learning system, for example, must clearly distinguish between data preprocessing, feature extraction, and loss computation stages, with connections that reveal the actual computational dependencies. Second, \textbf{visual clarity and consistency}: figures must maintain professional aesthetics with proper alignment, spacing, color coding, and typography while ensuring all elements are legible and distinguishable. Third, \textbf{scientific accuracy}: the visualization must faithfully represent the described methodology, using appropriate terminology and conventions specific to the research domain. Achieving any single property is difficult; satisfying all three simultaneously while maintaining editability and flexibility has proven tough.
% The lack of automatic solutions forces researchers into time-consuming workflows. Creating a single pipeline figure typically requires 4-8 hours for experienced researchers and potentially weeks for those less familiar with design tools like Adobe Illustrator, PowerPoint, or specialized diagram software. This process involves iterative cycles of concept making, sketching, digital rendering, improvement based on co-author feedback, and final polishing—all while maintaining consistency in the paper. The cognitive burden of switching between technical writing and visual design further makes the challenge harder. Moreover, the lack of systematic evaluation frameworks makes it difficult to objectively evaluate figure quality or compare different design choices, leading to subjective quarrels and additional revision cycles.
% \textbf{Why can't existing methods solve this problem?} Current methods fall into three categories, each with critical limitations. (1) General image generation models such as DALL-E, Stable Diffusion, and Midjourney are good at generating natural images from text descriptions but fundamentally lack the structural reasoning required for hierarchical diagrams. When prompted to generate pipeline figures, these models produce visually plausible but structurally incoherent outputs with inconsistent components, incorrect arrows, and illegible text. (2) Layout generation systems designed for document design, user interfaces, or graphic layouts cannot handle the domain-specific requirements of scientific figures. These methods treat all elements uniformly without understanding the hierarchical relationships and logical flow essential to research pipelines. (3) Diagram generation tools like GraphViz and Mermaid require precise formal specifications, offering no ability to automatically interpret natural language descriptions or make design decisions. They produce rigid, template-based outputs lacking the flexibility and aesthetic quality expected in publications. Furthermore, none of these methods provides systematic evaluation mechanisms, making it impossible to benchmark progress or ensure consistent quality.
This paper introduces \textbf{SciFig}, the first end-to-end AI agent for scientific figure generation and evaluation. SciFig transforms textual descriptions from manuscripts into publication-ready, editable figures. Inspired by the human design process, our system makes the following improvements. First, SciFig performs hierarchical layout generation with a multi-agent architecture. The system composes of a description agent that parses research text to extract component relationships and functional groupings, a layout agent that applies hierarchical reasoning to organize components into coherent modules, and a component generation agent that renders individual elements with consistent styling. SciFig generates inter-module connections rather than arbitrary point-to-point arrows, improving logical clarity. This hierarchy-aware design simplifies functional relationships and makes understanding easier for users.
% with related components naturally grouped and information flow clearly indicated.
% \textbf{Hierarchical Layout Generation with Multi-Agent Architecture.} We propose a new hierarchical layout strategy that fundamentally differs from existing methods by explicitly modeling the hierarchical structure of research pipelines. Our system uses specialized agents: a Description Agent that parses research text to extract component relationships and functional groupings, a Layout Agent that applies hierarchical reasoning to organize components into coherent modules, and a Layout Component Generation Agent that renders individual elements with consistent styling. Our method generates inter-module connections rather than arbitrary point-to-point arrows, improving logical clarity. This hierarchy-aware design ensures that functional relationships are visually apparent, with related components naturally grouped and information flow clearly indicated.
% \textbf{Iterative Chain-of-Thought Feedback Mechanism.} Creating high-quality layouts requires iterative improvement—a process that human designers perform naturally but automatic systems struggle to replicate.
Imitating human's iterative design workflows, we introduce an iterative CoT feedback mechanism. Within SciFig, a feedback agent analyzes rendered layouts, detects issues such as misaligned components, unclear arrows, spacing problems, labeling errors, etc., and provides detailed natural language guidance. A layout agent then applies CoT reasoning to understand the feedback and generate improved layouts. Repeating this cycle progressively enhances both visual quality and structural correctness of the figures.
% This process iterates multiple times, progressively improving visual quality and structural correctness.
% \textbf{Rubric-Based Automatic Evaluation Framework.}
A critical gap in automatic figure generation research is the lack of systematic evaluation methods. How can we objectively evaluate whether a generated pipeline figure is good? To this end, we propose a comprehensive evaluation framework that analyzes 2K+ scientific methodology figures collected from top-tier conferences to identify common patterns and introduce six key quality rubrics: technical accuracy, visual clarity, structural coherence, design consistency, interpretability, and implementation quality. Our evaluation agent automatically generates rubric-specific questions for each dimension and provides detailed scoring criteria. Based on such an agent, meaningful comparisons between different generation methods can be done to establish quality benchmarks for the field.
% This framework enables automatic evaluation, facilitating meaningful comparisons between different generation methods and establishing quality benchmarks for the field.
Our contributions can be summarized as follows:
\begin{itemize}
\item \textbf{Hierarchical Layout System:} We introduce a multi-agent AI system to generate scientific methodology figures, with explicit module-level organization and inter-module connection strategies.
% to improve structural clarity over flat layout methods.
\item \textbf{Iterative CoT Feedback Mechanism:} We propose a multi-round improvement process that leverages visual feedback and Chain-of-Thought reasoning to progressively improve layout quality.
% , achieving professional-level aesthetics and low-level consistency at the same time.
\item \textbf{Comprehensive Evaluation Framework:} We propose a rubric-based evaluation that scores figure quality across multiple dimensions, providing both quantitative scores and qualitative feedback. Our framework includes a curated dataset of 2K+ annotated methodology figures serving as a benchmark resource for the community.
% \item \textbf{First End-to-End Automatic System:} SciFig represents the first complete pipeline from research paper text to publication-ready, editable figures, achieving x\% reduction in creation time and x\% accuracy in matching human-designed figures. A human evaluation study with 20+ participants demonstrates that SciFig produces figures comparable to or exceeding manually created ones.
% \item \textbf{Comprehensive experiments} including quantitative metrics, ablation studies, and human evaluation studies with 20+ participants demonstrating that SciFig produces figures comparable to or exceeding manually created ones.
\end{itemize}
% \begin{figure*}
% \centering
% \begin{subfigure}{0.68\linewidth}
% \fbox{\rule{0pt}{2in} \rule{.9\linewidth}{0pt}}
% \caption{An example of a subfigure.}
% \label{fig:short-a}
% \end{subfigure}
% \hfill
% \begin{subfigure}{0.28\linewidth}
% \fbox{\rule{0pt}{2in} \rule{.9\linewidth}{0pt}}
% \caption{Another example of a subfigure.}
% \label{fig:short-b}
% \end{subfigure}
% \caption{Example of a short caption, which should be centered.}
% \label{fig:short}
% \end{figure*}
\section{Related Work}
\label{sec:rw}
\subsection{Agentic AI}
\textbf{Multi-Agent Systems with Specialized Roles.}
MetaGPT \cite{hong2023metagpt}, ChatDev \cite{qian2024chatdev}, AgentVerse \cite{chen2023agentverse}, and AutoGen \cite{wu2023autogen} introduce multi-agent frameworks where agents assume specialized roles, translating actions into structured prompts for collaborative reasoning and reduced hallucinations. These systems demonstrate the effectiveness of role-based task decomposition for complex generative workflows, forming the architectural foundation upon which SciFig builds.
\textbf{Scientific Discovery Systems.}
AI Scientist \cite{lu2024ai}, Robin \cite{ghareeb2025robin}, Agent Laboratory \cite{schmidgall2025agent}, and AgentRxiv \cite{schmidgall2025agentrxiv} present comprehensive frameworks for automated scientific discovery—covering ideation, experimentation, visualization, and paper writing. While parallel in multi-agent design, they focus on end-to-end research automation rather than visualization, establishing the broader AI-driven scientific workflow patterns that SciFig extends to figure generation.
\textbf{Agents for Visual and Diagram Generation.}
DiagrammerGPT \cite{zala2023diagrammergpt}, PlotGen \cite{goswami2025plotgen}, CoDA \cite{chen2025coda}, and GenArtist \cite{wang2024genartist} employ planner–generator–feedback agents for visual creation, showing that agent-based decomposition improves visual quality over end-to-end generation. However, they primarily target flowcharts and statistical plots (e.g., bar, line, scatter), lacking the domain-specific and hierarchical reasoning needed for complex scientific pipeline figures.
% Unlike them, SciFig solves the unique challenge of paper-to-pipeline figure generation, which requires hierarchical component relationships, domain-specific scientific accuracy, and publication-quality aesthetics.
\subsection{Layout Generation}
Layout generation aims to automatically generate harmonious arrangements of visual elements characterized by attributes such as category, position, and size.
% One SciFig's fundamental goal is arranging visual elements of scientific pipeline figures.
Recent diffusion models have achieved visually pleasing results by formulating layout generation as a denoising process. Dolfin \cite{wang2024dolfin}, Layout-Corrector \cite{iwai2024layout}, LayoutDM \cite{inoue2023layoutdm}, LayoutDiffusion \cite{zheng2023layoutdiffusion}, DLT \cite{levi2023dlt}, and PLay \cite{cheng2023play} transit tokens on the diffusion space, where inharmonious elements persist during generation, to remove distort geometircal features like bounding box alignment, and solve the layout sticking problem. Large language models (LLMs) also made a new progress for layout generation through structured text representations. LayoutGPT \cite{feng2023layoutgpt}, PosterLLaVa \cite{yang2024posterllava}, LayoutPrompter \cite{lin2023layoutprompter}, and LayoutNUWA \cite{tanglayoutnuwa} demonstrate LLM capability for layout generation from text conditions using in-context demonstrations in CSS, JSON, or code language, processing both visual and textual conditions end-to-end.
% While these methods are good at generating layouts for documents, user interfaces, posters, and general diagrams, they are fundamentally not suitable to scientific pipeline figure generation. They cannot distinguish between functional stages (data preprocessing, feature extraction, loss computation) or generate appropriate module (hierarchy groupings and connections). Most importantly, these systems optimize for visual balance rather than logical clarity and scientific accuracy—a figure may be beautifully aligned yet completely misrepresent the method's information flow. SciFig fixes these fundamental gaps through hierarchical layout modeling and CoT feedback that explicitly captures functional relationships between pipeline components, enabling the first system capable of generating publication-quality research method figures.
\subsection{LLM-Based Evaluation Systems}
For scientific visualizations, research remains few but shows promising directions. GPT-4 \cite{zhang2023gpt}, LLaVA-Critic \cite{xiong2024llava}, Prometheus-Vision \cite{lee2024prometheus}, and MatPlotAgent \cite{yang2024matplotagent} can serve as automatic evaluators for visual and scientific figures, achieving high correlation with humans, demonstrating feasibility of LLMs as cost-effective, reference-free evaluation methods for scientific content. DiagramEval \cite{liang2025diagrameval} conceptualizes diagrams as graphs, providing more discriminative and explainable evaluation than traditional image-level metrics. LIDA \cite{dibia2023lida} uses LLMs to evaluate visualizations through self-evaluation across six dimensions, achieving visualization error rates below 3.5\%.
% While existing LLM-based evaluators are promising for general visual content, they lack the domain-specific understanding required for scientific pipeline figures. Current systems cannot evaluate whether a pipeline figure accurately represents method logic, correctly conveys information flow between functional modules, or appropriately groups related components into hierarchical stages. SciFig fixes this gap by developing a rubric-based evaluation framework that analyzes 2K+ real pipeline figures to extract domain-specific quality dimensions specifically tailored to research method visualization. Our evaluation agent generates customized evaluation criteria that capture both the geometric properties evaluated by existing systems and the semantic accuracy, enabling the first comprehensive automatic evaluation specifically designed for pipeline figure quality.
% evaluation
% LightVA implements a multi-agent framework with planner, executor, and controller for visual analytics, while Visualization Generation with Large Language Models evaluates LLM effectiveness for generating Vega-Lite specifications from natural language, both demonstrating LLM proficiency in generating declarative visualization specifications.
% **Vision-Language Agents.** Recent advances in vision-language models enable agents that both understand text descriptions and generate visual content—core to SciFig's functionality. VisualAgentBench provides a comprehensive benchmark for training and evaluating LMMs as visual foundation agents across Embodied, GUI, and Visual Design scenarios, testing visual grounding, understanding, and interaction for both analysis and generation tasks including layout planning and visual creation. ViGoR addresses hallucination issues and incorrect attribute/relationship inference in LVLMs through fine-grained reward modeling that significantly improves visual grounding—critical for accurate figure generation from text descriptions.
% LayoutVLM leverages VLM semantic knowledge with differentiable optimization for physical plausibility in layout generation, using visually marked images for spatial grounding and self-consistent decoding, demonstrating VLM capabilities applicable to SciFig's Layout Agent. VQAScore/GenAI-Bench introduces VQAScore for evaluating text-to-visual generation using VQA models, with GenAI-Bench providing 1,600 compositional prompts with over 15,000 human ratings that significantly outperform CLIPScore, providing evaluation methodology essential for assessing SciFig's figure generation quality. UGround trains on 10M GUI elements over 1.3M screenshots, outperforming existing visual grounding models by up to 20% in identifying and localizing visual elements from text descriptions—critical for figure understanding and generation.
\section{Method}
\label{sec:method}
\subsection{Problem Formulation}
We formalize scientific pipeline figure generation as a multi-stage transformation task with quality constraints.
\textbf{Input:} Let $\mathcal{T} = \{t_1, t_2, \ldots, t_n\}$ denotes a paper's method description with $n$ sentences.
\textbf{Output:} The objective is to generate publication-ready figure $\mathcal{F}$ consists of:
\begin{itemize}
\item Layout $\mathcal{L} = \{(p_i, s_i, \text{style}_i)\}_{i=1}^{m}$ specifying position $p_i \in \mathbb{R}^2$, size $s_i \in \mathbb{R}^2$, and styling $\text{style}_i$ for $m$ components;
\item A connection set $\mathcal{C} = \{(i, j, \theta_{ij})\}$ representing arrows from component $i$ to $j$ with connection type $\theta_{ij}$;
\item Visual elements $\mathcal{V} = \{v_1, \ldots, v_m\}$ where each $v_i$ is a rendered component (image or text).
\end{itemize}
The final figure $\mathcal{F}$ is constructed using a composition function $\Phi(\cdot)$:
\vspace{-3mm}
\begin{align}
\mathcal{F} = \Phi(\mathcal{L}, \mathcal{C}, \mathcal{V}) = \bigcup_{i=1}^{m} \pi(v_i, p_i, s_i) \cup \bigcup_{(i,j) \in \mathcal{C}} \alpha(p_i, p_j, \theta_{ij}),
\label{eq:final_composition}
\end{align}
where $\pi(v_i, p_i, s_i)$ positions visual element $v_i$ at location $p_i$ with size $s_i$, and $\alpha(p_i, p_j, \theta_{ij})$ renders connections between components. The result is a cohesive, publication-quality figure.
\begin{figure*}
\centering
\includegraphics[width=\linewidth]{agentfig-pipeline.pdf}
\vspace{-3mm}
\caption{\textbf{SciFig Multi-Agent System Architecture.} Our system consists of three stages: \textbf{Hierarchical Layout Generation} where the Description Agent parses research text, followed by the Layout Agent that organizes components into hierarchical structure. \textbf{Iterative CoT Feedback} where the Feedback Agent analyzes rendered layouts to identify specific issues, while the Layout Agent applies Chain-of-Thought reasoning to generate improved layouts across multiple rounds. \textbf{Component Generation} where the Component Agent renders individual visual elements with consistent styling to produce the final figure.
% This multi-agent decomposition enables systematic quality improvement from initial layout to professional output.
This figure is drafted by our system and polished manually.}
\label{fig:pipeline}
\vspace{-3mm}
\end{figure*}
\subsection{SciFig: Multi-Agent System Architecture}
SciFig is an end-to-end multi-agent framework that converts method descriptions into publication-ready figures. As illustrated in Fig.~\ref{fig:pipeline}, our system consists of four specialized agents working collaboratively, each solving a specific sub-problem that requires specific reasoning patterns and domain expertise.
\textbf{Description Agent ($\mathcal{A}_D$}) takes natural language method description $\mathcal{T}$ as input and produces hierarchical structure $\mathcal{H} = (\mathcal{M}, \mathcal{R})$ where $\mathcal{M}$ represents modules and $\mathcal{R}$ represents their relationships. This agent specializes in semantic parsing of technical text.
\textbf{Layout Agent ($\mathcal{A}_L$)} takes $\mathcal{H}$ and feedback $\text{FB}_t$ to produce layout $\mathcal{L}_t = \{(p_i, s_i, \text{style}_i)\}_{i=1}^{m}$ and connections $\mathcal{C}$. It performs spatial reasoning—grouping related modules, arranging hierarchy, and defining directional flows for conceptual clarity.
\textbf{Feedback Agent ($\mathcal{A}_F$)} receives rendered layout $\mathcal{F}_{t-1}$ and $\mathcal{T}$, generating feedback $\text{FB}_t$ identifying issues in alignment, spacing, arrow clarity, label readability, and visual balance. Operating independently, it mimics peer-review-style critique to avoid confirmation bias.
\textbf{Component Agent ($\mathcal{A}_C$}) takes optimized layout $\mathcal{L}^*$ as input and produces visual elements $\mathcal{V} = \{v_1, \ldots, v_m\}$. The agent ensures overall style consistency, technical precision, and publication-ready quality.
\subsubsection{Hierarchical Layout Generation}
We define a two-level hierarchy for scientific pipeline figures: \textit{modules} are high-level functional blocks that segment the overall pipeline (e.g., ``Stage 1: Feature Extraction",``Stage 2: Processing", ``Stage 3: Output Generation"), while \textit{components} are the individual visual elements within each module (e.g., boxes, icons, arrows, text labels). For example, in Fig.~\ref{fig:pipeline}, the three colored stage blocks are modules, and all the icons, shapes, and text within each stage are components. This hierarchical decomposition mirrors how researchers conceptually organize complex methods—first dividing the pipeline into major functional stages, then showing the specific operations within each stage.
Given method description $\mathcal{T} = \{t_1, \ldots, t_n\}$, the Description Agent $\mathcal{A}_D$ generates hierarchical structure $\mathcal{H} = (\mathcal{M}, \mathcal{R})$ by parsing $\mathcal{T}$ to extract components, analyze their relationships, and group functionally-related elements into modules. The resulting structure consists of $k$ modules $\mathcal{M} = \{M_1, \ldots, M_k\}$ representing major pipeline stages, where each module $M_i = \{c_{i1}, \ldots, c_{im_i}\}$ contains $m_i$ components (e.g., encoder icons, loss function boxes) that collectively perform a coherent sub-task. Module-level relationships $\mathcal{R} \subseteq \mathcal{M} \times \mathcal{M}$ define how these stages connect (sequential, parallel, or hierarchical).
Given hierarchical structure $\mathcal{H}$, the Layout Agent $\mathcal{A}_L$ generates spatial arrangement $\mathcal{L}$ that visually represents this organization. Unlike flat layout methods~\cite{inoue2023layoutdm, zheng2023layoutdiffusion} that treat all elements uniformly without recognizing functional groupings, our hierarchical method explicitly organizes the figure into visually different module blocks. The agent uses a top-down strategy that first arranges module blocks $M_1, \ldots, M_k$ according to relationships $\mathcal{R}$, creating visually separated stages (as shown in Fig.~\ref{fig:pipeline}), then arranges individual components $\{c_{i1}, \ldots, c_{im_i}\}$ within each module $M_i$ following their functional relationships.
Critically, we generate connections exclusively between modules rather than between individual components:
\begin{align}
\mathcal{C} = \{(M_i, M_j, \theta_{ij}) \mid (M_i, M_j) \in \mathcal{R}\}.
\end{align}
This module-level connection strategy means arrows connect stage blocks rather than every individual icon, largely reducing visual clutter. For example, instead of drawing arrows from each component in "Stage 1" to multiple components in "Stage 2" (potentially dozens of arrows), we draw a single arrow from the "Stage 1" module to the "Stage 2" module. The agent outputs baseline layout $\mathcal{L}_0$ consisting of module positions, component arrangements within modules, and inter-module connections.
% \subsubsection{Hierarchical Layout Generation}
% Given method description $\mathcal{T} = \{t_1, \ldots, t_n\}$, $\mathcal{A}_D$ produces hierarchical structure $\mathcal{H} = (\mathcal{M}, \mathcal{R})$ where $\mathcal{M} = \{M_1, \ldots, M_k\}$ is a set of modules, each module $M_i = \{c_{i1}, \ldots, c_{im}\}$ contains atomic components (i.e., icons, shapes, textboxes), and $\mathcal{R} \subseteq \mathcal{M} \times \mathcal{M}$ defines module-level relationships (sequential, parallel, or hierarchical).
% % The agent first performs component identification by parsing $\mathcal{T}$ to extract atomic components $\{c_1, \ldots, c_m\}$ with semantic types such as encoder, loss function, or data preprocessing. Next, relationship analysis determines for each component pair $(c_i, c_j)$ their hierarchical relationship (parent-child or sibling), functional relationship (sequential, parallel, or conditional), and information flow (data dependency or control flow). The module formation step groups components with strong relationships into modules according to $M_i = \{c_j \mid \text{functionally-related}(c_j, M_i)\}$. Finally, $\mathcal{A}_D$ generates a hierarchical description where sections represent modules and subsections represent components.
% The agent first parses $\mathcal{T}$ to extract atomic components $\{c_1, \ldots, c_m\}$ with semantic types such as encoder, loss function, or data preprocessing. Next, it analyzes each component pair $(c_i, c_j)$ to determine hierarchical (parent–child or sibling), functional (sequential, parallel, or conditional), and informational (data or control flow) relationships. Components with strong functional ties are grouped into modules according to $M_i = \{c_j \mid \text{functionally-related}(c_j, M_i)\}$. Finally, $\mathcal{A}_D$ generates a hierarchical structure where sections represent modules and subsections represent components.
% Given the hierarchical structure $\mathcal{H}$, the layout agent ($\mathcal{A}_L$) generates a spatial arrangement $\mathcal{L}$ that visually represents the logical organization. This agent solves a limitation: flat layout methods cannot capture multi-level abstractions in scientific pipelines.
% Unlike existing layout generation methods ~\cite{inoue2023layoutdm, zheng2023layoutdiffusion} that treat all components uniformly, we use a top-down strategy. The agent first determines the primary flow direction $d \in \{\text{horizontal}, \text{vertical}\}$ based on pipeline structure, then positions modules $M_1, \ldots, M_k$ along the flow direction according to relationships $\mathcal{R}$. Within each module $M_i$, components $\{c_{i1}, \ldots, c_{im}\}$ are arranged following their relationships: sequential components are placed along the flow direction, parallel components are placed perpendicular to the flow direction, and hierarchical components use visual nesting. Visual containerization draws bounding boxes around each module.
% We generate connections exclusively between modules rather than between individual components:
% \begin{align}
% \mathcal{C} = \{(M_i, M_j, \theta_{ij}) \mid (M_i, M_j) \in \mathcal{R}\}.
% \end{align}
% This strategy reduces visual clutter by replacing potentially dozens of component-level arrows with single module-level connections, improves logical clarity by emphasizing high-level information flow rather than low-level implementation details, and enables editability where module-level connections remain valid even when internal components are modified.
% % For example, instead of drawing 12 arrows from individual components in a "feature extraction" module to individual components in a "classification" module, we draw a single arrow between the modules.
% The agent outputs a layout $\mathcal{L}_0$.
\subsubsection{Iterative Chain-of-Thought Feedback}
High-quality figures require iterative improvement, a process humans perform naturally. We introduce a Feedback Agent $\mathcal{A}_F$ that enables this capability.
Beginning with baseline layout $\mathcal{L}_0$, we iterate for $T$ rounds:
\begin{align}
\mathcal{L}_t = \mathcal{A}_L(\mathcal{H}, \mathcal{L}_{t-1}, \text{FB}_t),
\end{align}
where $\text{FB}_t = \mathcal{A}_F(\text{Render}(\mathcal{L}_{t-1}), \mathcal{T})$ is structured feedback.
The Feedback Agent analyzes rendered layout $\mathcal{F}_{t-1}$ and identifies issues across multiple dimensions: alignment issues where components fail to follow consistent grid structure, spacing problems characterized by inconsistent or insufficient gaps between elements, etc.
For each issue, the agent generates structured natural language feedback. Upon receiving feedback $\text{FB}_t$, the Layout Agent applies Chain-of-Thought reasoning by first understanding the feedback to identify affected components and specific issues, then diagnosing the root cause (e.g., alignment issue caused by inconsistent anchor points), planning systematic corrections solving the root cause, and finally executing the corrections to generate $\mathcal{L}_t$. Iteration stops when $|\{\text{issues}\}| = 0 \text{ OR } t = T$.
\subsubsection{Component Generation and Rendering}
After layout optimization, the Component Agent $\mathcal{A}_C$ renders individual visual elements with consistent professional styling. The agent uses a multi-modal method where geometric elements such as boxes, arrows, and connectors are rendered. Complex visual elements including neural network diagrams, sample images, and domain-specific icons are generated via controlled image generation. Text elements consisting of labels and annotations are rendered with consistent typography. Final figure is composite of all elements according to Eq.~\ref{eq:final_composition}. The output is a fully editable figure with proper layering and grouping.
\subsection{Evaluation Framework}
Figure generation quality is not a single scalar but a multi-dimensional construct. A figure may be good in visual clarity while failing in technical accuracy, or achieve perfect structural coherence while lacking professional design consistency. To this end, we first introduce a consistent evaluation framework for figure generation.
\subsubsection{Rubric Generation from Real Scientific Figures}
We curate a comprehensive dataset $\mathcal{D} = \{\mathcal{F}_1, \mathcal{F}_2, \ldots, \mathcal{F}_N\}$ of $N = 2,219$ method figures from top-tier conferences (CVPR, NeurIPS, ICLR, etc.). Each figure $\mathcal{F}_i$ is associated with its source paper text $\mathcal{T}_i$ and metadata (conference, domain, year). To establish systematic evaluation criteria, our Evaluation Agent analyzes what makes figures in $\mathcal{D}$ high-quality or low-quality by examining their visual characteristics and structural properties. Through this analysis across the entire dataset, the agent identifies and summarizes six fundamental quality dimensions: \textbf{Technical Accuracy and Correctness $R_1$}, \textbf{Visual Clarity and Readability $R_2$}, \textbf{Structural Coherence $R_3$}, \textbf{Design Consistency $R_4$}, \textbf{Interpretability and Accessibility $R_5$}, and \textbf{Technical Implementation Quality $R_6$}. Formally, each rubric $R_k$ defines a scoring function:
\begin{align}
q_k: (\mathcal{F}, \mathcal{T}) \rightarrow [0, 10] \times \mathbb{T},
\end{align}
where the output is a numerical score and textual justification $\mathbb{T}$.
% \subsubsection{Rubric Generation from Real Scientific Figures}
% We curate a comprehensive dataset $\mathcal{D} = \{\mathcal{F}_1, \mathcal{F}_2, \ldots, \mathcal{F}_N\}$ of $N = 2,219$ methodology figures from top-tier conferences (CVPR, NeurIPS, ICLR, etc.). Each figure $\mathcal{F}_i$ is associated with its source paper text $\mathcal{T}_i$ and metadata (conference, domain, year).
% % \textbf{Rubric Extraction:}
% Our Evaluation Agent analyzes $\mathcal{D}$ and extracts six fundamental quality dimensions to determine the quality of a methodology figure: \textbf{Technical Accuracy and Correctness $R_1$}, \textbf{Visual Clarity and Readability $R_2$}, \textbf{Structural Coherence $R_3$}, \textbf{Design Consistency $R_4$}, \textbf{Interpretability and Accessibility $R_5$}, \textbf{Technical Implementation Quality $R_6$}.
% Formally, each rubric $R_k$ defines a scoring function:
% \begin{align}
% q_k: (\mathcal{F}, \mathcal{T}) \rightarrow [0, 10] \times \mathbb{T},
% \end{align}
% where the output is a numerical score and textual justification $\mathbb{T}$.
\subsubsection{Automatic Question Generation}
We automatically generate evaluation questions at two granularities:
\textbf{Dataset-Level Questions $\mathcal{Q}^{\text{common}}$:} These are universal questions applicable to all pipeline figures. For each rubric $R_k$, we generate $3-5$ questions that systematically cover the rubric's key aspects.
% For example, for Visual Clarity ($R_2$), questions are how effectively the figure uses color coding to distinguish functional components, whether spatial organization and alignment support clear visual flow, and if component sizes and proportions appropriately reflect their relative importance in the pipeline.
\textbf{Paper-Specific Questions $\mathcal{Q}_i^{\text{paper}}$:} For each paper $i$ in the dataset, we generate $\sim$40 tailored questions about unique characteristics of that method. These questions capture fine-grained aspects such as whether specific components mentioned in $\mathcal{T}_i$ appear in $\mathcal{F}_i$, and appropriate representation of method-specific architectural patterns.
\subsubsection{Automatic Evaluation Process}
Given a generated figure $\mathcal{F}$ and source description $\mathcal{T}$, the Evaluation Agent systematically evaluates quality through two independent evaluation processes, as formalized in Alg.~\ref{alg:evaluation}.
\textbf{Rubric-Based Evaluation (Common Questions):} For each of the six quality rubrics $R_k$ (where $k = 1, \ldots, 6$), the agent evaluates common questions $\mathcal{Q}^{\text{common}}_k$ that are generated based on rubric $R_k$ and applicable to all pipeline figures. For each question $q \in \mathcal{Q}^{\text{common}}_k$, the agent analyzes the figure $\mathcal{F}$ visually to extract relevant features such as component arrangement, arrow patterns, color schemes, and text readability. It then compares the figure against the source description $\mathcal{T}$ to verify consistency. The AnswerQuestion function produces a detailed answer $a_q$ that is with the specific rubric. After collecting answers for all questions within a rubric, the agent aggregates them into a numerical score $s_k \in [0, 10]$ and a textual justification $j_k$ through AggregateScores. This process repeats for all six rubrics, producing six dimension-specific scores.
\textbf{Paper-Specific Evaluation:} Independently, the agent evaluates paper-specific questions $\mathcal{Q}^{\text{paper}}$ that are generated directly from the dataset to capture unique characteristics of individual methods. These questions are not tied to any rubric dimension and evaluate method-specific aspects such as whether particular components mentioned in the paper appear in the figure, correct use of domain-specific notation, and appropriate representation of unique architectural patterns. The evaluation process follows the same question-answering procedure, producing a separate overall paper-specific score.
The overall quality scores are computed as:
\begin{align}
Q^{\text{common}}(\mathcal{F}, \mathcal{T}) &= \frac{1}{K}\sum_{k=1}^{K} s_k, \\
Q^{\text{paper}}(\mathcal{F}, \mathcal{T}) &= \text{AggregateScores}(\{a_q \mid q \in \mathcal{Q}^{\text{paper}}\})
\end{align}
where $Q^{\text{common}}$ represents the dataset-level quality averaged across rubrics, and $Q^{\text{paper}}$ represents the paper-specific quality score computed independently.
\begin{algorithm}[t!]
\small
\caption{Automatic Figure Evaluation}
\label{alg:evaluation}
\begin{algorithmic}[1]
\Require Figure $\mathcal{F}$, description $\mathcal{T}$, question sets $\mathcal{Q}^{\text{common}}, \mathcal{Q}^{\text{paper}}$
\Ensure Rubric scores $\{s_1, \ldots, s_K\}$, paper-specific score $s^{\text{paper}}$
\State \textbf{// Rubric-Based Evaluation}
\For{each rubric $R_k$, $k = 1, \ldots, K$}
\State $\text{answers}_k \gets \emptyset$
\For{each question $q \in \mathcal{Q}^{\text{common}}_k$}
\State Analyze $\mathcal{F}$ visually to extract relevant features
\State Compare $\mathcal{F}$ against $\mathcal{T}$ for consistency
\State $a_q \gets \text{AnswerQuestion}(q, \mathcal{F}, \mathcal{T})$
\State $\text{answers}_k \gets \text{answers}_k \cup \{a_q\}$
\EndFor
\State $(s_k, j_k) \gets \text{AggregateScores}(\text{answers}_k)$
\EndFor
% \State
\State \textbf{// Paper-Specific Evaluation}
\State $\text{answers}^{\text{paper}} \gets \emptyset$
\For{each question $q \in \mathcal{Q}^{\text{paper}}$}
\State Analyze $\mathcal{F}$ visually to extract relevant features
\State Compare $\mathcal{F}$ against $\mathcal{T}$ for consistency
\State $a_q \gets \text{AnswerQuestion}(q, \mathcal{F}, \mathcal{T})$
\State $\text{answers}^{\text{paper}} \gets \text{answers}^{\text{paper}} \cup \{a_q\}$
\EndFor
\State $s^{\text{paper}} \gets \text{AggregateScores}(\text{answers}^{\text{paper}})$ \\
\Return $\{s_1, \ldots, s_K\}$, $s^{\text{paper}}$
\end{algorithmic}
\end{algorithm}
\vspace{-3mm}
\section{Experiments}
\label{sec:exp}
% \subsection{Experimental Setup}
% \textbf{Dataset.} We curate a comprehensive dataset of pipeline figures from top-tier conferences to support evaluation benchmarking. Our collection process begins with gathering 2,219 pipeline figures from papers published in 2023 across major AI and machine learning conferences including CVPR, NeurIPS, ICLR, ICML, AAAI, ACL, EMNLP, ICASSP, and others. To ensure dataset quality and diversity, we perform balanced sampling across both conference conferences and research domains, selecting 435 papers that are published in specific conferences with broad domain coverage spanning 35 research areas.
% % The 500-paper subset exhibits broad domain coverage spanning 35 research areas. The distribution includes major representation from core Computer Science domains: Computer Vision (cs.CV, 9.4\%), Natural Language Processing (cs.CL, 9.4\%), Machine Learning (cs.LG, 9.2\%), Information Retrieval (cs.IR, 9.2\%), Speech and Audio (cs.SD, 9.0\%; eess.AS, 9.0\%), and Artificial Intelligence (cs.AI, 9.4\%). The dataset also includes substantial coverage of specific domains such as Image and Video Processing (eess.IV, 5.3\%), Signal Processing (eess.SP, 2.5\%), Robotics (cs.RO, 2.8\%), Cryptography and Security (cs.CR, 3.4\%), and Systems and Control (cs.SI, 3.7\%). Additional domains represented include Graphics (cs.GR), Human-Computer Interaction (cs.HC), Neural and Evolutionary Computing (cs.NE), Multimedia (cs.MM), Quantum Physics (quant-ph), Quantitative Biology (q-bio), and others, ensuring comprehensive coverage across the AI research landscape.
% From this balanced 435-paper collection, we select 30 papers as our final test set for comprehensive evaluation. Each test paper contains detailed method descriptions and corresponding human-designed pipeline figures. This rigorous curation process ensures our evaluation reflects real-world challenges in scientific figure generation across diverse research contexts.
% \textbf{Evaluation Metrics.} We use our proposed rubric-based evaluation framework with six quality dimensions.
% % : Technical Accuracy and Correctness ($R_1$), Visual Clarity and Readability ($R_2$), Structural Coherence ($R_3$), Design Consistency ($R_4$), Interpretability and Accessibility ($R_5$), and Technical Implementation Quality ($R_6$).
% Each dimension is scored on a 0-10 scale (reported in percentage), with the overall quality computed as the average across all dimensions. For the 30 test papers, we generate both common evaluation questions applicable to all pipeline figures and paper-specific questions tailored to each method's unique characteristics. Additionally, we have human evaluation with 20+ participants (See Supplemental Material for details) from various backgrounds.
% \textbf{Baselines.} We compare against several representative methods of figure generation. For single-agent LLM methods, we evaluate GPT-5-Image \cite{openai2025gpt5}, Gemini-2.5 \cite{comanici2025gemini}, and Qwen-Image \cite{wu2025qwen}. For image generation methods, we include Stable Diffusion V1.5 \cite{rombach2022high} and XL with diagram-specific fine-tuning \cite{podellsdxl}. For layout-based methods, we compare against Paper2Poster \cite{pang2025paper2poster} as an LLM-based multi-agent layout generation system.
\textbf{Dataset.}
We curate a dataset of 2,219 pipeline figures from papers published in 2023 across top AI conferences, including CVPR, NeurIPS, ICLR, ICML, AAAI, ACL, EMNLP, and ICASSP. To ensure quality and diversity, we perform balanced sampling across venues and research domains, selecting 435 papers spanning 35 areas such as computer vision, NLP, machine learning, speech, and signal processing. From this balanced set, 30 papers with detailed method descriptions and human-designed figures form our final test set, reflecting real-world diversity and structural complexity in scientific visualization. More details in Supplementary.
\textbf{Evaluation Metrics.}
We employ a rubric-based framework covering six dimensions: technical accuracy, visual clarity, structural coherence, design consistency, interpretability, and implementation quality. Each dimension is rated on a 0–10 scale (reported in percentage), with overall quality as their mean. For the 30 test papers, we design both general and paper-specific evaluation questions. Human assessment is conducted with 20+ participants from diverse research backgrounds (details in Supplemental Material).
\textbf{Baselines.}
We compare \textsc{SciFig} with representative figure generation approaches. Single-agent LLMs include GPT-5-Image~\cite{openai2025gpt5}, Gemini-2.5~\cite{comanici2025gemini}, and Qwen-Image~\cite{wu2025qwen}. Image generation baselines include Stable Diffusion v1.5~\cite{rombach2022high} and SDXL with diagram-specific fine-tuning~\cite{podellsdxl}. For layout generation, we benchmark against Paper2Poster~\cite{pang2025paper2poster}, a multi-agent pipeline.
\begin{table*}[t!]\small
\centering
\setlength{\tabcolsep}{4pt}
\renewcommand{\arraystretch}{0.7}
\resizebox{\linewidth}{!}{\begin{tabularx}{\textwidth}{lXXXXXXXc}
\toprule
& \multicolumn{7}{c}{\bf Dataset-Level Questions} & \bf Paper-Specific \\
\cmidrule{2-8}
\multirow{-2.5}{*}{\bf Model} & $R_1$ & $R_2$ & $R_3$ & $R_4$ & $R_5$ & $R_6$ & Overall & Overall \\ \midrule
GPT-5-Image \cite{openai2025gpt5} & 36.3 & 45.1 & 52.5 & 44.4 & 34.4 & 45.9 & 43.1 & 30.2 \\
% Claude-4.5 & & & & & & & & \\
Gemini-2.5-Flash \cite{comanici2025gemini} & \underline{64.4} & 56.9 & \underline{64.5} & 64.1 & 57.3 & 57.9 & 60.9 & \textbf{67.3} \\
Qwen-Image \cite{wu2025qwen} & 6.2 & 12.4 & 14.4 & 12.9 & 9.0 & 11.9 & 11.1 & 5.6 \\
Stable Diffusion V1.5 \cite{rombach2022high} & 8.9 & 13.4 & 13.7 & 13.3 & 11.1 & 12.8 & 12.2 & 7.0 \\
Stable Diffusion XL \cite{podellsdxl} & 26.6 & 29.5 & 38.9 & 32.9 & 27.6 & 28.4 & 30.6 & 18.6 \\
Paper2Poster \cite{pang2025paper2poster} & 62.5 & \underline{64.4} & 62.2 & \textbf{76.2} & \underline{61.8} & \underline{67.4} & \underline{65.7} & 63.0 \\ \midrule
\rowcolor{Light}
\textbf{SciFig (Ours)} & \textbf{67.5} & \textbf{68.8} & \textbf{71.5} & \underline{75.6} & \textbf{67.2} & \textbf{69.9} & \textbf{70.1} & \underline{66.2} \\
\bottomrule
\end{tabularx}}
% \vspace{-2mm}
\caption{\textbf{Quantitative Comparison on Scientific Figure Generation.} Dataset-level questions evaluate common quality aspects applicable to all pipeline figures, while paper-specific questions assess method-specific characteristics. Best results in \textbf{bold}, second-best \underline{underlined}.}
\label{tab:main_results}
\vspace{-1em}
\end{table*}
\begin{table*}[t!]\small
\centering
\setlength{\tabcolsep}{4pt}
\renewcommand{\arraystretch}{0.7}
\resizebox{\linewidth}{!}{\begin{tabularx}{\textwidth}{lXXXXXXXc}
\toprule
& \multicolumn{7}{c}{\bf Dataset-Level Questions} & \bf Paper-Specific \\
\cmidrule{2-8}
\multirow{-2.5}{*}{\bf Model} & $R_1$ & $R_2$ & $R_3$ & $R_4$ & $R_5$ & $R_6$ & Overall & Overall \\ \midrule
Baseline & 68.9 & 65.1 & 68.5 & 72.4 & 67.2 & 63.9 & 67.7 & 68.1 \\
+ Naive Layout Agent & 67.2 & 65.7 & 69.2 & 74.1 & 66.5 & 66.9 & 68.3 & 68.5 \\
+ Hierarchical Layout & 67.1 & 68.1 & 70.2 & 76.5 & 65.0 & 69.7 & 69.4 & 67.2 \\
+ CoT Feedback & 69.5 & 70.2 & 71.5 & 78.4 & 68.5 & 71.1 & 71.6 & 69.3 \\ \midrule
w/ CoT Feedback round 1 & 67.0 & 65.1 & 68.6 & 72.7 & 65.7 & 64.9 & 67.3 & 67.3 \\
w/ CoT Feedback round 2 & 67.3 & 65.9 & 69.7 & 73.1 & 65.9 & 66.7 & 68.1 & 66.6 \\
w/ CoT Feedback round 3 & 63.5 & 61.5 & 66.8 & 68.8 & 64.3 & 61.0 & 64.3 & 65.6 \\
w/ CoT Feedback round 4 & 67.9 & 66.2 & 69.2 & 74.0 & 67.7 & 66.1 & 68.5 & 65.1 \\
w/ CoT Feedback round 5 & 65.0 & 62.9 & 66.8 & 70.6 & 63.5 & 62.7 & 65.3 & 63.9 \\ \bottomrule
\end{tabularx}}
% \vspace{-2mm}
\caption{\textbf{Ablation Study: Component Contributions.} We systematically evaluate the contribution of each component in SciFig's architecture. The table is divided into two parts: (1) \textit{Cumulative Component Analysis} progressively adds components to isolate individual contributions—starting from a baseline, adding Naive Layout Agent, then Hierarchical Layout strategy, and finally CoT Feedback mechanism; (2) \textit{Isolated CoT feedback analysis} tests different numbers of feedback rounds (1-5) on the Naive Layout Agent baseline \textit{without} hierarchical structure to isolate feedback effects independent of hierarchy.}
% \vspace{-5mm}
\label{tab:ablation}
\end{table*}
\subsection{Quantitative Evaluation}
Table~\ref{tab:main_results} shows comprehensive quantitative comparison between SciFig and baseline methods across all six quality dimensions.
\textbf{Overall Performance.} SciFig achieves the highest overall quality score of 70.1 on dataset-level questions, substantially outperforming the best baseline (Paper2Poster at 65.7) by 4.4. On paper-specific evaluation questions, our system scores 66.2, demonstrating robust performance on fine-grained method details. This shows the effectiveness of our system.
\textbf{Dimension-Specific Analysis.} SciFig achieves the highest scores in five of six dimensions, demonstrating comprehensive quality advantages. In Structural Coherence ($R_3$: 71.5), our hierarchical layout generation with explicit module-level organization substantially outperforms the best baseline by 7.0\%+, validating this as our core novelty. Visual Clarity ($R_2$: 68.8) and Technical Implementation Quality ($R_6$: 69.9) reflect our iterative CoT feedback mechanism's effectiveness in achieving professional-level aesthetics with publication-ready graphics. Technical Accuracy ($R_1$: 67.5) and Interpretability ($R_5$: 67.2) demonstrate our system's ability to maintain scientific correctness while ensuring accessibility. Paper2Poster achieves the highest Design Consistency ($R_4$: 76.2), the only dimension where it exceeds SciFig (75.6), reflecting its capability in document formatting—though this comes at the cost of structural coherence (62.2 vs. 71.5) and technical accuracy (62.5 vs. 67.5). Gemini-2.5-Flash and GPT-5-Image achieve moderate scores (36.3-64.5 range) but struggle with hierarchical organization and professional styling. Stable Diffusion XL/V1.5 and Qwen-Image score consistently lowest across all dimensions (6.2-38.9 range), producing blurry, non-editable outputs with poor text rendering, pixelation artifacts, and inability to capture logical structure inherent in scientific pipelines.
% \textbf{Technical Accuracy ($R_1$).} AgentFig scores 67.5 on technical accuracy, outperforming all baselines. Gemini-2.5-Flash achieves 64.4 and Paper2Poster reaches 62.5, demonstrating that LLM methods can maintain reasonable scientific accuracy. GPT-5-Image scores only 36.3, Stable Diffusion XL achieves 26.6, and Qwen-Image reaches just 6.2, showing that they struggle with scientific accuracy in structured technical diagrams.
% \textbf{Visual Clarity ($R_2$).} Our method achieves 68.8 in visual clarity, the highest among all methods. Paper2Poster scores 64.4 and Gemini-2.5-Flash reaches 56.9, showing that they can achieve reasonable readability. However, AgentFig systematically identifies and corrects alignment issues, spacing problems, and label readability across multiple rounds, achieving professional-level clarity. GPT-5-Image (45.1), Stable Diffusion XL (29.5), Stable Diffusion V1.5 (13.4), and Qwen-Image (12.4) all suffer from blurry text, poor contrast, and inconsistent visual encoding.
% \textbf{Structural Coherence ($R_3$).} AgentFig scores 71.5, the highest across all dimensions. Gemini-2.5-Flash achieves 64.5 and Paper2Poster reaches 62.2, demonstrating that LLMs can produce reasonably organized layouts. However, the 7-9\% gap demonstrates that explicit hierarchical modeling—with module-level organization and inter-module connections—provides substantial advantages over flat layout methods. GPT-5-Image produce notably less coherent arrangements (52.5). Stable Diffusion XL (38.9), V1.5 (13.7), and Qwen-Image (14.4) score lowest, showing they cannot capture the logical structure inherent in scientific pipelines.
% \textbf{Design Consistency ($R_4$).} Our method achieves 75.6, substantially outperforming all baselines. Notably, Paper2Poster scores 76.2 in this single dimension—the only case where a baseline exceeds AgentFig. This reflects Paper2Poster's specialization in document formatting and visual consistency for poster generation. However, Paper2Poster's strength in design consistency comes at the cost of structural coherence (62.2 vs. our 71.5) and technical accuracy (62.5 vs. our 67.5), as it focuses on visual uniformity rather than scientific correctness. Gemini-2.5-Flash achieves 64.1, while other methods score substantially lower (GPT-5-Image: 44.4, Stable Diffusion XL: 32.9). AgentFig ensures professional styling across diverse element types while maintaining scientific accuracy.
% \textbf{Interpretability and Accessibility ($R_5$).} AgentFig scores 67.2, demonstrating strong performance in making figures understandable and accessible. Paper2Poster achieves 61.8 and Gemini-2.5-Flash reaches 57.3. Our hierarchy strategy is critical—replacing dozens of component-level arrows with clean module-level connections dramatically improves interpretability by emphasizing high-level information flow. GPT-5-Image generates cluttered figures requiring significant cognitive effort to understand (34.4). Stable Diffusion XL (27.6), V1.5 (11.1), and Qwen-Image (9.0) perform poorly, producing outputs that lack self-contained explanatory structure.
% \textbf{Technical Implementation Quality ($R_6$).} Our method achieves 69.9, reflecting publication-ready vector graphics with clean lines, proper scaling, and high resolution. Paper2Poster scores 67.4 and Gemini-2.5-Flash reaches 57.9. GPT-5-Image (45.9), Stable Diffusion XL (28.4), V1.5 (12.8), and Qwen-Image (11.9) score lowest due to pixelation and artifacts.
\textbf{Paper-Specific Analysis.} On paper-specific questions, SciFig maintains strong performance (66.2). Gemini-2.5-Flash shows 67.3, performing better. Paper2Poster achieves 63.0. In contrast, other models show dramatic degradation: GPT-5-Image 30.2, Stable Diffusion XL 18.6, and Qwen-Image 5.6. This pattern indicates that while SciFig successfully captures fine-grained method details specific to individual papers, these methods produce general outputs that fail to represent method-specific architectural patterns, domain-specific notation, or unique component relationships.
The results reveal three different performance levels: (1) SciFig (70.1) achieve the highest quality through specialized agent collaboration and hierarchical reasoning; (2) Gemini-2.5-Flash (60.9) and Paper2Poster (65.7) produce reasonable quality but cannot balance competing objectives without explicit architectural decomposition; (3) GPT-5-Image (43.1), Stable Diffusion XL (30.6), and others ($<15.0$) fundamentally struggle with structured technical visualization, achieving low scores across all dimensions.
\begin{figure*}
\centering
\includegraphics[width=\linewidth]{agentfig-visualization.pdf}
\caption{\textbf{Qualitative Comparison Across Research Domains.} We compare SciFig against four baseline methods with same method text input across three diverse research domains: Artificial Intelligence (left column), Graphics (middle column), and Material Science (right column). \textbf{GPT-5-Image} produces figures with poor hierarchical organization, arbitrary arrow connections, inconsistent styling, and bad text rendering quality. \textbf{Qwen-Image} generates text-heavy outputs that completely fail to create visual pipeline representations, essentially producing unreadable document layouts instead of technical diagrams. \textbf{Stable Diffusion XL} produces blurry, non-editable images that lack scientific accuracy and semantic coherence. \textbf{SciFig (Ours)} generates publication-ready figures with clear hierarchical organization, clean module-level connections, consistent professional styling, and domain-appropriate scientific accuracy. Our method maintains high quality across diverse research domains, demonstrating the effectiveness of multi-agent architecture with hierarchical layout generation and iterative CoT feedback.}
% \vspace{-4mm}
\label{fig:qualitative}
\end{figure*}
\subsection{Ablation Study}
We evaluate the contribution of each component in SciFig's multi-agent architecture. Table~\ref{tab:ablation} shows systematic ablation results on our test set. We remove or modify components to isolate their individual contributions.
\textbf{Cumulative Component Analysis.} Starting from a baseline system with single agent (67.7 overall), we progressively add components to isolate their contributions. Adding a naive flat layout agent improves quality to 68.3 (+0.6), demonstrating that even basic spatial arrangement provides benefits. Incorporating hierarchical layout generation further improves overall quality to 69.4 (+1.1 from naive layout), with substantial gains in Visual Clarity ($R_2$, 65.7 → 68.1, +2.4), Implementation Quality ($R_6$, 66.9 → 69.7, +2.8), and Structural Coherence ($R_3$, 69.2 → 70.2, +1.0), validating that explicit module-level organization and inter-module connections are essential for translating hierarchical structure into effective visual organization. The complete iterative CoT feedback mechanism on top of hierarchical layout achieves 71.6 overall (+2.2), representing our full system, with improvements distributed across all six dimensions: Technical Accuracy (67.1 → 69.5, +2.4), Visual Clarity (68.1 → 70.2, +2.1), Structural Coherence (70.2 → 71.5, +1.3), Design Consistency (76.5 → 78.4, +1.9), Interpretability (65.0 → 68.5, +3.5), and Implementation Quality (69.7 → 71.1, +1.4). Paper-specific performance follows a similar progressive pattern (68.1 → 68.5 → 67.2 → 69.3), demonstrating that while each component provides benefits, the complete system with hierarchical layout and iterative feedback is necessary to achieve professional-level quality across all evaluation dimensions.
\textbf{Isolated CoT Feedback Analysis.} To isolate feedback effects independent of hierarchical structure, we test 1-5 feedback rounds on the naive flat layout baseline (68.3). Results reveal fundamental instability with a non-monotonic pattern: round 1 degrades to 67.3 (-1.0), round 2 recovers to 68.1, round 3 catastrophically drops to 64.3 (-4.0 from baseline) with dramatic dimension-specific degradation ($R_2$ -4.4, $R_4$ -4.3, $R_6$ -5.7), round 4 partially recovers to 68.5 (+0.2 above baseline), and round 5 drops again to 65.3 (-3.2). This unstable trajectory demonstrates a critical insight: \textit{iterative feedback and hierarchical layout are not independent improvements but collaborative components that depend on each other}. When CoT feedback applies to hierarchical layouts (full system: 71.6), it achieves substantial gains (+2.2 from hierarchy-only baseline of 69.4) because hierarchical structure provides stable module boundaries that guide corrections with limited cross-module impact. In contrast, feedback on flat layouts struggles because corrections lack organizational context—improving alignment in one region disrupts spacing elsewhere, rerouting arrows creates new overlaps, and local optimizations conflict with global coherence, causing the observed non-monotonic instability. Without hierarchical structure providing stable organization, iterative feedback cannot reliably converge to high-quality outputs.
% \textbf{Baseline + Naive Layout Agent (No Hierarchy).} When we replace our hierarchical layout generation with a naive flat layout agent that treats all components uniformly, the system achieves 68.3 overall quality on dataset-level questions and 68.5 on paper-specific questions. While this still shows strong performance due to the Description Agent's structure extraction, the absence of explicit hierarchical layout modeling causes notable degradation in Structural Coherence (69.2 vs. 71.5 in full system) and Design Consistency (74.1 vs. 75.6). Without module-level organization and inter-module connections, the layout generator cannot effectively translate the hierarchical structure into visual organization. Components are arranged spatially but lack clear functional grouping and visual containerization that emphasize logical relationships.
% \textbf{+ Hierarchical Layout Strategy.} Incorporating our hierarchical layout generation strategy improves overall quality to 69.4 (+1.1 overall). Structural Coherence increases (69.2 $\rightarrow$ 70.2, +1.0), validating that explicit hierarchical modeling in the layout phase is essential beyond just having hierarchical structure from the Description Agent. The hierarchical strategy also improves Visual Clarity (65.7 $\rightarrow$ 68.1, +2.4) and Implementation Quality (66.9 $\rightarrow$ 69.7, +2.8) as module boundaries and consistent organization help better component rendering and alignment. Notably, paper-specific performance shows more modest improvement (68.5 $\rightarrow$ 67.2, -1.3), suggesting that hierarchical layout provides consistent benefits across general cases but requires careful tuning for method-specific architectural patterns.
% \textbf{+ CoT Feedback Mechanism.} Adding the complete iterative CoT feedback mechanism on top of hierarchical layout achieves 71.6 overall on dataset-level questions (+2.2 from hierarchical layout), representing our full system performance. The improvement is distributed across all dimensions: Technical Accuracy (67.1 $\rightarrow$ 69.5, +2.4), Visual Clarity (68.1 $\rightarrow$ 70.2, +2.1), Structural Coherence (70.2 $\rightarrow$ 71.5, +1.3), Design Consistency (76.5 $\rightarrow$ 78.4, +1.9), Interpretability (65.0 $\rightarrow$ 68.5, +3.5), and Implementation Quality (69.7 $\rightarrow$ 71.1, +1.4). The Feedback Agent's objective analysis combined with the Layout Agent's CoT reasoning enables systematic improvement across multiple quality dimensions. Paper-specific performance reaches 69.3 (+2.1 from hierarchical layout), demonstrating that iterative feedback helps capture fine-grained method details.
% \textbf{Isolating CoT Feedback Contribution.} To understand the specific contribution of iterative CoT feedback independent of hierarchical layout, we show additional experiments starting from the naive layout agent baseline (without hierarchical structure) and adding only different numbers of feedback rounds. This isolates the effect of iterative improvement from hierarchical organization.
% SY: Definitely cut below
% \textit{With 1 Feedback Round:} Starting from naive layout (68.3 baseline) and adding one CoT feedback round achieves \textbf{67.3} overall, representing a \textbf{-1.0 point degradation}. Paper-specific score is \textbf{67.3}. This counterintuitive result—where adding feedback actually harms performance—reveals that a single correction pass on a flat layout can introduce inconsistencies without sufficient iterations to reconcile them. Visual Clarity drops significantly (65.7 → 65.1, -0.6) and Implementation Quality decreases (66.9 → 64.9, -2.0). Without hierarchical structure, the Feedback Agent identifies issues in a flat arrangement, but one correction round is insufficient to achieve coherent improvements.
% \textit{With 2 Feedback Rounds:} Two feedback rounds on naive layout achieve \textbf{68.1} overall (+0.8 from round 1), with paper-specific score of \textbf{66.6}. This matches the naive layout baseline (68.3) approximately, suggesting that two rounds can recover from the single-round degradation but provide minimal net benefit without hierarchical structure. Visual Clarity (65.1 → 65.9, +0.8) and Structural Coherence (68.6 → 69.7, +1.1) improve modestly, but the system still struggles to achieve professional quality on flat layouts.
% \textit{With 3 Feedback Rounds:} Three rounds achieve \textbf{64.3} overall, a significant \textbf{-4.0 point drop} from the naive baseline. Paper-specific performance is \textbf{65.6}. This substantial degradation demonstrates that without hierarchical structure to guide corrections, intermediate feedback rounds can over-correct and introduce local optimizations that harm global coherence. Visual Clarity drops dramatically (65.9 → 61.5, -4.4), Design Consistency decreases (73.1 → 68.8, -4.3), and Implementation Quality falls sharply (66.7 → 61.0, -5.7). The Feedback Agent attempts to improve a fundamentally flat organization, but corrections in one area create cascading issues elsewhere without the stabilizing structure that hierarchy provides.
% \textit{With 4 Feedback Rounds:} Four rounds achieve \textbf{68.5} overall (+4.2 from round 3), recovering from the degradation and slightly exceeding the naive baseline. Paper-specific score is \textbf{65.1}. Design Consistency improves substantially (68.8 → 74.0, +5.2), suggesting that additional rounds help reconcile previous over-corrections. However, performance remains well below what hierarchical layout achieves (69.4) and far below the full system (71.6).
% \textit{With 5 Feedback Rounds:} Five rounds on naive layout achieve \textbf{65.3} overall, unexpectedly dropping again. Paper-specific score is \textbf{63.9}. Technical Accuracy (67.9 → 65.0, -2.9), Visual Clarity (66.2 → 62.9, -3.3), and Implementation Quality (66.1 → 62.7, -3.4) all degrade significantly. This demonstrates that iterative feedback alone, without hierarchical structure, cannot reliably converge to high quality—additional rounds may continue to introduce corrections that lack global coherence.
% cut ends
% The isolated CoT experiments reveal a fundamental insight: \textit{iterative feedback and hierarchical layout are not independent improvements but collaborative parts that depend on each other}.
% When CoT feedback applies to hierarchical layouts (full system: 71.6), it achieves substantial gains (+2.2 from hierarchy-only baseline of 69.4). The hierarchical structure provides stable module boundaries and logical organization that guide feedback corrections. The Feedback Agent can identify issues within well-defined modules, and corrections in one module have limited impact on others.
% In contrast, when CoT feedback applies to flat layouts (best: 68.5 with 4 rounds, often worse than naive baseline: 68.3), it struggles to achieve consistent improvements. Without hierarchical structure, corrections lack organizational context—improving alignment in one region may disrupt spacing elsewhere, rerouting arrows may create new overlaps, and local optimizations conflict with global coherence. The non-monotonic pattern (degradation at rounds 1 and 3, recovery at rounds 2 and 4, degradation again at round 5) demonstrates instability.
This explains why both components are necessary: hierarchical layout (69.4) provides structure but lacks the improvement needed for professional quality, while feedback alone on flat layouts ($< 68.5$) lacks stable organization. Together, they achieve 71.6, substantially outperforming either component in isolation.
% This collaboration explains why our full system requires both components: hierarchical layout alone (69.4) provides good structural organization but lacks the improvement needed for professional quality; iterative feedback alone on flat layouts (typically $<68.5$, often worse than baseline) cannot overcome the fundamental limitations of non-hierarchical organization. Together, they achieve 71.6, subatantially outperforming either component in isolation.
% cut temporarily; 8:30 pm 11/13: move to sm
% Fig.~\ref{fig:iterative_progression} visualizes the iterative improvement process for our full system (hierarchical layout + CoT feedback). Round 0 shows the hierarchical layout baseline with clear module boundaries and logical organization but imperfect alignment and spacing. Rounds 1-2 progressively correct these issues while maintaining the hierarchical structure. Rounds 3-5 achieve professional polish with consistent styling and balanced composition. In contrast, the same visualization for flat layout + feedback (not shown) would demonstrate the instability observed in the ablation—corrections that improve one aspect while degrading others, lacking the organizing principle that hierarchy provides.
% \begin{table*}[h]
% \centering
% \begin{tabularx}{\textwidth}{lXXXXXXXc}
% \hline
% & \multicolumn{7}{c}{Dataset-Level Questions} & Paper-Specific \\
% Model & $R_1$ & $R_2$ & $R_3$ & $R_4$ & $R_5$ & $R_6$ & Overall & Overall \\ \hline
% Baseline & & & & & & & & \\
% + Naive Layout Agent & 67.2 & 65.7 & 69.2 & 74.1 & 66.5 & 66.9 & 68.3 & 68.5 \\
% + Hierarchical Layout & 67.1 & 68.1 & 70.2 & 76.5 & 65.0 & 69.7 & 69.4 & 67.2 \\
% + CoT Feedback & 69.5 & 70.2 & 71.5 & 78.4 & 68.5 & 71.1 & 71.6 & 69.3 \\ \hline
% \end{tabularx}
% \caption{system ablation}
% \label{tab:placeholder}
% \end{table*}
% \begin{table*}[h]
% \centering
% \begin{tabularx}{\textwidth}{lXXXXXXXc}
% \hline
% & \multicolumn{7}{c}{Dataset-Level Questions} & Paper-Specific \\
% Model & $R_1$ & $R_2$ & $R_3$ & $R_4$ & $R_5$ & $R_6$ & Overall & Overall \\ \hline
% Baseline & 67.2 & 65.7 & 69.2 & 74.1 & 66.5 & 66.9 & 68.3 & 68.5 \\
% w/ Hierarchical Layout & 67.1 & 68.1 & 70.2 & 76.5 & 65.0 & 69.7 & 69.4 & 67.2 \\
% w/ CoT Feedback round 1 & 67.0 & 65.1 & 68.6 & 72.7 & 65.7 & 64.9 & 67.3 & 67.3 \\
% w/ CoT Feedback round 2 & 67.3 & 65.9 & 69.7 & 73.1 & 65.9 & 66.7 & 68.1 & 66.6 \\
% w/ CoT Feedback round 3 & 63.5 & 61.5 & 66.8 & 68.8 & 64.3 & 61.0 & 64.3 & 65.6 \\
% w/ CoT Feedback round 4 & 67.9 & 66.2 & 69.2 & 74.0 & 67.7 & 66.1 & 68.5 & 65.1 \\
% w/ CoT Feedback round 5 & 65.0 & 62.9 & 66.8 & 70.6 & 63.5 & 62.7 & 65.3 & 63.9 \\
% w/ Hierarchy and CoT & 67.5 & 68.8 & 71.5 & 75.6 & 67.2 & 69.9 & 70.1 & 66.2 \\ \hline
% \end{tabularx}
% \caption{layout ablation}
% \label{tab:placeholder}
% \end{table*}
% \begin{table*}[h]
% \centering
% \begin{tabularx}{\textwidth}{lXXXXXXXX}
% \hline
% & \multicolumn{7}{c}{Common Dimensions for All Papers} & Paper-Specific \\
% Model & R1 & R2 & R3 & R4 & R5 & R6 & Overall & Overall \\ \hline
% baseline: no component polish & & & & & & & & \\
% w/ svg & & & & & & & & \\
% w/ png & & & & & & & & \\
% w/ text placement & & & & & & & & \\
% w/ shape adjustment & & & & & & & & \\
% w/ mix svg and png & & & & & & & & \\ \hline
% \end{tabularx}
% \caption{component ablation}
% \label{tab:placeholder}
% \end{table*}
\subsection{Qualitative Results}
Fig.~\ref{fig:qualitative} presents representative results comparing SciFig with baselines on identical method text inputs across 3 domains: Artificial Intelligence, Graphics, and Material Science.
GPT-5-Image produces figures with weak hierarchical structure, arbitrary arrow connections, and inconsistent visual styles, lacking clear module boundaries and professional consistency. Qwen-Image generates text-heavy outputs that completely fail to create visual pipeline representations, essentially producing unreadable document layouts instead of technical diagrams. Stable Diffusion XL generates blurry, non-editable, and scientifically inaccurate pixel-level outputs, demonstrating limitations of general image generation models for structured technical visualization.
% Single-agent LLMs (GPT-5-Image, Claude-4.5-Sonnet) produce figures with weak hierarchical structure, excessive component-level arrows, and inconsistent visual styles. While Claude-4.5-Sonnet improves layout slightly, both lack clear module boundaries and professional consistency. Image-generation models (e.g., Stable Diffusion XL) perform worst, yielding blurry, non-editable, and scientifically inaccurate outputs.
\textbf{SciFig (Ours)} delivers consistently high-quality figures across all domains. Its hierarchical layout generation yields clear module boundaries, uniform styling, and concise module-level connections that highlight high-level information flow. The iterative CoT feedback further enhances visual polish, optimizing spacing, alignment, text readability, and color contrast, producing publication-ready, fully editable outputs adaptable to domain-specific needs.
% \textbf{GPT-5-Image} produces outputs with inconsistent visual structure across domains. In the AI domain example, the generated figure shows poor hierarchical organization with components scattered without clear functional grouping. Arrow connections appear arbitrary, failing to convey the logical flow of information. Text labels suffer from readability issues with inconsistent font sizes and overlapping elements. The Graphics domain example demonstrates similar problems—while individual components are recognizable, they lack coherent spatial arrangement and professional styling. The Material Science example shows technical inaccuracies where the molecular structure representation deviates from standard conventions.
% \textbf{Claude-4.5-Sonnet} demonstrates improved layout organization compared to GPT-5-Image but still exhibits fundamental limitations of single-agent approaches. The AI domain figure shows attempts at modular organization, but module boundaries are unclear and connections between modules remain cluttered with excessive component-level arrows. In the Graphics domain, the figure achieves better visual balance but fails to maintain consistent styling—color schemes, box shapes, and arrow types vary inconsistently across the figure. The Material Science example reveals the challenge of scientific accuracy, where complex domain-specific representations (crystal lattice structures, atomic positions) are oversimplified or incorrectly depicted.
% \textbf{Stable Diffusion XL}, despite fine-tuning for diagram generation, produces the least satisfactory results. The image generation paradigm creates pixel-level outputs that appear superficially plausible but lack both scientific accuracy and editability. The AI domain example shows a blurry, visually inconsistent figure where text is barely readable and component relationships are semantically meaningless. In the Graphics domain, the generated output resembles a photograph of a hand rather than a technical diagram, demonstrating the fundamental challenge of using general image generation models for structured technical visualization. The Material Science example produces a graph-like structure with nodes and edges that bears no correspondence to the actual crystalline lattice structure described in the method text. These outputs are neither scientifically accurate nor editable for publication.
% \textbf{Paper2Poster}, designed specifically for converting research papers to poster layouts, produces text-heavy outputs that fail to generate proper visual pipeline representations. The AI domain example shows primarily textual content with minimal visual abstraction of the methodology. While text is well-formatted and readable, it does not provide the visual clarity that pipeline figures require. The Graphics and Material Science examples demonstrate similar issues—the system essentially extracts and reformats text from papers rather than creating visual representations of pipelines. This approach fundamentally mismatches the requirements of methodology figure generation.
% \textbf{AgentFig (Ours)} demonstrates substantial improvements across all domains and quality dimensions. In the AI domain, our hierarchical layout generation produces clear module boundaries with distinct functional groups. The figure uses consistent visual styling—uniform box shapes, coherent color schemes, and professional typography. Module-level connections replace cluttered component-level arrows, creating clean visual flow that emphasizes high-level information progression. Component alignment follows a consistent grid structure, and spacing is optimized through our iterative CoT feedback mechanism.
% The Graphics domain example showcases AgentFig's ability to handle domain-specific visual requirements. The figure correctly represents the volumetric reasoning and super-resolution pipeline stages with appropriate visual metaphors. Mathematical notations are properly formatted, and the spatial arrangement reflects the sequential nature of the algorithm. The Material Science example demonstrates scientific accuracy—atomic positions, lattice structures, and vector field representations conform to domain conventions. The hierarchical organization clearly delineates different stages of the crystalline material generation process.
% Across all three domains, AgentFig maintains publication-ready aesthetic quality. Text is consistently readable with appropriate font sizes and contrast. Color schemes are coherent and purposeful, using visual encoding to distinguish different component types. The overall composition is balanced with effective use of white space. Most importantly, most outputs are fully editable, allowing researchers to make minor adjustments while benefiting from the automatically generated structure.
% \textbf{Key Observations.} The qualitative comparison reveals three critical advantages of our approach: (1) \textit{Hierarchical organization} enables clear visual structure that single-agent and image generation methods cannot achieve; (2) \textit{Scientific accuracy} is maintained through our Description Agent's semantic parsing and domain-aware component generation; (3) \textit{Professional aesthetics} emerge from iterative CoT feedback that progressively refines visual quality. These advantages are consistent across diverse research domains, validating the generality of our multi-agent architecture.
\section{Conclusion}
% We introduce SciFig, the first end-to-end AI agent system for automatic scientific pipeline figure generation, transforming textual method descriptions into publication-ready figures through systematic multi-agent decomposition. Our key contributions are: First, we propose hierarchical layout generation with explicit module-level organization and inter-module connections, achieving overall improvement with particularly strong gains in Structural Coherence and Implementation Quality. Second, we introduce an iterative Chain-of-Thought feedback mechanism that enables professional-level aesthetics through systematic correction across multiple quality dimensions. Third, we establish the first systematic evaluation framework through analyzing 2,219 real scientific figures from top-tier conferences, introducing six-dimensional rubrics (Technical Accuracy, Visual Clarity, Structural Coherence, Design Consistency, Interpretability, Implementation Quality) that enable meaningful comparison between generation methods. SciFig achieves 70.1 overall quality on dataset-level evaluation and 66.2 on paper-specific evaluation, substantially outperforming the best baseline (Paper2Poster: 65.7) by 4.4\% and demonstrating consistent superiority across diverse research domains.
We proposed SciFig, a multi-agent AI system for automatic generation of scientific pipeline figures from textual method descriptions. By decomposing the problem into specialized agents, SciFig achieves a structured and interpretable workflow that mirrors human design reasoning. The hierarchical layout generation captures module-level organization and logical flow, while the iterative Chain-of-Thought feedback mechanism improves visual quality through multi-round corrections. In addition, our rubric-based evaluation framework provides the first systematic benchmark for evaluating scientific figure generation across diverse research domains. Together, these enable SciFig to produce publication-ready figures that are both technically accurate and aesthetically consistent, paving the way toward more intelligent, collaborative, and automatic tools for scientific communication.
% We introduce SciFig, the first end-to-end AI agent system for automatic scientific pipeline figure generation and evaluation. Through systematic multi-agent decomposition with hierarchical layout generation and iterative Chain-of-Thought feedback, SciFig transforms textual method descriptions into publication-ready figures that match or exceed the quality of manually created visualizations.
% Our work addresses three fundamental challenges in automatic figure generation. First, we demonstrate that hierarchical layout generation—with explicit module-level organization, visual containerization, and inter-module connections—is essential for capturing the multi-level abstractions inherent in scientific pipelines. Our ablation studies show that hierarchical layout provides the largest single quality improvement (+1.1\%), with particularly strong gains in Structural Coherence (+1.0) and Implementation Quality (+2.8). Second, we establish that iterative CoT feedback mechanisms enable iterative improvement across multiple quality dimensions, achieving professional-level aesthetics through systematic correction of alignment, spacing, arrow clarity, and label readability issues. The feedback mechanism contributes +2.2\% to overall quality when applied to hierarchical layouts. Third, we reveal that hierarchy and feedback are not independent improvements but collaborative components—feedback on flat layouts achieves inconsistent results and often degrades quality, while feedback on hierarchical layouts converges reliably to high-quality outputs.
% Our comprehensive evaluation framework, developed through analysis of 2,219 real scientific figures from top-tier conferences, establishes the first systematic benchmark for pipeline figure quality. The six-dimensional rubric—covering Technical Accuracy, Visual Clarity, Structural Coherence, Design Consistency, Interpretability, and Implementation Quality—provides standardized metrics enabling meaningful comparison between generation methods. SciFig achieves 70.1 overall quality on dataset-level evaluation and 66.2 on paper-specific evaluation, substantially outperforming the best baseline (Paper2Poster: 65.7) and demonstrating consistent superiority across diverse research domains including Artificial Intelligence, Graphics, and Material Science. The multi-agent architecture proves essential for this complex generative task. Removing any component results in measurable quality degradation, validating the necessity of the complete system.
\section{Author Contributions}
Siyuan Huang: Project Lead, everything \\
Yutong Gao: Idea discussion, Layout Agent, Hierarchical Layout Generation \\
Juyang Bai: Idea discussion, Component Agent \\
Yifan Zhou: Evaluation system, questionnaire \\
Zi Yin: Experiments, code repo, website, demo, deployment, parallel, distribution \\
Xinxin Liu: Experiments \\
Rama Chellappa: Supervisor, idea discussion, paper discussion \\
Chun Pong Lau: Supervisor, idea discussion \\
Sayan Nag: Supervisor, dataset, idea discussion, paper discussion \\
Cheng Peng: Supervisor, idea discussion, paper discussion, paper modification \\
Shraman Pramanick: Supervisor, Feedback Agent, dataset, idea discussion, paper discussion
\clearpage
\setcounter{page}{1}
\maketitlesupplementary
\section{Limitations of Existing Methods}
\label{sec:rationale}
%
While existing layout methods are good at generating layouts for documents, user interfaces, posters, and general diagrams, they are fundamentally not suitable to scientific pipeline figure generation. They cannot distinguish between functional stages (data preprocessing, feature extraction, loss computation) or generate appropriate module (hierarchy groupings and connections). Most importantly, these systems optimize for visual balance rather than logical clarity and scientific accuracy. A figure may be beautifully aligned yet completely misrepresent the method's information flow. SciFig fixes these fundamental gaps through hierarchical layout modeling and CoT feedback that explicitly captures functional relationships between pipeline components, enabling the first system capable of generating editable publication-quality research method figures.
While existing LLM-based evaluators are promising for general visual content, they lack the domain-specific understanding required for scientific pipeline figures. Current systems cannot evaluate whether a pipeline figure accurately represents method logic, correctly conveys information flow between functional modules, or appropriately groups related components into hierarchical stages. SciFig fixes this gap by developing a rubric-based evaluation framework that analyzes 2K+ real pipeline figures to extract domain-specific quality dimensions specifically tailored to research method visualization. Our evaluation agent generates customized evaluation criteria that capture both the geometric properties evaluated by existing systems and the semantic accuracy, enabling the first comprehensive automatic evaluation specifically designed for pipeline figure quality.
\section{Overview: Why Multi-Agent Architecture?}
Scientific pipeline figure generation shows unique challenges that require a multi-agent collaborative system rather than a single model. Unlike general image generation or simple diagram creation, this task demands:
\textbf{(1) Hierarchical Understanding:} Research descriptions contain multi-level abstractions from atomic operations to functional modules to complete pipelines. A single agent cannot effectively parse, maintain, and reason about these nested structures simultaneously while also handling visual design constraints.
\textbf{(2) Domain Knowledge:} The task requires different capabilities: semantic parsing of technical text, spatial layout reasoning, visual design principles, and multi-dimensional quality evaluation. Each capability benefits from specific optimization that cannot be effectively combined in a single agent.
\textbf{(3) Iterative Improvement Cycles:} Human designers naturally separate creation from feedback: generating initial designs, giving feedback, and iteratively improving. This separation of concerns is critical for quality but cannot be achieved with a single agent that simultaneously generates and evaluates its own outputs without explicit architectural separation.
\textbf{(4) Verifiable Quality Evaluation:} Automatic evaluation requires independence from generation to avoid confirmation bias. An evaluation agent must analyze figures objectively without access to generation process, mimicking human expert review.
We demonstrate empirically in Sec. 4.2 that single-agent methods fail to achieve acceptable quality. The fundamental limitation is not model capacity but architectural design: complex generative tasks with quality constraints require explicit role separation, reasoning, and iterative feedback loops.
\section{Discussion of Each Agent}
\textbf{Layout Agent:} Separating layout generation from semantic parsing enables focused spatial reasoning about hierarchical structures. Our ablation (Table~\ref{tab:ablation}) shows that hierarchical layout improves Structural Coherence by 1.0\% (69.2→70.2) and Visual Clarity by 2.4\% (65.7→68.1) compared to naive layout, demonstrating that explicit module-level organization is essential for translating functional relationships into effective visual structure.
\textbf{Feedback Agent:} Independent feedback generation avoids confirmation bias inherent in self-evaluation. Table~\ref{tab:ablation} shows that adding CoT feedback on top of hierarchical layout improves overall quality by 2.2\% (69.4→71.6), with gains distributed across all dimensions, particularly Interpretability (+3.5\%) and Technical Accuracy (+2.4\%). The isolated feedback analysis reveals that without hierarchical structure, feedback becomes unstable (rounds 1-5: 67.3→68.1→64.3→68.5→65.3), confirming that structural organization is necessary for effective iterative improvement.
\textbf{Component Agent:} Separating rendering from layout enables specific focus on visual consistency, technical precision, and publication-ready quality. This agent handles diverse element types (geometric shapes, neural network diagrams, domain-specific icons, text annotations) while maintaining uniform styling, a task requiring different expertise from spatial reasoning or semantic parsing.
\section{Details of Rubrics}
\textbf{$R_1$: Technical Accuracy and Correctness} measures the scientific fidelity of the figure, including mathematical consistency in notations and equations, algorithmic fidelity in representing operation sequences and data flow, and architectural precision in describing model components and connections.
\textbf{$R_2$: Visual Clarity and Readability} evaluates the figure's visual effectiveness through clear differentiation between elements, unambiguous flow direction via arrows or sequential arrangement, appropriate visual hierarchy emphasizing primary versus secondary elements, readable fonts and labels, balanced information density avoiding both overcrowding and oversimplification, and effective use of visual encoding (shapes, colors, sizes) to present information.
\textbf{$R_3$: Structural Coherence} evaluates the logical organization of the pipeline, including logical order showing coherent operation sequences, clear module boundaries between functional components or processing stages, explicit connection clarity between components, and grouping of related components.
\textbf{$R_4$: Design Consistency} examines the uniformity and professionalism of visual presentation through visual language consistency in shapes, colors, and symbols, notation and terminology consistency throughout the figure, and professional aesthetic quality with balanced composition and appropriate white space.
\textbf{$R_5$: Interpretability and Accessibility} measures how easily the figure can be understood and accessed by diverse readers, including intuitive representation using familiar visual metaphors and conventions, self-containment allowing understanding with minimal text reference, color accessibility for color-blind readers, and legend completeness with necessary elements.
\textbf{$R_6$: Technical Implementation Quality} evaluates the production quality of the figure, including vector graphics quality with clean lines and proper scaling, typography quality with professional font choices, layout efficiency using space effectively without unnecessary elements, and high resolution rendering without artifacts or pixelation.
The original generated rubrics by our agent is shown in Fig. 4.
\section{Algorithm of the Complete System}
The complete system is formalized in Algorithm~\ref{alg:agentfig}.
\begin{algorithm}[t]
\caption{SciFig: Multi-Agent Figure Generation}
\label{alg:agentfig}
\begin{algorithmic}[1]
\Require Research description $\mathcal{T}$, maximum feedback rounds $T$
\Ensure Publication-ready figure $\mathcal{F}$
\State \textbf{Stage 1: Initial Layout Generation}
\State $\mathcal{H} \gets \mathcal{A}_D(\mathcal{T})$ \Comment{Extract hierarchical structure}
\State $\mathcal{L}_0 \gets \mathcal{A}_L(\mathcal{H})$ \Comment{Generate baseline layout}
\State
\State \textbf{Stage 2: Iterative CoT Feedback Improvement}
\For{$t = 1, \ldots, T$}
\State $\mathcal{F}_{t-1} \gets \text{Render}(\mathcal{L}_{t-1})$ \Comment{Render layout to image}
\State $\text{FB}_t \gets \mathcal{A}_F(\mathcal{F}_{t-1}, \mathcal{T})$ \Comment{Analyze and generate feedback}
\If{$|\text{issues}| = 0$}
\State \textbf{break} \Comment{Convergence achieved}
\EndIf
\State $\mathcal{L}_t \gets \mathcal{A}_L(\mathcal{H}, \mathcal{L}_{t-1}, \text{FB}_t)$ \Comment{Improve with CoT reasoning}
\EndFor
\State $\mathcal{L}^* \gets \mathcal{L}_t$ \Comment{Final optimized layout}
\State
\State \textbf{Stage 3: Component Generation and Rendering}
\State $\mathcal{V} \gets \mathcal{A}_C(\mathcal{L}^*)$ \Comment{Generate visual elements}
\State $\mathcal{F} \gets \text{Composite}(\mathcal{L}^*, \mathcal{V})$ \Comment{Make final figure} \\
\Return $\mathcal{F}$
\end{algorithmic}
\end{algorithm}
\begin{figure*}[t]
\label{fig:rubric}
\fcolorbox{black}{gray!10}{%
\begin{minipage}{\dimexpr\textwidth-2\fboxsep-2\fboxrule\relax}
\#\# Technical Accuracy and Correctness\\
- Mathematical consistency in notations, equations, and transformations\\
- Algorithmic fidelity in representing sequences of operations and data flow\\
- Architectural precision in depicting model components and connections\\
- Terminological precision with domain-appropriate technical terms and labels\\
\#\# Visual Clarity and Readability\\
- Component distinction with clear visual differentiation between elements\\
- Unambiguous flow direction through arrows or sequential arrangement\\
- Appropriate visual hierarchy emphasizing primary vs. secondary elements\\
- Text legibility with readable fonts and clear labels\\
- Balanced information density avoiding overcrowding and oversimplification\\
- Effective use of visual encoding (shapes, colors, sizes) to convey information\\
\#\# Structural Coherence\\
- Logical progression showing a coherent sequence of operations\\
- Clear module boundaries between functional components or processing stages\\
- Explicit connection clarity between components\\
- Proper representation of feedback loops and iterative processes\\
- Modular organization with logical grouping of related components\\
\#\# Design Consistency\\
- Visual language consistency in shapes, colors, and symbols\\
- Notation and terminology consistency throughout the figure\\
- Stylistic coherence with unified visual appearance\\
- Professional aesthetic quality with balanced composition and white space\\
\#\# Interpretability and Accessibility\\
- Intuitive representation using familiar visual metaphors and conventions\\
- Self-containment allowing understanding with minimal reference to text\\
- Color accessibility for color-blind readers\\
- Legend completeness with necessary explanatory elements\\
- Consistent symbolism using standard or clearly defined visual conventions\\
\#\# Technical Implementation Quality\\
- Vector graphics quality with clean lines and proper scaling\\
- Typography quality with professional font choices\\
- Layout efficiency using space effectively without unnecessary elements\\
- High resolution rendering without artifacts or pixelation
\end{minipage}%
}
\caption{Detailed Evaluation Rubrics. Our Evaluation Agent automatically derives six quality dimensions by analyzing 2,219 real scientific figures. Each rubric contains specific criteria that capture both visual properties (clarity, consistency, implementation) and semantic accuracy (technical correctness, structural coherence, interpretability). These rubrics form the foundation for our comprehensive figure evaluation framework, enabling systematic quality evaluation across diverse research domains.}
\end{figure*}
\begin{figure*}[t]
\centering
\includegraphics[width=\linewidth]{evaluation_set_stats_final.pdf}
\caption{Evaluation Set Statistics. Distribution of 435 papers by conference (left) and research domain (right, top 20 shown). The dataset spans 15 top-tier AI conferences and 37 domains, with balanced representation across major research areas (cs.CV, cs.CL, cs.AI each with 41 papers).}
\label{fig:evalset}
\end{figure*}
\section{Evaluation Set Statistics}
Our evaluation set consists of 435 papers sampled from publications in 2023, spanning 15 top-tier AI conferences and 37 different research domains. This carefully curated dataset ensures comprehensive coverage across the AI research landscape while maintaining balanced representation of different methods and communities.
\textbf{Conference Distribution.} The dataset samples from top-tier conferences across multiple research communities (Fig.~\ref{fig:evalset}, left). ICASSP contributes the largest share with 99 papers (22.8\%), reflecting the substantial role of signal processing and audio research in scientific visualization. General AI conferences show strong representation: AAAI (43 papers, 9.9\%), IJCAI (42 papers, 9.7\%), and NeurIPS (40 papers, 9.2\%). Computer vision venues contribute 56 papers total, with ICCV (37 papers, 8.5\%) and CVPR (19 papers, 4.4\%). Natural language processing is represented through ACL (34 papers, 7.8\%) and EMNLP (2 papers, 0.5\%). The dataset also includes papers from specialized domains: KDD for data mining (33 papers, 7.6\%), WWW for web research (29 papers, 6.7\%), SIGIR for information retrieval (24 papers, 5.5\%), ICLR and ICML for machine learning (18 and 9 papers respectively), and SIGGRAPH and CIKM (3 papers each).
\textbf{Domain Distribution.} The 37 research domains exhibit balanced coverage across core areas (Fig.~\ref{fig:evalset}, right). The three largest domains—Computer Vision (cs.CV), Natural Language Processing (cs.CL), and Artificial Intelligence (cs.AI)—each contribute exactly 41 papers (9.4\%), demonstrating our sampling strategy for equal representation of major fields. Machine Learning (cs.LG) and Information Retrieval (cs.IR) each account for 40 papers (9.2\%). Audio and speech processing domains are well-represented through Audio Signal Processing (eess.AS, 39 papers, 9.0\%) and Speech (cs.SD, 39 papers, 9.0\%). Additional substantial coverage includes Image and Video Processing (eess.IV, 23 papers, 5.3\%), Systems and Control (cs.SI, 16 papers, 3.7\%), Cryptography and Security (cs.CR, 15 papers, 3.4\%), Robotics (cs.RO, 12 papers, 2.8\%), and Signal Processing (eess.SP, 11 papers, 2.5\%). The dataset further encompasses emerging and interdisciplinary areas including statistical machine learning (stat.ML, 10 papers), game theory (cs.GT), neural computing (cs.NE), graphics (cs.GR), multimedia (cs.MM), human-computer interaction (cs.HC), computational biology (q-bio.BM, q-bio.NC, q-bio.QM), quantum physics (quant-ph), and others, ensuring comprehensive representation of diverse research methods and visualization requirements across the scientific landscape.
\begin{table*}[t!]\small
\centering
\setlength{\tabcolsep}{4pt}
\renewcommand{\arraystretch}{0.7}
\resizebox{\linewidth}{!}{\begin{tabularx}{\textwidth}{lXXXXXXX}
\toprule
\bf Model & $R_1$ & $R_2$ & $R_3$ & $R_4$ & $R_5$ & $R_6$ & Overall \\ \midrule
GPT-5-Image \cite{openai2025gpt5} & 44.8 & 42.5 & 52.2 & 52.0 & 43.0 & 39.9 & 45.7 \\
Qwen-Image \cite{wu2025qwen} & 7.9 & 4.2 & 7.1 & 7.4 & 5.2 & 5.4 & 6.2 \\
Original & 70.0 & 71.4 & 71.6 & 72.1 & 68.7 & 67.4 & 70.2 \\
\midrule
\rowcolor{Light}
\textbf{SciFig (Ours)} & 58.8 & 54.0 & 61.2 & 60.5 & 55.2 & 52.8 & 57.1 \\
\bottomrule
\end{tabularx}}
% \vspace{-2mm}
\caption{\textbf{Human Evaluation Results: Rubric-Based Quality Evaluation.} We compare SciFig against baseline methods and original human-designed figures through human evaluation. Each dimension is rated on a 7-point scale by 32 participants and converted to percentages (0-100). }
\label{tab:human_ai_comparison}
\vspace{-1em}
\end{table*}
\begin{table*}[t!]\small
\centering
\setlength{\tabcolsep}{4pt}
\renewcommand{\arraystretch}{0.7}
\resizebox{\linewidth}{!}{\begin{tabularx}{\textwidth}{lXXXXXXXXXXc}
\toprule
& \multicolumn{11}{c}{\bf Paper} \\
\cmidrule{2-12}
\multirow{-2.5}{*}{\bf Model} & $1$ & $2$ & $3$ & $4$ & $5$ & $6$ & $7$ & $8$ & $9$ & $10$ & Overall \\ \midrule
GPT-5-Image \cite{openai2025gpt5} & 1.156 & 1.313 & 1.875 & 1.313 & 1.688 & 1.125 & 1.469 & 1.188 & 1.406 & 1.438 & 1.397 \\
Qwen-Image \cite{wu2025qwen} & 0.125 & 0.094 & 0 & 0.031 & 0 & 0.094 & 0.063 & 0.156 & 0.031 & 0.031 & 0.063 \\
Original & 2.531 & 2.219 & 2.563 & 2.625 & 2.188 & 2.906 & 2.656 & 2.531 & 2.688 & 2.563 & 2.547 \\ \midrule
\rowcolor{Light}
\textbf{SciFig (Ours)} & 2.188 & 2.375 & 1.563 & 2.031 & 2.125 & 1.875 & 1.813 & 2.125 & 1.875 & 1.969 & 1.994 \\
\bottomrule
\end{tabularx}}
% \vspace{-2mm}
\caption{\textbf{Blind Preference Ranking Results: Condorcet Scores Across Test Papers.} We show a blind preference study where 32 participants rank figures without knowing their source. For each of the 10 test papers, participants view all 4 figures in randomized order and rank them from best to worst based on overall quality. Condorcet scores represent the average number of victories against other figures, where 3 indicates winning all comparisons (consistently ranked first) and 0 indicates losing all comparisons (consistently ranked last).}
\label{tab:condorcet}
\vspace{-1em}
\end{table*}
\section{Human Evaluation Study}
To validate our automatic evaluation framework and evaluate the practical effectiveness of SciFig, we did a comprehensive human evaluation study with 32 participants. The study consisted of three experiments: (1) rubric-based scoring to compare human and agent evaluation consistency, (2) blind preference ranking using Condorcet method, and (3) time efficiency analysis comparing SciFig against manual figure creation.
\subsection{Participant Demographics}
We recruited 32 participants (21 male, 8 female, 3 preferred not to disclose) with substantial research experience in AI-related fields. All participants reported having research experience and publication records in areas including AI, Computer Vision (CV), Natural Language Processing (NLP), Machine Learning, Deep Learning, and Reinforcement Learning.
The participant pool showed strong academic credentials: 14 PhD students, 14 master's students, 2 postdoctoral researchers, and 2 industry research scientists. Research experience ranged from 1 to 9 years, with the majority having 2-3 years (n=17) or 4-5 years (n=9) of experience, ensuring they are familiar with scientific figure creation and evaluation. Regarding figure creation experience, participants reported spending considerable time on pipeline figures: 12 participants typically require several days, 9 require one week, 7 require two weeks or longer, and only 2 can complete figures within several hours. This distribution reflects the substantial time cost required for high-quality scientific visualization, motivating the need for automatic solutions like SciFig.
\subsection{Rubric-Based Evaluation}
In the first experiment, participants evaluated 40 pipeline figures (10 papers × 4 figures per paper) using our six quality rubrics: Technical Accuracy ($R_1$), Visual Clarity ($R_2$), Structural Coherence ($R_3$), Design Consistency ($R_4$), Interpretability ($R_5$), and Implementation Quality ($R_6$). Each figure was rated on a 7-point scale (1=lowest, 7=highest), which was then converted to percentages (0-100) for comparison with our automatic evaluation framework.
For each paper, the four figures are: (1) Original human-designed figure from the published paper, (2) SciFig-generated figure, and (3-4) Baseline methods (GPT-5-Image, Qwen-Image). Figure order was randomized to avoid position bias. This setup served two purposes: comparing models against human-designed figures to evaluate quality gaps, and validating consistency between human ratings and our automatic agent-based evaluation.
\textbf{Results Summary.} Table~\ref{tab:human_ai_comparison} presents the human evaluation results. Original human-designed figures achieved the highest overall score of 70.2\%, demonstrating the quality standard set by expert researchers. Among AI-generated methods, SciFig achieved 57.1\% overall quality, substantially outperforming GPT-5-Image (45.7\%) and Qwen-Image (6.2\%). SciFig's performance represents an 11.4\% improvement over the best baseline (GPT-5-Image) and a 50.9\% improvement over the worst baseline (Qwen-Image).
Breaking down by individual rubrics, SciFig achieved its highest scores in Structural Coherence (R3: 61.2\%) and Design Consistency (R4: 60.5\%), directly validating our hierarchical layout generation and iterative feedback mechanisms described in Section 3.2. The system also performed well in Technical Accuracy (R1: 58.8\%) and Interpretability (R5: 55.2\%), demonstrating balanced performance across semantic and visual quality dimensions. The relatively lower scores in Implementation Quality (R6: 52.8\%) and Visual Clarity (R2: 54.0\%) suggest opportunities for future improvement in rendering improvement and visual polish.
\textbf{Comparison with Original Figures.} The 13.1\% gap between SciFig (57.1\%) and original human figures (70.2\%) represents the current quality upper bound for automatic methods. This gap reflects several factors: (1) domain-specific visual conventions that expert researchers incorporate through years of experience, (2) iterative improvement based on co-author feedback during the paper writing process, and (3) paper-specific design decisions that require deep understanding of the technical contribution. Nevertheless, SciFig's ability to achieve 81.3\% of human-level quality (57.1/70.2) while requiring only minutes instead of days represents a significant practical progress.
\textbf{Human-Agent Evaluation Consistency.} Comparing these human ratings with our agent-based evaluation scores (Table~\ref{tab:main_results}, Overall: 70.1 for SciFig) reveals important insights. While the absolute scores differ due to evaluation granularity and rater strictness, the relative rankings remain highly consistent: both evaluations identify SciFig as substantially superior to baseline methods, with particularly strong performance in structural organization. This consistency validates that our rubric-based evaluation framework captures the quality dimensions that human researchers consider important when evaluating scientific figures.
\begin{figure*}
\centering
\includegraphics[width=\linewidth]{time_efficiency_single_plot.pdf}
\caption{\textbf{Time Efficiency Comparison: SciFig Multi-Agent Pipeline vs. Human Creation.} We measure the execution time for each agent in our multi-agent system and compare against human creation time collected from 32 participants. SciFig achieves a \textbf{1,407× speedup} compared to human mean creation time, representing a \textbf{99.93\% time reduction} while maintaining 57.1\% quality (81.3\% of human-level performance as validated in Tables~\ref{tab:human_ai_comparison}-\ref{tab:condorcet}). This huge efficiency gain transforms figure creation from a multi-day bottleneck into a minutes-long interactive process, enabling researchers to rapidly generate multiple design alternatives and iterate on figure designs in ways that would be impractical with manual creation. The log-scale visualization emphasizes the three orders of magnitude improvement in efficiency.}
\label{fig:placeholder}
\end{figure*}
\subsection{Blind Preference Ranking}
We did a blind preference study where participants ranked figures without knowing their source. For each of the 10 papers, participants viewed all 4 figures in randomized order and ranked them from best to worst based on overall quality.
We computed Condorcet scores for each figure, representing the average number of victories against other figures. A score of 3 indicates a figure won all comparisons (ranked first consistently), while 0 indicates it lost all comparisons (ranked last consistently). This method provides robust preference aggregation that is resistant to ranking inconsistencies and accurately reflects relative quality across the entire participant pool.
\textbf{Results.} Table\ref{tab:condorcet} shows the Condorcet scores for each method across all 10 test papers. The results show clear and consistent quality preferences: Original human-designed figures achieved the highest average score of 2.547 (84.9\% win rate), dominating the rankings by winning 9 out of 10 papers. This demonstrates that expert researchers can still detect quality differences even in blind evaluation, with human-designed figures maintaining a clear advantage.
Among AI-generated methods, SciFig achieved an average score of 1.994 (66.5\% win rate), securing second place overall and winning 1 of 10 papers (Paper 2). GPT-5-Image scored 1.397 (46.6\% win rate), while Qwen-Image scored lowest at 0.063 (2.1\% win rate), losing nearly all comparisons. The gap between SciFig and GPT-5-Image (0.597) is larger than the gap between Original and SciFig (0.553), indicating that SciFig represents a substantial step forward among automatic methods even as it approaches human-level performance.
\textbf{Consistency with Rubric Evaluation.} The blind preference rankings show perfect rank correlation with the detailed rubric-based evaluation from Experiment 1: both place methods in the order Original $>$ SciFig $>$ GPT-5-Image $>$ Qwen-Image. This strengthens confidence in both evaluation methods and confirms that SciFig generates figures that human researchers generally prefer when evaluated without knowledge of generation method. The consistency between blind preferences and detailed rubric scores suggests that our six quality rubrics effectively capture the factors that researchers consider when judging overall figure quality.
\subsection{Time Efficiency Analysis}
A critical practical consideration for automatic figure generation is time efficiency compared to manual creation. We measured the time required for SciFig to generate figures and compared it against human creation time.
\textbf{SciFig Generation Time.} We measured the execution time for each agent in our multi-agent pipeline: Description Agent (20.1s), Layout Agent (72.9s), Feedback Agent per round (99.8s), Component Agent (187.2s), and final Rendering (4.0s). With our default configuration of 3 feedback iterations, total generation time is approximately 583.6 seconds (9.7 minutes) per figure. The component generation phase dominates the total time (32.1\%), as it involves rendering diverse visual elements with consistent styling and technical precision.
\textbf{Human Creation Time.} We collected detailed time estimates from our 32 participants regarding their typical figure creation workflow. The data reveals substantial time cost: participants reported times ranging from few hours (minimum) to more than a month (maximum), with a mean of 227.5 hours (approximately 9.5 days) and a median of 168 hours (7 days). The distribution shows that 37.5\% of researchers typically require around several days (72 hours), 28.1\% require approximately one week (168 hours), while 12.5\% need a month or longer ($>=$720 hours). The standard deviation of 222.2 hours reflects high variability in creation time, likely due to differences in figure complexity, researcher experience, tool proficiency, and iterative improvement cycles.
This distribution indicates that figure creation is a time-intensive process for the majority of researchers. The median of 7 days (168 hours) provides a robust central estimate, while the mean of 9.5 days (227.5 hours) accounts for the substantial tail of researchers requiring extended time.
\textbf{Efficiency Gains.} Using the mean human creation time of 227.5 hours as our baseline, SciFig achieves a \textbf{1,407× speedup} (227.5 hours × 60 min/hour ÷ 9.7 min = 1,407×), reducing figure generation from days to minutes while maintaining 57.1\% quality (81\% of human level) as validated in Tables~\ref{tab:human_ai_comparison}-\ref{tab:condorcet}. Even using the more conservative median estimate of 168 hours (7 days), SciFig still provides \textbf{1,039× speedup}, representing over three orders of magnitude improvement. This 99.9\% time reduction (9.7 min / 13,650 min = 0.071\%) enables researchers to generate multiple design alternatives rapidly, experiment with different visual organizations, and iterate on figure designs in ways that would be impractical with manual creation.
The time savings are particularly impactful for researchers with limited design expertise or access to professional tools. While experienced researchers with mastered workflows might approach the lower end of our distribution (3-7 days), the substantial fraction requiring 2-4 weeks (27.1\% combined) would benefit most from our system. Furthermore, SciFig's rapid iteration capability transforms the figure creation process from a sequential bottleneck in paper writing to an interactive exploration phase where researchers can evaluate multiple design options within an afternoon.
\section{More Qualitative Results}
We proudly show more qualitative results in Fig.~\ref{fig:sm-vis1}-\ref{fig:sm-vis3}. Fig.~\ref{fig:sm-vis1} and \ref{fig:sm-vis2} are extended comparison to other methods of Fig.~\ref{fig:qualitative}. Fig.~\ref{fig:sm-vis3} is wide-range showcases of our system. It's worth noting that all figures generated by our system in Fig.~\ref{fig:qualitative}, \ref{fig:sm-vis1}, \ref{fig:sm-vis2}, and \ref{fig:sm-vis3} are \textbf{fully automatic, no manual modification at all, while keep full editability.}
\begin{figure*}
\centering
\includegraphics[width=\linewidth]{cvpr26-sm-vis1.pdf}
\caption{\textbf{Additional Qualitative Comparisons Across Methods and Research Domains.} We show additional qualitative results comparing SciFig against four baseline methods (GPT-5-Image, Qwen-Image, Stable Diffusion XL, Paper2Poster) across diverse research papers, demonstrating consistent performance patterns. Each column shows results from the same method text input. \textbf{SciFig (Ours)} (row 5) \textbf{is the only method that generates fully editable, vector-based figures,} allowing researchers to improve layouts, adjust component styling, modify text labels, and reorganize elements as needed. Beyond editability, SciFig consistently generates publication-ready figures across all examples with clear hierarchical organization, explicit module-level connections, consistent professional styling, and domain-appropriate scientific accuracy. Our method maintains high quality across diverse complexity levels and research domains, demonstrating the robustness and generalization of our multi-agent architecture with hierarchical layout generation and iterative CoT feedback mechanisms.}
\label{fig:sm-vis1}
\end{figure*}
\begin{figure*}
\centering
\includegraphics[width=\linewidth]{cvpr26-sm-vis2.pdf}
\caption{\textbf{(Cont'd) Additional Qualitative Comparisons Across Methods and Research Domains.} We show additional qualitative results comparing SciFig against four baseline methods (GPT-5-Image, Qwen-Image, Stable Diffusion XL, Paper2Poster) across diverse research papers, demonstrating consistent performance patterns. Each column shows results from the same method text input. \textbf{SciFig (Ours)} (row 5) \textbf{is the only method that generates fully editable, vector-based figures,} allowing researchers to improve layouts, adjust component styling, modify text labels, and reorganize elements as needed. Beyond editability, SciFig consistently generates publication-ready figures across all examples with clear hierarchical organization, explicit module-level connections, consistent professional styling, and domain-appropriate scientific accuracy. Our method maintains high quality across diverse complexity levels and research domains, demonstrating the robustness and generalization of our multi-agent architecture with hierarchical layout generation and iterative CoT feedback mechanisms.}
\label{fig:sm-vis2}
\end{figure*}
\begin{figure*}
\centering
\includegraphics[width=0.9\linewidth]{cvpr26-sm-vis3.pdf}
\caption{\textbf{Comprehensive SciFig Results Showcase Across Diverse Research Methods and Domains.} We show an extensive collection of publication-ready figures generated by SciFig across a wide range of research papers, demonstrating the system's consistent high quality. Each example represents a complete end-to-end generation from natural language method descriptions to fully editable, vector-based scientific figures. }
\label{fig:sm-vis3}
\end{figure*}