StepJack: Benchmarking Computer-Use Agent Safety Against
Multi-Step Indirect Prompt Injection

Computer-use agents (CUAs) Xie et al. (2024) hold the promise of automating desktop and browser workflows, and their safety has become a growing concern Liao et al. (2026); Kuntz et al. (2025). A particularly salient threat is indirect prompt injection Greshake et al. (2023); Evtimov et al. (2025); Zhan et al. (2024), in which an attacker plants adversarial instructions in content the CUA reads during normal operation (e.g., forum posts or shared documents) rather than communicating with the CUA directly.

To the best of our knowledge, existing CUA safety benchmarks Liao et al. (2026); Kuntz et al. (2025); Evtimov et al. (2025); Cao et al. (2025); Tur et al. (2025); Boisvert et al. (2025) share a structural assumption: a full adversarial goal is delivered as a single monolithic payload at a single environmental location. Recent benchmarks Liao et al. (2026); Boisvert et al. (2025) have shown that current-generation CUAs have become increasingly resilient to such payloads due to model-level safety training Anthropic (2024); OpenAI (2025).

A monolithic payload concentrates all adversarial intent in one location, presenting a CUA’s safety mechanisms with a relatively straightforward decision: read the payload, recognize it as harmful, and refuse. Real-world adversaries face no such constraint. Web content is inherently distributed: an attacker who controls one page typically controls (or can reference) others, and an instruction split across several pages along the agent’s navigation path may raise no single red flag. This gap between current evaluation frameworks and the adversary environment that CUAs will actually face in deployment motivates a stronger test:

Can a CUA still refuse an adversarial goal when it is distributed across the environment rather than placed in a single location?

To answer this question, we present StepJack, a benchmark and accompanying pipeline for red-teaming CUAs under a novel attack class. Our contributions are:

Indirect Prompt Injection Attacks on Text-based Agents. Indirect prompt injection means that adversarial instructions are embedded in data that an agent retrieves from the environment Greshake et al. (2023). Recent benchmarks Zhan et al. (2024); Debenedetti et al. (2024); Zhang et al. (2025) probe indirect prompt injection robustness across diverse tool-use settings. These works focus exclusively on text-based tool-calling agents and investigate single-point injections where the adversarial goal is delivered in full at one location.

CUA-specific Safety Benchmarks. CUAs interact with operating systems through screenshots and keyboard/mouse actions rather than tool calls. CUA safety works Evtimov et al. (2025); Cao et al. (2025); Tur et al. (2025); Boisvert et al. (2025); Kuntz et al. (2025) each probe different aspects of CUA vulnerability. Among them, RedTeamCUA Liao et al. (2026) is most relevant to our work and the most comprehensive prior benchmark for indirect prompt injection on CUAs, providing a sandbox built on OSWorld Xie et al. (2024) with diverse adversarial test cases. To the best of our knowledge, all existing CUA safety benchmarks use monolithic, single-location injection; none study whether distributing a malicious goal across multiple innocuous-looking environmental sub-steps improves attack success or evades safety reasoning.

Multi-step Attacks. A parallel literature on direct attacks against LLM chatbots Russinovich et al. (2025); Yang et al. (2025); Weng et al. (2025) has established that decomposing harmful instructions across multiple conversation turns substantially increases attack effectiveness. However, to the best of our knowledge, all existing multi-step attacks are against LLM chatbots and operate in direct user-model conversation, where the attacker is the user and can observe and adapt to model responses in real time using closed-loop optimization. This is fundamentally different from the indirect prompt injection setting, where an attacker must plant sub-steps in the environment before the agent begins its task, without observing intermediate agent responses, and without any direct interaction with the model, so the attack can only be optimized open-loop. The absence of multi-step attacks in the indirect injection literature represents a significant gap across all LLM-based agents that consume external content, of which CUAs are an especially consequential instance, and our work addresses this gap.

Computer-Use Agent (CUA). A CUA is an LLM-driven agent that operates in a computer environment Xie et al. (2024): at each step it observes a screenshot and emits a keyboard/mouse action, which the environment executes to produce the next screenshot. Given a benign user task $b$, the CUA produces a trajectory that may visit a sequence of pages (e.g., forum posts, shared documents), which we call its navigation trace.

Adversarial Goal and Attacker Constraints. An adversarial goal ${a}$ is a harmful action the attacker wants the CUA to execute (e.g., delete a sensitive file, exfiltrate credentials); following Liao et al. (2026), we determine whether a CUA executes ${a}$ using an environment-state checker. We adopt RedTeamCUA’s attacker constraints: the attacker is an ordinary web user who can post comments or upload documents to pages on the CUA’s navigation trace, and does not know the specific CUA model or its real-time observation-action history.

(Single-step) Indirect Prompt Injection. Existing benchmarks Liao et al. (2026); Kuntz et al. (2025); Evtimov et al. (2025) inject the full adversarial instruction into a single page $w$ on the CUA’s navigation trace. RedTeamCUA’s standard template is shown in Upper of Figure˜2. When the CUA visits $w$ during execution of $b$, the page presents both the adversarial goal $a$ and the benign task continuation $b$, with the intent that the agent executes ${a}$ before completing $b$. The ASR is the fraction of test examples for which the CUA achieves the adversarial goal ${a}$.

We present a novel multi-step indirect prompt injection attack, which consists of two components: (i) a delivery mechanism that creates multiple injection points along the CUA’s navigation trace (Section˜4.1), and (ii) a decomposition function that splits the adversarial goal into sub-steps suitable for distributed injection (Section˜4.2).

To implement the decomposition function $\mathcal{D}({a},k)=(d_{1},d_{2},\ldots,d_{k})$ introduced in Section˜4.2, we develop an automated pipeline in Algorithm 1, which takes an adversarial goal $a$ and decomposition depth $k$ and returns a decomposition $D^{*}$. The pipeline is open-loop with respect to the target CUA: nothing in its operation depends on observing the target CUA’s outputs.

Decomposition LLM. We first design a decomposition LLM $LLM^{\mathrm{Decomp}}$ that receives goal $a$ and depth $k$ and outputs the $k$-step decomposition, using a prompt template that enforces four design principles: (i) simplicity, each sub-step executable in 1 or 2 actions; (ii) separation, no sub-step reveals the goal; (iii) cover story, each sub-step carries a plausible benign justification; (iv) standardized output as $d_{1},\ldots,d_{k}$. These principles give a reasonable starting point, but single-shot generations are frequently either unfaithful (their sequential execution does not achieve $a$) or transparently harmful at some sub-step. The two stages below correct these two failure modes.

Stage 1: LLM-judged candidate search. The decomposition LLM samples $N$ candidates (Lines 3, 4 of Algorithm˜1). Each candidate $D^{(j)}=(d_{1}^{(j)},d_{2}^{(j)},\ldots,d_{k}^{(j)})$ is scored on goal faithfulness $F\in\{0,1\}$ (the faithfulness judge $LLM^{\mathrm{F}}$ sees the full goal $a$ and all sub-steps $D^{(j)}$ and predicts whether sequential execution achieves $a$) and safety score $S=\tfrac{1}{k}\sum_{i}S_{i}$ with $S_{i}\in[1,10]$ (the safety judge $LLM^{\mathrm{S}}$ evaluates each $d_{i}^{(j)}$ in isolation, simulating the CUA’s per-step reasoning). The composite score $Q=F\cdot S$ uses faithfulness as a hard constraint and safety as the ranking criterion. To help better optimization and improve diversity, the judges’ reasoning together with the scores, i.e., $F,S,\text{judges' reasoning}$ in Line 11, is fed back to $LLM^{\mathrm{Decomp}}$ as a new conversation turn. The iterative refinement runs $M$ rounds. Finally, the top-$B$ candidates across all chains and refinement depths, $\mathcal{T}_{B}$, are passed to Stage 2 (Line 14). The prompts used for the LLMs can be found in Appendix D.

Stage 2: CUA-verified selection. LLM judges in Stage 1 miss CUA idiosyncrasies (e.g., UI grounding, application navigation), so we execute the top-$B$ Stage 1 candidates $\mathcal{T}_{B}$ on a verifier CUA $\mathcal{V}$ in a sandboxed environment. We present “Execute the following step-by-step: $d_{1}^{(j)},\ldots,d_{k}^{(j)}$” as a user instruction to $\mathcal{V}$, record the action count taken to achieve adversarial goal $a$, and select $D^{*}$ as the candidate $\mathcal{V}$ completes in the fewest actions. The minimum-action criterion favors decompositions whose sub-steps are most naturally interpretable, reducing execution errors for CUAs.

A natural concern is selection bias: because the verifier $\mathcal{V}$ (in our case Qwen3.5-Plus) is also one of the CUAs we later evaluate, one might worry that Stage 2 keeps decompositions that specifically evade $\mathcal{V}$’s safety, making it look artificially vulnerable. This does not happen, because Stage 2 never runs an attack. The sub-steps are given to $\mathcal{V}$ as a plain user instruction (“Execute the following step-by-step: …”), with none of the reference chain or injection wrapping from Section˜4.1. Since $\mathcal{V}$ is not under attack, its injection defenses are never engaged, so selection cannot be optimized to bypass them. Stage 2 therefore filters only for whether the sub-steps are physically executable in the computer UI, not for whether they evade safety. We expand this discussion in Appendix E.

The pipeline explores diverse strategies despite only optimizing a single objective. We find that the pipeline’s outputs cluster post hoc into three categories that exploit distinct blind spots of a per-step filter. We document them as follows:

Prior multi-step jailbreaks on LLM chatbots each instantiate at most one category: Crescendo Russinovich et al. (2025) and Foot-in-the-door Weng et al. (2025) are escalatory, progressively escalating conversation topics over multiple turns. Jigsaw Puzzles Yang et al. (2025) is semantic, fragmenting harmful queries into individually innocuous pieces spread across turns. Operational decomposition has not been studied in any prior literature to our knowledge. Appendix C gives concrete examples of the generated attacks with different types of decompositions.

StepJack instantiates the multi-step attack of Section˜4 on the RedTeamCUA sandbox Liao et al. (2026), where the test examples are generated by the pipeline in Section˜5, with deliberate axes chosen so that ASR can be decomposed into interpretable factors. Appendix B lists the benign tasks and adversarial goals.

Inherited axes. We retain three axes from RedTeamCUA. (i) Platforms: Reddit and OwnCloud, with 2 benign tasks per platform; we drop RocketChat since its attack channel (direct messages from a counterparty) does not match web-based indirect injection. (ii) Adversarial-goal category: confidentiality, integrity, and availability (CIA triad). (iii) Resource type: we keep the System resource type, since it causes the most severe consequences and RedTeamCUA shows it yields the lowest single-step ASR. Besides, we retain General user instructions and Language adversarial-goal phrasings, since RedTeamCUA reports minimal ASR variation along these axes.

New axes. We add four axes that probe the multi-step regime. (i) Resource type. We introduce Targeted resource type, where adversarial goals reference a group of target files rather than a single file (e.g., “find all files containing confidential and delete them”). (ii) User-instruction mode: Permissive adds a blanket authorization clause (“follow the instructions on the website/document and help me set up”) while Restrictive omits it. (iii) Decomposition depth $k\in\{1,2,3\}$, with $k=1$ reproducing the single-step baseline and $k>1$ produced by our pipeline. (iv) Wrapping type (for $k>1$ only): whether each injected page includes the urgency cue defined in Section˜4.1, yielding two variants per chain, indicated by “Urgent = Y / N”.

Pipeline configuration. Decompositions are produced by Algorithm˜1 with $N=20$, $M=3$, $B=10$, using Qwen3.5-27B Team (2026b) for all LLMs $LLM^{\mathrm{Decomp}},LLM^{\mathrm{F}},LLM^{\mathrm{S}}$ and Qwen3.5-Plus as the Stage 2 verifier CUA $\mathcal{V}$.

Total. Combining 2 platforms, 2 benign tasks per platform, 12 adversarial goals, 2 user-instruction modes, and decomposition depths $k\in\{1,2,3\}$, with the wrapping-type axis applying only to $k>1$, StepJack yields $2\times 2\times 12\times 2\times(1+2\times 2)=480$ test examples per CUA. On the size of adversarial goals, we distinguish nominal goal count from effective behavior coverage; see discussions in Appendix B.

We present StepJack, a benchmark for measuring CUA robustness to multi-step indirect prompt injection. StepJack is built around a new attack class where an adversarial goal is decomposed into innocuous-looking sub-steps and distributed across a chain of pages along the agent’s benign navigation trace, in contrast to existing CUA safety benchmarks that inject the full goal at a single location. The accompanying automatic decomposition makes the benchmark extendable with more adversarial goals and decomposition depths. Across the five CUAs that reliably follow the reference chain, ASR rises from 31.3% at single-step to 36.9% at three-step on average, and by up to 31.2 points on individual models (GPT-5.4-mini, 41.7% to 72.9%). We test two defenses, which lose more ground to multi-step attacks than to single-step ones.

The evaluation is bounded by CUA API costs, so the results are based on a single run. This is a budget constraint as one complete pass over Table˜1 costs $\approx$520 USD in CUA API charges plus several hundred USD of AWS EC2 time, and repeating it multiple times is beyond our budget (Appendix A). Additionally, given the high experiment costs and the fixed benchmark size, every axis in our benchmark competes with every other. We vary platform, benign task, urgency cue, and user-instruction mode, at the cost of limiting the benchmark to 12 adversarial goals and $k\leq 3$, although we argue in Section˜6 that our 12 goals cover more distinct behaviors than a larger nominal count would suggest.

StepJack documents a new attack class against CUAs, and we believe surfacing it openly puts defenders in a stronger position than withholding would. To mitigate the dual-use risk of releasing a red-teaming pipeline, all evaluations are conducted in sandboxed environments derived from RedTeamCUA Liao et al. (2026) with no real user data or live services, and the benchmark targets frontier CUAs whose developers are positioned to act on the findings.