Coding Agents Beat Long Context: What the Duke Paper Actually Shows

Long context is one of those features that vendors love to benchmark and users struggle to actually use. Two million tokens sounds like magic — until you feed a million-token corpus to a frontier model and watch precision collapse somewhere past 60k. This week's paper argues something that sounds almost too clean: don't put the long context in the model. Put it in the filesystem, and let a coding agent walk it with grep and sed.

The paper is Coding Agents are Effective Long-Context Processors (Cao, Yin, Dhingra, Zhou; Duke University, March 2026). Across five benchmarks covering context lengths from 188K to three trillion tokens, off-the-shelf coding agents outperformed published SOTA on four of them, averaging 17.3% relative improvement. No fine-tuning, no new architecture — just a Claude-Code-shaped agent given a working directory and a task.

If you build with Claude, this is the most useful piece of long-context research I've read this year.

The 60-second TL;DR

• Setup. Take a coding agent (SWE-agent, Aider-style, or Claude Code) with the usual tools: read_file, write_file, grep, run_bash, list_dir. Give it a long-context task — QA over a 200k-token document, RAG over a 3T-token corpus, whatever. Instead of stuffing the corpus into the model's window, drop it as files in a workspace.
• Result. The agent explores files, writes small filter scripts, greps for patterns, iterates. On LongBench-v2, ∞Bench, HELMET, and BABILong-3T (yes, three trillion tokens), it beats every published long-context baseline including Gemini 1.5 Pro at 2M tokens.
• Why it works. Two mechanisms: native tool proficiency (agents already know how to use grep and awk because they were trained on code) and file system familiarity (browsing a corpus feels like browsing a repo).
• What to steal. Don't scale your context window. Externalize your context to disk. Turn long-context problems into agentic search + short-context reasoning.

The problem the paper is really solving

There are two things people mean by "long context." The paper cleanly separates them:

1. Long-context access — can the model see the tokens at all? Modern frontier LLMs solved this. Gemini 1.5 handles 2M tokens, Claude 3.5/4.5 handles 200k+.
2. Long-context processing — can the model effectively reason over those tokens? This is where the wheels fall off.

There's a growing literature on what the authors call context rot: as you push past 30-60k tokens, precision on needle-in-haystack, multi-hop QA, and code understanding degrades sharply. It's not about the theoretical limit; it's that attention gets fuzzier, and the model gets more confident about the wrong answer. RAG partially fixes this by externalizing retrieval, but standard RAG pipelines are one-shot: they retrieve once, hand the result to the model, and that's it. Multi-hop reasoning where later queries depend on earlier findings? RAG breaks down.

The paper's core observation: coding agents already solved this problem for code repositories. When Claude Code answers "where does the auth logic live in this monorepo," it doesn't stuff the whole repo into context. It runs grep -rn "auth", opens the top hits, follows imports, refines its query. That's iterative, programmatic, multi-hop long-context processing — and it's been the default for coding agents for two years.

The bet in the paper is that this same behavior transfers to any long text, not just code.

What they actually did

The experiment is refreshingly boring, which is the point. They took three off-the-shelf agents:

• SWE-agent (the original Princeton coding agent)
• Claude Code (Anthropic's terminal agent)
• Aider (community-favorite pair-programming CLI)

For each long-context task, they:

1. Wrote the corpus to a workspace directory (chunked by document, one file per doc for the multi-doc benchmarks; single file for single-doc).
2. Gave the agent a task prompt: "Answer the following question. The relevant text is in the current directory. Feel free to use grep, cat, or write scripts."
3. Let it run.

That's it. No custom retrieval, no fine-tuning, no RAG pipeline. The agent decides how to explore.

The five benchmarks

The numbers are the interesting part. All comparisons are against the best published SOTA at time of submission:

The Loong result is the most striking — 23.8 percentage points is enormous for a benchmark where the SOTA is Gemini 1.5 Pro with full 2M-token context. The agent isn't seeing all 250k tokens; it's actively choosing which parts to read. That constraint apparently helps.

The BABILong-3T result is where the paper stops being a curiosity and starts being a paradigm shift. Three trillion tokens is not a context window any model will ever have. But the agent doesn't care — it just walks a directory tree. Total tokens sent to the model per task: usually under 20k.

Why does this work so well?

The paper identifies two mechanisms, both of which map cleanly onto how you'd think about your own agent.

Mechanism 1: Native tool proficiency

Coding agents were pre-trained and fine-tuned on massive code corpora that include shell commands, Python scripts, and file manipulation. grep, awk, sed, find, xargs — the model knows these tools cold. When it needs to find "all mentions of a spell being cast by character X in a long fantasy transcript," it writes:

grep -n "cast" episode_*.txt | grep -iE "gandalf|dumbledore" | head -50

...and gets an answer in one turn. A retrieval-based RAG system would need to embed every sentence, run k-NN, rerank, and hope the query embedding lines up. The agent gets literal string matching for free, and can iterate when the first query misses.

This is a specific instance of a general principle worth internalizing: structured operations always beat semantic search when the operation is exact. Regex, path traversal, line ranges, byte offsets — anything that has a deterministic answer should not be handed to an embedding model.

Mechanism 2: File system familiarity

Coding agents treat directories the way humans treat books: hierarchical, browsable, indexable. When the corpus is 500 documents about company earnings reports, an agent naturally does:

ls -la earnings/          # what's here?
grep -l "revenue growth" earnings/*.txt | head -5   # which files matter?
cat earnings/Q3-2025-google.txt | grep -A 3 "revenue growth"   # what do they say?

Compare that to a RAG system that embeds every chunk once, retrieves top-k, and calls it done. The agent's search adapts to intermediate findings; RAG's doesn't.

The uncomfortable implication for the "bigger context window" arms race

If you take this paper seriously, the industry's investment in 2M / 10M / 100M-token context windows starts to look misallocated. The paper doesn't say "long context doesn't work" — it says "long context doesn't work as well as coding agents with short context and a filesystem."

That has real consequences:

• Cost. A 3T-token BABILong task ran on the agent using ~15k tokens of actual model input per turn, cumulatively maybe 200k tokens across the task. The Gemini 1.5 Pro baseline that scored 42% burned 250k+ tokens per query and lost.
• Latency. Agent turns are sequential but each turn is fast. Long-context prefill on 2M tokens takes 30+ seconds before the first output token.
• Cache. Prompt caching works beautifully for a code repo you keep re-querying. It works badly for a fresh 2M-token corpus you'll only touch once.

The paper doesn't claim long context is dead. It claims that for problems where you can externalize the corpus to disk, agents win. That covers approximately all real production workloads: internal knowledge bases, customer support archives, code repositories, legal documents, research corpora.

What this doesn't work for

Honest limitations, per the paper:

• True in-context reasoning where the whole context matters. If your task requires holding 200k tokens simultaneously in working memory — say, tracking every character's emotional arc across a novel — file-based agents don't help.
• Latency-critical single-turn. Agents take 30-120 seconds of wall clock per task. If you need sub-second responses, this isn't the tool.
• Tasks where the "documents" are already inside the model's training distribution. Asking an agent to grep through a corpus the base model already memorized doesn't add value.

But those constraints don't cover much of what people are actually building.

What to steal for your own agent

Three concrete moves you can make this week.

1. Externalize your context

If you're currently stuffing user history, retrieved chunks, or knowledge base excerpts into your Claude prompt: stop. Write them to disk in a workspace directory the agent can read_file and grep. Your prompt goes from 80k tokens to 2k tokens. Precision goes up, not down.

2. Add a run_bash tool with grep, head, sed, cat

The paper's agents used surprisingly minimal tooling: read_file, write_file, list_dir, run_bash. You already have all of these in Mini Claude Code Ep.03. If you don't, this paper is the strongest possible argument for adding them.

3. Prompt for iteration

Include in your system prompt: "When answering questions about the corpus, prefer grep and file exploration over trying to read entire documents. Refine your queries based on intermediate findings." This nudge alone accounts for a measurable chunk of the paper's gains.

The bigger picture

There's a pattern here worth naming. Every year, the frontier lab paradigm makes the model bigger, longer, and more expensive. Every year, someone at Duke or Berkeley or Princeton publishes a paper showing that a smaller, dumber system with the right tools beats it. Retrieval-augmented generation beat long context in 2023. Chain-of-thought prompting beat fine-tuning in 2022. And now coding-agent-style filesystem access beats long-context attention in 2026.

The lesson is not "smaller is better." The lesson is that capability and context are two different variables, and the community keeps confusing them. A model with a 200k context and a filesystem is more capable than one with 2M context and no tools, because the filesystem is not just storage — it's a structured, addressable, programmable form of memory.

If you take one thing from this paper: your agent is only as long-context as its most powerful tool. Give it grep and a directory, and it's effectively longer than any frontier LLM's window. Give it only its input buffer, and it's short-context by definition, no matter how many tokens fit.

Reproducing this yourself

The paper's code is at github.com/coding-agents-longctx (repo referenced in Section 7 of the paper). But you can reproduce the core finding in about 30 minutes:

1. Pick a long doc (say, a 100k-token novel from Project Gutenberg).
2. Write it to workspace/novel.txt.
3. Give Claude Code the prompt: "The file workspace/novel.txt contains a long novel. Answer this question about it: [your question]. Use grep and cat, don't try to read the whole file."
4. Compare against pasting the whole novel into Claude's context window and asking the same question.

The gap will be obvious. Precision, cost, and latency all move in your favor.

Where I'd push next

The paper doesn't cover a few things I'd love to see:

• Adversarial corpora. What if the "long context" is deliberately confusing (contradictory documents, near-duplicates with subtle differences)? Filesystem agents might actually degrade here because they see less context per turn.
• Multi-agent long context. Could a "planner" agent + a "searcher" sub-agent pipeline hit even higher performance? (See our Ep.05 on sub-agents for the tradeoffs.)
• Structured corpora. JSON, SQL, graph data. jq, sqlite, and dot should give agents even bigger advantages than plain-text grep.

If you build any of those, I'd love to read your writeup.

Paper: Cao, W., Yin, X., Dhingra, B., & Zhou, S. (2026). Coding Agents are Effective Long-Context Processors. arXiv:2603.20432.

Cite as:

@article{cao2026codingagents,
  title={Coding Agents are Effective Long-Context Processors},
  author={Cao, Weili and Yin, Xunjian and Dhingra, Bhuwan and Zhou, Shuyan},
  journal={arXiv preprint arXiv:2603.20432},
  year={2026}
}