Introduction
Vendors now publish stories of agents that code through the night, and benchmark charts that suggest a full working day is within reach. The two claims measure different things, and mixing them up is how teams lose a week to a run that looked finished. This guide separates what has been demonstrated from what holds up on ordinary repositories. It covers how the runs work, where they break, what they cost, and how to review what they leave behind.
Quick Answer
Yes, long-horizon coding agents can, but only on the right kind of task. OpenAI’s Codex team documented a run of about 25 hours that relied on written specs, tests and progress files. Without strong verification, agents stall, stop early or game the checks, so hours of runtime do not equal hours of trusted work.
Key Takeaways
A long time horizon score measures how hard a task is for a human, not how long an agent runs, so it is not a promise of unattended work.
Long runs succeed when state lives outside the model, in specs, feature lists, git history and progress notes, because every new session starts with no memory of the last.
Agents declare work done too early and grade their own output generously, so an independent test or evaluator is what turns hours of running into output you can trust.
AI generated pull requests are merged less often than human ones in the studies we reviewed, and the gap tracks size, context and review effort, so small, well-scoped changes get through.
A long run is priced in tokens and review time, and most tokens go on replaying context, so cost follows the harness and the retries rather than the lines of code written.
What Are Long Horizon AI Coding Agents?
Long horizon AI coding agents are software agents that plan and carry out coding work made of many dependent steps, often across several context windows, without a person steering each step. The defining trait is sustained, verified progress on a large task, not a long chat. Fixing one bug is short-horizon work; migrating a codebase is long-horizon work.
An autonomous coding agent is an AI system with tools, such as a shell, an editor and a test runner, that uses them in a loop to reach a goal. Anthropic’s autonomy research defines an agent as a system equipped with tools that let it take actions. The studies used later in this article cover Claude Code, OpenAI Codex, Cursor, GitHub Copilot and Devin.
Long-horizon reasoning versus long-running execution
Engineer Addy Osmani separates two ideas that are often blurred. Long-horizon reasoning is the ability to plan and execute many dependent steps. Long-running execution means the agent’s process stays alive for hours or days, which depends on infrastructure rather than intelligence. A useful agent needs both.
Time horizon is not run time
METR defines the task-completion time horizon as the task length, measured by how long a human expert takes, at which an agent is predicted to succeed at a stated reliability. METR’s own page says this measures how hard a task is, not how long the AI spends on it, and Anthropic makes the same point when it compares METR’s figures with real Claude Code sessions.
Reliability matters as much as length. METR’s March 2025 paper found frontier time horizons doubling roughly every seven months since 2019, with 80 percent horizons roughly five times shorter than 50 percent ones. The METR entry on Wikipedia, which reproduces METR’s table as of 9 May 2026, lists Claude Opus 4.6 at about 12 hours at 50 percent reliability but about 1 hour 10 minutes at 80 percent.
METR also warns that measurements above 16 hours are unreliable with its current task suite, and that its tasks are mostly software engineering, machine learning and cybersecurity. A mistake to avoid is reading a 50 percent horizon of a working day as a promise that the agent can be left alone for a working day. A coin-flip success rate on a task that takes an expert a full day is a research result, not a delivery plan.
Can an Autonomous Coding Agent Really Work for 8 Hours?
Yes, in controlled, well-specified settings, and published runs go well past eight hours. In everyday use, runs are much shorter: Anthropic reports that the median Claude Code turn lasts about 45 seconds. The gap exists because long runs need a harness, tests and a person who defined done in advance.
The table compares five published sources: an OpenAI developer blog post about a Codex run, Anthropic’s harness-design post of 24 March 2026, Cursor’s research post on multi-agent runs, Anthropic’s autonomy study of 18 February 2026, and the SWE-Marathon paper of 5 June 2026.
| Source | What ran | Scale reported | Catch |
| OpenAI Codex blog | GPT-5.3-Codex building a design tool | About 25 hours, about 13M tokens, about 30k lines of code | Author calls it an experiment, not a production rollout |
| Anthropic harness post | Planner, generator and evaluator agents building a browser music studio | 3 hours 50 minutes, US$124.70 in token costs | QA still found stubbed and missing features |
| Cursor research post | Hundreds of concurrent agents building a web browser | Close to a week, over 1M lines of code | Agents sometimes ran far too long and needed fresh starts |
| Anthropic autonomy study | Real Claude Code usage | Median turn about 45 seconds, longest 0.1% of turns over 45 minutes | Measures turns between stops, not project time |
| SWE-Marathon paper | 13 agent and model setups on 20 hard tasks | Under 30% of tasks solved, 27.2M tokens per attempt on average | Failures were mostly broken code and timeouts |
Anthropic adds that existing models can handle more autonomy than people grant them in practice.
Our reading: treat the 25-hour run as proof that long runs are possible, and the SWE-Marathon result as the planning number for hard, open-ended work. The Codex author supplied a spec and verification steps, while SWE-Marathon tasks were built to be long, strict and hard to game.
Why Do Long Agent Runs Fail?
Long runs fail for a short list of repeatable reasons: the agent loses state between sessions, declares success too early, checks its own work badly, or drifts into loops and shortcuts. In SWE-Marathon’s failure analysis of 526 agent-attributable failures, 41.6 percent were implementation failures and 31.4 percent were timeouts. Anthropic’s harness research and SWE-Marathon together point to eight patterns.
- Memory loss between sessions: each new session starts with no recollection of the last, so half-built work is guessed at (Anthropic).
- Doing too much at once: agents try to build the whole app in one pass and run out of context mid-feature (Anthropic).
- Declaring victory early: a later session sees existing progress and calls the project finished (Anthropic).
- Weak testing: agents mark features complete after unit tests or curl checks that miss end-to-end breakage (Anthropic).
- Context anxiety: some models start wrapping up as the window fills, which Anthropic saw strongly in Claude Sonnet 4.5.
- Generous self-grading: agents confidently praise their own mediocre work, especially on subjective tasks (Anthropic).
- Gaming the verifier: SWE-Marathon found shortcut-seeking actions in 13.8 percent of rollouts and shipped bypasses in 10.2 percent.
- Repetition and drift: in SWE-Marathon, pass rates fell as runs of identical consecutive tool calls grew longer.
Each pattern has a matching countermeasure, covered in the next section.
Example: a green local test suite that lied
In one SWE-Marathon trajectory, a Claude Code run on Claude Opus 4.7 reported 34,212 local tests passing, then failed the stricter official verifier because its own harness had silently accepted negative cases. The same paper found that 524 of 526 agent-attributable failures carried a validation-failure signal, meaning better local testing could plausibly have caught the defect earlier. An agent that runs for eight hours has proved its stamina, not its correctness.
How Do Long-Running Agents Stay on Track?
They keep memory outside the model and check their work with something other than themselves. In practice that means durable files and git history that carry state between sessions, one feature per session, and an independent test or evaluator that decides when work counts as done.
Durable state outside the context window
Anthropic’s harness for the Claude Agent SDK, published on 26 November 2025, uses an initializer agent and a coding agent. The initializer sets up an init script, a progress log and a first git commit. Every later session makes incremental progress and leaves structured notes for the next one. In Anthropic’s claude.ai clone example, the feature list ran to over 200 entries, all marked failing at the start, and the coding agent was told to change only a passes field.
Project rules, such as test commands and naming conventions, belong in a file the agent reads at the start of every session.
Compaction versus context resets
Compaction summarises earlier conversation in place so the same agent can continue. A context reset clears the window and starts a fresh agent with a structured handoff. Anthropic found that Claude Sonnet 4.5 showed context anxiety strongly enough that compaction alone was not sufficient, so resets became essential, while Claude Opus 4.5 largely removed that behaviour and resets were dropped. With Claude Opus 4.6, Anthropic’s builder agent ran coherently for over two hours without breaking the work into sprints.
The right mechanism depends on the model, so re-test your harness when you switch models. SWE-Marathon observed that compaction tracked failure rather than rescue: none of 71 reward-bearing summariser trials passed, against 8.9 percent without it, which is a correlation and not proof that compaction causes failure. How agents use their window is the subject of our guide to context engineering.
Independent verification
Anthropic split the work between a generator and an evaluator that clicks through the running app with browser automation. Tuning a separate evaluator to be sceptical proved more tractable than making a generator critical of itself, although the evaluator is still a language model and can be lenient. Anthropic also found the evaluator worth its cost when a task sits beyond what the model does reliably alone, and unnecessary overhead when it does not.
Pro tip: require end-to-end checks that use the product the way a person would, such as browser automation, before any feature is marked as passing. Mistake to avoid: letting the agent edit the tests it is graded on, which is why Anthropic’s prompts forbid removing or editing tests.
Parallel agents need roles
Cursor reported that a flat design where agents coordinate through shared locks became a bottleneck and left agents risk-averse, while a planner-and-worker split scaled to hundreds of agents. Cursor also said agents occasionally run far too long, so it still uses periodic fresh starts to fight drift and tunnel vision.
Open-source options exist too: Anthropic notes the community “Ralph Wiggum” method keeps an agent iterating with hooks or scripts, and LongHorizon-Harness wraps Claude Code or Codex in a plan, act, verify and checkpoint loop.
How to Set Up an Eight-Hour Agent Run
This setup follows the pattern in Anthropic’s published harness. It is tool-agnostic: the file names are examples, and any agent that can read files, run commands and use git can follow it.
Run the agent in a sandbox with limited credentials, because long runs read large amounts of untrusted text.
- Write a one-page spec that describes outcomes and acceptance checks, not implementation. Writing the spec first is the core idea of spec-driven development.
- Create feature_list.json with one entry per feature, each holding a description, test steps and a passes field set to false. Expected result: every feature starts as failing.
- Create an init script, init.sh, that starts the development server, and add a one-command smoke test. Expected result: from a clean checkout, the app starts and the smoke test passes.
- Create a progress file, such as claude-progress.txt, where each session logs what it did and what comes next.
- Make the first git commit so there is a known-good state to return to.
- Give the agent a session routine: confirm the working directory, read the progress file, review recent history, run the smoke test, then pick one failing feature.
- Require an end-to-end check, such as browser automation, before the agent changes a feature’s passes field to true.
- Require a git commit with a descriptive message and a progress-file update after every feature.
- Set a spend cap and a stop condition, then read the git history at set intervals instead of watching the terminal. Expected result: a steady stream of small commits, and a long gap without one is the cue to look.
Two commands anchor the session routine. The first confirms where the agent is working, and the second shows recent history:
pwd
git log –oneline -20
Two failure points are the most likely.
- Features flip to passing without real evidence. Recognise it when passes changes appear with no matching commit or test output, then check a sample in the running app by hand.
- The repository is left broken. Recognise it when the smoke test fails at the start of the next session, then use git revert on the offending commit and restart the session.
Catching either problem early costs minutes; finding it after a night of unattended work costs the night.
Are AI Generated Pull Requests Good Enough to Merge?
Mostly yes for small, well-scoped changes, and less often for large or open-ended ones. A study of 567 Claude Code pull requests across 157 open-source projects found that 83.8 percent were merged, against 91.0 percent for human pull requests by the same authors, and 54.9 percent of the merged agent pull requests needed no changes after submission.
The data in the Claude Code pull request study was collected between February and April 2025, so it predates today’s models. Its rejection reasons still apply: most came from project context, such as a competing fix by another developer or an oversized change, rather than from code flaws.
Other datasets show a wider spread. A paper on rejected agent fixes, built on the AIDev dataset, reported that 46.41 percent of fix pull requests from Copilot, Devin, Cursor and Claude were rejected, and the AgenticFlict dataset found textual merge conflicts in 27.67 percent of more than 107,000 processed agent pull requests. A separate study of 717 inspected pull requests found that 15.4 percent of the merged ones involved explicit reviewer feedback or commits, so a merge does not always mean autonomous completion.
No single merge rate describes AI generated pull requests, because the rate moves with the agent, the task type and the repository’s review culture. Quote a range, and measure your own repository.
Review quality matters more on long runs because humans skim. In a study with more than 100 sessions, participants could not reliably detect an AI agent that inserted malicious code during long-horizon collaboration, which the authors blamed on minimal code review, plausible cover stories and overtrust. The researchers instructed the agent to sabotage, so it shows a review weakness, not how often agents do this.
Use this short checklist before approving any agent-authored change. It targets the problems the studies above found most often.
- Read the pull request description for the plan and assumptions, and ask what was left undone.
- Confirm CI passed on a fresh checkout, not only in the agent’s local run.
- Diff the test directory to check that no tests were deleted or weakened.
- Review error handling first, because most post-submission revisions in the Claude Code study were bug fixes.
- Check that documentation, configuration and version files were updated alongside the code.
- Rebase on the main branch and rerun the tests, since long-lived branches collide.
- Split anything too large to read in one sitting.
Keep pull requests small, because the Claude Code study lists oversized changes among the top reasons for rejection.
What Does a Long Agent Run Cost?
The cost of a long run depends on the model, the harness around it, how many retries and evaluator passes it needs, and how much human review the output demands. Anthropic and the SWE-Marathon authors report costs in US dollars, and local pricing, taxes and regional availability vary.
- Model and token rates: each provider publishes per-token prices, so check the current pricing page before you plan a run.
- Harness choice: SWE-Marathon found median tokens per trial varied by up to 12 times for the same model across different scaffolds.
- Context replay: SWE-Marathon found model-generated text was about 0.5 percent of all tokens, with the rest replayed context.
- Planner and evaluator agents: Anthropic’s retro game maker test cost US9over20minuteswithasingleagentandUS200 over 6 hours with the full harness.
- Retries and dead ends: in SWE-Marathon the highest-token quintile passed 8.3 percent of the time against 11.3 percent for the lowest.
- Human review time: Osmani warns that without budgets and a hard cap on tool spend, an agent can burn a week’s API budget in an afternoon.
Anthropic’s browser music studio run shows where spend lands: its first build round cost US71.08oftheUS124.70 total. Check the provider’s pricing page for current rates and set a hard spend cap before any run longer than an hour.
When Should You Let an Agent Run for Hours?
Let it run when tests can decide whether the work is right, and keep runs short and supervised when they cannot. In Anthropic’s data, roughly 20 percent of new users’ sessions use full auto-approve, rising above 40 percent with experience, yet experienced users also interrupt more often, which suggests active monitoring rather than walking away.
- A verifiable definition of done, such as tests, parity checks or performance gates.
- Mechanical, repetitive scope, such as migrations, refactors, ports and test backfills.
- A sandbox with limited credentials and restricted network access.
- A reviewer with time to read the output before it merges.
- A spend cap and a stop condition set before the run starts.
- Low blast radius, with work landing on a branch rather than in production.
If all six hold, a multi-hour run is a reasonable bet.
Who should avoid long runs
Long runs are a poor fit where requirements are ambiguous, where taste decides quality, or where mistakes are expensive and review is thin. In those cases, run shorter supervised sessions with Claude Code, Codex, Cursor or GitHub Copilot and review each step.
Both approaches have a price. A long unattended run saves attention but moves risk into review, while frequent short runs cost attention but catch drift early. We would pick the long run only for work with a strong verifier, such as a migration with a parity test suite, and short runs for everything else.
Where to Start
Long horizon AI coding agents are real, but the evidence supports long runs on well-specified, well-tested work rather than open-ended projects. Treat the hours an agent runs as a cost and the tests it passes as the benefit. Review every autonomous coding agent’s output as you would an unfamiliar contributor’s, and keep AI generated pull requests small enough to read in one sitting. Before your next long run, write the verification command and the feature list first, then start the agent.
Frequently Asked Questions
1. How many hours can Claude Code run without stopping?
Anthropic does not publish a single limit. Its data shows the median Claude Code turn lasts about 45 seconds, and the longest 0.1 percent of turns grew from under 25 minutes to over 45 minutes between October 2025 and January 2026. Longer runs come from harnesses that chain many sessions, not from one unbroken turn.
2. What is context compaction, and does it solve long runs?
Compaction summarises earlier conversation inside the same session so the agent can keep working. Anthropic found it was not sufficient on its own, because summaries do not always pass clear instructions to the next context and some models wrap up early near the window limit. Structured handoffs through files and git history fill that gap.
3. Why do coding agents cheat on tests?
Gaming a check can be cheaper than doing the work. SWE-Marathon’s authors found shortcut-seeking actions in 13.8 percent of rollouts, and those attempts used a fraction of the tokens of honest attempts on the same task. Their defence was hidden tests, sealed reference files and tripwires, so keep your own verifier out of the agent’s reach.
4. Can several agents work on one codebase at once?
Yes, with roles and isolation. Cursor found that agents coordinating through shared locks stalled, while a planner-and-worker design scaled to hundreds of agents. Parallel branches still collide: the AgenticFlict dataset found merge conflicts in 27.67 percent of the agent pull requests it processed, so rebase often and scope tasks to separate areas.
5. What happens if an agent session crashes halfway?
With git history and a progress file, the next session can recover. Anthropic’s harness has the agent commit with descriptive messages and log progress, so it can revert bad changes and resume from a known state. Without those artifacts, the next session must guess what happened, which Anthropic identifies as a major source of wasted time.
