LongHorizon-Harness

A harness from AMAP-ML (alibaba's mapping/ML group) for running agents across desktop apps and the command line — MIT-licensed, and explicitly a layer above an existing agent backend rather than an agent of its own. It plugs into claude-code, Codex and OpenClaw, and its published results run on Qwen. Routed here 2026-08-05 from Telegram. Its pitch is duration — "work across desktop apps and the command line for dozens of hours."

Its framing sentence is the sharpest statement of this wiki's founding bet that the corpus holds:

The model determines what an agent can do in one round. LongHorizon-Harness determines whether that work can be verified, preserved, and continued until the task is actually complete.

Three roles, and the boundary between them is the point

The design splits the loop into Manager, Executor and Auditor. The Manager keeps the goal, the verified progress so far, and the next step. The Executor does one focused task per round, and gets a fresh context each time. The Auditor checks the result independently, and only what it passes enters persistent state.

Most of the agent-loops-verification thread treats verification as something bolted onto the end of a loop. Here it is the write gate on state. An unverified result does not merely get flagged, it never becomes part of what the next round builds on — which is how the design claims to stop drift accumulating over dozens of hours. Compare statewright, which restricts which tools a phase may call, and agent-guardrails, which restricts what damage a call may do. This restricts what is allowed to become true.

The fresh-context-per-round choice puts it against the grain of the long-context direction: the Executor is deliberately denied the history, and the Manager's distilled state is the only channel by which anything survives. Read against context-rot, that is the architectural answer to intrinsic rot rather than a mitigation of it — if the context never grows, the budget never erodes. The cost is that everything the Manager fails to write down is gone, which the design pays for with the Auditor.

The numbers, and exactly what they measure

All three results hold the backbone fixed at Qwen 3.7-Plus and switch the harness on:

Coverage is claimed across 13+ task domains — web, data analysis, debugging, design, documents, CAD, healthcare forms.

Two things are worth separating here, because the spoke has been burned by running them together.

What is new: the task counts are stated. 114 and 108 are named, which is more than agentsys has ever offered and more than statewright's 5-task SWE-bench subset. The Terminal-Bench row is the weakest of the three: no N, and the smallest gain.

The OSWorld figure is the one that should set a reader's expectations, because 2.8% → 8.3% is a 3× improvement on a number that is still terrible. A project selling its own harness had every reason to lead with WeaveBench's +28.9 and bury that row. It didn't.

What is not new: this is self-reported, and it varies the wrong thing for our purposes. The measurements are the project's own, with no third party in the loop, no trial count per task, and no described rubric. And structurally the design holds the model fixed and varies the harness — the identical shape as cc-thinking-skills, jetbrains-ponytail-tested and context-files-ablation-paper before it.

Why it does not close growth edge #2 — and what it does do

The spoke's second growth edge wants a comparison that varies model and harness in the same design, because that is the only shape that can test agentsys's "Sonnet + harness beats raw Opus." This is the seventh measurement to go past that question without asking it. The edge stands.

What it does supply is a magnitude for one arm of the substitution the edge is about. If a harness is worth +28.9 points on a fixed mid-tier backbone, then the question of whether that exceeds the gap between two model tiers is now a comparison between two numbers rather than between a number and a README claim. Someone still has to run the fourth cell. But the edge is better posed than it was yesterday, and it is worth recording that the harness arm keeps coming back large: statewright 2/10 → 10/10, this at +28.9 points, agentsys claiming 73–83% cost savings.

The caution that applies to all three is the same and it is not a small one: every one of these numbers was produced by the people selling the harness. The corpus has no independent replication of any of them.

Re-checked 2026-09-05 and still true, with the nearest miss recorded. The field has moved toward the question without answering it for any of these three: Stop Comparing LLM Agents Without Disclosing the Harness (arXiv 2605.23950) argues for harness cards and variance decomposition, and harness-bench — already held here since 2026-08-08 — measures harness effects across models. Both study the harness as a variable; neither re-runs a vendor's own reported figure. So the shape of the gap is now specific: the discipline has a method for the question and nobody has pointed it at a seller's headline number.

Superseded the next day — and the answer went against the harness (added 2026-08-30). The section above was written on 2026-08-05 and has been stale since 2026-08-06. Growth edge 2 did not stand: it was closed three times over in the following week, and "someone still has to run the fourth cell" is no longer true.

• scaffold-effects-gaia (2026-08-06) — a pre-registered crossed 5 models × 3 scaffolds design. Structure carries a model one tier, not two, and the best scaffold is model-specific.
• claw-swe-bench (2026-08-07) — 350 SWE-bench-class instances, prompt/container/timeout/evaluator fixed: harness spread 12.5–27.4 pp against a model spread of 29.4 pp.
• harness-bench (2026-08-08) — 8 model backends × 6 harnesses, 106 tasks, 5,194 trajectories, frontier models included. The spread narrows at the top and does not vanish (23.8 pp).

Read those against this page's +28.9 on a fixed backbone and the comparison it invited resolves the unflattering way: the model gap is the larger of the two. A big single-arm harness number is exactly what you would expect from a design that never varies the other arm, and three independent designs that do vary it put the harness second. Nothing here retracts the measurements above — they are still the clearest self-reported figures in the corpus, and the OSWorld row is still admirably unflattering. What is retracted is the framing: this was not evidence for the founding bet, it was one arm of it.

Adoption and release cadence (added 2026-08-30)

The first ingest recorded no repository numbers at all. From the GitHub API on 2026-08-30: 1,396 stars / 144 forks, 37 open issues, Python, MIT, homepage lh-harness.pages.dev. Created 2026-08-04 — so those stars accumulated in 26 days, which is fast even for this spoke.

It is being worked on, not parked. Five tagged releases since the first ingest — v0.1.3 (08-07) through v0.1.7 (08-20) — and 38 commits in that window. The commit titles are the tell that this is a real system meeting real conditions rather than a benchmark artifact: "Guard-only failures fail the round, not the run," "Recover from isolated EBADF in control-bus reads; keep the watcher alive," "Recover from local role timeouts." Those are the failure modes a Manager/Executor/Auditor split actually produces once it runs for dozens of hours, and they are being fixed one at a time.

Last push 2026-08-20, ten days before this refresh. Nothing here is measured by anyone outside the project.