Environment Evolution for Terminal Agents

Zhiyuan Fan, Tinghao Yu, Yuanjun Cai, Jiang Zhou, Jiangtao Guan, Jincheng Liu, Yun Yang, Dingxin Hu, Zhuo Han, Xing Wu, Feng Zhang and Lilin Wang. arXiv:2609.04128 (cs.AI), v1 3 September 2026. No affiliations are given on the listing page. Arrived by Telegram 2026-09-05.

The problem it names

Training a terminal agent needs interactive, verifiable environments, and the paper's premise is that the supply of hard ones runs out: "As frontier models become more capable, environments synthesized from scratch become less challenging and thus provide limited learning signals."

Co-evolution methods answer that by generating environments near the model's learnable frontier, using weaknesses the model exposes during rollouts. The paper's objection is to the on-policy dependence: because the next environment is derived from what this model just failed at, generalisation is limited and the signal thins as the model improves. Which is a scaling complaint about the curriculum, not about the model.

What it proposes

Environment evolution: raise environment difficulty off-policy, and schedule the evolved environments generation by generation across training, so the learning signal keeps arriving instead of decaying. Three evolution directions are derived from the multi-turn learning objective — the paper argues them from the objective rather than picking them — and implemented through a "loop-engineered multi-agent harness."

That last phrase is why this page is in this spoke rather than in ../machine-learning-wiki. The generator of the training curriculum is itself an agent harness: the same construct this spoke tracks as a runtime for doing work is here a runtime for manufacturing the difficulty another model is trained against.

The numbers

Rollout experiments with Hy4 preview, Claude Opus 5 and GPT-5.6 Sol are used to show that the evolved environments are "consistently" harder. Three frontier models as difficulty meters, not as the thing being trained.
Training, via "simple long-horizon RL": Qwen3.6-27B +14.4 points and Qwen3.6-35B-A3B +18.0 points on Terminal-Bench 2.1.

What to hold it at

Every figure is the authors' own, on their own method, and this is a v1 preprint three days old with no affiliations stated on the listing and no independent replication — the standing pattern the hub backlog tracks. Two specific gaps worth naming rather than assuming: the abstract gives the gains as percentage points and no baseline score, so how much of Terminal-Bench 2.1 remains is not stated here; and "simple long-horizon RL" is the training half of a two-part claim, unspecified at abstract level, so the improvement cannot yet be attributed to the environment scheduling rather than to the RL setup around it. Read the paper body before either number is carried anywhere.

Where it sits in the spoke

It is the counterpart to the harness-measurement line. harness-bench measures how much of an agent's score belongs to the harness rather than the model; this paper uses a harness to change what the model can be trained on at all. And it inverts the usual direction of self-improving-agents: the thing that improves generation over generation is not the agent, it is the environment — deliberately, because the authors think making the agent generate its own curriculum is what limits the existing approach.

harness-bench · longhorizon-harness · long-horizon-agency · self-improving-agents · · scienceide agent-evals · skillsbench