Dream-RSI: Recursive Self-Improvement through Evolving Worlds

Tong Zheng, Xidong Wu, Zheng Zhang, Zhankui He, Chaoyi Zhang, Benjamin Coleman, Ruoqiao Wei, Di Bai, Haolin Liu, Rui Liu, Xue Wang, Yue Zhuan, Wang-Cheng Kang, Renkai Xiang, Heng Huang, Xinwu Cheng and Yunsong Guo — University of Maryland, College Park; google-deepmind; University of Virginia. arXiv:2609.14858 (cs.CL), v1 14 September 2026, 12 pages. Code at github.com/zhengkid/Dream-RSI, site dream-rsi.com (neither checked). Arrived by Telegram via the Hugging Face papers page, 2026-09-16.

The problem it names

The paper treats "recursive self-improvement" as a discovery problem: an agent expands a tree of candidate solutions (a Lasso solver, a circle packing, a GPU kernel), and what limits it is the exploration strategy that decides which node to expand next. Fixed strategies, the authors say, stop fitting as the search space grows; tuning the strategy online means paying for full long-horizon rollouts every time a candidate strategy is tried. That is the same objection environment-evolution raised against on-policy curriculum generation, aimed one level down at the search policy.

The move

A lightweight orchestration layer makes exploration "explicit and programmable while leaving the underlying coding agent unchanged." Three stages repeat:

1. Online exploration — the current policy drives a coding agent to grow discovery trees; every trace is logged.
2. Replay simulator construction — the finished trees become a pool of simulators over the realized search space.
3. Dreaming — candidate exploration policies are scored by replaying them against that pool, which gives off-policy feedback without new agent calls. The best policy goes back online, which grows the pool.

The word "dreaming" is the authors' own and means offline replay, not generation; it shares nothing but the name with claude-dreams, which consolidates a memory store.

What they report

Authors' numbers, v1 preprint, no independent replication (re-checked 2026-10-11: a v2 went up on 2026-10-06 from the same authors, and no outside replication has appeared):

• Algorithm engineering (Lasso regularization path): mean runtime 3,587.1 ms → 2,931.0 ms with 317 agent calls against 550 for a fixed strategy.
• Mathematical optimization (sum–difference problem, circle packing, autocorrelation inequalities): competitive results within 1k generations, claimed 50× budget saving against a baseline named SimpleTES.
• GPU kernels (VGG16, LayerNorm, ConvDiv, ConvMax from KernelBench): matching performance in 1.79×–2.43× fewer generations, or 1.44×–2.09× better at equal budget.

The abstract hedges its own claim ("competitive or improved … in several settings") and that is the right summary; the per-task table decides which settings are which.

What it adds here

• The loop improves a policy over the agent, not the agent, its skills, or its environment. That is a fourth site for the self-improvement loop in this spoke, after skills (self-improving-agents), the harness (harnessdev) and the training environment (environment-evolution) — and the only one of the four that never touches the model's weights or its prompts.
• History reused as a simulator is the cheap-feedback trick. It works because a discovery tree is a record of what each expansion cost and returned, so a new policy's choices can be priced against what actually happened. The unstated limit is that replay only covers nodes the old policies reached; a policy that would have gone somewhere new gets no credit for it.
• The claim is cost, not quality: most of the headline numbers are fewer generations to the same result. Read beside harnessdev's wobbling gains, that is the more defensible thing for a self-improvement paper to claim.

Runner-up spoke: optimization-algorithms (the tasks are metaheuristic benchmarks and the exploration policy is a search heuristic); routed here because the object being improved is the agent's orchestration layer.

Tier T1 — arXiv preprint with named affiliations and code; results are the authors' own.

Related

self-improving-agents · environment-evolution · harnessdev · self-evolving-agents-survey · loop-engineering · claude-dreams · google-deepmind