Code-as-World-VL

mirros's family of vision-language models for quantitative physical reasoning from video — the released consequence of code-as-world. Given a monocular video and a question like "what is the speed of the blue ball as it leaves the ramp?", the model returns a number with a unit.

Three sizes trained, two released:

BF16, 16 frames sampled per video, 4,608-token context, served through vLLM with a Qwen3.5 no-think chat template. Built on Qwen's open-weight line, so the Apache-2.0 grant is conditional on the base model's terms.

The training recipe

Two phases on eight H100s — and both halves belong to this spoke's fine-tuning corner rather than to pretraining.

1. Image-space measurement, by SFT. Supervised fine-tuning on pixel-level questions built by converting existing annotations into quantities: object width/height/diagonal from RefCOCO/RefCOCO+/RefCOCOg/RefCLEF bounding boxes, and displacement, velocity and acceleration from GOT-10K tracks via finite differences over the trajectory. This teaches localisation, measurement and tracking — no physics yet.
2. World-space calibration, by GRPO. Reinforcement post-training on question–answer pairs generated from verified executable worlds, where the answer is read directly off the simulator's state trajectory.

The checkpoints after phase one are called the Image-Space variants, and the report notes the full models improve over them on every image-space benchmark too — world-space training did not cost the grounding it was built on.

What the numbers support, and what they don't

The comparison that carries weight is against the model's own base: Qwen3.5-4B scores 31.2 on quantiphy and Code-as-World-VL-4B scores 50.6 — same size, same benchmark, same protocol. That is a self-run measurement of a self-made intervention, but it is a controlled one, and it does not require believing the proprietary rows in the same table.

The headline claim — that the 9B beats Gemini-3.1 Flash (55.4 vs 54.8) — is the weakest number in the release, not the strongest. It is a 0.6-point margin on one benchmark run by an interested party against an API model configured by that party, in a table where ChatGPT-5 Pro lands below a 0.26B SmolVLM and MiniCPM-V 4.5 scores an implausible 0.0 on two subsets. Record the base-model delta; treat the leaderboard position as a claim.

The 27B is not released and changes scale and response format together, so it establishes that the recipe scales, not how much reasoning contributed.

Adoption

40 downloads in the month before ingest for the 9B — this is a research release, not a deployed model, and nothing here has been independently reproduced.

Related

• code-as-world · code-as-world-report · quantiphy
• executable-world-representation — where its training labels come from
• fine-tuning — the SFT-then-RL curriculum