Package-manager benchmark (cheezone, 2026)
A benchmark harness by an individual developer (Cheez Lin, GitHub cheezone) that times npm, pnpm,
Bun, nub and aube on pinned open-source projects inside GitHub Actions, with hyperfine reporting
mean ± standard deviation. Found by the 2026-09-22 research pass for Most-wanted #41 / coverage edge 4,
which asked for an install-time measurement someone outside the project ran, with the cache state
stated. Read from the repository's README and its raw result files the same day.
T2: someone who builds none of the five tools measured all five, and published the harness so the numbers can be re-run. Graded with three weaknesses, below, that matter more than the tier suggests.
Method
- Fixtures pinned by commit:
handle(antfu, Vue 3 + Vite, ~697 packages installed) andvitesse(Vite + Vue starter, ~1,180 packages), plus a small synthetic monorepo. Before timing, lockfiles,node_modules,packageManagerpins, pnpm-only config andpostinstallhooks are stripped so every manager resolves the same tree. - Cache state stated per scenario, which is what the edge asked for:
install_cold(no cache, no lockfile, nonode_modules),install_warm(cache and lockfile primed,node_modulesremoved),install_frozen(npm ci/--frozen-lockfile), andrun_noop(script spawn overhead). - Linux x64 GitHub runner, Node 24.20. Yarn is out of scope by the author's choice.
Results, latest version of each, recorded 2026-09-22 (seconds, mean ± sd)
| Fixture | Manager | Cold | Warm | Frozen |
|---|---|---|---|---|
| handle | npm 12.0.2 | 26.24 ± 0.67 | 5.50 ± 0.06 | 5.51 ± 0.05 |
| handle | pnpm 12.5.1 | 3.40 ± 2.68 | 0.28 ± 0.01 | 0.29 ± 0.01 |
| handle | Bun 1.4.2 | 1.69 ± 0.21 | 0.21 ± 0.02 | 0.20 ± 0.01 |
| vitesse | npm 12.0.2 | 38.74 ± 2.13 | 11.45 ± 0.05 | 11.49 ± 0.06 |
| vitesse | pnpm 12.5.1 | 3.42 ± 1.91 | 1.01 ± 0.17 | 0.89 ± 0.17 |
| vitesse | Bun 1.4.2 | 2.69 ± 0.76 | 0.65 ± 0.06 | 0.60 ± 0.02 |
(nub 0.9.3 and aube 2.2.4 are in the data too; aube's warm install is slower than its frozen one.)
On this harness, pnpm installs these projects roughly 8–11× faster than npm cold and 11–20× faster warm. Bun is faster again. The warm and frozen columns have small spread; the cold column does not.
Why it only half-closes the edge
- It is one day old. The repository was created on 2026-09-22 and every result file carries that date. It has not been re-run on another day, by anyone else, or on another machine.
- Uneven repetition. Bun and nub cold installs were sampled 10 times; npm, pnpm and aube only twice. pnpm's cold standard deviation (2.68 s on a 3.40 s mean) says two samples are not enough.
- CI runner only. A shared GitHub runner's network and disk dominate cold installs; the README itself says local numbers will differ.
The warm/frozen gap between pnpm and npm is large and stable enough to hold up. The cold ratios are not yet something to quote. There is still no disk-use measurement, the other half of pnpm's pitch.
Related
pnpm · package-manager · monorepo · synthesis