@clickhouse/rowbinary (library as parser compiler)

A ClickHouse engineering-blog post introducing @clickhouse/rowbinary, a Node.js package for reading and writing ClickHouse’s RowBinary wire format from JavaScript. Its thesis — “when your library is also a parser compiler” — is where it earns attention: instead of shipping one generic, runtime-dispatched parser, the library ships densely commented, monomorphizable primitives plus a bundled Agent Skill (SKILL.md) that teaches an AI coding assistant to assemble a query-specific parser for the exact column types in hand. The serialization / data-interchange layer of developer-tooling, and the spoke’s second serialization source beside yaff.

The problem — why RowBinary gets abandoned for JSON

RowBinary is efficient but awkward to consume from JS. It has nested types (Nullable, Array, Map, Tuple), self-describing types (Variant, Dynamic, JSON), and precision-aware types (DateTime64). A generic parser that dispatches on type per column produces megamorphic call sites that defeat V8’s inlining — the slow path. So most developers reach for JSON instead and accept silent data corruption when 64-bit integers exceed Number.MAX_SAFE_INTEGER. The library exists to close that gap: keep RowBinary’s efficiency without the hand-written-parser tax.

The format (RowBinary)

Little-endian primitives, LEB128 varints for variable widths, no per-row overhead — a compact, self-describing binary encoding that ClickHouse (clickhouse, in analytical-databases-wiki) speaks as one of its wire formats. Reading it correctly means walking the type tree in the right order with the right widths; that walk is exactly what the library specializes away.

The technique — “parser compiler” without a compiler

Rather than build a protoc-style schema compiler, compilation is decomposed into artifacts an AI agent can read and recombine (see monomorphization):

Monomorphizable primitives — small, single-purpose, inlinable read functions with no megamorphic dispatch, so V8’s optimizer can inline an entire specialized parser.
Agent-driven codegen — the bundled SKILL.md teaches Claude to read the library’s comments and emit type-specialized TypeScript: a composed reader with per-field bounds checks collapses into, e.g., a single 26-byte row check followed by constant-offset reads.
The “compiler” is the agent + the skill, not a build tool — the library is the compiler’s input.

Numbers (vendor-reported)

RowBinary vs JSON: 2.1–3.3× faster across schemas.
Agent-generated specialized parser vs the generic composed reader: 1.5–3.4× (financial ledger / wide integers 1.55×; IoT telemetry 2.46×; fixed-width orders 3.41×).
Generation cost ~$0.20 per parser with prompt caching.
Benchmark hardware: Apple M4 Max, AMD EPYC 7763.

Cross-spoke context (routed here, runners-up noted)

This source legitimately touches three spokes; the dominant substance is the serialization format + its parser-compilation technique, which is this spoke’s corner — so it routes here, with the rest recorded as cross-spoke context (not fragmented):

agentic-tooling-wiki (runner-up) — the SKILL.md-as-compiler mechanism is a concrete Agent Skill instance (progressive disclosure, prompt caching); the skills machinery lives there, the library it compiles here.
analytical-databases-wiki (runner-up) — clickhouse is the database whose wire format this is; RowBinary-the-format routes here per that spoke’s own boundary note (“a data format, not a database → dev-tooling”).

Tier

T1 — first-party ClickHouse engineering blog on their own library. freshness: volatile — a fresh release; the speedups are the vendor’s own benchmarks, not independently verified here, and both the API and the skill will move.

developer-tooling · monomorphization · yaff · flat-ast · clickhouse · synthesis