How to Utilize OKF Efficiently to Enable Knowledge Exchange Among LLMs (Towards Data Science)

Anubhab Banerjee, Towards Data Science, 13 Aug 2026 — the same author as build-llm-runtime-from-scratch, and the same method: build the thing, pin the checkpoints, publish the numbers. T2: a hand-run measurement with a stated protocol (median of 7 trials, greedy decoding, 64 new tokens), by an independent author with nothing to sell.

Three Qwen2.5-Coder agents (7B → 3B → 1.5B) read one 5G design document in sequence. The upstream agent tokenizes once and writes the token IDs to a .npy array in /dev/shm; the downstream agents follow a pointer to that array and call model.generate(input_ids=…) without ever running their own tokenizer. The handoff rides on research-wiki/open-knowledge-format markdown files whose frontmatter carries a non-standard token_pointer field.

Banerjee's framing: "the tokenizer is not the bottleneck… But the tokenizer is redundant work." Three models from one family re-derive the same integers from the same bytes, and nothing in the stack notices.

What it measures

Time-to-first-token on three document blocks of 1,948–3,437 tokens:

Read the absolute numbers, not the percentages

The two savings are 19.4 ms and 18.7 ms. They are the same number. Skipping the tokenizer removes a roughly constant ~19 ms, and the percentage moves only because the denominator does — the smaller model's forward pass is shorter, so the same fixed cost is a bigger share of it. Banerjee says as much ("tokenizer cost… is a larger fraction"), but the table is the part that gets quoted, and 37.8% reads like a scaling win rather than a constant.

Against the pipeline's own clock the constant is small: the full three-agent run takes 41.3 seconds, so the tokenizer work this removes is well under a tenth of a percent of end-to-end time. The win is real and it is a TTFT win — first-token latency in an interactive or chained setting — not a throughput win, and the article does not claim otherwise.

The guardrail is the contribution

Feeding a model integers minted under a different vocabulary does not fail loudly: it "generates a fluent, coherent-looking, completely wrong report." So before any agent trusts another's integers the pipeline compares the full ~151,936-entry get_vocab() dictionaries for equality — "not a vocab_size comparison, the real thing" — and refuses to run on mismatch.

That check is the price of the optimization and defines its blast radius. Token IDs are only portable inside one vocabulary, so the handoff works within a model family and stops at its edge. Banerjee verifies equivalence for "the exact three checkpoints this repo pins, not a family-wide standing guarantee."

Where it does not reach

The author's own list: only few-thousand-token blocks were tested, the agents run as three sequential OS processes to force CUDA memory release rather than concurrently, everything sits on the transformers API with no custom kernels, and /dev/shm pins every agent to one GPU host. Semantic fidelity is checked by two crude heuristics (printable-character ratio ≥ 0.98, unique-word ratio ≥ 0.25) rather than by comparing outputs against the baseline runs.

Cross-spoke context

• OKF is the wrapper, not the mechanism. research-wiki/open-knowledge-format is Google Cloud's spec for knowledge as a directory of markdown files, and that spoke owns it. Banerjee borrows its shape and admits his bundles are "not valid Google-OKF bundles as-is" because token_pointer is his own field. Useful over there as the format's first sighting outside its own repo; useful here only as the envelope the array pointer travels in.
• MCP and A2A appear in the scenario the agents reason about — a 5G Service-Based Interface — not in the pipeline itself. No agent protocol is used; the exchange is a NumPy file and a YAML key. ../agentic-tooling-wiki owns those protocols.
• qwen (../llm-providers-wiki) is the model family; linked, not re-paged here.