Tokenmining

Tokenmining (Sam Black, Towards Data Science, 2 July 2026) is the named discipline of minimizing token consumption while holding or improving output quality — the deliberate inverse of "tokenmaxxing" (spending tokens freely). Tier T3 — a practitioner how-to with anecdotal cost figures ($40/day personal spend; "some engineers $10k+/week"), not a controlled study. It crystallizes the cost/token levers this spoke had been tracking scattered across headroom, tail-control-agentic-workflows, claw-code, and claude-apps-gateway into one framing.

"Tokenmaxxing" does not hold still (2026-09-01)

This page's framing depends on tokenmaxxing meaning spend tokens freely, since tokenmining is defined as its inverse. inferoa ships under the tagline "Inference-native Tokenmaxxing Agent Harness" and means the opposite by it — shaping each turn "to preserve cacheable prefixes, bound mutable context, expose token pressure," with stack targets that read "spend fewer prompt and tool-output tokens." That is Black's discipline wearing the name of the thing it corrects.

Both readings are kept. Which is dominant is not something two sources can settle, and the practical consequence is small but real: the contrast that makes tokenmining legible as a name stops working if the word it inverts has flipped.

The thesis — more tokens ≠ better

Black's core claim is that maximizing token use is a false proxy for quality — "like ranking engineers by lines of code." Beyond cost and latency, quality itself declines: context rot degrades effectiveness at large context sizes and attention deteriorates in the middle of long prompts (lost-in-the-middle), so irrelevant context actively competes for the model's focus. Token discipline is therefore a quality argument, not only a cost one — the generative-side echo of the retrieval-side finding in ../research-wiki's long-context-vs-chunking (a longer window is quadratically expensive and usually loses to cheaper segmentation).

The two levers

Routing (an LLM gateway). Classify each prompt by intent and complexity, then route simple requests to a cheaper or local quantized model and only hard ones to the frontier tier. Black reports ~60% cost reduction from routing alone. This is the same routing dial the spoke sees elsewhere pulled for different motives: tail-control-agentic-workflows downshifts to a faster model on a stall (latency), claude-apps-gateway routes through one identity with spend caps (governance/cost), and claw-code is provider-agnostic by construction — tokenmining pulls it for per-request cost/complexity fit.
Context compaction. Lossy summarization as an agent nears its token limit — preserving architectural decisions and key details while discarding redundant tool outputs, with structured memory extraction to avoid hallucinating during compression. This is the runtime cousin of headroom (compress tool outputs/logs/RAG before the model, 60–95% fewer tokens) and the flip side of memory-vault (externalize the context otherwise lost to auto-compaction).

Why it's here

It gives the spoke's token-budget / progressive-disclosure thread a name and a why: progressive disclosure (agentskills-spec), token-efficient DSLs (openui), and pre-model compression (headroom) are all tokenmining moves, and "context rot / lost-in-the-middle" is the mechanism that makes them quality-preserving rather than merely cheap. It sits alongside agentsys's "structure substitutes for capability" (Sonnet+harness > raw Opus on cost) as the cost face of the spoke's through-line: the value is in the disciplined layer around the model, not in spending more tokens through it.

Cross-spoke

../llm-inference-wiki owns the mechanism the quality argument rests on — attention cost and the context-window internals behind context rot / lost-in-the-middle. ../research-wiki's long-context-vs-chunking makes the same "long context isn't free or better" case on the retrieval/embedding side. Here the subject is the agent/LLM-app cost discipline (routing + compaction); the mechanism and the retrieval angle are cross-linked.

headroom · tail-control-agentic-workflows · claw-code · claude-apps-gateway · memory-vault · agentsys · cache-aware-model-routing · synthesis