Safe prompt pruning (Emmimal P Alexander, TDS)

A Towards Data Science build-log by Emmimal P Alexander (July 2026) describing a deterministic prompt-pruning layer that strips redundant state out of a long-running conversation before it reaches the model. It’s the spoke’s clearest instance of a fourth token-budget lever: removal, distinct from compression (headroom) and modality arbitrage (pxpipe).

The problem

Long agent/RAG conversations accrete context that never gets cleared — stale tool outputs, duplicate RAG retrievals, superseded preferences. The bloat costs money and latency and, the author argues, degrades reasoning (the same context-rot / lost-in-the-middle case tokenmining makes). The naive fix — keep the last N messages — silently drops facts a later turn still needs. Pruning has to be selective, and safe.

The mechanism — a deterministic three-pass pipeline

Each message carries metadata (role, content, turn number, optional dependency markers). No LLM, no embeddings — only the standard library (dataclasses, regex, dict lookups):

1. Expired-context elimination. When a tool is called repeatedly with the same key (same search query, same SQL lookup), only the newest result is trustworthy; earlier ones are dropped.
2. Duplicate-context elimination. Near-duplicate retrieved documents are detected by normalizing whitespace/casing and only the first occurrence is kept.
3. Dependency restoration — the safety pass. Messages mark themselves with DEFINE tags; later ones reference them with REF tags. If an earlier pass removed something a downstream message depends on, this pass puts it back.

Why “safe”

Determinism is the whole pitch: same input, same output, no model in the loop to hallucinate about what to keep, and idempotence (running the pruner twice yields identical output) so it’s safe to re-run every turn. Dependency tracking is what stops passes 1–2 from silently breaking a conversation chain.

Results (synthetic, self-reported)

Three synthetic workloads across five sizes (50–2,000 turns):

• Normal chat — 2–4% token reduction (little repetition to exploit).
• RAG assistant — 27–32% (overlapping document retrieval).
• Tool agent — 33–34% (repeated tool calls).

All 15 configurations preserved 100% of labeled required facts, and processing stayed under 50 ms even at 131k tokens / 2,000 turns.

The honest caveat

The author admits the dependency-restoration pass was initially untested and never actually fired: the synthetic corpus only attached dependency markers to plain user messages, while pruning only removed tool outputs and documents — categories that never overlapped. After fixing the benchmark so tool outputs carried dependencies, restoration activated and caught 2–127 messages per configuration. A useful reminder that a safety mechanism you never exercise is not a safety mechanism.

Why it matters here

The spoke’s token-budget thread now has a clean taxonomy of how you cut tokens before the model: compress what stays (headroom, reversible content-typed compression), re-count it via a cheaper modality (pxpipe, render-as-PNG), or remove what’s redundant (this — deterministic pruning of expired/duplicate state). tokenmining frames its second lever as “context compaction” via lossy summarization; this is the lossless, deterministic cousin — nothing is summarized, only provably redundant messages are dropped, with dependencies restored. That determinism is itself the spoke’s recurring structure-over-tokens discipline pointed at context management: no LLM call, no embedding, reproducible, idempotent — cheaper and more auditable than a model-judged compactor, at the cost of only catching exact redundancy (same tool key, near-identical docs), not semantic overlap. (T3: a single practitioner’s synthetic self-benchmarks, no independent validation.)