Introducing Lev (Dmitri Sotnikov, 2026-09-24)

Launch post for lev, written by the tool's own author, Dmitri Sotnikov (dmitri-sotnikov, known online as yogthos). Arrived by Telegram 2026-09-28 (sender: Lucsw). Read directly from the blog. The post pitches Lev as an open, offline answer to TypeSafe's hosted Jev (system-one-models-and-jev): same wire shape — free text plus typed questions in, typed answers with calibrated probabilities out — run on local hardware with open weights instead of a paid API call.

The problem it names in Jev

Jev returns a probability with every answer but cannot say how sure it should be: a single forward pass through an encoder either gets a question right or wrong, with no mechanism to try harder on the ones it gets wrong. The post cites independent numbers for that gap — Jev abstaining on only 49.7% of forced-uncertainty items against 97–100% for LLMs, and a 0.246 worst-case calibration error, 0.480 on the DAIR Emotion set with zero probability assigned to the true label 16% of the time — and treats "sometimes wrong with total confidence" as the failure mode worth designing around, not raw accuracy.

Lev's answer: escalate, don't just classify

Lev keeps the fast encoder for the easy majority of questions and adds an escape hatch: when the encoder's own confidence falls under a threshold, the question re-runs on a local LLM that scores the same candidate answers by token log-probability rather than generating free text. On the post's own 144-case adversarial three-way set:

Where the set is easier — a 120-case mix of AG News/BoolQ/SST-5 — everything stayed on the encoder at 65.8% and ~0.13 s/case, so the gate's cost is paid only where the questions are genuinely hard. The post also names its own blind spot: a yes/no ("noul") question's confidence is max(p, 1-p), which can never read below 0.5, so a naive single threshold never escalates a single BoolQ case (72.5% accuracy, un-escalated, even at a 0.7 threshold). The fix stated on the page is a separate confidence threshold per question type, not one global cutoff.

Calibration, debiasing, constraints

• Temperature-scaling refit: a tool that fits one temperature per question-type/option-count bucket from the operator's own labeled traffic, minimizing negative log-likelihood without changing any answer. Applied to laya's published numbers it takes mean calibration error from 0.466 to 0.081; on Lev's own buckets, 20-way-choice error falls from 0.57 to 0.10.
• Debias mode: repeats a wide-choice question once per rotation of its option order and averages the probabilities — a direct answer to option-order sensitivity, measured on the post's own sets at about 13% of Jev's choices and about 14% of Lev's encoder choices flipping under a reshuffle.
• Constraints decoder: a post-pass that links related typed questions so contradictory answers can't both stand — the post's example is a phishing-flagged email that the constraint forbids also labeling "needs reply."

Architecture and packaging

The fast tier routes by content and language across three checkpoints from Convai Innovations's laya (Apache-2.0, ModernBERT-large and mmBERT-base backbones) — reused directly rather than retrained. The slow tier is any GGUF chat model through llama.cpp, statically linked; the post's default is Qwen3.5-4B (Qwen). A named fork by GitHub user thecodacus caches the base instructions and answer schema in the KV cache once at load time and scores every field of a batched question in one forward pass, closing most of the speed gap to the encoder while still needing a GPU. The whole stack is written in Jolt — Clojure compiled to Scheme, with a native FFI the post contrasts favorably with Project Panama's ceremony — and ships via jolt binary: a single statically-linked executable with llama.cpp built in, deployed by copying it next to the model weights, no interpreter or container required.

What Lev gives up

The post is direct about the trade: the decision space has to be declared up front (no open-ended generation, no summaries or drafts), the English encoder checkpoint caps context at 512 tokens with silent end-truncation past that, and a calibration refit needs upkeep as traffic drifts. Against that, Jev keeps auto-classification, a much larger context for sorting unstructured data, and a built-in escalation path to hosted "System Two" models Lev does not attempt to reproduce.

Tier and provenance

T3 — the maker's own launch post for his own open-source project, reporting benchmarks he ran himself on sets he built himself (the 144-case adversarial set, the AG News/BoolQ/SST-5 mix). The architecture description is checkable against the linked repository; the accuracy and calibration numbers are not independently verified anywhere in this ingest. No license is stated for Lev itself in the post (only the reused laya checkpoints carry one, Apache-2.0).

Cross-spoke context

The router's runner-up was ../agentic-tooling-wiki: Lev is a harness a coding agent could call for in-workflow typed decisions, the same shape as jev-code-reviewer's use of TypeSafe's Jev for PR triage. It is filed here instead because the post's substance is inference architecture — a confidence-gated two-tier pipeline, GGUF log-probability scoring instead of generation, static llama.cpp packaging — not an agent-tooling harness; nothing in the post is about coding-agent review workflows. The Jev/System-One ecosystem this post responds to already has pages in two other spokes: system-one-models-and-jev and typesafe-ai (../llm-providers-wiki), and laya, open-jev-9b, typed-decision (../machine-learning-wiki). This page adds the inference-mechanics angle — confidence gating, calibration refit, option-order debiasing — that none of those cover.

Related

lev · dmitri-sotnikov · model-routing · llama-cpp · local-llm-stack · quantization · laya · system-one-models-and-jev · typesafe-ai · qwen · jolt-clojure · typed-decision