Abliteration

Editing a model's weights to suppress its refusal behaviour, rather than retraining it or prompting around it. The name is a portmanteau of ablate + obliterate. It presumes that refusal is carried by an identifiable direction in activation space — find it, remove it from the weights, and the model stops declining.

The presumption is a published finding, not a folk theory: Arditi et al. 2024 show that across 13 open chat models refusal is mediated by a one-dimensional subspace, and that the direction both suppresses refusal when erased and causes it when added.

The technique only exists where weights are downloadable, which makes it a property of the open-weight tier specifically. A closed model's refusals can be routed around (claude-refusals-and-fallback) but not edited out.

The trade it is named for

The standard complaint about abliteration is collateral damage: removing the direction wholesale costs coherence and general capability, because the direction was never carrying refusal alone. Every refinement of the method is an attempt to pay less of that cost.

scotoma is one such attempt — a Jacobian-lens projection that keeps ~22% of the abliteration magnitude and re-applies it at 1.5× across a layer band, on the theory that the discarded 78% was never refusal. Whether that works is unverified; the model ships with no evals.

How the edit is actually made

Per heretic, the standard recipe is two steps and no training. For each layer, compute a residual direction as the difference of means between the first-token hidden states over a set of "harmful" prompts and a set of "harmless" ones. Then take the layer's attention out-projection and MLP down-projection matrices and orthogonalize them against that direction, so nothing multiplied through them can express it again. The weights change; the architecture, the tokenizer and the training data do not.

Everything that distinguishes one abliteration from another is a choice on top of that: which layer's direction, how hard to apply it, over which band of layers, and whether attention and MLP get the same treatment (they should not — MLP interventions do more damage).

The cost, finally with a number

heretic makes the trade measurable by putting both halves in one objective: refusal count on harmful prompts, and KL divergence from the original model on harmless ones. On gemma-3-12b-it — which refuses 97 of 100 harmful prompts unedited — three abliterations all reach 3/100 refusals, at KL divergences of 1.04 (mlabonne v2), 0.45 (huihui-ai) and 0.16 (Heretic's automated search). Same suppression, sixth of the collateral damage.

Two things follow. Refusal suppression is not the hard part and has not been for a while; the whole competition is over the second number. And KL divergence is a proxy — it says the output distribution moved less, not that the model can still reason. No capability eval accompanies those numbers in the source.

The interpretability seam

Abliteration is downstream of mechanistic interpretability: it is what you do once you believe you have located a behaviour inside the network. The instruments are shared — the scotoma edit uses Anthropic's Jacobian lens, a technique built for auditing models, to modify one. A tool for finding out what a model is doing is also a tool for changing it, and the second use needs no permission from whoever built the first.

heretic ships both halves in one binary and labels the auditing one "research": install the extra and --plot-residuals gives you PaCMAP projections of the harmful/harmless residual clusters layer by layer, animated, while --print-residual-geometry prints cosine similarities, norms and a cluster silhouette coefficient per layer. The decensoring tool is also the best interpretability instrument for the thing it decensors, because it has to be — you cannot delete a direction you cannot see.

From expertise to pip install

The 2024 paper describes a white-box jailbreak that needs someone who understands residual streams. The 2026 tool describes itself as usable by "anyone who knows how to run a command-line program," takes 20–30 minutes on a 4B model on a consumer GPU, and has been used to publish well over 5,000 models on Hugging Face. Nothing about the underlying safety training got weaker; the tooling around it got finished. That is the fact this page is now really about, and it belongs beside open-weight-models's point that a permissive licence is also permission to rewrite.

What the word has come to mean outside this page (2026-09-04)

The term has escaped its definition. uncensored-ai-rekhi, a widely-read popular account, uses "abliteration" for safety removal in general — and quotes a description of it as "a method that erodes safety alignments via targeted fine-tuning." On this page's own primary sources that is simply wrong: heretic does the edit with no post-training and no training at all, which is the whole reason it takes 20–30 minutes on a consumer GPU rather than a fine-tuning run.

The distinction is worth defending because the three things now travelling under one word have different requirements and different defences:

Only the first two are properties of the open-weight tier. The third works against hosted frontier models and says nothing about open weights at all — which matters, because the popular argument that open weights are the problem gets much of its force from evidence drawn from the third row. uncensored-ai-rekhi stacks all three: a fine-tuning attack in which one benign prompt ("create a fake news article…") reportedly made 15 models more permissive across all 44 SorryBench categories, and Tenable's multi-turn "crescendo" escalation against a hosted GPT-5, sit in the same essay as the abliteration it opens with.

The popular account also drops the cost. In it, the technique yields "a model that simply doesn't have the machinery for refusal anymore" — a clean subtraction, no trade named. That is the opposite of what the section above records, and the disagreement is not a detail: the entire refinement history of the method, from mlabonne's v2 at 1.04 KL through Heretic's automated search at 0.16, exists because the subtraction is not clean. Nobody neutral has measured what it costs, so the corpus cannot settle it — but it can note that the version circulating outside the field asserts the answer without measuring either.

A number arrives from the other side (2026-09-06). Chasing the essay's citation to Microsoft's GRP-Obliteration work (Microsoft Security Blog, 2026-02-09) turns up the first figure in this corpus that grades abliteration against an alternative and is not published by anyone who made an abliterated model: an average score of 81% for GRP-Obliteration, 69% for abliteration, 58% for TwinBreak, with utility reported "within a few percent of the aligned base model." Read it for what it is — a comparison of attack success, run by the party proposing the winning attack. It does not answer the capability question below, and the fine-tuning route beating the weight-editing route on permissiveness says nothing about what either costs the model.

Gaps

The mechanism is now sourced to a primary paper (arditi-refusal-direction, T1) and to a working implementation (heretic, T1 for its method). What remains missing is independent capability evaluation of abliterated models: every effectiveness number in this corpus is self-reported by the party that made the edit. Heretic's author points at third-party MMLU/GSM8K comparisons on Reddit; they are unread here.

The second gap, opened 2026-09-04, is the fine-tuning lineage the page has never covered. Eric Hartford's Dolphin models are named by uncensored-ai-rekhi as heavily-downloaded uncensored fine-tunes, and nothing in this corpus describes how that route differs from weight editing in cost, in capability retained, or in how a defender would tell them apart. A source on Dolphin — or on ../machine-learning-wiki's side of the seam — would close it.

Related

heretic · arditi-refusal-direction · gemma-4-31b-it-scotoma · open-weight-models · claude-refusals-and-fallback · readyart · gemma-4 · philipp-emanuel-weidmann · uncensored-ai-rekhi · maxime-labonne · dark-llm-services · llm-abliteration