llm-abliteration (NousResearch fork)

A Python toolkit that turns the finding in arditi-refusal-direction into a weight edit. It measures the activation direction that separates "harmful" prompts from "harmless" ones, picks layers where that direction is clean, and orthogonalizes the model's weights against it so the network can no longer write along it. The README's tagline is "Make abliterated models with transformers, easy and fast." Held here as an interpretability result used as an engineering tool: the causal direction is the science, the edit is the product. The technique itself is paged in llm-providers-wiki as abliteration; this page covers one implementation of it. Artifact facts below are T1 (the repo's own README); what the edit does to a model is the README's claim and nothing here measures it.

What the repo is

A public fork of jim-plus/llm-abliteration, hosted under nous-research. At fetch time: 150 stars, 24 forks, 155 commits on main. The README gives no licence name, no dates and no release tags, so no published: is recorded.
Pitched as a proof of concept that works without TransformerLens, using plain transformers with GPU acceleration and batch inference. It requires CUDA and a model that fits in VRAM; the measurement pass can run on a 4-bit or 8-bit bitsandbytes quant, but the ablation must be applied to full-weight models.
Credits the original paper ("Refusal in LLMs is mediated by a single direction"), Orion-zhen's abliteration, Sumandora's remove-refusals-with-transformers, FailSpy's abliterator, AUGMXNT/deccp and huihui-ai.

The pipeline

Measure (measure.py): run harmful and harmless prompt sets, record activations, write a measurement file. Options include a quantised load, custom prompt files (.txt, .parquet, .json, .jsonl) and a --deccp set that adds topics Chinese-origin models decline.
Analyze (analyze.py -c): chart per-layer quality of the direction. The README suggests middle to late-middle layers as candidates.
Configure: a YAML file picks one or more source layers and a strategy per destination layer.
Ablate (sharded_ablate.py): apply the edit shard by shard.
Test (chat.py, compare.py): inherited scripts the README itself flags as needing updates.

Where it differs from plain abliteration

Three switches, each independent; with none on you get the conventional method.

--projected orthogonalizes the refusal direction against the harmless direction at measurement time, so the part of the refusal vector that merely tracks ordinary content is not removed.
A projected variant of the removal itself during ablation.
--normpreserve keeps each weight's norm unchanged during the edit.

The README points to Jim Lai's (grimjim) Hugging Face write-up for "norm-preserving biprojected abliteration" and to maxime-labonne's article for the general method. These are the same collateral-damage mitigations abliteration describes as the method's defining trade: removing the direction wholesale costs capability, and every refinement tries to pay less.

It also records a detail that matters for reproducing results: implementations read activations at different points. The original code used resid_pre, resid_mid and resid_post; FailSpy's used resid_pre and resid_post; Sumandora's transformers version uses hidden_states, which is resid_post. A "refusal direction" from one tool is not automatically comparable to one from another.

Evidence and caveats

Tested on (the README's list, no scores attached): Llama-3.2, Qwen2.5-Coder, Ministral-8b, Mistral-7B-Instruct-v0.2, gemma-3-27b-it, Mistral-Nemo-Instruct-2407, plus an example config for gemma-3-12b-it. "Tested" means it runs; the repo publishes no refusal rate, perplexity or benchmark before and after.
The README says abliteration "does not guarantee full removal of censorship": a good run stops explicit refusals only for the kinds represented in the prompt sets used. That is a limit on the direction's generality, in line with the one-dimensional-subspace finding being a statement about measured refusals, not about every way a model can decline.
The bundled prompts are described as illustrative only.
Compare heretic, which automates the same family of edit with a search over parameters rather than hand-picked layers.

Why it sits in this spoke

It is a clean example of the model-science corner producing a tool: a causal direction found by contrast (as in induction-head work, where a mechanism is isolated by ablating or adding it) is used to edit weights with no training. See synthesis for how it bears on the question of what a trained network is doing inside.