Fine-tuning

Adapting an already-pretrained model to a task by continuing to train it on a smaller, higher-quality dataset. Pretraining teaches next-word prediction across a large corpus; fine-tuning moves the weights toward one job finetuning-explained-kdnuggets.

Two ways to do it

Full fine-tuning — every parameter updates. Memory-hungry, and risks catastrophic forgetting: the model loses general ability while gaining the specific one.
PEFT (parameter-efficient fine-tuning) — freeze the base weights, train a small set of new ones. LoRA and QLoRA (quantized base plus low-rank adapters) and prompt tuning are the named methods finetuning-explained-kdnuggets.

PEFT is what makes the memory claims in unsloth-amd-support possible: an 8 B model fine-tuned on consumer hardware is an adapter being trained, not the model.

The adapter can become an object in its own right. skillsmith takes a PEFT artifact — prefix-tuning weights — and treats it as data: an LLM reads existing prefix weights plus a text description and synthesizes new prefix weights for a target skill. In ordinary PEFT the adapter is something you train and load; there it is both an input and an output of another model. The reframing matters for this page because it points past "train one adapter per task" toward composing existing ones — an early, preprint-stage idea, held with the caveats on its own page.

Don't reach for it first

The most useful line in the founding explainer is a caution against its own subject: better prompting or retrieval may solve the problem without training anything, and the techniques combine rather than compete finetuning-explained-kdnuggets. The retrieval branch of that fork is paged in ../research-wiki as retrieval-augmented-generation.

Unresolved here: the corpus states the decision rule and gives no basis for applying it. Nothing in it says when prompting stops being enough: no task taxonomy, no data-volume threshold, no cost comparison. An open question for synthesis.

The limit case (2026-08-03). timesfm is what "don't fine-tune first" looks like taken all the way: a pretrained forecaster meant to be used zero-shot on series it has never seen, in a field whose classical answer was to fit a fresh model per series. Adaptation isn't skipped because it's premature — the product claim is that it isn't needed. Whether that holds against a per-series ARIMA is exactly the evidence time-series-forecasting says the corpus doesn't have.

The corpus's first RL post-training curriculum (2026-08-31)

Until now RL post-training appeared here only as a line item — unsloth supports it, and unsloth-amd-support timed a GRPO step at 1.75 s. code-as-world-report is the first source that says what an RL stage is for and shows the result of running one.

The recipe behind code-as-world-vl is two phases, and the split is the point:

SFT teaches the measurement. Supervised fine-tuning on pixel-level questions built from existing bounding boxes and object tracks — where things are, how big, how fast in pixels.
GRPO teaches the calibration. Reinforcement post-training on world-space questions whose answers come from simulated worlds (executable-world-representation), where the ground truth is exact.

So SFT is used where a large corpus of cheap derived labels already exists, and RL where the labels are few, exact and expensive to manufacture. That is a reason to choose a stage, which the page's "unresolved" note above has been asking for from the other direction. It is one data point from a first-party report, not a rule.

The reported effect is worth recording precisely because it is the modest kind: the 4B goes from 31.2 to 50.6 on quantiphy against its own Qwen3.5-4B base after both phases, and the full models beat their phase-one Image-Space checkpoints on every image-space benchmark too — the second stage did not cost the grounding the first one bought. The report does not ablate the phases against each other, so how the gain divides between SFT and GRPO is unknown.

RL for calibration, a second instance (2026-09-24)

laya trains a small decision head on top of a frozen-then-unfrozen encoder with a recipe that rhymes with code-as-world-vl's second stage: REINFORCE-style updates, GRPO-flavoured (group-mean baseline), against a reward built from a strictly proper scoring rule rather than plain correctness. Under a proper scoring rule, honest probability reporting is the only policy that maximises expected reward, so the RL stage is aimed at calibration directly instead of accuracy as a proxy for it. Different domain — typed classification questions, not simulated-world measurements — but the same shape: GRPO teaches calibration, twice now, in two unrelated projects three weeks apart. Laya's own card reports the effect size (mean ECE 0.466 → 0.081 after also adding temperature scaling), self-measured, and the ablation between the RL stage and the temperature-scaling step is not separated out any more than code-as-world-vl's SFT/GRPO split was.

A same-day counter-case: calibration without the RL. open-jev-9b targets the identical typed- decision interface — a LoRA adapter plus a scalar decision head, initialised from a base model's own Yes-minus-No readout — and its card never names REINFORCE or RLCD, only "trained jointly with LoRA." The wider project documents a sibling checkpoint trained with a supervised soft cross-entropy plus a Brier-score term instead — a proper-scoring-rule shaped loss, not a proper-scoring-rule shaped reward. If that recipe extends to the checkpoint this spoke holds, it's a third data point for calibration-as-training-target, and evidence the policy-gradient step in RLCD/GRPO may not be load-bearing for calibration specifically, only for the harder cases the reward formulation was built to handle (multi-turn credit assignment via TD(λ), not one-shot classification). Unconfirmed — the 9B card doesn't state its loss function outright.

Adapters and precision — a live gotcha

Serving a LoRA adapter by merging it into the base weights (W + sAB) and subtracting it later to resume training is not safe in BF16: rounding is not associative, so the base weights drift across merge/de-merge cycles. Unsloth avoids the round trip entirely and serves through vLLM's LoRA path unsloth-amd-support. A numerical-precision constraint on the whole adapter workflow, not a quirk of one framework.

Tooling

Hugging Face PEFT, TRL, Unsloth and Axolotl are the named implementations finetuning-explained-kdnuggets. Only unsloth is paged so far.

unsloth · machine-learning · classical-ml-algorithms · timesfm · · lfm2-5-230m-chess time-series-forecasting · finetuning-explained-kdnuggets · unsloth-amd-support · code-as-world-vl · code-as-world-report · laya · open-jev-9b · synthesis