Synthesis — Machine Learning

The evolving thesis. Spun out of the hub _inbox on 2026-07-30 from five sources parked under four different tags across two months, clustered on the human’s instruction after the router flagged that 29 spokes existed and none of them owned machine learning.

Current thesis

The hub’s ML strays scattered because a model’s life is split across spokes, and the split had no centre. Serving mechanics had a home, the provider market had a home, agent frameworks had a home — but the model itself, how it is fitted, adapted and designed into a product, did not. Each stray was correctly declined by the spoke nearest it, and four separate tags accumulated as a result. That is what a missing spoke looks like from the inside.

The corpus that fell out of it covers three layers of one discipline:

• Algorithm — the fitting procedures, in code, from regression to LSTMs mlalgorithms.
• Adaptation — taking someone else’s pretrained model and turning it to a task (fine-tuning, PEFT/LoRA), with one explainer for the concepts finetuning-explained-kdnuggets and one toolkit release for the costs unsloth-amd-support.
• System — what gets built around a model in production ml-system-design, as 300+ shipped case studies ml-system-design-case-studies and one headline about the ones that didn’t ship ai-projects-fail-infrastructure-people.

A floor under the three layers

The sixth source is not a fourth layer — it is underneath all three. information-theory-inference-learning-algorithms arrives as the corpus’s first textbook, first T1, and first source that explains why fitting works rather than how to run it: the loss a network minimizes is a log likelihood, its regularizer is a log prior, and what learning produces is an ensemble of parameter values rather than a point (bayesian-inference). Model complexity is priced by the evidence without anyone adding a penalty term. The same message passing that marginalizes a probabilistic model decodes an error-correcting code (information-theory).

It changes the corpus’s shape in three ways.

• The evidence spectrum now has a top. Five sources of vendor posts, catalogs, explainers and one unread headline, and now a university-press textbook with derivations and worked solutions. The other five are about the present; this one is about the mathematics, which is why it can be twenty-three years old and still be the strongest thing here.
• It half-fills the middle gap, from the wrong end. The gap was training a model from data end to end — data, labelling, pretraining, evaluation. MacKay supplies the theory of fitting and model comparison and nothing at all of the practice: no data pipeline, no compute, no cost. The hole is now specifically empirical, not conceptual, and that is a sharper statement of it than before.
• It splits the corpus’s centre of gravity. Every earlier source is engineering under time pressure. This one has no cost model and no deployment, and its neural-network chapters stop a decade before the scaling era — Hopfield nets and Boltzmann machines, no ReLU, no attention. The theory is durable, Part V is a historical document, and the spoke should not let the two be read as one.

The founding gap — nothing in the middle

The corpus jumps from how a fitting algorithm works to how to adapt a finished model, with no source on training a model from data end to end: no pretraining, no data collection or labelling, no evaluation methodology. That absence is itself informative about where practitioner attention has gone — adapting a pretrained model is now the common case, and building one from scratch is not — but it is a hole in the corpus before it is a finding about the field. First priority for the next ingest.

One third of it closed, 2026-08-03. timesfm-2-5-forecasting-tutorial supplies the evaluation methodology the gap named: a baseline that isn’t zero (seasonal-naive, and MASE which bakes the comparison into the metric), rolling-origin backtesting across folds instead of one holdout, scoring the interval with pinball loss and coverage rather than only the point forecast, and ablating a configuration knob against both accuracy and cost (time-series-forecasting). None of that is specific to the model under test, which is why it counts as method rather than result. Pretraining and the data/labelling half of the gap are still unsourced.

A second third, 2026-08-04. transformers-tutorials supplies the training loop — not described, executable: fine-tuning on real datasets across several dozen models, written three ways (native PyTorch, PyTorch Lightning, the Trainer API) with preprocessing guidance and a stated basis for choosing among them. Until now fine-tuning was a page of concepts from an explainer with no measurements, and unsloth was a tool that hides the loop; this is the loop itself.

So the founding gap has closed from both ends and the remaining hole is specific: data collection and labelling, plus pretraining proper. Worth naming what the pattern of closure shows. The evaluation methodology arrived inside a tutorial for one model, and the training loop arrived inside a tutorial collection for many — both times as method carried incidentally by teaching material, never as a source about the pipeline itself. The middle layer appears to have no literature of its own in this corpus’s reach; it is documented only where someone was demonstrating something else.

That also caps what either source settles. Neither reports a number. Knowing how to run a training loop and knowing whether the result is any good remain separate acquisitions, and the corpus keeps paying for that lesson one source at a time.

The last third closes 2026-08-08, and the reading above was wrong. data-cascades (Sambasivan et al., CHI 2021) and pervasive-label-errors (Northcutt et al., NeurIPS 2021) supply data collection and labelling — both peer-reviewed, both about the data layer itself rather than about a model that needed data. So the founding gap is closed except for pretraining proper.

The correction matters more than the closure. This page claimed the middle layer “appears to have no literature of its own in this corpus’s reach” and is “documented only where someone was demonstrating something else.” False. It has a literature; it sits in HCI and in the datasets-and-benchmarks track, studies practitioners and labels rather than architectures, and no amount of searching the ML side would have returned it. The absence was in where this wiki looked.

What the two say, briefly. Data Cascades interviewed 53 practitioners in high-stakes domains and found 92% had hit a compounding data failure, 45.3% more than one in one project — failures seeded upstream and surfacing late. Pervasive Label Errors measured the labels: at least 3.3% wrong on average across ten standard test sets, at least 6% of ImageNet validation, and correcting them can flip a ranking (ResNet-18 over ResNet-50, VGG-11 over VGG-19). Both are consolidated at training-data-quality.

The second result reaches back into this page’s own method. The evidence spectrum below sorts sources by what they can establish and treats a neutral benchmark as the strong rung. A neutral benchmark on a 6%-mislabeled test set is weaker than that ordering implies — the ruler has a measured error rate. Growth edge 2 (no independent benchmark) is not retired but its bar moves: what would close it is a benchmark that states its label quality, and almost none do.

The evidence spectrum, stated up front

Three of the six carry a structural bias worth naming, in different directions. The case-study catalog is survivorship — every entry is a company describing a system that worked, so the corpus cannot say how often those same designs failed. The AMD post is first-party benchmarking — the vendor chose the hardware, the baseline and the configuration. The TimesFM tutorial adds a third and sharper form: synthetic data, where the author chose the problem. A vendor benchmark at least runs on a real workload; a forecast evaluated on a series built from a weekly cycle plus noise measures whether the code executes.

The pretrain-once move arrives in a second modality

timesfm is a 200M-parameter foundation model for numeric sequences, used zero-shot, in a field whose classical answer was to fit a fresh ARIMA or ETS per series. That is the language-model pattern — pretrain once, apply everywhere, adapt rarely — landing on a task that already had decades of per-series statistics to displace.

It sharpens this corpus’s “don’t fine-tune first” rule (fine-tuning) into a stronger claim: not adapt later, but don’t adapt. And it makes forecasting the clearest place in the hub to watch whether the pattern generalizes past text, because the comparison is unusually clean — the classical baselines are cheap, well understood, and decades old.

Nothing here settles it. The single source measured the model on data it invented, against one naive baseline, with no ARIMA, no exponential smoothing and no gradient-boosted tabular comparison (time-series-forecasting). The corpus now holds a method for answering the question and no answer.

A fourth layer arrives, and it is after the model

supervision does not fit the three layers, and that is the interesting thing about it. It is not an algorithm, not adaptation, and not system design — it is the code that runs once a vision model has produced its output: a common detection structure, annotators, multi-object tracking, dataset format conversion, and detection metrics. Nothing in it touches backprop.

It stretches this spoke’s declared boundary (train-time and design-time) and was still routed here, because the alternatives fit worse: ../llm-inference-wiki is serve-time mechanism for LLMs specifically, and ../dev-tooling-wiki declines application-level runtime libraries. Recorded as a stretch rather than a precedent — if serve-time vision tooling keeps arriving, the honest response is to widen the boundary deliberately or admit that computer vision as a modality is unowned across the hub, which it currently is.

What it contributes is evidence about demo-to-production-gap, which until now had been a claim with no mechanism: a success catalog with no denominator ml-system-design-case-studies and an unreadable headline about failures ai-projects-fail-infrastructure-people. Here is the gap itemized, in the form of what a company had to write and 48.7k stars’ worth of people needed. Format conversion between YOLO, COCO and Pascal VOC. Identity across frames. Drawing a box. Counting what crossed a line. Computing mAP.

That is a modest, one-sided piece of evidence — a library’s existence proves demand, not that projects fail without it. But it is the first source in the corpus that says anything concrete about what the distance consists of, as opposed to how wide it is.

Weights as a modality

skillsmith is the corpus’s first primary research on adaptation (every prior adaptation source was an explainer or a toolkit) and it arrives with an unusual stance. It treats model weights as a thing a model reads and writes: prefix-tuning stores a skill as parameters, and an LLM composes new prefix weights from old ones plus a text description, reportedly beating both text-only and weight-space-only baselines.

This is a third position on the seam this synthesis opened between the textbook and the practice. information-theory-inference-learning-algorithms says learning produces a distribution over parameters; the fine-tuning/unsloth toolchain ships the single point estimate and calls it training. SkillSmith does neither — it takes trained weights as objects to manipulate, making the parameters themselves the unit of analysis rather than the loss or the data. It does not settle the disagreement, and as a preprint on the authors’ own evaluation it is held loosely (T2, the domain’s benchmark caveat in full force). But it widens the corpus’s account of what “adapting a model” can mean: not only moving weights toward a task, but composing weights as reusable parts.

Note the standing homonym it sharpens: a parametric skill (prefix weights) is not an agent skill (a markdown prompt an agent loads) — the same week both senses arrived in the hub, in different spokes.

Training becomes a subject, and open question 6 gets a partial answer

Every source before this one treats training as something you run: a loop, a tool, a cost in VRAM. phase-transitions-in-attention treats it as something to explain — why a capability appears at the data size it appears at — and it is the corpus’s first T1 primary research rather than a textbook, tutorial or vendor post.

What it does. Marginalise the weights of a single-head attention layer trained on a copy task, get a closed-form posterior over attention patterns, and reduce it by the task’s permutation symmetry to two order parameters: attend uniformly, or attend one position back. The negative log-posterior is then an effective landscape whose minima say which pattern the data prefers, “separately from whether an optimizer reaches it.”

Open question 6 — “does the Bayesian account survive at scale?” — moves, without closing. bayesian-inference had a textbook saying learning produces an ensemble and a toolchain shipping point estimates, neither citing the other. This paper puts them in the same equation in one tractable setting: the posterior’s MAP is where Adam lands, because minimising the action at large χ corresponds to gradient flow with weight decay, and SGLD sampling agrees with Adam closely. The point estimate is the mode, not a different activity. What buys that result is a symmetry, not a scalable method — nobody integrated over billions of parameters — so the question survives in a sharper form: the correspondence holds where the posterior collapses to two numbers, and no one has shown what happens where it doesn’t.

The finding that travels furthest is about monitoring, not training. The order of the transition is set by the attention activation: softmax gives a first-order jump with no precursor, linear attention gives a second-order transition preceded by diverging fluctuations and then a gradual crossover. If that generalises, capability onset in the architecture everyone actually ships is “intrinsically unpredictable from observable quantities,” and there are “fundamental limits to what can be monitored” (capability-emergence). That is a claim about AI risk practice derived from statistical mechanics, and this spoke should hold it as a conjecture with a derivation behind it: one layer, one head, i.i.d. uniform tokens, a supervised copy task standing in for next-token training, and asymptotic approximations. Copying is the minimal circuit under in-context learning, not in-context learning.

It also sits oddly against the corpus’s evidence habits. Everything else here is unreplicated first-party numbers; this is a derivation whose numerics match its own theory. Both are single-source, and neither kind of weakness cures the other.

Open questions

1. How often do AI projects actually fail, and by what definition? A success catalog with no denominator on one side, an unevidenced failure claim on the other. Nothing measures the rate.
2. When is fine-tuning the right tool? The corpus states the rule — try prompting and retrieval first — and gives no threshold, task taxonomy or cost comparison for applying it.
3. Do the Unsloth speedups survive independent measurement? The identical loss curve is the load-bearing claim, and it is self-reported.
4. What does the middle layer look like? Pretraining, data, evaluation — see the founding gap. Evaluation now has a method (time-series-forecasting); pretraining and the data/labelling half remain unsourced.
5. Does the ranking/matching/forecasting concentration reflect the field, or only what companies blog about? The catalog cannot distinguish these.
6. Does the Bayesian account survive at scale? bayesian-inference says the product of learning is an ensemble; nobody samples a posterior over billions of parameters. Whether the point estimate everyone actually ships is an approximation with known cost or a different activity wearing the same vocabulary, no source here answers. Partly answered 2026-08-07 in one tractable case (phase-transitions-in-attention): the MAP of an analytically-derived posterior is where Adam lands, so there the point estimate is the mode. The reduction that made it computable was a permutation symmetry, so the open half is now specific — what holds when the posterior does not collapse to two order parameters.
7. Is capability onset detectable before it happens? (added 2026-08-07) One derivation says the answer depends on the activation function, and that softmax — the one in production — gives no precursor (capability-emergence). Nothing here tests that on a real training run, and the claim would matter a great deal if it held.
8. Do time-series foundation models beat per-series classical models? (added 2026-08-03) The corpus has timesfm evaluated against seasonal-naive on synthetic data and nothing else. The settling comparison is against ARIMA/ETS and a gradient-boosted tabular baseline on real series; the M-competitions are the standard venue and none of that literature is here. Highest-value next source in this corner, and the answer bears directly on question 2.

The tabular default finally has sources

The corpus argued about foundation models for a month with no page on the method a practitioner would actually use first. gradient-boosted-trees fixes that, and the two sources behind it say different things worth keeping apart.

xgboost is a story about systems, not statistics. Friedman’s algorithm was a decade old; what changed in 2016 was an implementation with sparsity-aware splits, a weighted quantile sketch and out-of-core blocks, hitting scikit-learn’s accuracy on Higgs-1M at roughly forty times the speed. The method became the default because it became cheap to try. That is the same mechanism this spoke keeps finding elsewhere — capability follows tooling — and it is worth noting that the dominance evidence (17 of 29 Kaggle winners) is competition results, a venue with its own selection effects.

tabular-deep-learning-benchmark is the durable half. Six years later, with 45 datasets and ~20,000 tuning hours per learner, tree ensembles are still ahead at ~10K samples, and the explanation is structural: uninformative columns, rotation invariance, irregular targets. Those are properties of messy real data, which lines this up with data-cascades — the deciding variable keeps turning out to be the data, and the method that tolerates bad columns keeps winning.

Open question 7 is now precisely stated and still unanswered. The baseline it needs exists here; the comparison against a time-series foundation model does not.

Tensions

Still no head-on contradiction — the sources don’t overlap enough to disagree, which remains a weakness in a six-source corpus.

The first real seam has opened, though, and it runs between the textbook and the practice. information-theory-inference-learning-algorithms frames learning as producing a distribution over parameters and treats the single most probable setting as a fallback for when the integral is intractable. Every other source in this corpus ships that fallback and calls it training — fine-tuning and unsloth are entirely point-estimate machinery, and the vocabulary of priors and evidence appears nowhere in them. Neither side argues with the other because neither cites the other. Filed as open question 6 rather than a contradiction: a twenty-three-year-old textbook and a 2026 toolchain disagreeing about what learning produces is a gap in the corpus’s coverage before it is a dispute in the field.

The nearest thing to a live tension is between ml-system-design-case-studies and ai-projects-fail-infrastructure-people: one says ML systems ship routinely across dozens of industries, the other that most AI projects fail. Both can be true (different denominators, different eras — the catalog stops at 2023), and neither source cites the other. Recorded as an unresolved question rather than a conflict, because the failure side is a headline and not yet an argument.

A boundary the hub drew and this spoke inherits: the parked ai-test-automation-paradox record argues that AI generating tests at scale mostly scales brittleness — the same demo-to-production shape as this corpus, applied to software testing rather than ML systems. It was deliberately left in _inbox rather than folded in here. If a third source lands on that seam, the boundary should be revisited rather than defended.

Growth edges

Ranked; each names the kind of source that would close it (see ../QUALITY.md → Growth edges).

1. The missing middle layer — partly closed 2026-08-04 by transformers-tutorials, which supplies the training loop as working code across three frameworks and several dozen models. The method half is answered. Data curation and evaluation are not — the repo teaches how to run the pipeline and reports nothing about whether the result is good. — needs: T1/T2 on data curation and on evaluation, which have separated out as edge 3 and are the live half of this one. Touched, not closed, 2026-08-05: supervision ships a dataset module (YOLO/COCO/Pascal VOC conversion) and a metrics module (mAP), so the corpus now holds curation and evaluation as installable tooling. It still holds neither as a result — nobody has reported what good looks like on a task. A library is where the answer would be computed, not the answer. CLOSED 2026-08-08 (research pass) by data-cascades and pervasive-label-errors, both T1, both about the data layer as a subject: a 92% cascade prevalence across 53 practitioners, and at least 3.3% wrong labels across ten standard test sets. Successor written as edge 1a. 1a. How the data work is actually done well. New 2026-08-08. Both closing sources are diagnoses: one measures how often the practice fails, the other how wrong the labels are, and neither says what competent labelling looks like. Partly closed the same day by measuring-annotator-agreement (Braylan, Alonso & Lease, T1), which supplies the instrument — and reports that inter-annotator agreement is well-founded for categorical labels and shaky for the bounding boxes, sequence tags, ranked lists and free text that modern datasets actually use. Still missing the practice: annotation guidelines, annotator training, sampling design, cost. Hunted again 2026-08-10 (research pass) — and the hunt returned the reason, not the artifact. who-annotates-in-nlp (Kunilovskaya et al., T1) audits 2,667 annotation tasks from 1,603 ACL-venue papers and finds the practice mostly unpublished: annotator training reported for 18.7% of tasks, released guidelines 34.1%, an agreement value 35% — against recruitment 90.4% and total items 86.0%. Papers say who was hired and how much was labelled, and omit what would let a reader judge whether the labels mean anything. So this edge was never a search failure; the base rate is low. Flagged, because it undercuts the instrument above. measuring-annotator-agreement analyses a statistic that a third of tasks report a value for, under conditions — who annotated, trained how, told what — that are documented less often still. A kappa without its guidelines describes an unspecified procedure. Re-specified rather than closed. — needs: a single labelling operation documented end to end (guidelines, training, sampling, cost) — a dataset paper’s full annotation appendix, an industrial data-operations write-up, or a datasheet that actually fills the fields. A second meta-study of reporting would not close it.
2. No independent benchmark, and the bar just moved. Every speed/VRAM figure is a vendor first-party snapshot — but pervasive-label-errors shows a neutral benchmark inherits its test set’s label error rate, at least 6% on ImageNet validation, enough to reverse model rankings. — needs: a T2 neutral benchmark that states its label quality, not merely a third-party one.
3. Evaluation as a discipline — the demo→production gap page implies it and nothing covers it. — needs: T1/T2 on model evaluation and monitoring.

Coverage edges (added 2026-08-08, at the curator’s request for a wider backlog). These widen what the spoke covers instead of answering an open question above; one ordinary solid source closes any.

4. Gradient-boosted trees. CLOSED 2026-08-09 (research pass) — xgboost (Chen & Guestrin, KDD 2016) and tabular-deep-learning-benchmark (Grinsztajn et al., NeurIPS D&B 2022), both T1, with gradient-boosted-trees as the concept page. Successor: run the comparison question 7 actually needs. The baseline now exists on paper and nobody has pointed it at a time-series foundation model. — needs: an M-competition-style evaluation, or any published result putting a foundation model against boosted trees and ARIMA/ETS on real series.
5. Model selection and validation. Cross-validation, leakage and the split protocol are what pervasive-label-errors and every benchmark claim here silently depend on. — needs: a textbook chapter or a paper on leakage in published work.
6. The machinery under demo-to-production-gap. Feature stores, drift monitoring and retraining triggers are named by implication and never described. — needs: a system report from a team running one, or a tool’s architecture docs.
7. Interpretability. SHAP, LIME and partial-dependence plots are how deployed models get explained to the people affected by them, and no page covers any of it. — needs: the SHAP paper (T1) plus one critique of its limits.

Index — Machine Learning Wiki

Catalog of every page, grouped by schema.org @type. Spine: synthesis (thesis), log.md (history), this file (catalog). Spun out of the hub _inbox on 2026-07-30 (5 sources, four tags). Most speed and memory numbers in this domain are first-party and unreplicated — record the setup beside the number, or say the source gave none.

DefinedTerm (concepts)

• training-data-quality — the founding gap’s last third: how training and test data get collected, labelled and checked. Two T1 halves — a 92% cascade prevalence across practitioners, and ≥3.3% wrong labels across the standard test sets · practice-area

• machine-learning — the root page: fitting a model to data, treated as an engineering discipline; the three layers this corpus covers and the one it skips · root

• classical-ml-algorithms — the pre-deep-learning core (regression, SVM, trees, boosting, k-means, PCA); currently a pointer to an implementation rather than a conceptual account · algorithm-family

• fine-tuning — adapting a pretrained model: full vs PEFT (LoRA/QLoRA), catastrophic forgetting, the “don’t fine-tune first” rule, and the BF16 merge/de-merge drift gotcha · technique

• ml-system-design — designing the system around the model (target, features, serving, retraining); what actually ships is ranking, matching and forecasting over behavioural data · practice

• demo-to-production-gap — the distance between a demo that works once and a system that runs; the corpus’s biggest hole — a success catalog with no denominator · problem

• bayesian-inference — learning read as inference: the loss is a log likelihood, the regularizer a log prior, and the product an ensemble rather than a point; Occam’s razor comes free with the evidence · framework

• information-theory — entropy, compression and channel capacity, and the claim that they are the same subject as learning; coding theory proper is unowned across the hub · foundation

• time-series-forecasting — predicting a sequence from its own past, and the corpus’s first full evaluation design: seasonal-naive + MASE, rolling-origin backtesting, pinball loss and interval coverage, context ablation; plus forecasting reused as anomaly detection · task

• monte-carlo-methods — sampling instead of integrating, because the posterior is intractable; the ML-side half of the standing homonym with metaheuristic search · technique

• capability-emergence — whether an ability arrives abruptly or gradually, and whether the abruptness is real or a metric artifact; the transition’s order (first vs second) decides whether anything is visible before it, which makes it a monitoring question · phenomenon

• induction-head — the … A, B … A → B circuit behind in-context token statistics, and its first-layer copy sub-circuit, the part that is hard to learn; held here as how it forms in training, not as a serving concern · mechanism

• gradient-boosted-trees — sequential ensembles of shallow trees, each fitted to the last one’s residual; the default for tabular data, and the missing baseline in open question 7. Wins on inductive bias (ignores uninformative columns, orientation-preserving, handles irregular targets), not on maturity · method

SoftwareApplication

• timesfm — Google Research’s 200M-parameter foundation model for time-series forecasting; zero-shot, ten quantiles per step, covariates via XReg — the limit case of “don’t fine-tune first,” in a field that used to fit a model per series · model
• unsloth — fine-tuning framework (Triton kernels, ~80% VRAM reduction claimed); AMD/ROCm support since 2026-07; refuses LoRA merge/de-merge on BF16 drift grounds · tool

SoftwareSourceCode (sources)

• mlalgorithms — rushter, MIT, ~11.1k★: from-scratch NumPy implementations from linear regression to LSTMs and deep Q-learning, written for legibility over speed · source · T2 · github.com

• transformers-tutorials — NielsRogge, MIT, 11.7k★: one folder per model, notebooks for inference/fine-tuning/evaluation across DETR, ViT, SAM, DINOv2, LayoutLM v1–v3, Donut, TrOCR, BLIP-2, LLaVA, TAPAS and more; fine-tunes three ways (native PyTorch / Lightning / Trainer). The spoke’s first source on the middle layer — the training loop as executable code, and no numbers anywhere · source · T2 · github.com

• supervision — Roboflow, MIT, 48.7k★: model-agnostic Python utilities for the layer after a vision model — sv.Detections, annotators, multi-object tracking, YOLO/COCO/VOC dataset conversion, mAP metrics. Stretches the spoke’s train-time boundary; the corpus’s most concrete evidence on demo-to-production-gap · source · T2 · github.com

Collection (sources)

• awesome-agentic-time-series — TROUBADOUR000, MIT, 188★: 243 papers on LLMs and agents for time series, companion to an unpublished survey. Taxonomy of agent roles (perception / reasoning / planning-action / memory / world-model) and 31 foundation models beside timesfm — but 16 papers on evaluation against 133 on building, which is demo-to-production-gap seen from the literature side · source · T3 · github.com
• ml-system-design-case-studies — Engineer1999, ~10.9k★: 300+ production ML case studies from 80+ companies (2017–2023), indexed by industry and use case; a bibliography, and a survivorship-biased one · source · T3 · github.com

Book (sources)

• information-theory-inference-learning-algorithms — David MacKay, Cambridge University Press 2003 (held copy: v7.2, 4th printing, 2005): 50 chapters unifying Shannon’s information theory with Bayesian inference and neural networks; the spoke’s only textbook and only T1 · source · T1 · inference.org.uk

Article (sources)

• finetuning-explained-kdnuggets — KDnuggets (Kanwal Mehreen, 2026-07-10): pretraining vs fine-tuning, full vs PEFT, and the caution against reaching for fine-tuning first; vocabulary without measurements · source · T3 · kdnuggets.com
• timesfm-2-5-forecasting-tutorial — MarkTechPost (2026-08-01): end-to-end timesfm walkthrough — zero-shot forecast, rolling-origin backtest, XReg covariates, interval-based anomaly detection, context ablation, throughput and robustness, plus a production checklist. All of it on synthetic data the author generated, so the method transfers and the accuracy numbers don’t · source · T3 · marktechpost.com
• unsloth-amd-support — AMD developer blog (2026-07-20): Unsloth on Instinct/Radeon/Ryzen AI via ROCm; 1.39× faster and 1.33× leaner than TRL+FA2 on Llama-3.1-8B LoRA SFT, all first-party; plus the BF16 adapter-drift argument · source · T3 · amd.com
• ai-projects-fail-infrastructure-people — The New Stack (Meredith Shubel, 2026-07-06): failure is infrastructure and people, not model quality. Body never readable — four extraction routes failed; the wiki holds the headline only · source · T4 · thenewstack.io

ScholarlyArticle (sources)

• xgboost — Chen & Guestrin, KDD 2016 (T1): the system that made boosting the default — 17 of 29 Kaggle 2015 winning solutions, every KDDCup 2015 top-10 team; sparsity-aware splits, weighted quantile sketch, cache-aware out-of-core blocks; Higgs-1M 0.68 s/tree vs scikit-learn’s 28.51 s at equal AUC, 1.7 billion Criteo examples on one machine · source · T1 · arxiv.org
• tabular-deep-learning-benchmark — Grinsztajn, Oyallon & Varoquaux, NeurIPS D&B 2022 (T1): 45 tabular datasets, ~20,000 tuning hours per learner — tree ensembles remain state-of-the-art at ~10K samples, and the three reasons are inductive biases (uninformative features, rotation invariance, irregular targets) · source · T1 · arxiv.org
• phase-transitions-in-attention — Lavie/Fischer et al., arXiv 2606.12058: a closed-form Bayesian posterior over an attention layer, reduced by symmetry to two order parameters, showing the copy sub-circuit appears as a phase transition in training-set size — first-order (no precursor) for softmax, second-order plus crossover for linear attention. Adam lands on the MAP, so the point estimate is the posterior mode here. One layer, one head, synthetic copy task · source · T1 · arxiv.org
• data-cascades — nithya-sambasivan, Kapania, Highfill, Akrong, Paritosh, Aroyo (Google Research), CHI 2021: interviews with 53 AI practitioners in India, East/West Africa and the USA on high-stakes AI (cancer detection, suicide prevention, landslide detection). Names data cascades — compounding downstream failures seeded by upstream data decisions — and measures them: 92% prevalence, 45.3% with two or more in one project; “pervasive, invisible, delayed, but often avoidable.” Closes the founding gap’s data half, and it is HCI, not ML, which is why searching the ML side never found it · source · T1 · research.google
• measuring-annotator-agreement — Braylan, Alonso & Lease, arXiv 2212.09503 (ACM, 2022): inter-annotator agreement works for categorical and ordinal labels and breaks down on the seven task shapes real datasets use — bounding boxes, keypoints, sequence tags, ranked lists, free-text translation, numeric vectors, syntax trees — because each needs a distance function chosen before agreement means anything. Proposes two more interpretable measures. The instrument for catching the failures the other two sources measure · source · T1 · arxiv.org
• pervasive-label-errors — curtis-northcutt, Athalye, Mueller, NeurIPS 2021 (Datasets & Benchmarks), arXiv 2103.14749: label errors in 10 standard CV/NLP/audio test sets found by confident learning and confirmed by crowdworkers (51% of flagged candidates real). ≥3.3% wrong on average, ≥6% of ImageNet validation. Correcting labels reverses rankings — ResNet-18 beats ResNet-50, VGG-11 beats VGG-19 — so the field’s ruler has a measured error rate · source · T1 · arxiv.org
• who-annotates-in-nlp — Kunilovskaya et al., arXiv 2606.02255 (2026): task-level audit of annotation reporting across ACL venues 2018–2025, 2,667 tasks from 1,603 papers. Operational facts get reported (recruitment 90%, expertise 86%, total items 86%); the facts that let a reader judge validity do not — training 19%, guidelines released 34%, IAA value 35%. Explains why the corpus could never find a documented labelling operation: the base rate is low · source · T1 · arxiv.org
• skillsmith — Dery et al., arXiv preprint: weights as a modality — an LLM composes new prefix-tuning weights from existing prefix weights + text, beating text-only and weight-space-only baselines; the spoke’s first primary research on adaptation, and a third stance on the weights-as-distribution seam · source · T2 · arxiv.org

Person (entities)

• niels-rogge — HuggingFace Transformers contributor (ViT, DETR, LUKE and others) and author of the tutorials; expert on the code, interested party on the library choice · author
• nithya-sambasivan — lead author of data-cascades, published through Google Research; HCI method — 53 practitioner interviews, no model trained · author
• curtis-northcutt — first author of pervasive-label-errors; confident learning, the method behind the label-error estimates · author
• david-mackay — Cambridge information theorist; author of the 2003 textbook, and a co-rediscoverer of the LDPC codes it teaches · author
• itay-lavie — Hebrew University / Harvard SEAS; co-first and corresponding author of phase-transitions-in-attention · author
• zohar-ringel — Racah Institute, Hebrew University; senior author, statistical-mechanics methods applied to learning · author
• moritz-helias — Jülich (IAS-6) / RWTH Aachen; senior author, field-theoretic and Bayesian feature-learning theory · author

Organization (entities)

• amd — Corporation: Instinct/Radeon/Ryzen AI silicon and the ROCm stack — the non-CUDA training path · vendor
• roboflow — Corporation: computer-vision platform company; maintainer of supervision, with a commercial inference product the library integrates · vendor
• kdnuggets — NewsMediaOrganization: data-science practitioner outlet; orientation-tier explainers · publisher
• marktechpost — NewsMediaOrganization: AI-news outlet running long runnable Colab walkthroughs; checkable API surface, synthetic datasets, no visible editorial verification · publisher
• cambridge-university-press — the publisher that let a full university textbook stay free on the author’s site · publisher
• university-of-cambridge — CollegeOrUniversity: where the textbook was lectured for nine years before it was a book · institution

Linked cross-wiki, not duplicated: The New Stack (the-new-stack, ../ai-governance-wiki) · Google (google, ../llm-providers-wiki) as timesfm‘s maker · backtesting (backtesting, ../quant-trading-wiki) as the trading-side sibling of rolling-origin evaluation.