Transformers-Tutorials

niels-rogge‘s collection of Jupyter notebooks demonstrating HuggingFace Transformers models, in PyTorch. 11.7k stars, 1.7k forks, MIT. Organized as one folder per model, each holding notebooks for a specific task — inference, fine-tuning, evaluation — with Google Colab badges so they run without local setup.

This is the corpus’s first source on the middle layer (synthesis growth edge 1): the stretch between “here is what the algorithm does” and “here is a tool that fine-tunes for you” that the spoke had been jumping over.

Tier note

T2. A personal MIT-licensed teaching repository, not a vendor artifact — the same shape as mlalgorithms, which this spoke also grades T2. The affiliation is worth recording rather than penalizing: Rogge contributed many of these model implementations to the Transformers library itself, which is why coverage tracks that library’s catalogue closely. That makes him expert on the code and not a neutral party on the choice of library; nothing here compares Transformers to an alternative.

What it actually covers

Breadth first, because the list is the point — this is one of the widest single-repo model surveys anywhere:

• Detection and segmentation — DETR and its variants (Conditional, Deformable, RT-DETR), YOLOS; SegFormer, MaskFormer, Mask2Former, OneFormer, UPerNet.
• Vision classification and features — Vision Transformer, ConvNeXT, BEiT, DINOv2; SAM for promptable segmentation; DPT and GLPN for depth; Swin2SR for super-resolution.
• Documents and layout — LayoutLM v1–v3, LayoutXLM, Donut, TrOCR, MarkupLM. An unusually deep corner, and a commercially load-bearing one.
• Multimodal — BLIP-2, Idefics2, LLaVA variants, ViLT, GroupViT, CLIP-family models.
• Text — BERT, T5, CANINE, LUKE.
• Audio and video — Audio Spectrogram Transformer, VideoMAE, X-CLIP.
• Structured and other tasks — TAPAS and Table Transformer for tables, GIT for captioning, Perceiver.

The part that closes the gap

The notebooks are not only inference demos. They fine-tune on actual datasets, and they do it three ways — native PyTorch, PyTorch Lightning, and HuggingFace’s Trainer API — with README guidance on data preprocessing and on picking among the three.

That is the training pipeline as executable code: load and preprocess a dataset, define the loop or delegate it, train, evaluate. fine-tuning here has been a page about concepts (full versus PEFT, catastrophic forgetting, the don’t-fine-tune-first rule) sourced from an explainer with no measurements, plus unsloth, a tool that hides the loop entirely. This shows the loop.

What it does not give you

No numbers, and it does not claim any. There is no benchmark, no accuracy table, no throughput comparison — it is a teaching resource, and the spoke’s standing rule (record the setup beside the number, or say the source gave none) resolves trivially here because there are no numbers to record.

So it closes the middle-layer gap in the method sense and leaves growth edges 2 and 3 untouched: still no independent benchmark, still nothing on evaluation as a discipline. Knowing how to run a training loop is not knowing whether the result is any good, and this corpus keeps discovering that those are separate acquisitions.

Coverage is also frozen at whatever was current when each notebook was written, which for a repository organized by model is a per-folder question rather than a repo-wide one. Marked volatile.

Related

niels-rogge · fine-tuning · machine-learning · mlalgorithms · unsloth · ml-system-design · demo-to-production-gap