Classical ML algorithms
The pre-deep-learning core of the field: regression, support vector machines, decision trees and their ensembles (random forests, gradient boosting), k-means and Gaussian mixtures, kNN, naive Bayes, and the dimensionality-reduction methods (PCA, t-SNE). Still the default choice on tabular data, and still where the vocabulary of machine-learning comes from.
What the corpus holds
One source, and it is code rather than exposition: mlalgorithms implements the whole list from scratch on NumPy/SciPy/Autograd, deliberately trading speed for legibility. Its span reaches past the classical set into neural networks and deep Q-learning, written in the same style — so the break between the eras looks smaller in that repo than the field’s own narrative suggests.
No conceptual account of any single algorithm is paged yet — there is no gradient-descent explainer, no boosting derivation, nothing on the bias-variance trade-off. This page is currently a pointer to an implementation, and says so rather than paraphrasing textbook material the corpus doesn’t contain.
Partly answered since. information-theory-inference-learning-algorithms gives derivations for one branch of the list — clustering and mixture models, maximum likelihood, independent component analysis, and neural network training read as bayesian-inference — plus the model-comparison account this page was missing. It leaves the other branch untouched: no trees, boosting, random forests or SVMs, all of which existed by 2003 and none of which the book covers — a choice of tradition rather than a gap in the corpus, and the branch that still dominates tabular data.
Fitting is not metaheuristic optimization
Gradient descent, EM and boosting all minimize a loss, which makes “optimization” the natural word
and creates the hub’s sharpest homonym. ../optimization-algorithms-wiki owns metaheuristic,
population-based global optimizers — CMA-ES, PSO, genetic algorithms — searching a space with no
usable gradient. Model fitting exploits a gradient it does have. Both spokes carry the warning;
cross-link the two traditions, never merge them.