Synthesis — Optimization Algorithms

The evolving thesis. Spun out 2026-06-09 from the hub optimization-algorithms cluster (4 founding sources, all from Andrey Dik's MQL5 optimizer-benchmark series); expanded the same day with 5 more entries from that series — the canonical swarm/EA classics particle-swarm-optimization (PSO), grey-wolf-optimizer (GWO), ant-colony-optimization (ACO), artificial-bee-colony (ABC), and the foundational evolution-strategies ((μ,λ)-ES & (μ+λ)-ES) — then with 5 independent, authoritative (Wikipedia) sources to break single-author dependence: differential-evolution, genetic-algorithm, simulated-annealing, test-functions-for-optimization, and bayesian-optimization. Now 14 algorithms/concepts.

Scope

Metaheuristic, population-based optimizers — algorithms that search a multidimensional space for a global optimum without gradients, using a population of candidate solutions that explore and exploit over iterations. The founding four span the main flavors:

• exchange-market-algorithm (EMA) — behavior-inspired (stock-market traders: elite/middle/ beginner; balanced vs. fluctuating phases + opposition-based learning).
• backtracking-search-algorithm (BSA, Civicioglu 2013) — evolutionary, with a historical (archive) population as memory; mutation + crossover + greedy selection.
• cma-es (Hansen & Ostermeier, late 1990s) — the theory-heavyweight evolution strategy: samples from a multivariate normal and adapts the covariance matrix / step size.
• deterministic-oscillatory-search (DOS) — fully deterministic: oscillatory "bounce" movement + slope tracking + swarming toward the global best; reproducible, no randomness.

Current thesis — No Free Lunch, made concrete

The sources share one comparative benchmark (Dik's ~45-algorithm MQL5 suite over standard test functions at several dimensionalities), and the result is a textbook illustration of the no-free-lunch-theorem: no optimizer dominates. The famous, mathematically sophisticated cma-es ranked only ~38/45 here; the pragmatic BSA landed mid-pack (20/45); the conceptually elegant EMA and the deterministic DOS sat near the bottom (45th). Sophistication and reputation don't predict benchmark rank — performance is problem-, dimension-, and tuning-relative. So the wiki's first rule: record rankings with their benchmark context; treat "best optimizer" as benchmark-relative.

The +5 expansion sharpens the point in two ways:

• Old beats new (held the top slot 2026-06 → 2026-07-25; see the leader change below). (μ+λ)-ES at 72.18% led the suite — a 1970s evolution strategy out-ranking every modern, metaphor-laden method (and its own sophisticated descendant cma-es). Worse, its sibling (μ,λ)-ES scores only 51.22%: a single design choice — keep parents (plus) vs. discard them (comma) — swings rank ~20 points. The mechanism, not the metaphor or the vintage, is what pays.
• Famous can lose to random. In Dik's earlier, small-field scoring, PSO (0.47695) ranked below the random-search baseline (0.51254) — the bluntest NFL datum in the corpus. The other classics (GWO, ACO, ABC) all shine on smooth low-dim functions and collapse on high-dim / discrete ones — the same scalability cliff, in five different costumes.

The leader changed, and how it changed is the story

DOAm now tops Dik's suite at 73.63%, 1.45 points above (μ+λ)-ES. Both numbers stand as recorded. Three things about how it won matter more than the rank.

First, the edit was tiny. DOAm is the published Dingo algorithm with a sign flipped in the group- attack move (toward the leader, not away), a survival threshold dropped 0.3 → 0.01, and an abs() removed from the scavenging formula. No new mechanism, no new metaphor — three corrections to the direction and gating of moves the original already had. That is the same lesson (μ,λ) vs (μ+λ) taught at ~20 rank points: the mechanism detail is load-bearing and the story wrapped around it is not. It also gives the corpus's standing "metaphor ≠ novelty" critique (Sörensen, via wikipedia-metaheuristic) an unusually sharp instance — the dingo story survived the modification untouched while the thing that actually moved the score was arithmetic.

Second, the winner is a self-modification by the benchmark's author. Dik owns the suite and entered a variant he tuned; it now leads. Nothing here suggests bad faith — the code is published and he states the weaknesses himself — but the standing open question "how neutral is the benchmark?" has stopped being abstract. Every earlier entrant was somebody else's algorithm scored by Dik; this one is Dik's algorithm scored by Dik, and no independent suite has run it. Treat 73.63% as the least-portable number in the corpus.

Third, its own weakness profile argues for the No-Free-Lunch reading anyway. DOAm is strong where the landscape is rugged or discrete (Forest 0.912, Megacity 0.848) and comparatively weak on smooth (Hilly 0.454), with high run-to-run variance and a tendency to stick. Dik's own recommendation is to run it as an exploration front-end and hand off to a precise optimizer — exploration-vs-exploitation resolved by switching algorithms mid-solve rather than by tuning one. The suite leader is, by its author's account, not a general-purpose winner but half of a pipeline. That is NFL restated from the top of the table instead of the bottom.

Benchmark-comparability caveat (new, important)

Dik's scoring changed over the years, so cross-article numbers can mislead. The mature rating is a % of MAX over Hilly/Forest/Megacity across ~45 algorithms (founding four + ES); the early articles (PSO/GWO/ACO/ABC, ~2022–23) use an absolute 0–1 score over Skin/Forest/Megacity in a ~6-algorithm field. The two scales are not interchangeable — the benchmark page now keeps them in separate tables. The benchmark's own drift is itself an NFL lesson: even one author's "score" isn't a fixed yardstick.

Independent corroboration — beyond Dik (5 authoritative sources)

The corpus was broadened with five independent, authoritative (Wikipedia) sources so the thesis no longer rests on one author's suite: differential-evolution (Storn & Price), genetic-algorithm (Holland — the family ancestor), simulated-annealing (Kirkpatrick 1983), the academic test-functions-for-optimization basis, and bayesian-optimization (Mockus/Jones). They both widen the field (see new axes below) and echo No Free Lunch from outside the MQL5 world: GA's own article cites Skiena's doubt that GAs are ever clearly best; SA's global-convergence guarantee is provably useless in practice; benchmark test functions are designed to expose where each method breaks. Now 14 algorithms/concepts.

And the theorem itself is now primary-sourced. nfl-original-paper (Wolpert & Macready, No Free Lunch Theorems for Optimization, IEEE TEC 1997) anchors the spoke's organizing claim in the canonical paper rather than Dik's blog — separating the two cleanly: the theorem rests on the T1 primary (the ∑_f formulation, the time-varying NFL, the alignment-with-P(f) interpretation, head-to-head minimax), while Dik's MQL5 suite is its empirical illustration. The paper also tightens the standing caveat: NFL's uniform-distribution premise is the precise reason structure-exploiting methods do win on real problem classes — improvement requires advance knowledge of P(f), which is exactly what the model-based / exact methods encode.

And now a textbook sits above both (2026-07-26). Algorithms for Optimization (Kochenderfer & Wheeler, MIT Press 2019, free CC BY-NC-ND PDF, Julia implementations) is the first academic reference in a corpus otherwise built from one blog series and Wikipedia. It doesn't contradict anything here; what it does is re-proportion the field. In Dik's world metaheuristics are optimization. In the textbook they are one chapter — "population methods" — sitting between local descent, stochastic methods, linear/constrained programming and surrogate models, in a book organized around engineering design rather than around ranking optimizers against each other. The spoke's own map (structure ⇄ generality) survives that reframing intact, which is mild corroboration that the map was drawn correctly. The book also names two areas the corpus has no page and no source for: multi-objective optimization and optimization under uncertainty.

The axes that organize the field

• exploration-vs-exploitation — every algorithm is a different answer to the same tension (diversify the search vs. refine the best). EMA literally splits it into "balanced" (exploit) and "fluctuating" (explore) phases; BSA injects diversity via a shuffled historical population; CMA-ES shapes exploration with the covariance matrix; DOS oscillates then swarms.
• Stochastic vs. deterministic — almost all metaheuristics are stochastic; DOS is the outlier, trading the robustness of randomness for full reproducibility (same seed-free inputs → same result). A clean natural experiment on whether randomness is load-bearing (its weak rank suggests it often is).
• Scalability / cost — cma-es is O(n²) memory, O(n³) ops; excellent on ill-conditioned low-dim problems (invariance to affine transforms) but fails by n>100–500. Mechanism richness trades against dimensional reach.
• Population vs. single-solution (new) — almost all the corpus evolves a set of candidates; simulated-annealing and tabu-search are the outliers, walking one state through the space. They split on how to escape local optima while moving alone: SA accepts worsening moves via a stochastic cooling schedule, tabu-search (Glover, 1986) via deterministic memory (a tabu list + short/intermediate/long-term memory) — randomness vs. memory as the two single-solution escape strategies, and the corpus's clearest case of explicit memory driving search. Population parallelism vs. trajectory simplicity is its own design axis.
• Direct search vs. sampling (new) — nelder-mead (Nelder & Mead, 1965, the downhill simplex) occupies a quadrant nothing else does: derivative-free but local and largely deterministic — an n+1-vertex simplex crawling downhill by reflection/expansion/contraction/shrink, geometric rather than sampling. It is the bridge between the gradient-free black-box corner and the classical methods: gradient-free like the metaheuristics, but local with no global ambition. Both single-solution classics reinforce No-Free-Lunch by niche — Nelder–Mead for cheap smooth low-dim, tabu search for combinatorial/discrete problems the continuous-space population-optimization-benchmark doesn't even test.
• Model-free vs. model-based (new) — the metaheuristics are model-free (sample blindly, spend many cheap evaluations); bayesian-optimization is model-based (a Gaussian-process surrogate + acquisition function reasons about where to look, spending few expensive evaluations). This reframes NFL across evaluation budget, not just landscape shape — different regime, different winner.

Open questions

• How neutral is the benchmark? (now well-grounded, 2026-06-12) The Dik rankings still come from one author's MQL5 suite, but the corpus now carries both the independent academic basis (test-functions-for-optimization) and the standard platform that uses it — coco-bbob (COCO/BBOB, the GECCO benchmarking framework since 2009, with a fixed-target/ERT methodology and standard bbob/bbob-largescale/bbob-biobj/mixed-integer suites). COCO is the neutral referee for the recorded CMA-ES contradiction (~38/45 in Dik vs near-top on BBOB) — it doesn't overturn Dik (record-don't-overwrite) but gives the academic side a reproducible home. Still want the literal head-to-head: Dik's exact algorithm set re-run inside COCO. The platform exists; the cross-run doesn't yet. Sharpened 2026-07-25: the suite is now led by DOAm, the author's own modification of a published algorithm. Every previous entrant was someone else's algorithm scored by Dik; this is Dik's variant scored by Dik, unreplicated anywhere. The neutrality question now has a specific, checkable target — run DOAm on COCO/BBOB.
• What happens to the ranking question when the objective is a vector? (opened 2026-07-26) Every benchmark in this spoke scores one number per run, so "best optimizer" means best scalar fitness. algorithmsbook devotes a section to multi-objective optimization, where the output is a Pareto front and no total order exists to rank against. The suite's whole apparatus assumes a scalar, and nothing in the corpus says what the metaheuristics do once that assumption goes. Related and equally unpaged: optimization under uncertainty, where the objective is noisy and a single evaluation no longer settles anything. Instrument now in place (2026-08-14): hypervolume-indicator (Guerreiro/Fonseca/Paquete, ACM Computing Surveys 2021, T1) supplies the standard way to turn a Pareto front back into one comparable number — strictly monotonic with dominance, #P-hard to compute exactly above ~4–5 objectives. That's the tool a multi-objective re-run would be scored with; no page here has run one yet. Still open: does the "no optimizer dominates" result survive once ranking is by hypervolume instead of scalar fitness.
• Behavior-metaphor inflation. EMA, GWO (wolves), ACO (ants), ABC (bees) — many "novel" metaheuristics dress standard explore/exploit mechanics in a fresh metaphor; do the metaphors add anything over the mechanics? Evidence says no: the metaphor-free, decades-old (μ+λ)-ES led the suite for a year, while the elaborate EMA sits 45/45 and PSO trails random search — and GA's own article concedes plain SA/hill-climbing "often outperform" it. Mechanism > metaphor. The 2026-07 leader change makes the same case from the other direction: DOAm beat (μ+λ)-ES by flipping a sign, lowering a threshold, and deleting an abs() — the dingo story went through the modification completely untouched, which is as clean a demonstration as the corpus has that the metaphor was never the working part.
• Where do classical/exact and ML-style optimizers fit? (now answered, 2026-06-09) bayesian-optimization brought the model-based regime and simulated-annealing the single-solution lineage; now the gradient-based half (gradient-descent, SGD/Adam) and the exact half (convex-optimization) are on the board. This completes the map: the founding corpus is the gradient-free, black-box, no-guarantee quadrant; the new pages are its complements — use derivatives (gradient descent — the optimizer ML training actually runs on), exploit structure (convex — local = global, polynomial-time). The unifying frame is now explicit: structure ⇄ generality (convex-optimization's phrase) — every optimizer trades how much it assumes about the problem against how broadly it applies, which is the no-free-lunch-theorem restated. Metaheuristics own the assumption-poor extreme.

The exact half arrives, and it reframes the metaheuristic half

linear-programming, integer-programming, miplib and mittelmann-benchmarks give the spoke its first real contact with the part of optimization that industry runs, and the contrast with 36 pages of population search is sharper than expected.

What structure buys. LP has a global optimum at a polytope vertex, a polynomial-time guarantee since Khachiyan 1979, and strong duality — a solution arrives with a certificate that nothing better exists. Integer programming loses the polynomial guarantee (NP-complete; 0–1 ILP is one of Karp's 21) and keeps something the metaheuristics here never have: branch and bound reports a bound, so an early stop yields a feasible solution and a measured distance from optimal.

That is the honest reading of no-free-lunch-theorem the spoke has been half-making. NFL says no search dominates across all problems, and this corpus has treated that mainly as licence for variety. The exact side is the other half of the argument: where structure exists, exploiting it buys guarantees that no black-box search can offer at any budget. The question worth asking of a genetic-algorithm applied to scheduling or routing is whether the problem was a disguised MIP — and this corpus has never seen a metaheuristic compared against a MIP solver at equal time budget.

Benchmarking is healthier here, and under attack. miplib is built from real submitted instances whose difficulty is discovered rather than designed, with a public list of open instances marking the frontier — a stronger design than the synthetic suites in test-functions-for-optimization and coco-bbob. But mittelmann-benchmarks, the field's independent comparison, records its own hollowing out: IBM and FICO demanded their solvers be removed after Gurobi's 2018 action, and Gurobi and MindOpt withdrew in 2024. The reason the comparison is incomplete is a licence term, not a methodological limit — and it is worth noting that this spoke's other benchmark complaints have all been about method.

The exact half arrives, and it is faster than everything here (quality cycle, 2026-08-10)

The corpus is a metaheuristics corpus: GA, PSO, simulated-annealing, and a long tail of nature-inspired variants, all returning good answers with no optimality proof. It had no solver in it at all — linear-programming and integer-programming sat on Wikipedia and nothing was runnable.

highs fills that (T1, MIT-licensed, four solvers in one library: dual revised simplex, interior point, active-set QP, branch-and-cut). The placement matters more than the description, and mittelmann-benchmarks supplies it: on 240 MIPLIB2017 instances with a two-hour limit, HiGHS solves 158 where COPT solves 219, at a 7.55× scaled geometric mean. On LP it is 12.8× off the same baseline while beating every other free solver by 2–8×.

Two things follow. Open-source owns the default, not the frontier — HiGHS is what SciPy ships and what most people run without knowing it, and it is also nowhere near the commercial ceiling. And the ceiling itself is unmeasurable here: the baseline in both tables is COPT because Gurobi, CPLEX, Xpress and MindOpt have all been withdrawn from publication, so every gap this spoke records is a lower bound.

For the no-free-lunch-theorem thread, this is the sharpest available instance of the theorem's practical reading. NFL equalizes algorithms averaged over all problems; inside the class where LP and branch-and-cut apply, exploiting the structure beats searching it, and the margin is orders of magnitude. Nothing in the metaheuristics half of this spoke competes on MIPLIB, and none of it tries.

Freshness housekeeping. Seventeen pages carried freshness: volatile and ten were past the 60-day window — all of them algorithm concept pages or summaries of dated MQL5/Wikipedia articles, which QUALITY.md grades stable (re-reading returns the same text; whether the claims hold is a synthesis question). Regraded. coco-bbob stays volatile: it is a live benchmark platform with moving results. The queue went 10 → 0 without refreshing anything, because nothing in it was ever re-verifiable.

The comparisons are run by the people who wrote the algorithm (quality cycle, 2026-08-22)

Two primary papers arrived together and said the same thing about how this field measures itself. nsga-ii reports better convergence and better spread than PAES and SPEA on nine test problems — scored by Υ and Δ, two metrics the same paper defines, on problems the first author had written the design guidelines for. adam-optimizer reports favourable results against SGD with Nesterov momentum, AdaGrad and RMSProp, on models and datasets its authors picked. Neither result has been overturned, and neither was ever an independent test.

That is the third and fourth instance in this spoke, after DOAm leading Dik's suite as Dik's own modification, and it is the reason coco-bbob and mittelmann-benchmarks matter more than their page counts suggest: they are the only two things in the corpus where the scorer is not the author. The no-free-lunch-theorem argument makes this worse rather than better. If no optimizer dominates across problem distributions, then a result measured on the author's chosen distribution carries even less beyond it than usual.

One asymmetry is worth keeping. Adam was replicated anyway, by a decade of the field adopting it and not going back, which is a harsher test than any benchmark suite and one NSGA-II has arguably also passed. What neither has is a number produced by somebody with nothing at stake.

The tier was grading the venue, not the claim (quality cycle, 2026-08-25)

This spoke carried the corpus's second-weakest tier mix — ten T3 pages out of 38 tiered — and every one of the ten was an andrey-dik MQL5 article. Read against QUALITY.md's rule that a tier records what a page's load-bearing claim rests on, nine of them were graded on the venue: MQL5 is a broker's community article site, so the pages inherited T3 from where they were published.

Nine of the ten describe an algorithm somebody else published — Kennedy and Eberhart, Dorigo, Karaboga, Mirjalili, Hansen, Civicioglu, Ghorbani and Babaei, Rechenberg and Schwefel — and score it on a shared suite. Dik did not write those algorithms and gains nothing from where they land, which is a third party measuring, and the ladder puts that at T2. All nine moved, each with the reason written onto the page.

The tenth did not. DOAm is Dik's own modification topping Dik's own suite, and there the section above — the comparisons are run by the people who wrote the algorithm — applies exactly. It stays T3, and it is now the only MQL5 page here that is.

The pass nearly conflated two separate things, so they are worth keeping apart. The spoke's finding about who scores what is real and unchanged; it was simply attached to nine pages it was never true of, which is the failure QUALITY.md describes as a tier proxying a finding. And the remaining objection to Dik's numbers is not self-interest but implementation — he re-codes each optimizer in MQL5, so the figure grades his code alongside the method, and his scoring scheme changed mid-series. That is a methodological limit, it stays flagged in prose on every page, and it is not what a tier records.

The floor did not move, and could not: T1 ÷ tiered counts the same T1 pages against the same denominator whichever side of the T2/T3 line the nine sit on. It stands at 0.18, which is the honest reading — this spoke rests on one author's implement-and-measure series and on two neutral platforms (coco-bbob, mittelmann-benchmarks) it has barely drawn on.

Growth edges

Ranked; each names the kind of source that would close it (see ../QUALITY.md → Growth edges).

1. DOAm on COCO/BBOB. The suite is now led by the author's own modification of a published algorithm, scored by the author, replicated nowhere — and the neutral platform to check it against already exists in this corpus (coco-bbob). — needs: a COCO/BBOB run of DOAm, or any independent scoring of it.
2. The literal head-to-head. Dik's exact algorithm set re-run inside COCO would settle the recorded cma-es contradiction (~38/45 there, near-top on BBOB) without overturning either. — needs: a published cross-run, T1/T2.
3. Ranking when the objective is a vector — instrument found, not yet run. hypervolume-indicator (T1) supplies the standard Pareto-front quality metric. — needs now: an actual multi-objective benchmark run (DTLZ/WFG suites) scoring this spoke's algorithms by hypervolume, not another survey of the metric.

Coverage edges (added 2026-08-08, at the curator's request for a wider backlog). These widen what the spoke covers instead of answering an open question above; one ordinary solid source closes any.

4. Linear and integer programming — mostly closed 2026-08-09 (research pass). linear-programming and integer-programming are written (T2, Wikipedia as tertiary survey, matching this spoke's existing practice), with miplib (T1) and mittelmann-benchmarks (T2) for the benchmark half. Two things stayed open. The edge asked for a text chapter, and the free T1 texts were not reachable (the Kochenderfer PDF now sits behind a Drive redirect, Bixby's history is paywalled) — so the concept pages rest on a tertiary source, which is weaker than this spoke's better pages. And no solver has a page: HiGHS, CBC, SCIP and Gurobi are named nowhere. — needs: a T1 chapter on simplex/branch-and-bound, and one solver's own documentation or release report. Both halves now closed, and the edge text was stale when the second one was hunted. The solver half closed 2026-08-10 by highs (T1, project documentation plus mittelmann-benchmarks rankings) and this entry went on saying "no solver has a page" for three weeks. The chapter half closed 2026-09-03 by huangfu-hall-parallel-simplex (T1, Mathematical Programming Computation 2018) — not a textbook chapter but the peer-reviewed engine paper for the solver already held, which is better: it supplies the mechanism highs could only name. Successor, edge 4a: branch-and-cut is still undescribed. integer-programming rests on Wikipedia, highs names Leona Gottwald's MIP solver and says nothing about how it works, and the LP half now has a primary while the IP half does not. — needs: a T1 paper or chapter on branch-and-cut, ideally HiGHS's own or SCIP's.
5. Ranking vectors, the standard answers. CLOSED 2026-08-22 — nsga-ii (Deb et al., IEEE TEC 6(2) 2002, T1) is read from the paper: fast nondominated sorting O(MN²), crowding distance replacing the sharing parameter, elitism through the combined parent+offspring pool, and constrained domination. With hypervolume-indicator the spoke now holds both the algorithm and the metric. Successor (new 2026-08-22): NSGA-II's own comparison used Υ and Δ, two metrics defined in that same paper; hypervolume arrived nineteen years later and nobody in this corpus has re-scored the 2002 result with it — needs: a published re-run of NSGA-II against PAES/SPEA (or its modern successors) scored by hypervolume, T1/T2.
6. Quasi-Newton methods. gradient-descent and stochastic-gradient-descent are the whole deterministic side; BFGS and L-BFGS, which most numerical optimizers default to, are absent. — needs: a text chapter or the original papers. CLOSED 2026-10-07 by l-bfgs (Liu & Nocedal, Mathematical Programming 1989, T1, read from the authors' PDF): keep the last m step and gradient-change pairs instead of a dense n² matrix; beats Buckley–LeNir; a per-iteration scalar scaling helps most; partitioned quasi-Newton wins when the problem structure is supplied. Successor, edge 6a: full-matrix BFGS still has no source of its own, and the 1989 comparison is the authors' own on mostly cheap test functions. — needs: an independent benchmark of L-BFGS against CG and Newton-type methods (e.g. a CUTEst study), T1/T2.
7. The optimizers that run most often. CLOSED 2026-08-22 — adam-optimizer (Kingma & Ba, ICLR 2015, T1) replaces the Wikipedia-mediated account the spoke had. Successor (new 2026-08-22): the paper predates AdamW by two years, and the decoupled-weight-decay correction that every framework now defaults to is still held here only through wikipedia-stochastic-gradient-descent — needs: Loshchilov & Hutter's Decoupled Weight Decay Regularization (ICLR 2019), T1. Cross-spoke: the training context sits in ../machine-learning-wiki. Successor CLOSED 2026-10-05 by adamw (T1, read from the ICLR 2019 PDF): for Adam, L2 regularization and weight decay differ, and decoupling them gave a 15% relative test-error gain on CIFAR-10 and ImageNet32x32. Same caveat as Adam: the authors ran the comparison on two image datasets, and the language-model use where AdamW is now standard rests on practice, not on this paper.

Contradictions / tensions

• CMA-ES: Dik rank vs. academic standing (recorded contradiction). cma-es ranks only ~38/45 on Dik's population-optimization-benchmark yet is widely held a top general-purpose optimizer and rates near the top on BBOB/COCO. Per record-don't-overwrite, we keep both: the disagreement is the no-free-lunch-theorem point made literal — same algorithm, different problem distribution / implementation / tuning. Hold any single ranking loosely (and see the volatile-data caveat in CLAUDE.md). The academic side now has a concrete home: coco-bbob (the BBOB platform on which CMA-ES rates near-top) — the neutral place a head-to-head re-run could one day adjudicate this against Dik's MQL5 suite.

Cross-spoke adjacency

• ../research-wiki — owns formal methods / theorem proving (proving truths) and the mechanizing-reasoning lineage; optimization (searching for optima) is a sibling computational discipline, not the same. Its parked-cluster neighbor computational-foundations (Wolfram; computational irreducibility) is the limits-of-computation angle — adjacent, distinct.
• MQL5 / algorithmic trading is the delivery context of the founding sources, not the subject; no trading spoke exists and the optimizers are general-purpose.

Index — Optimization Algorithms Wiki

Catalog of every page, grouped by schema.org @type. Spine: synthesis (thesis), log.md (history), this file (catalog). Spun out of the hub optimization-algorithms cluster 2026-06-09. Benchmark rankings are benchmark-relative — recorded with context, not absolute (no-free-lunch-theorem).

DefinedTerm (concepts / mechanisms)

• metaheuristic-optimization — umbrella: gradient-free global optimization (population, single-solution, model-based) · domain
• exploration-vs-exploitation — the core tension every optimizer balances · theory
• no-free-lunch-theorem — no optimizer dominates across all problems; rankings are relative · theory
• population-optimization-benchmark — Andrey Dik's ~45-algorithm MQL5 comparative test suite; leader table updated 2026-07-25 · standard
• test-functions-for-optimization — the independent academic benchmark basis (Rastrigin/Rosenbrock/Ackley…) · source · standard
• coco-bbob — COCO/BBOB: the standard academic black-box-optimization benchmarking platform (GECCO workshops since 2009); fixed-target/ERT methodology; the neutral referee · source

Book (sources)

• algorithmsbook — algorithmsbook.com: Kochenderfer's three MIT Press books; Algorithms for Optimization (2019 + 2nd-ed preview, free CC BY-NC-ND PDF, Julia) is the spoke's first textbook source · source · T1 · algorithmsbook.com

ScholarlyArticle (sources)

• nfl-original-paper — Wolpert & Macready, No Free Lunch Theorems for Optimization (IEEE TEC 1997); the primary source for the spoke's central theorem · source · T1 · cs.ubc.ca
• hypervolume-indicator — Guerreiro/Fonseca/Paquete, ACM Computing Surveys 54(6) 2021; the standard quality indicator for scoring a Pareto front — answers "what does ranking mean when the objective is a vector" · source · T1 · arxiv.org
• nsga-ii — Deb/Pratap/Agarwal/Meyarivan, IEEE TEC 6(2) 2002; fast nondominated sort, crowding distance, constrained domination — the standard multi-objective GA · source · T1 · ieeexplore.ieee.org
• adam-optimizer — Kingma & Ba, ICLR 2015; the bias-corrected moment update every deep-learning framework ships, and its own bounded-step argument · source · T1 · arxiv.org
• l-bfgs — Liu & Nocedal, Mathematical Programming 1989: limited-memory BFGS keeps the last m vector pairs instead of an n² matrix; beats Buckley–LeNir; dynamic scaling helps most; partitioned quasi-Newton wins when structure is supplied. Authors' own tests · source · T1 · northwestern.edu
• adamw — Loshchilov & Hutter, ICLR 2019: L2 and weight decay are equivalent for SGD but not for Adam, where L2 under-regularizes weights with large gradient history; decay applied outside the gradient step fixes it. 15% relative test-error gain over Adam on CIFAR-10 and ImageNet32x32, learning rate and decay made near-independent, AdamWR up to 10× better anytime performance. Authors' own comparison, image tasks only · source · T1 · arxiv.org
• huangfu-hall-parallel-simplex — Huangfu & Hall, Mathematical Programming Computation 10(1):119–142, 2018. The engine paper behind highs: PAMI (parallelism across iterations, via suboptimization) and SIP (within one). On a 30-problem Mittelmann reference set, PAMI on 8 cores is 1.51× faster than serial hsol, which is 2.29× faster than Clp; PAMI's mean parallel speedup is 2.34× and it is slower than hsol in serial. Huangfu was at FICO and the abstract says the technique became FICO Xpress promotional material — a declared conflict, which is what makes it usable · source · T1 · arXiv 1503.01889

TechArticle (source summaries — authoritative references)

• wikipedia-convex-optimization — standard form, LP⊂QP⊂SOCP⊂SDP ladder, KKT, Nesterov–Nemirovskii barriers · source · T2
• wikipedia-gradient-descent — Cauchy 1847, O(1/k)→O(1/k²) Nesterov, condition number, momentum · source · T2
• wikipedia-simulated-annealing — Metropolis 1953 lineage, Černý co-credit, adaptive SA, restarts · source · T2
• wikipedia-stochastic-gradient-descent — Robbins–Monro 1951, decay conditions, Adam (Kingma–Ba), AdamW · source · T2
• wikipedia-artificial-bee-colony — Karaboga 2005 phases, vᵢₖ formula, the limit knob · source · T2
• wikipedia-ant-colony-optimization — Dorigo's Ant System, τ/η rule, MMAS/ACS, combinatorial native · source · T2
• wikipedia-metaheuristic — Glover-1986 term, the Sörensen "metaphor ≠ novelty" critique · source · T2

DefinedTerm (algorithms — Dik MQL5 series)

• dingo-optimization-algorithm — DOA / DOAm (Dik's modification): four-behaviour pack metaheuristic; suite leader at 73.63% (2026-07) off a sign flip, a threshold, and a deleted abs(); author-modified entrant on the author's own benchmark · source · mechanism
• evolution-strategies — (μ,λ)-ES & (μ+λ)-ES (Rechenberg/Schwefel, 1960s–70s); 72.18%, suite leader 2026-06 → 2026-07 · source · mechanism
• cma-es — Covariance Matrix Adaptation Evolution Strategy (Hansen); adapts a covariance matrix · source · mechanism
• backtracking-search-algorithm — BSA (Civicioglu 2013); historical/archive population as memory · source · mechanism
• exchange-market-algorithm — EMA; stock-market-behavior-inspired (elite/middle/beginner traders) · source · mechanism
• deterministic-oscillatory-search — DOS; fully deterministic oscillatory + swarming search · source · mechanism
• particle-swarm-optimization — PSO (Kennedy & Eberhart 1995); cognitive + social velocity pull · source · mechanism
• grey-wolf-optimizer — GWO; alpha/beta/delta/omega hierarchy, a decays 2→0 · source · mechanism
• ant-colony-optimization — ACO (Dorigo 1992), recast for continuous spaces; pheromone trails · source · mechanism
• artificial-bee-colony — ABC (Karaboga 2005); employed/onlooker/scout bees · source · mechanism

DefinedTerm (algorithms — independent / authoritative sources)

• genetic-algorithm — GA (Holland, 1970s); the canonical evolutionary algorithm, family ancestor · source · mechanism
• differential-evolution — DE (Storn & Price 1997); vector-difference mutation; CEC baseline · source · mechanism
• simulated-annealing — SA (Kirkpatrick 1983); single-solution trajectory, cooling schedule · source · mechanism
• bayesian-optimization — BO (Mockus/Jones); model-based GP surrogate + acquisition function · source · mechanism
• nelder-mead — Nelder–Mead (1965); derivative-free direct-search downhill simplex; local, single-solution · source · mechanism
• tabu-search — Tabu search (Glover 1986); memory-based single-solution metaheuristic; tabu list escapes local optima · source · mechanism

DefinedTerm (gradient-based & exact — the non-metaheuristic half)

• gradient-descent — first-order gradient steps; the canonical gradient-based optimizer (local, needs derivatives) · source · mechanism
• stochastic-gradient-descent — SGD + momentum/AdaGrad/RMSProp/Adam; the optimizer ML training runs on · source · mechanism
• convex-optimization — convex objective ⇒ local = global; polynomial-time, guaranteed; the structure⇄generality trade · source · mechanism
• linear-programming — max cᵀx s.t. Ax ≤ b: optimum at a vertex of a convex polytope. Simplex (Dantzig 1947, exponential worst case, still open whether a variant is polynomial), ellipsoid (Khachiyan 1979 — LP is in P), interior point (Karmarkar 1984); strong duality gives a certificate no metaheuristic here can produce · source · T2 · en.wikipedia.org
• integer-programming — integrality makes it NP-complete (0–1 ILP is one of Karp's 21) while the relaxation stays polynomial; LP relaxation → branch and bound → cutting planes → branch and cut, with total unimodularity as the free case. Returns a solution and a bound on how far from optimal it is · source · T2 · en.wikipedia.org

SoftwareApplication (solvers)

• highs — HiGHS (Edinburgh, MIT licence): dual revised simplex + interior point + active-set QP + branch-and-cut MIP; the default solver behind SciPy's linprog. The spoke's first solver. Strongest fully-open entry in both Mittelmann tables and still 12.8× (LP) / 7.55× (MILP) off the publishable commercial baseline · source · T1 · highs.dev

Dataset / WebPage (benchmarks)

• miplib — MIPLIB 2017 (Zuse Institute Berlin, Gleixner & Turner): 240-instance benchmark set plus a larger collection set, instances labelled easy / hard / open. Real submitted instances with discovered difficulty, unlike this spoke's synthetic suites · source · T1 · miplib.zib.de
• mittelmann-benchmarks — Hans Mittelmann (ASU): the field's long-running independent solver comparison across LP/MILP/SDP/NLP/MINLP, shifted geometric mean. Also documents its own hollowing-out — IBM and FICO demanded removal, Gurobi and MindOpt withdrew (2018, 2024). Current LPopt/MILP tables pulled 2026-08-10 · source · T2 · plato.asu.edu

Person

• mykel-kochenderfer — author of the three MIT Press Algorithms textbooks; hardcover + free CC BY-NC-ND PDF + Julia notebooks
• andrey-dik — author of the MQL5 population-optimizer series & its comparative benchmark

Synthesis

• synthesis — the thesis: gradient-free optimizers across benchmarks; No Free Lunch made concrete (now corroborated by independent sources)