optimization-algorithms-wiki

Log — Optimization Algorithms Wiki

Append-only history. Entries start with ## [YYYY-MM-DD] <op> | <title>.

[2026-06-09] split | optimization-algorithms-wiki created from _inbox (4 sources)

Spun out of the hub optimization-algorithms _inbox cluster once it blew past 3 — the user dropped four entries from Andrey Dik’s MQL5 optimizer-benchmark series in quick succession: exchange-market-algorithm (EMA), backtracking-search-algorithm (BSA), cma-es, and deterministic-oscillatory-search (DOS). EMA & BSA had been parked in _inbox (cluster 1→2); CMA-ES completed the cluster and DOS arrived during the build. Scaffolded raw/+wiki/, CLAUDE.md from the hub template, and these spine files; added the registry block to ../wikis.md; bumped the hub spoke count 12 → 13. Founding page set: 4 source/algorithm pages + concepts (metaheuristic-optimization, exploration-vs-exploitation, population-optimization-benchmark, no-free-lunch-theorem) + andrey-dik. All sources URL-only (raw/ empty).

[2026-06-09] ingest | founding sources (EMA, BSA, CMA-ES, DOS — MQL5 series)

Ingested the four cluster sources per Ingest. See synthesis: these are all population-based metaheuristics scored on one comparative benchmark (population-optimization-benchmark), and the headline is No Free Lunch — even the celebrated cma-es only ranked 38/45 on this suite, while simple ideas varied wildly (BSA 20/45; EMA & DOS near the bottom). Axes that emerge: stochastic-vs-deterministic (deterministic-oscillatory-search is fully reproducible), scalability (cma-es is O(n³), fails at high dimensions), and the universal exploration-vs-exploitation balance.

[2026-06-09] ingest | +5 classics from the Dik series (PSO, GWO, ACO, ABC, ES)

Expanded the spoke by five more entries from andrey-dik‘s MQL5 series, adding the canonical swarm/EA classics the founding four lacked: particle-swarm-optimization (PSO, Kennedy & Eberhart 1995), grey-wolf-optimizer (GWO), ant-colony-optimization (ACO, Dorigo — recast for continuous spaces), artificial-bee-colony (ABC, Karaboga 2005), and evolution-strategies ((μ,λ)-ES & (μ+λ)-ES, Rechenberg/Schwefel). Spoke now 9 algorithms / 14 pages. Two findings sharpen the no-free-lunch-theorem thesis (see synthesis): (μ+λ)-ES leads the entire mature suite at 72.18% — a 1970s strategy beating every modern method, with the comma/plus survival rule alone swinging rank ~20 points — and PSO ranked below the random-search baseline in Dik’s early scoring. Also surfaced a benchmark-comparability caveat: Dik’s scoring evolved (early small-field 0–1 over Skin/Forest/Megacity vs. mature %-of-MAX over Hilly/Forest/Megacity), so PSO/GWO/ACO/ABC numbers are not comparable to the founding-four percentages — population-optimization-benchmark now keeps the two schemes in separate tables.

[2026-06-09] ingest | +5 independent authoritative sources (Wikipedia) — break single-author dependence

At the user’s request, expanded with 5 non-Dik, authoritative (Wikipedia) sources so the thesis no longer rests on one MQL5 series: differential-evolution (Storn & Price), genetic-algorithm (Holland — family ancestor), simulated-annealing (Kirkpatrick 1983 — the corpus’s first single-solution / trajectory method), test-functions-for-optimization (the independent academic benchmark basis — Rastrigin/Rosenbrock/Ackley behind CEC/BBOB), and bayesian-optimization (the corpus’s first model-based / surrogate optimizer). Spoke now 14 algorithms/concepts / 19 wiki pages. Effects on the thesis (see synthesis): (a) two new axes — population-vs-single-solution and model-free-vs-model-based; (b) independent corroboration of no-free-lunch-theorem (GA’s article concedes SA/hill-climbing “often outperform” it; SA’s convergence guarantee is practically useless); (c) the CMA-ES Dik-vs-BBOB tension promoted from open question to a recorded contradiction (≈38/45 on Dik vs. near-top on BBOB — kept both, flagged, not overwritten). Refreshed no-free-lunch-theorem, metaheuristic-optimization, and population-optimization-benchmark in place with the independent grounding.

[2026-06-09] ingest | +3 the non-metaheuristic half (gradient descent, SGD/Adam, convex) — all-spokes cron test

Closed the “where do gradient-based / exact methods fit?” open question with three Wikipedia-sourced foundations: gradient-descent (DefinedTerm, src — first-order, derivative-using, local), stochastic-gradient-descent (DefinedTerm, src — SGD + momentum/AdaGrad/RMSProp/Adam; the optimizer ML training runs on; bridge to llm-providers/llm-inference), convex-optimization (DefinedTerm, src — local=global, polynomial-time; the structure⇄generality trade). Reframed the whole wiki: founding corpus = the gradient-free/black-box/ no-guarantee quadrant; these are its complements. Synthesis open question answered + unifying “structure ⇄ generality” frame made explicit (an NFL restatement). 19 → 22 pages.

[2026-06-10] ingest | Nelder–Mead + Tabu search — all-spokes pass (two classic missing quadrants)

Two canonical methods filling gaps in the gradient/exact/metaheuristic map (spoke heavily grown 06-09, so kept to 2 high-value classics). nelder-mead (DefinedTerm, source, Wikipedia) — Nelder & Mead 1965 downhill-simplex; derivative-free direct search that is local + (largely) deterministic + single-solution (n+1 simplex via reflection/expansion/contraction/shrink) — a quadrant neither the global-stochastic metaheuristics nor the gradient/exact methods occupied. tabu-search (DefinedTerm, source, Wikipedia) — Glover 1986; memory-based single-solution metaheuristic completing the single-solution axis beside simulated-annealing (both accept worsening moves; SA=stochastic cooling vs TS=deterministic tabu-list memory), strong on combinatorial/discrete problems the continuous population-optimization-benchmark doesn’t test. Both reinforce no-free-lunch-theorem by niche. Folded into synthesis (new 2026-06-10 section) + index (2 DefinedTerm rows). No contradictions. 22 → 24 pages.

[2026-06-12] ingest | COCO / BBOB benchmarking platform — numbbo

All-spokes daily expansion. Added coco-bbob (@type SoftwareApplication) — the neutral academic benchmarking platform the “how neutral is the benchmark?” open question wanted. COCO (“Comparing Continuous Optimisers”) runs the GECCO BBOB workshops (2009–present): standard suites (bbob, bbob-largescale, bbob-biobj, mixed-integer), fixed-target/ERT methodology, multi-language bindings, automated experiment→figure pipeline — reproducible and cross-comparable, the counterweight to one author’s MQL5 suite. Gives the recorded CMA-ES contradiction (~38/45 Dik vs near-top BBOB) a concrete academic home without overturning Dik (record-don’t-overwrite); also covers discrete/mixed-integer regimes the continuous-only MQL5 suite omits. Open question reframed (platform now paged; literal Dik-in-COCO cross-run still pending). Wired to test-functions-for-optimization / cma-es / no-free-lunch-theorem; synthesis (open Q + contradiction) + index updated. 1 new page. Authoritative (academic standard platform).

[2026-06-15] ingest | NFL primary paper (Wolpert & Macready 1997) — T1 anchor for the central theorem

Quality cycle, T1-floor raise. The spoke’s organizing claim (no-free-lunch-theorem) was grounded only in Andrey Dik’s MQL5 blog (T3) + Wikipedia echoes; added the canonical primary as a source page: nfl-original-paper (ScholarlyArticle, T1) — Wolpert & Macready, No Free Lunch Theorems for Optimization, IEEE TEC 1(1):67–82, 1997. Captures Theorem 1 (∑_f P(d^y_m|f,m,a) identical across algorithms), the cost-value-sequence framework, the time-varying NFL, the alignment-with-P(f) / geometric reading, and head-to-head minimax. Separates theorem (T1 primary) from empirical illustration (Dik suite, T3). Sourcing note: found via WebSearch; the IEEE PDF didn’t machine-extract (no pdftotext/pypdf in env), so formal statements were corroborated against the Wikipedia NFL article and the well-established canonical formulation — recorded transparently on the page. Linked from the concept page, synthesis (“Independent corroboration”), and index (new ScholarlyArticle section). 1 new page.