optimization-algorithms-wiki

Metaheuristic (population-based) optimization

Metaheuristic optimization is the family of gradient-free, general-purpose search algorithms that seek a global optimum of an objective function over a multidimensional space by iteratively improving one or many candidate solutions. The population-based subfamily (evolutionary, swarm, and nature/behavior-inspired) maintains a set of candidates that explore the space and exploit the best regions over generations. The umbrella concept of this wiki.

Why they exist

They make almost no assumptions about the objective — it can be non-differentiable, noisy, discontinuous, multi-modal, black-box — so they apply where classical/exact methods can’t. The price: no optimality guarantee, and performance that is problem- and tuning-dependent (no-free-lunch-theorem).

Common anatomy

• A population of candidate solutions, scored by a fitness/objective function.
• A variation step (mutation/crossover/sampling/movement) that generates new candidates.
• A selection step (often greedy/elitist) that keeps the better ones.
• A balance of exploration-vs-exploitation governing diversity vs. refinement.

The corpus, by flavor

Population-based (a set of candidates evolving together):

• Evolution strategies / EAs: genetic-algorithm (the ancestor) · evolution-strategies ((μ,λ)/(μ+λ)-ES) · cma-es (covariance adaptation) · differential-evolution (vector-difference mutation) · backtracking-search-algorithm (archive memory).
• Swarm / nature-inspired: particle-swarm-optimization · grey-wolf-optimizer · ant-colony-optimization · artificial-bee-colony.
• Behavior-inspired: exchange-market-algorithm.
• Deterministic outlier: deterministic-oscillatory-search.

Beyond population-based (the corpus now spans wider):

• Single-solution / trajectory: simulated-annealing — one state, a cooling schedule.
• Model-based / surrogate: bayesian-optimization — a learned model picks the next sample (a different regime: few, expensive evaluations).

The Dik subset is scored on his population-optimization-benchmark; the rest are grounded in independent, authoritative sources and standard test-functions-for-optimization (see synthesis).

Related

exploration-vs-exploitation · no-free-lunch-theorem · population-optimization-benchmark · test-functions-for-optimization · genetic-algorithm · evolution-strategies · cma-es · differential-evolution · backtracking-search-algorithm · exchange-market-algorithm · deterministic-oscillatory-search · particle-swarm-optimization · grey-wolf-optimizer · ant-colony-optimization · artificial-bee-colony · simulated-annealing · bayesian-optimization