Claude Shannon
The founder of information-theory, and the author of both limits this spoke now holds: source-coding-theorem and noisy-channel-coding-theorem. Discovered while building out the coding-theory half of information-theory-inference-learning-algorithms, which is built on his results and cites his 1948 paper throughout.
What the book establishes about his method rather than his life: he proved that good codes exist without exhibiting one, by averaging the error probability over all randomly chosen codes — the move MacKay illustrates by weighing a class of babies at once. That style of argument is why fifty years separate the theorem from a code that meets it, and why the definitions in coding-theory are about families of codes rather than individual constructions.
Not from the ingested source: Shannon published A Mathematical Theory of Communication at Bell Labs in 1948, and his 1937 MIT master's thesis applied Boolean algebra to switching circuits. Neither biographical fact appears in MacKay's text, which treats him as a set of theorems.
Connections
- originator of information-theory
- author of the results on source-coding-theorem and noisy-channel-coding-theorem
- cited throughout information-theory-inference-learning-algorithms