Claude Elwood Shannon (1916–2001) was an American mathematician and electrical engineer. His 1948 paper “A Mathematical Theory of Communication” (cite: Shannon, 1948) founded information theory as a mathematical discipline, providing the formal framework for quantifying, transmitting, and encoding information.
Core ideas
- Information as uncertainty reduction: Shannon defined information not as meaning but as the resolution of uncertainty. A message carries information to the extent that it narrows the set of possible states the receiver considers. This definition deliberately excludes semantics — Shannon’s information theory measures how much is communicated, not what. This exclusion is both the theory’s power (it applies universally to any communication channel) and its limit (it cannot address meaning, relevance, or significance).
- Entropy: Shannon adopted the term entropy from thermodynamics to name the average uncertainty in a random variable — equivalently, the average amount of information produced per message. A source with high entropy produces messages that are hard to predict; a source with low entropy produces messages that are easy to predict. Shannon entropy provides the theoretical limit on lossless data compression.
- Channel capacity: Shannon proved that every communication channel has a maximum rate — the channel capacity — at which information can be transmitted with arbitrarily low error probability. This theorem, the noisy channel coding theorem, is the foundational result of information theory: it guarantees that reliable communication is possible even over noisy channels, provided the transmission rate stays below capacity.
- Redundancy and noise: Shannon’s framework distinguishes signal (the intended message) from noise (unwanted disturbance in the channel). Redundancy — the inclusion of more symbols than strictly necessary — allows the receiver to detect and correct errors introduced by noise. Natural languages are highly redundant; mathematical codes can be designed with optimal redundancy for specific channels.
Notable works
- “A Mathematical Theory of Communication” (1948) (cite: Shannon, 1948)
- “Communication Theory of Secrecy Systems” (1949)
- “Prediction and Entropy of Printed English” (1951)
Related
- Norbert Wiener — fellow founder of the information-cybernetics nexus
- Cybernetics — the parallel discipline of feedback and control
- Gregory Bateson — extended information theory into ecology and mind
Shannon, C. E. (1948). A Mathematical Theory of Communication. Bell System Technical Journal, 27(3), 379–423. https://doi.org/10.1002/j.1538-7305.1948.tb01338.x