10: Entropy and Information
- Page ID
- 32050
\( \newcommand{\vecs}[1]{\overset { \scriptstyle \rightharpoonup} {\mathbf{#1}} } \) \( \newcommand{\vecd}[1]{\overset{-\!-\!\rightharpoonup}{\vphantom{a}\smash {#1}}} \)\(\newcommand{\id}{\mathrm{id}}\) \( \newcommand{\Span}{\mathrm{span}}\) \( \newcommand{\kernel}{\mathrm{null}\,}\) \( \newcommand{\range}{\mathrm{range}\,}\) \( \newcommand{\RealPart}{\mathrm{Re}}\) \( \newcommand{\ImaginaryPart}{\mathrm{Im}}\) \( \newcommand{\Argument}{\mathrm{Arg}}\) \( \newcommand{\norm}[1]{\| #1 \|}\) \( \newcommand{\inner}[2]{\langle #1, #2 \rangle}\) \( \newcommand{\Span}{\mathrm{span}}\) \(\newcommand{\id}{\mathrm{id}}\) \( \newcommand{\Span}{\mathrm{span}}\) \( \newcommand{\kernel}{\mathrm{null}\,}\) \( \newcommand{\range}{\mathrm{range}\,}\) \( \newcommand{\RealPart}{\mathrm{Re}}\) \( \newcommand{\ImaginaryPart}{\mathrm{Im}}\) \( \newcommand{\Argument}{\mathrm{Arg}}\) \( \newcommand{\norm}[1]{\| #1 \|}\) \( \newcommand{\inner}[2]{\langle #1, #2 \rangle}\) \( \newcommand{\Span}{\mathrm{span}}\)\(\newcommand{\AA}{\unicode[.8,0]{x212B}}\)
The concept of entropy is one of the more difficult concepts in physics. Historically, it emerged as a consequence of the second law of thermodynamics, as in Equation 3.4.13. Later, Boltzmann gave a general definition for it in terms of the number of ways of distributing a given number of particles, as in Equation 7.2.3. But a clearer understanding of entropy is related to its interpretation in terms of information. We will briefly discuss this point of view here.
- 10.1: Information
- We want to give a quantification of the idea of information. This is originally due to C. Shannon.
- 10.2: Maxwell’s Demon
- There is a very interesting thought experiment due to Maxwell which is perhaps best phrased as a potential violation of the second law of thermodynamics. The resolution of this problem highlights the role of entropy as information.
- 10.3: Entropy and Gravity
- There is something deep about the concept of entropy which is related to gravity. This is far from being well understood, and is atopic of ongoing research, but there are good reasons to think that the Einstein field equations for gravity may actually emerge as some some sort of entropy maximization condition. A point of contact between gravity and entropy is for spacetimes with a horizon, an example being a black hole.