23: Entropy

So far we have taken a purely empirical view of the properties of systems composed of many atoms. However, as previously noted, it is possible to understand such systems using the underlying principles of mechanics. The resulting branch of physics is called statistical mechanics. J. Willard Gibbs, a late 19th century American physicist from Yale University, almost single-handedly laid the groundwork for the modern form of this subject. Interestingly, the quantum mechanical version of statistical mechanics is much easier to understand than the version that Gibbs developed, which is based on classical mechanics. It also gives correct answers where the Gibbs version fails.

A system of many atoms has many quantum mechanical states in which it can exist. Think of, say, a brick. The atoms in a brick are not stationary; they are in a continual flurry of vibration at ordinary temperatures. The kinetic and potential energies associated with these vibrations constitute the internal energy of the brick.

Though the details of each state are unimportant, the number of states turns out to be a crucial piece of information. To understand why this is so, let us imagine two bricks identical in composition and mass. Brick A has internal energy between \(E\) and \(\mathrm{E}+\Delta \mathrm{E}\)and brick B has energy between 0 and \(\Delta \mathrm{E} \text { . }\). Think of \(\Delta \mathrm{E}\) as the uncertainty in the energy of the bricks; we can only observe a brick for a finite amount of time Δt, so the uncertainty principle asserts that the uncertainty in the energy is \(\Delta \mathrm{E} \approx \hbar / \Delta \mathrm{t}\).

The brick is a complex system consisting of many atoms, so in general there are many possible quantum mechanical states available to brick A in the energy range E to E + ΔE. It turns out, for reasons that we will see later, that significantly fewer states are available to brick B in the energy range 0 to ΔE than are available to brick A.

Roughly speaking, the larger the internal energy of an object per unit mass, the higher is its temperature. Thus, we infer that brick A has a much higher temperature than brick B. What happens when we bring the two bricks into thermal contact? Our experience tells us that heat (i. e., internal energy) immediately starts to flow from one brick to the other, ultimately resulting in an equilibrium state in which the temperature is the same in the two bricks.

We explain this process as follows. Statistical mechanics hypothesizes that any system of atoms (such as a brick) is free to roam through all quantum mechanical states that are energetically available to it. In fact, this roaming is assumed to be continually taking place. Given this picture and the assumption that the roaming between states is completely random, one would expect equal probabilities for finding the system in any particular state.

Of course, this probability argument assumes that we don’t know anything about the initial state of the system. If the system is known to be in some particular state at time t = 0, then it will take some time for the system to evolve in such a way that it has “forgotten” the initial state. During this interval our knowledge of the initial state and the quantum mechanical dynamics of the system can be used (in principle) to follow the evolution of the system. Eventually the uncertainty in our initial knowledge of the system catches up with us and we cannot predict the future evolution of the system beyond this point. The brick develops “amnesia” and its probability of being in any of the energetically allowed states is then uniform.

Something like this happens to the two bricks if they are brought into thermal contact. Initially brick A has virtually all of the energy and brick B has only a tiny amount. When the bricks are brought into contact, they eventually can be treated as a single brick of twice the size. However, it takes time for the new, larger brick to evolve to the point where it has forgotten the fact that it started out as two separate bricks at different temperatures. In this interval the temperature of brick A is decreasing while the temperature of brick B is increasing as a result of internal energy flowing from one to the other. This evolution continues until equilibrium is reached.

Even though the combined brick has forgotten its initial state, there is a small chance that it will return to this state, since the probability of finding the brick in any state, including the original one, is non-zero. Thus, according to the postulates of statistical mechanics, one might suddenly find the brick again in a state in which virtually all of the internal energy is concentrated in former brick A. Actually, the issue is slightly more complicated than this. Brick A actually had many states available to it before being brought together with brick B. Thus, a more interesting problem is to find the probability of the system suddenly finding itself in any of the states in which (virtually) all of the energy is concentrated in former brick A. Given the randomness assumption of statistical mechanics, this probability is simply the number of states that correspond to all of the energy being in brick A, divided by the total number of states available to the combined brick. Computing this number is the task we set for ourselves.