If you want to derive the Einstein field equations from scratch, you can do so without making very many assumptions. You must assume that
- the geometry of spacetime is dynamical;
- there are no extra fixed, nondynamical "background structures" that influence the geometry;
- special relativity becomes a good approximation when gravitational fields are weak;
- the field equations can be derived from a Lagrangian, or an action principle; and
- the field equations involve no more than second derivatives; that is, they determine "accelerations" rather than requiring accelerations as initial data.
These assumptions lead almost uniquely to a set of field equations with two undetermined constants. One of these is Newton's constant, which determines the strength of the gravitational interaction. The other is the cosmological constant, Λ ("Lambda").
In modern terms, the cosmological constant looks like a very peculiar kind of gravitating matter, one that pervades the Universe with a constant density and an extremely high, negative pressure. Einstein originally introduced this term in order to allow static cosmological solutions to the field equations, and it is said that he later called it his biggest blunder. But it's not so easy to put this genie back in the bottle. The cosmological constant can be interpreted as the energy density of the vacuum, and can arise for at least two reasons
- In quantum field theory, the vacuum is not really "empty," but is filled with fields that briefly appear because of quantum fluctuations. If you start with a theory with no cosmological constant, these vacuum fluctuations will create one. Conversely, to end up with no cosmological constant, you have to cancel off the vacuum fluctuations very precisely. This is tricky, because the fluctuations occur at all energy scales; you either need a mechanism that can simultaneously deal with all scales of distance and energy at once, or you have to postulate extraordinarily accurate "fine tuning" that allows effects at one scale to exactly cancel effects at very different scales.
- The vacuum energy changes when matter fields undergo phase transitions and spontaneous symmetry-breaking, essentially because spontaneous symmetry-breaking determines the nature of the physical ground state. Many "inflationary universe" models rely on this mechanism -- they require a cosmological constant in the very early Universe to produce exponential expansion, followed by a phase transition that eliminates Λ. The question is again why the present value should be exactly (or very nearly) zero.
We can measure Λ by looking at its effect on cosmology. A value significantly different from zero in "particle physics" units would lead, depending on sign, to exponential expansion and a cold, empty universe, or to a universe that would have recollapsed long before its present age. A nonzero value of Λ is not ruled out observationally, and in fact there is some pretty good evidence for a small positive cosmological constant (although there are other possible explanations for the observed acceleration of expansion of the Universe). But the value of the vacuum energy density must be, at most, comparable in magnitude to the density of ordinary matter in the Universe, and this is is tiny compared to the values one expects from particle physics -- it's some 120 orders of magnitude smaller than one would predict from simple dimensional analysis.
While a number of more or less exotic suggestions have been floated, no one really knows why Λ should be so small. This "cosmological constant problem" is one of the biggest mysteries in modern physics.
See Ned Wright's cosmology tutorial for more about Λ.\
- Steve Carlip (Physics, UC Davis)