6.1: Basic Facts about Eigenvalue Problems

Even if a matrix \(\mathbf{A}\) is real, its eigenvectors and eigenvalues can be complex. For example,

\[\begin{bmatrix}1&1\\-1&1\end{bmatrix} \begin{bmatrix}1\\i\end{bmatrix} = (1+i) \begin{bmatrix}1\\i\end{bmatrix}.\]

Eigenvectors are not uniquely defined. Given an eigenvector \(\vec{x}\), any nonzero complex multiple of that vector is also an eigenvector of the same matrix, with the same eigenvalue. We can reduce this ambiguity by normalizing eigenvectors to a fixed unit length:

\[\sum_{n=0}^{N-1} |x_n|^2 = 1.\]

Note, however, that even after normalization, there is still an inherent ambiguity in the overall complex phase. Multiplying a normalized eigenvector by any phase factor \(e^{i\phi}\) gives another normalized eigenvector with the same eigenvalue.

6.1.1 Matrix Diagonalization

Most matrices are diagonalizable, meaning that their eigenvectors span the \(N\)-dimensional complex space (where \(N\) is the matrix size). Matrices which are not diagonalizable are called defective. Many classes of matrices that are relevant to physics (such as Hermitian matrices) are always diagonalizable; i.e., never defective.

The reason for the term "diagonalizable" is as follows. A diagonalizable \(N\times N\) matrix \(\mathbf{A}\) has eigenvectors that span the \(N\)-dimensional space, meaning that we can choose \(N\) linearly independent eigenvectors, \(\{\vec{x}_0, \vec{x}_1, \cdots \vec{x}_{N-1}\}\), with eigenvalues \(\{\lambda_0, \lambda_1, \cdots \lambda_{N-1}\}\). We refer to such a set of \(N\) eigenvalues as the "eigenvalues of \(\mathbf{A}\)". If we group the eigenvectors into an \(N\times N\) matrix

\[\mathbf{Q} = [\vec{x}_0, \vec{x}_1, \cdots \vec{x}_{N-1}],\]

then, since the eigenvectors are linearly independent, \(\mathbf{Q}\) is guaranteed to be invertible. Using the eigenvalue equation, we can then show that

\[\mathbf{Q}^{-1} \,\mathbf{A} \, \mathbf{Q} = \begin{bmatrix}\lambda_0 & 0& \cdots & 0 \\ 0 & \lambda_1 & \cdots & 0 \\ \vdots& \vdots & \ddots & \vdots \\ 0&0&\cdots&\lambda_{N-1}\end{bmatrix}.\]

In other words, there exists a similarity transformation which converts \(\mathbf{A}\) into a diagonal matrix. The \(N\) numbers along the diagonal are precisely the eigenvalues of \(\mathbf{A}\).

6.1.2 The Characteristic Polynomial

One of the most important consequences of diagonalizability is that the determinant of a diagonalizable matrix \(\mathbf{A}\) is the product of its eigenvalues:

\[\det(\mathrm{A}) = \prod_{n=0}^{N-1} \lambda_n\]

This can be proven by taking the determinant of the similarity transformation equation, and using (i) the property of the determinant that \(\det(\mathbf{U}\mathbf{V}) = \det(\mathbf{U})\det(\mathbf{V})\), and (ii) the fact that the determinant of a diagonal matrix is the product of the elements along the diagonal.

In particular, the determinant of \(\mathbf{A}\) is zero if one of its eigenvalues is zero. This fact can be further applied to the following re-arrangement of the eigenvalue equation:

\[\Big(\mathbf{A} - \lambda\mathbf{I}\Big) \, \vec{x} = 0,\]

where \(\mathbf{I}\) is the \(N\times N\) identity matrix. This says that the matrix \(\mathbf{A}-\lambda\mathbf{I}\) has an eigenvalue of zero, meaning that for any eigenvalue \(\lambda\),

\[\det\left(\mathbf{A} - \lambda\mathbf{I}\right) = 0.\]

The left-hand side of the above equation is a polynomial in the variable \(\lambda\), of degree \(N\). This is called the characteristic polynomial of the matrix \(\mathbf{A}\). Its roots are eigenvalues of \(\mathbf{A}\), and vice versa.

For \(2\times 2\) matrices, the standard way of calculating the eigenvalues is to find the roots of the characteristic polynomial. However, this is not a reliable method for finding the eigenvalues of larger matrices. There is a well-known and important result in mathematics, known as Abel's impossibility theorem, which states that polynomials of degree \(5\) and higher have no general algebraic solution. (By comparison, degree-2 polynomials have a general algebraic solution, which is the familiar quadratic formula, and similar formulas exist for degree-3 and degree-4 polynomials.) A matrix of size \(N \ge 5\) has a characteristic polynomial of degree \(N \ge 5\), and Abel's impossibility theorem tells us that we can't calculate the roots of that characteristic polynomial by ordinary arithmetic.

In fact, Abel's impossibility theorem leads to an even stronger conclusion: there is no general algebraic method for finding the eigenvalues of a matrix of size \(N \ge 5\), whether using the characteristic polynomial or any other method. For suppose we had such a method for finding the eigenvalues of a matrix. Then, for any polynomial equation of degree \(N \ge 5\), of the form

\[a_0 + a_1 \lambda + \cdots + a_{N-1} \lambda^{N-1} + \lambda^N = 0,\]

we can construct an \(N\times N\) "companion matrix" of the form

\[\mathbf{A} = \begin{bmatrix}0 & 1 & 0 & \cdots & 0 \\ 0 & 0 & 1& \cdots & 0 \\ \vdots&\vdots&\ddots&\ddots& \vdots \\ 0&0&0&\ddots&1\\-a_0& -a_1& -a_2 & \cdots & -a_{N-1} \end{bmatrix}.\]

As you can check for yourself, each root \(\lambda\) of the polynomial is also an eigenvalue of the companion matrix, with corresponding eigenvector

\[\vec{x} = \begin{bmatrix}1\\\lambda\\ \vdots \\ \lambda^{N-1}\end{bmatrix}.\]

Hence, if there exists a general algebraic method for finding the eigenvalues of a large matrix, that would allow us to find solve polynomial equations of high degree. Abel's impossibility theorem tells us that no such solution method can exist.

This might seem like a terrible problem, but in fact there's a way around it, as we'll shortly see.

6.1.3 Hermitian Matrices

A Hermitian matrix \(\mathbf{H}\) is a matrix which has the property

\[\mathbf{H}^\dagger = \mathbf{H},\]

where \(\mathbf{H}^\dagger\) denotes the "Hermitian conjugate", which is matrix transposition accompanied by complex conjugation:

\[\mathbf{H}^\dagger \equiv \left(\mathbf{H}^T\right)^*, \quad \mathrm{i.e.}\;\;\left(H^\dagger\right)_{ij} = H_{ji}^*.\]

Hermitian matrices have the nice property that all their eigenvalues are real. This can be easily proven using index notation:

\[\begin{align}\sum_j H_{ij} x_j = \lambda x_i \;\;&\Rightarrow\;\; \sum_j x_j^* H_{ji} = \lambda^* x_i^*\\ &\Rightarrow \sum_{ij} x^*_i H_{ij} x_j = \lambda \sum_i |x_i|^2 = \lambda^* \sum_j |x_j|^2 \\ &\Rightarrow \lambda = \lambda^*.\end{align}\]

In quantum mechanics, Hermitian matrices play a special role: they represent measurement operators, and their eigenvalues (which are restricted to the real numbers) are the set of possible measurement outcomes.