Eigenvector, Eigenvalue, and Eigenspace

Every square matrix $A$ transforms vectors. It stretches them, rotates them, reflects them, or some combination. Most vectors change direction when $A$ acts on them, and for such a vector the effect of $A$ cannot be described by a single number. But some special vectors are mapped to scalar multiples of themselves. They may get longer or shorter or reversed in direction, but the line they lie on does not change. These are the eigenvectors of $A$. Along these lines, the action of $A$ reduces to multiplication by a scalar, and the whole transformation can be understood one line at a time.

If $\mathbf{x}$ is one of these special vectors, then $A\mathbf{x} = \lambda\mathbf{x}$ for some scalar $\lambda$. The image of the vector is the same vector, scaled by $\lambda$. That scalar is the eigenvalue. If $\lambda = 2$, the matrix doubles the vector. If $\lambda = -1$, it reverses the direction of the vector. If $\lambda = 0$, it sends the vector to the origin. The eigenvector determines which direction is preserved, and the eigenvalue determines the scaling along that direction.

How do you find these? Rearrange $A\mathbf{x} = \lambda\mathbf{x}$ into $(A - \lambda I)\mathbf{x} = \mathbf{0}$. This is a homogeneous system, and it has a nonzero solution $\mathbf{x}$ if and only if $A - \lambda I$ is singular, meaning its determinant is zero. The equation $\det(A - \lambda I) = 0$ is the characteristic equation. For an $n \times n$ matrix, expanding this determinant gives a polynomial of degree $n$ in $\lambda$, and the roots of that polynomial are the eigenvalues.

Take a concrete case. Let

\[A = \begin{bmatrix} 3 & 1 \\ 0 & 2 \end{bmatrix}.\]

Then

\[A - \lambda I = \begin{bmatrix} 3 - \lambda & 1 \\ 0 & 2 - \lambda \end{bmatrix}, \quad \det(A - \lambda I) = (3 - \lambda)(2 - \lambda).\]

Setting this to zero gives $\lambda = 3$ and $\lambda = 2$. For $\lambda = 3$, you solve $(A - 3I)\mathbf{x} = \mathbf{0}$, which reduces to $x_2 = 0$ with $x_1$ free. Every eigenvector for this eigenvalue is a scalar multiple of $(1,\,0)^\top$. For $\lambda = 2$, you get $x_1 = -x_2$, and every eigenvector is a multiple of $(-1,\,1)^\top$.

The set of all eigenvectors belonging to one eigenvalue, together with the zero vector, forms a subspace called the eigenspace for that eigenvalue. In the example above, the eigenspace for $\lambda = 3$ is $\operatorname{span}[(1,\,0)^\top]$, the $x_1$-axis. The eigenspace for $\lambda = 2$ is $\operatorname{span}[(-1,\,1)^\top]$, a line through the origin. Each eigenspace is simply the null space of $A - \lambda I$. Its dimension equals the number of independent directions the matrix leaves invariant at that scaling factor.