Lecture 4

Sep 25, 2026

Spectral theory: stationary vectors, eigenvalues, eigenspaces, and linear independence.

Spectral Theory

We will start this section with motivating what spectral theory is.

Note (Reminder: what we are looking for)

A matrix usually changes a vector’s direction as well as its length. Spectral theory looks for nonzero vectors that the matrix only scales:

M x = λ x .

When $λ = 1$ , the vector stays exactly the same. This is the case we will start with.

A stationary vector

Let

M = \frac{1}{4} [\begin{matrix} 2 & 1 \\ 2 & 3 \end{matrix}] \in ℝ^{2 \times 2}

be a Markov matrix. Determine a stationary vector

x = (\begin{matrix} x_{1} \\ x_{2} \end{matrix}), x_{1}, x_{2} \geq 0, x_{1} + x_{2} = 1,

such that $M x = x$ .

Note (Reminder: a probability vector)

Here we use column vectors, so the entries in each column of $M$ add up to $1$ . The entries of $x$ describe how the total probability is split between two states.

The equation $M x = x$ means that this split stays the same after one step. An eigenvector can be scaled, but a probability vector also needs nonnegative entries that add up to $1$ .

Solution

Let

A = 4 M = [\begin{matrix} 2 & 1 \\ 2 & 3 \end{matrix}] .

A stationary vector $x$ satisfies $M x = x$ . Since $A = 4 M$ ,

M x = x ⟺ A x = 4 x ⟺ (A - 4 I) x = 0.

Now,

A - 4 I = [\begin{matrix} - 2 & 1 \\ 2 & - 1 \end{matrix}] \sim [\begin{matrix} - 2 & 1 \\ 0 & 0 \end{matrix}] .

So,

- 2 x_{1} + x_{2} = 0 ⟺ x_{2} = 2 x_{1} .

We normalize $x$ using $x_{1} + x_{2} = 1$ :

x_{1} + x_{2} = x_{1} + 2 x_{1} = 3 x_{1} = 1.

This gives

x_{1} = \frac{1}{3}, x_{2} = \frac{2}{3} .

The stationary vector is

x = [\begin{matrix} \frac{1}{3} \\ \frac{2}{3} \end{matrix}] .

Note (Reminder: scaling the matrix)

We multiplied $M$ by $4$ to avoid fractions. The vector stays the same, but its eigenvalue changes: $M x = x$ becomes $A x = 4 x$ . That is why we subtract $4 I$ from $A$ , rather than $I$ .

Here $I$ is the identity matrix. The row operation above is $R_{2} \leftarrow R_{2} + R_{1}$ , and the symbol $\sim$ means that the matrices are row-equivalent.

Remarks

The equation
$M x = x$
is the eigenvalue equation $M x = λ x$ with $λ = 1$ .
We have
$M x = x ⟺ (M - I) x = 0,$
which is a homogeneous system of linear equations.
The eigenspace corresponding to $λ = 1$ is
$E_{1} (M) = \ker (M - I) = span {(\begin{matrix} 1 \\ 2 \end{matrix})} = {t (\begin{matrix} 1 \\ 2 \end{matrix}) : t \in ℝ} .$

Eigenvectors, eigenvalues, and eigenspaces

Definition

Let $V$ be a vector space over the field $𝕂$ , and let

T \in ℒ (V, V) .

A nonzero vector $v \in V$ is called an eigenvector of $T$ if there exists a scalar $λ \in 𝕂$ such that
$T v = λ v .$
The scalar $λ$ is called an eigenvalue of $T$ if there exists a nonzero vector $v \in V$ satisfying
$T v = λ v .$
The eigenspace corresponding to $λ$ is
$E_{λ} (T) = {x \in V : T x = λ x} = \ker (T - λ I) .$

If $λ$ is an eigenvalue, then $E_{λ} (T)$ consists of the zero vector together with all eigenvectors corresponding to $λ$ .

Note (Reminder: the notation)

$ℒ (V, V)$ means the set of linear maps from $V$ to itself. We need the same space on both sides because $T v$ is being compared with a multiple of $v$ .

The identity map satisfies $I v = v$ , so

T v = λ v ⟺ T v - λ I v = 0 ⟺ (T - λ I) v = 0.

This is why finding eigenvectors for a given $λ$ is a kernel problem. The kernel is a subspace, so it includes $0$ even though $0$ is not an eigenvector.

Note (The zero vector)

The zero vector is not an eigenvector. Indeed,

T 0 = 0 = λ 0 for every λ \in 𝕂 .

So, the equation $T v = λ v$ only defines an eigenvector when $v \neq 0$ .

The number $0$ can still be an eigenvalue. This happens when $T v = 0$ for some nonzero vector $v$ .

Eigenvectors with distinct eigenvalues

Theorem

Let $V$ be a vector space over $𝕂$ , and let

T \in ℒ (V, V) .

Suppose that $v_{1}, \dots, v_{n}$ are eigenvectors of $T$ corresponding to distinct eigenvalues $λ_{1}, \dots, λ_{n}$ . Then

{v_{1}, \dots, v_{n}}

is linearly independent.

Note (Reminder: what we need to prove)

Linear independence means that

α_{1} v_{1} + \dots + α_{n} v_{n} = 0

can only happen when every coefficient is zero. We will start with a relation like this and show that all the coefficients have to be zero.

Proof

We prove the statement by induction on $n$ .

For $n = 1$ , the set consists of a single eigenvector $v_{1}$ . By definition, $v_{1} \neq 0$ , and therefore ${v_{1}}$ is linearly independent.

Now assume that any $n$ eigenvectors corresponding to distinct eigenvalues are linearly independent. Let

v_{1}, \dots, v_{n + 1}

be eigenvectors corresponding to the distinct eigenvalues

λ_{1}, \dots, λ_{n + 1} .

Suppose that

α_{1} v_{1} + \dots + α_{n + 1} v_{n + 1} = 0.

Applying $T$ to both sides gives

α_{1} λ_{1} v_{1} + \dots + α_{n + 1} λ_{n + 1} v_{n + 1} = 0.

Multiplying the original equation by $λ_{1}$ gives

α_{1} λ_{1} v_{1} + \dots + α_{n + 1} λ_{1} v_{n + 1} = 0.

Subtracting this equation from the previous one gives

α_{2} (λ_{2} - λ_{1}) v_{2} + \dots + α_{n + 1} (λ_{n + 1} - λ_{1}) v_{n + 1} = 0.

Note (Reminder: why we subtract)

The $v_{1}$ terms cancel. We now have a relation involving only $v_{2}, \dots, v_{n + 1}$ : exactly $n$ eigenvectors with distinct eigenvalues. This lets us use the induction hypothesis.

Their coefficients in this new relation are $α_{j} (λ_{j} - λ_{1})$ , rather than just $α_{j}$ .

By the induction hypothesis, the vectors

v_{2}, \dots, v_{n + 1}

are linearly independent. Therefore,

α_{j} (λ_{j} - λ_{1}) = 0, j = 2, \dots, n + 1.

Since the eigenvalues are distinct,

λ_{j} - λ_{1} \neq 0,

and so

α_{2} = \dots = α_{n + 1} = 0.

Substituting this into the original equation gives

α_{1} v_{1} = 0.

Since $v_{1} \neq 0$ , it follows that $α_{1} = 0$ . All coefficients are zero, so $v_{1}, \dots, v_{n + 1}$ are linearly independent. $□$