Let V be an inner product space, and let $\text{T}:\text{ V}\to \text{V}$ be a projection. We say that T is an orthogonal projection if $\text{R}{(\text{T})}^{\perp }=\text{N}(\text{T})$ and $\text{N}{(\text{T})}^{\perp }=\text{R}(\text{T})$.

Note that by Exercise 13(c) of Section 6.2, if V is finite-dimensional, we need only assume that one of the equalities in this definition holds. For example, if $\text{R}{(\text{T})}^{\perp }=\text{N}(\text{T})$, then $\text{R}(\text{T})=\text{R}{(\text{T})}^{\perp \perp }=\text{N}{(\text{T})}^{\perp }$.

An orthogonal projection is not the same as an orthogonal operator. In Figure 6.5, T is an orthogonal projection, but T is clearly not an orthogonal operator because $||\text{T}(v)||\ne ||v||$.

Now assume that W is a finite-dimensional subspace of an inner product space V. In the notation of Theorem 6.6 (p. 347), we can define a function $\text{T}:\text{ V}\to \text{V}$ by $\text{T}(y)=u$. It is easy to show that T is an orthogonal projection on W. We can say even more—there exists exactly one orthogonal projection on W. For if T and U are orthogonal projections on W, then $\text{R}(\text{T})=\text{W}=\text{R}(\text{U})$. Hence $\text{N}(\text{T})=\text{R}{(\text{T})}^{\perp }=\text{R}{(\text{U})}^{\perp }=\text{N}(\text{U})$, and since every projection is uniquely determined by its range and null space, we have $\text{T}=\text{U}$. We call T the orthogonal projection of V on W.

To understand the geometric difference between an arbitrary projection on W and the orthogonal projection on W, let $\text{V}={\text{R}}^2$ and $\text{W}=\text{span}\{(1,1)\}$. Define U and T as in Figure 6.5, where T(v) is the foot of a perpendicular from v on the line $y=x$ and $\text{U}(a_1,a_2)=(a_1,a_1)$. Then T is the orthogonal projection of V on W, and U is a different projection on W. Note that $v-\text{T}(v)\in {\text{W}}^{\perp }$, whereas $v-\text{U}(v)\notin {\text{W}}^{\perp }$. From Figure 6.5, we see that T(v) is the “best approximation in W to v”; that is, if $w\in \text{W}$, then $||w-v||\ge ||\text{T}(v)-v||$. In fact, this approximation property characterizes T. These results follow immediately from the corollary to Theorem 6.6 (p. 348).

As an application to Fourier analysis, recall the inner product space H and the orthonormal set S in Example 9 of Section 6.1. Define a trigonometric polynomial of degree n to be a function $g\in \text{H}$ of the form

where $a_n$ or $a_{-n}$ is nonzero.

Let $f\in \text{H}$. We show that the best approximation to f by a trigonometric polynomial of degree less than or equal to n is the trigonometric polynomial whose coefficients are the Fourier coefficients of f relative to the orthonormal set S. For this result, let $\text{W}=\text{span}\left(\right\{f_j:|j|\le n\left\}\right)$, and let T be the orthogonal projection of H on W. The corollary to Theorem 6.6 (p. 348) tells us that the best approximation to f by a function in W is

For an application of this material to electronic music, visit goo.gl/EN5Fai.

An algebraic characterization of orthogonal projections follows in the next theorem.

Suppose that T is an orthogonal projection. Since ${\text{T}}^2=\text{T}$ because T is a projection, we need only show that T* exists and $\text{T}=\text{T*}$. Now $\text{V}=\text{R}(\text{T})\oplus \text{N}(\text{T})$ and $\text{R}{(\text{T})}^{\perp }=\text{N}(\text{T})$. Let $x,y\in \text{V}$. Then we can write $x=x_1+x_2$ and $y=y_1+y_2$, where $x_1,y_1\in \text{R}(\text{T})$ and $x_2,y_2\in \text{N}(\text{T})$. Hence

and

So $\langle x,\text{ T}(y)\rangle =\langle \text{T}(x),y\rangle$ for all $x,y\in \text{V}$ thus T* exists and $\text{T}=\text{T}*$.

Now suppose that ${\text{T}}^2=\text{T}=\text{T*}$. Then T is a projection by Exercise 17 of Section 2.3, and hence we must show that $\text{R}(\text{T})=\text{N}{(\text{T})}^{\perp }$ and $\text{R}{(\text{T})}^{\perp }=\text{N}(\text{T})$. Let $x\in \text{R}(\text{T})$ and $y\in \text{N}(\text{T})$. Then $x=\text{T}(x)=\text{T*}(x)$, and so

Therefore $x\in \text{N}{(\text{T})}^{\perp }$, from which it follows that $\text{R}(\text{T})\subseteq \text{N}{(\text{T})}^{\perp }$.

Let $y\in \text{N}{(\text{T})}^{\perp }$. We must show that $y\in \text{R}(\text{T})$, that is, $\text{T}(y)=y$. Now

Since $y-\text{T}(y)\in \text{N}(\text{T})$, the first term must equal zero. But also

Thus $y-\text{T}(y)=0$; that is, $y=\text{T}(y)\in \text{R}(\text{T})$. Hence $\text{R}(\text{T})=\text{N}{(\text{T})}^{\perp }$.

Using the preceding results, we have $\text{R}{(\text{T})}^{\perp }=\text{N}{(\text{T})}^{\perp \perp }\supseteq \text{N}(\text{T})$ by Exercise 13(b) of Section 6.2. Now suppose that $x\in \text{R}{(\text{T})}^{\perp }$. For any $y\in \text{V}$, we have $\langle \text{T}(x),y\rangle =\langle x,\text{ T*}(y)\rangle =\langle x,\text{ T}(y)\rangle =0$. So $\text{T}(x)=0$, and thus $x\in \text{N}(\text{T})$. Hence $\text{R}{(\text{T})}^{\perp }=\text{N}(\text{T})$.

Let V be a finite-dimensional inner product space, W be a subspace of V, and T be the orthogonal projection of V on W. We may choose an orthonormal basis $\beta =\{v_1,v_2,\dots ,v_n\}$ for V such that $\{v_1,v_2,\dots ,v_k\}$ is a basis for W. Then ${[\text{T}]}_{\beta }$ is a diagonal matrix with ones as the first k diagonal entries and zeros elsewhere. In fact, ${[\text{T}]}_{\beta }$ has the form

If U is any projection on W, we may choose a basis $\gamma$ for V such that ${[\text{U}]}_{\gamma }$ has the form above; however $\gamma$ is not necessarily orthonormal.

We are now ready for the principal theorem of this section.

(a) By Theorems 6.16 (p. 369) and 6.17 (p. 371), T is diagonalizable;

so

by Theorem 5.10 (p. 277).

(b) If $x\in {\text{W}}_i$ and $y\in {\text{W}}_j$ for some $i\ne j$, then $\langle x,y\rangle =0$ by Theorem 6.15(d) (p. 368). It follows easily from this result that ${\text{W}}_i^{\prime }\subseteq {\text{W}}_i^{\perp }$. From (a), we have

On the other hand, we have $\dim ({\text{W}}_i^{\perp })=\dim (\text{V})-\dim ({\text{W}}_i)$ by Theorem 6.7(c) (p. 349). Hence ${\text{W}}_i^{\prime }={\text{W}}_i^{\perp }$, proving (b).

(c) The proof of (c) is left as an exercise.

(d) Since ${\text{T}}_i$ is the orthogonal projection of V on ${\text{W}}_i$, it follows from (b) that $\text{N}({\text{T}}_i)=\text{R}{({\text{T}}_i)}^{\perp }={\text{W}}_i^{\perp }={\text{W}}_i^{\prime }$. Hence, for $x\in \text{V}$, we have $x=x_1+x_2+\cdots +x_k$, where ${\text{T}}_i(x)=x_i\in {\text{W}}_i$, proving (d).

(e) For $x\in \text{V}$, write $x=x_1+x_2+\cdots +x_k$, where $x_i\in {\text{W}}_i$. Then

The set $\{{\lambda }_1,{\lambda }_2,\dots ,{\lambda }_k\}$ of eigenvalues of T is called the spectrum of T, the sum $\text{I}={\text{T}}_1+{\text{T}}_2+\cdots +{\text{T}}_k$ in (d) is called the resolution of the identity operator induced by T, and the sum $\text{T}={\lambda }_1{\text{T}}_1+{\lambda }_2{\text{T}}_2+\cdots +{\lambda }_k{\text{T}}_k$ in (e) is called the spectral decomposition of T. The spectral decomposition of T is unique up to the order of its eigenvalues.

With the preceding notation, let $\beta$ be the union of orthonormal bases of the ${\text{W}}_i$’s and let $m_i=\dim ({\text{W}}_i)$. (Thus $m_i$ is the multiplicity of ${\lambda }_i$.) Then ${[\text{T}]}_{\beta }$ has the form

that is, ${[\text{T}]}_{\beta }$ is a diagonal matrix in which the diagonal entries are the eigenvalues ${\lambda }_i$ of T, and each ${\lambda }_i$ is repeated $m_i$ times. If ${\lambda }_1{\text{T}}_1+{\lambda }_2{\text{T}}_2+\cdots +{\lambda }_k{\text{T}}_k$ is the spectral decomposition of T, then it follows (from Exercise 7) that $g(\text{T})=g({\lambda }_1){\text{T}}_1+g({\lambda }_2){\text{T}}_2+\cdots +g({\lambda }_k){\text{T}}_k$ for any polynomial g. This fact is used below.

We now list several interesting corollaries of the spectral theorem; many more results are found in the exercises. For what follows, we assume that T is a linear operator on a finite-dimensional inner product space V over F.

Suppose first that T is normal. Let $\text{T}={\lambda }_1{\text{T}}_1+{\lambda }_2{\text{T}}_2+\cdots +{\lambda }_k{\text{T}}_k$ be the spectral decomposition of T. Taking the adjoint of both sides of the preceding equation, we have $\text{T*}={\overline{\lambda }}_1{\text{T}}_1+{\overline{\lambda }}_2{\text{T}}_2+\cdots +{\overline{\lambda }}_k{\text{T}}_k$ since each ${\text{T}}_i$ is self-adjoint. Using the Lagrange interpolation formula (see page 53), we may choose a polynomial g such that $g({\lambda }_i)={\overline{\lambda }}_i$ for $1\le i\le k$. Then

Conversely, if $\text{T*}=g(\text{T})$ for some polynomial g, then T commutes with T* since T commutes with every polynomial in T. So T is normal.

If T is unitary, then T is normal and every eigenvalue of T has absolute value 1 by Corollary 2 to Theorem 6.18 (p. 379).

Let $\text{T}={\lambda }_1{\text{T}}_1+{\lambda }_2{\text{T}}_2+\cdots +{\lambda }_k{\text{T}}_k$ be the spectral decomposition of T. If $|\lambda |=1$ for every eigenvalue $\lambda$ of T, then by (c) of the spectral theorem,

Hence T is unitary.

Suppose that T is normal and that its eigenvalues are real. Let $\text{T}={\lambda }_1{\text{T}}_1+{\lambda }_2{\text{T}}_2+\cdots +{\lambda }_k{\text{T}}_k$ be the spectral decomposition of T. Then

Hence T is self-adjoint.

Now suppose that T is self-adjoint and hence normal. That its eigenvalues are real has been proved in the lemma to Theorem 6.17 (p. 371).

Choose a polynomial $g_j(1\le j\le k)$ such that $g_j({\lambda }_i)={\delta }_{ij}$. Then