In previous chapters, we have seen the special role of the standard ordered bases for Cn
Let V be an inner product space. A subset of V is an orthonormal basis for V if it is an ordered basis that is orthonormal.
The standard ordered basis for Fn
The set
is an orthonormal basis for R2
The next theorem and its corollaries illustrate why orthonormal sets and, in particular, orthonormal bases are so important.
Let V be an inner product space and S={v1, v2, …, vk}
Write y=k∑i=1aivi
we have
So aj=〈y, vj〉||vj||2
The next corollary follows immediately from Theorem 6.3.
If, in addition to the hypotheses of Theorem 6.3, S is orthonormal and y∈span(S)
If V possesses a finite orthonormal basis, then Corollary 1 allows us to compute the coefficients in a linear combination very easily. (See Example 3.)
Let V be an inner product space, and let S be an orthogonal subset of V consisting of nonzero vectors. Then S is linearly independent.
Suppose that v1, v2, …, vk∈S
As in the proof of Theorem 6.3 with y=0
By Corollary 2, the orthonormal set
obtained in Example 8 of Section 6.1 is an orthonormal basis for R3
and
As a check, we have
Corollary 2 tells us that the vector space H in Section 6.1 contains an infinite linearly independent set, and hence H is not a finite-dimensional vector space.
Of course, we have not yet shown that every finite-dimensional inner product space possesses an orthonormal basis. The next theorem takes us most of the way in obtaining this result. It tells us how to construct an orthogonal set from a linearly independent set of vectors in such a way that both sets generate the same subspace.
Before stating this theorem, let us consider a simple case. Suppose that {w1, w2}
To find c, we need only solve the following equation:
So
Thus
The next theorem shows us that this process can be extended to any finite linearly independent subset.
Let V be an inner product space and S={w1, w2, …, wn}
Then S′
The proof is by mathematical induction on n, the number of vectors in S. For k=1, 2, …, n,
since 〈vj, vi〉=0
The construction of {v1, v2, …, vn}
In R4
Take v1=w1=(1, 0, 1, 0)
Finally,
These vectors can be normalized to obtain the orthonormal basis {u1, u2, u3}
and
Let V=P(R)
Take v1=1
Furthermore,
Therefore
We conclude that {1, x, x2−13}
To obtain an orthonormal basis, we normalize v1, v2,
and similarly,
Thus {u1, u2, u3}
Continuing to apply the Gram-Schmidt orthogonalization process to the basis {1, x, x2, …}
The following result gives us a simple method of representing a vector as a linear combination of the vectors in an orthonormal basis.
Let V be a nonzero finite-dimensional inner product space. Then V has an orthonormal basis β
Let β0
We use Theorem 6.5 to represent the polynomial f(x)=1+2x+3x2
and
Therefore f(x)=2√2u1+2√63u2+2√105u3
Theorem 6.5 gives us a simple method for computing the entries of the matrix representation of a linear operator with respect to an orthonormal basis.
Let V be a finite-dimensional inner product space with an orthonormal basis β={v1, v2, …, vn}
Proof. From Theorem 6.5, we have
Hence Aij=〈T(vj), vi〉
The scalars 〈x, vi〉
Let β
In the first half of the 19th century, the French mathematician Jean Baptiste Fourier was associated with the study of the scalars
or in the complex case,
for a function f. In the context of Example 9 of Section 6.1, we see that cn=〈f, fn〉
Let S={eint: n is an integer}
and, for n=0
As a result of these computations, and using Exercise 16 of this section, we obtain an upper bound for the sum of a special infinite series as follows:
for every k. Now, using the fact that ||f||2=43π2
or
Because this inequality holds for all k, we may let k→∞
Additional results may be produced by replacing f by other functions.
We are now ready to proceed with the concept of an orthogonal complement.
Let S be a nonempty subset of an inner product space V. We define S⊥
It is easily seen that S⊥
The reader should verify that {0}⊥=V
If V=R3
Exercise 18 provides an interesting example of an orthogonal complement in an infinite-dimensional inner product space.
Consider the problem in R3
The next result presents a practical method of finding u in the case that W is a finite-dimensional subspace of an inner product space.
Let W be a finite-dimensional subspace of an inner product space V, and let y∈V
Let {v1, v2, …, vk}
To show that z∈W⊥
To show uniqueness of u and z, suppose that y=u+z=u′+z′, where u′∈W and z′∈W⊥. Then u−u′=z′−z∈W∩W⊥={0}. Therefore u−u′ and z=z′.
In the notation of Theorem 6.6, the vector u is the unique vector in W that is “closest” to y; that is, for any x∈W,||y−x||≥||y−u||, and this inequality is an equality if and only if x=u.
Proof.
As in Theorem 6.6, we have that y=u+z, where z∈W⊥. Let x∈W. Then u−x is orthogonal to z, so, by Exercise 10 of Section 6.1, we have
Now suppose that ||y−x||=||y−u||. Then the inequality above becomes an equality, and therefore ||u−x||2+||z||2=||z||2. It follows that ||u−x||=0, and hence x=u. The proof of the converse is obvious.
The vector u in the corollary is called the orthogonal projection of y on W. We will see the importance of orthogonal projections of vectors in the application to least squares in Section 6.3.
Let V=P3(R) with the inner product
We compute the orthogonal projection f1(x) of f(x)=x3 on P2(R).
By Example 5,
is an orthonormal basis for P2(R). For these vectors, we have
and
Hence
It was shown (Corollary 2 to the replacement theorem, p. 48) that any linearly independent set in a finite-dimensional vector space can be extended to a basis. The next theorem provides an interesting analog for an orthonormal subset of a finite-dimensional inner product space.
Suppose that S={v1, v2, …, vk} is an orthonormal set in an n-dimensional inner product space V. Then
(a) S can be extended to an orthonormal basis {v1, v2, …, vk, vk+1, …, vn} for V.
(b) If W=span(S), then S1={vk+1, vk+2, …, vn} is an orthonormal basis for W⊥ (using the preceding notation).
(c) If W is any subspace of V, then dim(V)=dim(W)+dim(W⊥).
(a) By Corollary 2 to the replacement theorem (p. 48), S can be extended to an ordered basis S′={v1, v2, …, vk, wk+1, …, wn} for V. Now apply the Gram-Schmidt process to S′. The first k vectors resulting from this process are the vectors in S by Exercise 8, and this new set spans V. Normalizing the last n−k vectors of this set produces an orthonormal set that spans V. The result now follows.
(b) Because S1 is a subset of a basis, it is linearly independent. Since S1 is clearly a subset of W⊥, we need only show that it spans W⊥. Note that, for any x∈V, we have
If x∈W⊥, then 〈x, vi〉=0 for 1≤i≤k. Therefore
(c) Let W be a subspace of V. It is a finite-dimensional inner product space because V is, and so it has an orthonormal basis {v1, v2, …, vk}. By (a) and (b), we have
Let W=span({e1, e2}) in F3. Then x=(a, b, c)∈W⊥ if and only if 0=〈x, e1〉=a and 0=〈x, e2〉=b. So x=(0, 0, c), and therefore W⊥=span({e3}). One can deduce the same result by noting that e3∈W⊥ and, from (c), that dim(W⊥)=3−2=1.
Label the following statements as true or false.
(a) The Gram-Schmidt orthogonalization process produces an orthonormal set from an arbitrary linearly independent set.
(b) Every nonzero finite-dimensional inner product space has an orthonormal basis.
(c) The orthogonal complement of any set is a subspace.
(d) If {v1, v2, …, vn} is a basis for an inner product space V, then for any x∈V the scalars 〈x, vi〉 are the Fourier coefficients of x.
(e) An orthonormal basis must be an ordered basis.
(f) Every orthogonal set is linearly independent.
(g) Every orthonormal set is linearly independent.
In each part, apply the Gram–Schmidt process to the given subset S of the inner product space V to obtain an orthogonal basis for span(S). Then normalize the vectors in this basis to obtain an orthonormal basis β for span(S), and compute the Fourier coefficients of the given vector relative to β. Finally, use Theorem 6.5 to verify your result.
(a) V=R3, S={(1, 0, 1), (0, 1, 1), (1, 3, 3)}, and x=(1, 1, 2)
(b) V=R3, S={(1, 1, 1), (0, 1, 1), (0, 0, 1)}, and x=(1, 0, 1)
(c) V=P2(R) with the inner product 〈f(x), g(x)〉=∫10f(t)g(t) dt, S={1, x, x2}, and h(x)=1+x
(d) V=span(S), where S={(1, i, 0), (1−i, 2, 4i)} and x=(3+i, 4i, −4)
(e) V=R4, S={(2, −1, −2, 4), (−2, 1, −5, 5), (−1, 3, 7, 11)}, and x=(−11, 8, −4, 18)
(f) V=R4, S={(1, −2, −1, 3), (3, 6, 3, −1), (1, 4, 2, 8)}, and x=(−1, 2, 1, 1)
(g) V=M2×2(R), S={(35−11), (−195−1), (7−172−6)}, and A=(−127−48)
(h) V=M2×2(R), S={(2221), (11425), (4−123−16)}, and A=(8625−13)
(i) V=span(S) with the inner product 〈f, g〉=∫π0f(t)g(t) dt, S={sin t, cos t, 1, t}, and h(t)=2t+1
(j) V=C4, S={(1, i, 2−i, −1), (2+3i, 3i, 1−i, 2i), (−1+7i, 6+10i, 11−4i, 3+4i)}, and x=(−2+7i, 6+9i, 9−3i, 4+4i)
(k) V=C4, S={(−4, 3−2i, i, 1−4i), (−1−5i, 5−4i, −3+5i, 7−2i), (−27−i, −7−6i, −15+25i, −7−6i)}, and x=(−13−7i, −12+3i, −39−11i, −26+5i)
(l) V=M2×2(C), S={(1−i−2−3i2+2i4+i), (8i4−3−3i−4+4i), (−25−38i−2−13i12−78i−7+24i)}, and A=(−2+8i−13+i10−10i9−9i)
(m) V=M2×2(C), S={(−1+i−i2−i1+3i), (−1−7i−9−8i1+10i−6−2i), (−11−132i−34−31i7−126i−71−5i)}, and A=(−7+5i3+18i9−6i−3+7i)
In R2, let
Find the Fourier coefficients of (3, 4) relative to β .
Let S={(1, 0, i), (1, 2, 1)} in C3. Compute S⊥.
Let S0={x0}, where x0 is a nonzero vector in R3. Describe S⊥0 geometrically. Now suppose that S={x1, x2} is a linearly independent subset of R3. Describe S⊥ geometrically.
Let V be an inner product space, and let W be a finite-dimensional subspace of V. If x∉W, prove that there exists y∈V such that y∈W⊥, but 〈x, y〉≠0. Hint: Use Theorem 6.6.
Let β be a basis for a subspace W of an inner product space V, and let z∈V. Prove that z∈W⊥ if and only if 〈z, v〉=0 for every v∈β.
Prove that if {w1, w2, …, wn} is an orthogonal set of nonzero vectors, then the vectors v1, v2, …, vn derived from the Gram-Schmidt process satisfy vi=wi for i=1, 2, …, n. Hint: Use mathematical induction.
Let W=span({(i, 0, 1)}) in C3. Find orthonormal bases for W and W⊥.
Let W be a finite-dimensional subspace of an inner product space V. Prove that V=W⊕W⊥. Using the definition on page 76, prove that there exists a projection T on W along W⊥ that satisfies N(T)=W⊥. In addition, prove that ||T(x)||≤||x|| for all x∈V. Hint: Use Theorem 6.6 and Exercise 10 of Section 6.1.
Let A be an n×n matrix with complex entries. Prove that AA*=I if and only if the rows of A form an orthonormal basis for Cn. Visit goo.gl/
Prove that for any matrix A∈Mm×n(F), (R(LA*))⊥=N(LA).
Let V be an inner product space, S and S0 be subsets of V, and W be a finite-dimensional subspace of V. Prove the following results.
(a) S0⊆S implies that S⊥⊆S⊥0.
(b) S⊆(S⊥)⊥; so span(S)⊆(S⊥)⊥.
(c) W=(W⊥)⊥. Hint: Use Exercise 6.
(d) V=W⊕W⊥. (See the exercises of Section 1.3.)
Let W1 and W2 be subspaces of a finite-dimensional inner product space. Prove that (W1+W2)⊥=W⊥1∩W⊥2 and (W1∩W2)⊥=W⊥1+W⊥2. (See the definition of the sum of subsets of a vector space on page 22.) Hint for the second equation: Apply Exercise 13(c) to the first equation.
Let V be a finite-dimensional inner product space over F.
(a) Parseval’s Identity. Let {v1, v2, …, vn} be an orthonormal basis for V. For any x, y∈V prove that
(b) Use (a) to prove that if β is an orthonormal basis for V with inner product 〈⋅, ⋅〉, then for any x, y∈V
where 〈⋅, ⋅〉′ is the standard inner product on Fn.
(a) Bessel’s Inequality. Let V be an inner product space, and let S={v1, v2, …, vn} be an orthonormal subset of V. Prove that for any x∈V we have
Hint: Apply Theorem 6.6 to x∈V and W=span(S). Then use Exercise 10 of Section 6.1.
(b) In the context of (a), prove that Bessel’s inequality is an equality if and only if x∈span(S).
Let T be a linear operator on an inner product space V. If 〈T(x), y〉=0 for all x, y∈V, prove that T=T0. In fact, prove this result if the equality holds for all x and y in some basis for V.
Let V=C([−1, 1]). Suppose that We and Wo denote the subspaces of V consisting of the even and odd functions, respectively. (See Exercise 22 of Section 1.3.) Prove that W⊥e=Wo, where the inner product on V is defined by
In each of the following parts, find the orthogonal projection of the given vector on the given subspace W of the inner product space V.
(a) V=R2, u=(2, 6), and W={(x, y): y=4x}
(b) V=R3, u=(2, 1, 3), and W={(x, y, z): x+3y−2z=0}
(c) V=P(R) with the inner product 〈f(x), g(x)〉=∫10f(t)g(t) dt, h(x)=4+3x−2x2, and W=P1(R)
In each part of Exercise 19, find the distance from the given vector to the subspace W.
Let V=C([−1, 1]) with the inner product 〈f, g〉=∫1−1f(t)g(t) dt, and let W be the subspace P2(R), viewed as a space of functions. Use the orthonormal basis obtained in Example 5 to compute the “best” (closest) second-degree polynomial approximation of the function h(t)=et on the interval [−1, 1].
Let V=C([0, 1]) with the inner product 〈f, g〉=∫10f(t)g(t) dt. Let W be the subspace spanned by the linearly independent set {t, √t}.
(a) Find an orthonormal basis for W.
(b) Let h(t)=t2. Use the orthonormal basis obtained in (a) to obtain the “best” (closest) approximation of h in W.
Let V be the vector space defined in Example 5 of Section 1.2, the space of all sequences σ in F (where F=R or F=C) such that σ(n)≠0 for only finitely many positive integers n. For σ, μ∈V, we define 〈σ, μ〉=∞∑n=1σ(n)¯μ(n). Since all but a finite number of terms of the series are zero, the series converges.
(a) Prove that 〈⋅, ⋅〉 is an inner product on V, and hence V is an inner product space.
(b) For each positive integer n, let en be the sequence defined by en(k)=δnk, where δnk is the Kronecker delta. Prove that {e1, e2, …} is an orthonormal basis for V.
(c) Let σn=e1+en and W=span({σn: n≥2}.
(i) Prove that e1∉W, so W≠V.
(ii) Prove that W⊥={0}, and conclude that W≠(W⊥)⊥.
Thus the assumption in Exercise 13(c) that W is finite-dimensional is essential.