Skip to main content

Section7.4Orthogonal Sets

Objectives
  1. Understand which is the best method to use to compute an orthogonal projection in a given situation.
  2. Recipes: an orthonormal set from an orthogonal set, Projection Formula, B -coordinates when B is an orthogonal set, Gram–Schmidt process.
  3. Vocabulary: orthogonal set, orthonormal set.

In this section, we give a formula for orthogonal projection that is considerably simpler than the one in Section 7.3, in that it does not require row reduction or matrix inversion. However, this formula, called the Projection Formula, only works in the presence of an orthogonal basis. We will also present the Gram–Schmidt process for turning an arbitrary basis into an orthogonal one.

Subsection7.4.1Orthogonal Sets and the Projection Formula

Computations involving projections tend to be much easier in the presence of an orthogonal set of vectors.

Definition

A set of nonzero vectors { u 1 , u 2 ,..., u m } is called orthogonal if u i · u j = 0 whenever i A = j . It is orthonormal if it is orthogonal, and in addition u i · u i = 1 for all i = 1,2,..., m .

In other words, a set of vectors is orthogonal if different vectors in the set are perpendicular to each other. An orthonormal set is an orthogonal set of unit vectors.

Example

The standard coordinate vectors in R n always form an orthonormal set. For instance, in R 3 we check that

C 100 D · C 010 D = 0 C 100 D · C 001 D = 0 C 010 D · C 001 D = 0.

Since e i · e i = 1 for all i = 1,2,3, this shows that { e 1 , e 2 , e 3 } is orthonormal.

e 1 e 2 e 3
Example

We saw in the previous example that it is easy to produce an orthonormal set of vectors from an orthogonal one by replacing each vector with the unit vector in the same direction.

Recipe: An orthonormal set from an orthogonal set

If { v 1 , v 2 ,..., v m } is an orthogonal set of vectors, then

H v 1 B v 1 B , v 2 B v 2 B ,..., v m B v m B I

is an orthonormal set.

Example
Non-Example

A nice property enjoyed by orthogonal sets is that they are automatically linearly independent.

Proof

One advantage of working with orthogonal sets is that it gives a simple formula for the orthogonal projection of a vector.

Proof

If { u 1 , u 2 ,..., u m } is an orthonormal basis for W , then the denominators u i · u i = 1 go away, so the projection formula becomes even simpler:

x W =( x · u 1 ) u 1 +( x · u 2 ) u 2 + ··· +( x · u m ) u m .

Suppose that { u 1 , u 2 ,..., u m } is an orthogonal basis for a subspace W , and let L i = Span { u i } for each i = 1,2,..., m . Then we see that for any vector x , we have

x W = x · u 1 u 1 · u 1 u 1 + x · u 2 u 2 · u 2 u 2 + ··· + x · u m u m · u m u m = x L 1 + x L 2 + ··· + x L m .

In other words, for an orthogonal basis, the projection of x onto W is the sum of the projections onto the lines spanned by the basis vectors. In this sense, projection onto a line is the most important example of an orthogonal projection.

Example(Projection onto the xy -plane)
Example(Projection onto a plane in R 3 )
Example(Projection onto a 3 -space in R 4 )

Now let W be a subspace of R n with orthogonal basis B = { v 1 , v 2 ,..., v m } , and let x be a vector in W . Then x = x W , so by the projection formula, we have

x = x W = x · u 1 u 1 · u 1 u 1 + x · u 2 u 2 · u 2 u 2 + ··· + x · u m u m · u m u m .

This gives us a way of expressing x as a linear combination of the basis vectors in B : we have computed the B -coordinates of x without row reducing!

Recipe: B -coordinates when B is an orthogonal set

Let W be a subspace of R n with orthogonal basis B = { u 1 , u 2 ,..., u m } and let x be a vector in W . Then

[ x ] B = F x · u 1 u 1 · u 1 , x · u 2 u 2 · u 2 ,..., x · u m u m · u m G .

As with orthogonal projections, if { u 1 , u 2 ,..., u m } is an orthonormal basis of W , then the formula is even simpler:

[ x ] B = A x · u 1 , x · u 2 ,..., x · u m B .
Example(Computing coordinates with respect to an orthogonal basis)

The following example shows that the Projection Formula does in fact require an orthogonal basis.

Non-Example(A non-orthogonal basis)

You need an orthogonal basis to use the Projection Formula.

Subsection7.4.2The Gram–Schmidt Process

We saw in the previous subsection that orthogonal projections and B -coordinates are much easier to compute in the presence of an orthogonal basis for a subspace. In this subsection, we give a method, called the Gram–Schmidt Process, for computing an orthogonal basis of a subspace.

Proof
Example(Two vectors)
Example(Three vectors)
Example(Three vectors in R 4 )

We saw in the proof of the Gram–Schmidt Process that for every i between 1 and m , the set { u 1 , u 2 ,..., u i } is a an orthogonal basis for Span { v 1 , v 2 ,..., v i } .

If we had started with a spanning set { v 1 , v 2 ,..., v m } which is linearly dependent, then for some i , the vector v i is in Span { v 1 , v 2 ,..., v i − 1 } by the increasing span criterion in Section 3.2. Hence

0 =( v i ) Span { v 1 , v 2 ,..., v i − 1 } ⊥ =( v i ) Span { u 1 , u 2 ,..., u i − 1 } ⊥ = u i .

You can use the Gram–Schmidt Process to produce an orthogonal basis from any spanning set: if some u i = 0, just throw away u i and v i , and continue.

Subsection7.4.3Two Methods to Compute the Projection

We have now presented two methods for computing the orthogonal projection of a vector: this theorem in Section 7.3 involves row reduction, and the projection formula requires an orthogonal basis. Here are some guidelines for which to use in a given situation.

  1. If you already have an orthogonal basis, it is almost always easier to use the projection formula. This often happens in the sciences.
  2. If you are going to have to compute the projections of many vectors onto the same subspace, it is worth your time to run Gram–Schmidt to produce an orthogonal basis, so that you can use the projection formula.
  3. If you only have to project one or a few vectors onto a subspace, it is faster to use the theorem in Section 7.3. This is the method we will follow in Section 7.5.