INTRODUCTION TO VECTOR SPACES
LINEAR ALGEBRA
W W L CHEN
c W W L Chen, 1994, 2005.This chapter is available free to all individuals, on the understanding that it is not to be used for financial gain, and may be downloaded and/or photocopied, with or without permission from the author.
However, this document may not be kept on any information storage and retrieval system without permission from the author, unless such system is not accessible to any individuals other than its owners.
Chapter 5 INTRODUCTION TO VECTOR SPACES
5.1. Real Vector Spaces
Before we give any formal definition of a vector space, we shall consider a few concrete examples of such an abstract object. We first study two examples from the theory of vectors which we first discussed in
Chapter 4. Example 5.1.1. Consider the set R
1 , u
3 ), where u
(2.5) For every u ∈ R
2 , we have 1u = u.
Example 5.1.2.
Consider the set R
3 of all vectors of the form u = (u
1 , u
2 , u
1 , u
(2.4) For every a, b ∈ R and u ∈ R
2 , u 3 ∈ R.
Consider vector addition and also multiplication of vectors by real numbers. It is easy to check that we have properties analogous to (1.1)–(1.5) and (2.1)–(2.5) in the previous example, with reference to R
2 being replaced by R
3 .
We next turn to an example from the theory of matrices which we first discussed in Chapter 2.
1 , u
2 of all vectors of the form u = (u
2 , we have (ab)u = a(bu).
2 , we have (a + b)u = au + bu.
2 ∈ R. Consider vector addition and also multiplication of vectors by real numbers. It is easy to check that we have the following properties:
(1.4) For every u ∈ R
(1.1) For every u, v ∈ R
2 , we have u + v ∈ R
2 .
(1.2) For every u, v, w ∈ R
2 , we have u + (v + w) = (u + v) + w.
(1.3) For every u ∈ R
2 , we have u + 0 = 0 + u = u.
2 , we have u + (−u) = 0.
(2.3) For every a, b ∈ R and u ∈ R
(1.5) For every u, v ∈ R
2 , we have u + v = v + u.
(2.1) For every c ∈ R and u ∈ R
2 , we have cu ∈ R
2 .
2 ), where u
2 , we have c(u + v) = cu + cv.
(2.2) For every c ∈ R and u, v ∈ R Example 5.1.3.
Consider the set M 2,2 (R) of all 2 × 2 matrices with entries in R. Consider matrix addition and also multiplication of matrices by real numbers. Denote by O the 2 × 2 null matrix. It is easy to check that we have the following properties:
(2.5) For every P ∈ M 2,2 (R), we have 1P = P .
Remark.
(VA2) For every u, v, w ∈ V , we have u + (v + w) = (u + v) + w. (VA3) There exists an element 0 ∈ V such that for every u ∈ V , we have u + 0 = 0 + u = u. (VA4) For every u ∈ V , there exists −u ∈ V such that u + (−u) = 0. (VA5) For every u, v ∈ V , we have u + v = v + u. (SM1) For every c ∈ R and u ∈ V , we have cu ∈ V . (SM2) For every c ∈ R and u, v ∈ V , we have c(u + v) = cu + cv. (SM3) For every a, b ∈ R and u ∈ V , we have (a + b)u = au + bu. (SM4) For every a, b ∈ R and u ∈ V , we have (ab)u = a(bu). (SM5) For every u ∈ V , we have 1u = u.
A vector space V over R, or a real vector space V , is a set of objects, known as vectors, together with vector addition + and multiplication of vectors by element of R, and satisfying the following properties: (VA1) For every u, v ∈ V , we have u + v ∈ V .
Definition.
There are many more examples of sets where properties analogous to (1.1)–(1.5) and (2.1)–(2.5) in the four examples above hold. This apparent similarity leads us to consider an abstract object which will incorporate all these individual cases as examples. We say that these examples are all vector spaces over R.
(1.1) For every f, g ∈ A, we have f + g ∈ A. (1.2) For every f, g, h ∈ A, we have f + (g + h) = (f + g) + h. (1.3) For every f ∈ A, we have f + λ = λ + f = f . (1.4) For every f ∈ A, we have f + (−f ) = λ. (1.5) For every f, g ∈ A, we have f + g = g + f . (2.1) For every c ∈ R and f ∈ A, we have cf ∈ A. (2.2) For every c ∈ R and f, g ∈ A, we have c(f + g) = cf + cg. (2.3) For every a, b ∈ R and f ∈ A, we have (a + b)f = af + bf . (2.4) For every a, b ∈ R and f ∈ A, we have (ab)f = a(bf ). (2.5) For every f ∈ A, we have 1f = f .
f, g ∈ A, define the function f + g : R → R by writing (f + g)(x) = f (x) + g(x) for every x ∈ R. For every function f ∈ A and every number c ∈ R, define the function cf : R → R by writing (cf )(x) = cf (x) for every x ∈ R. Denote by λ : R → R the function where λ(x) = 0 for every x ∈ R. Then it is easy to check that we have the following properties:
Consider the set A of all functions of the form f : R → R. For any two functions
We also turn to an example from the theory of functions. Example 5.1.4.
(2.4) For every a, b ∈ R and P ∈ M 2,2 (R), we have (ab)P = a(bP ).
(1.1) For every P, Q ∈ M 2,2
(2.3) For every a, b ∈ R and P ∈ M 2,2 (R), we have (a + b)P = aP + bP .
(2.2) For every c ∈ R and P, Q ∈ M 2,2 (R), we have c(P + Q) = cP + cQ.
2,2 (R).
2,2 (R), we have cP ∈ M
(1.5) For every P, Q ∈ M 2,2 (R), we have P + Q = Q + P . (2.1) For every c ∈ R and P ∈ M
2,2 (R), we have P + (−P ) = O.
(1.3) For every P ∈ M 2,2 (R), we have P + O = O + P = P . (1.4) For every P ∈ M
(1.2) For every P, Q, R ∈ M 2,2 (R), we have P + (Q + R) = (P + Q) + R.
(R), we have P + Q ∈ M 2,2 (R).
The elements a, b, c ∈ R discussed in (SM1)–(SM5) are known as scalars. Multiplication of vectors by elements of R is sometimes known as scalar multiplication. n Example 5.1.5.
, . . . , u Let n ∈ N. Consider the set R of all vectors of the form u = (u 1 n ), where n u , . . . , u ∈ R. For any two vectors u = (u , . . . , u ) and v = (v , . . . , v ) in R and any number c ∈ R,
1 n 1 n 1 n write u + v = (u + v , . . . , u n + v n ) and cu = (cu , . . . , cu n ).
1
1
1 To check (VA1), simply note that u +v , . . . , u +v ∈ R. To check (VA2), note that if w = (w , . . . , w ),
1 1 n n 1 n then u + (v + w) = (u , . . . , u ) + (v + w , . . . , v + w ) = (u + (v + w ), . . . , u + (v + w ))
1 n
1 1 n n
1
1 1 n n n = ((u + v ) + w , . . . , (u + v ) + w ) = (u + v , . . . , u + v ) + (w , . . . , w )
1
1 1 n n n
1 1 n n 1 n = (u + v) + w.
If we take 0 to be the zero vector (0, . . . , 0), then u + 0 = 0 + u = u, giving (VA3). Next, writing −u = (−u , . . . , −u ), we have u + (−u) = 0, giving (VA4). To check (VA5), note that
1 n u , . . . , u , . . . , v
- v = (u 1 + v
1 n + v n ) = (v 1 + u 1 n + u n ) = v + u.
, . . . , cu To check (SM1), simply note that cu 1 n ∈ R. To check (SM2), note that c , . . . , u
(u + v) = c(u 1 + v 1 n + v n ) = (c(u 1 + v 1 ), . . . , c(u n + v n )) , . . . , cu , . . . , cu , . . . , cv
= (cu 1 + cv 1 n + cv n ) = (cu 1 n ) + (cv 1 n ) = cu + cv. To check (SM3), note that
, . . . , , . . . , au (a + b)u = ((a + b)u (a + b)u n ) = (au + bu n + bu n )
1
1
1 , . . . , au , . . . , bu = (au n ) + (bu n ) = au + bu.
1
1 To check (SM4), note that
, . . . , , . . . , bu (ab)u = ((ab)u (ab)u n ) = (a(bu ), . . . , a(bu n )) = a(bu n ) = a(bu).
1
1
1 Finally, to check (SM5), note that 1u = (1u , . . . , 1u n ) = (u , . . . , u n ) = u.
1
1 n It follows that R is a vector space over R. This is known as the n-dimensional euclidean space.
Example 5.1.6.
Let k ∈ N. Consider the set P of all polynomials of the form k k p (x) = p + p x + . . . + p x , where p , p , . . . , p ∈ R.
1 k 1 k In other words, P k is the set of all polynomials of degree at most k and with coefficients in R. For any k k x x x x two polynomials p(x) = p + p
1 + . . . + p k and q(x) = q + q 1 + . . . + q k in P k and for any number c ∈ R, write k k p cp x x .
(x) + q(x) = (p + q ) + (p 1 + q 1 )x + . . . + (p k + q k )x and (x) = cp + cp 1 + . . . + cp k , . . . , p
To check (VA1), simply note that p + q k + q k ∈ R. To check (VA2), note that if we write k r x x
(x) = r + r 1 + . . . + r k , then we have k k p x x
(x) + (q(x) + r(x)) = (p + p 1 + . . . + p k ) + ((q + r ) + (q 1 + r 1 )x + . . . + (q k + r k )x ) k
= (p + (q + r )) + (p 1 + (q 1 + r 1 ))x + . . . + (p k + (q k + r k ))x k
= ((p + q ) + r ) + ((p 1 + q 1 ) + r 1 )x + . . . + ((p k + q k ) + r k )x k k x x
= ((p + q ) + (p 1 + q 1 )x + . . . + (p k + q k )x ) + (r + r 1 + . . . + r k ) = (p(x) + q(x)) + r(x). k If we take 0 to be the zero polynomial 0 + 0x + . . . + 0x , then p(x) + 0 = 0 + p(x) = p(x), giving (VA3). k
Next, writing −p(x) = −p − p x − . . . − p x , we have p(x) + (−p(x)) = 0, giving (VA4). To check 1 k
(VA5), note that k p
(x) + q(x) = (p + q ) + (p 1 + q 1 )x + . . . + (p k + q k )x k = (q + p ) + (q + p )x + . . . + (q + p )x = q(x) + p(x).
1 1 k k , . . . , cp
To check (SM1), simply note that cp k ∈ R. To check (SM2), note that k c
(p(x) + q(x)) = c((p + q ) + (p + q )x + . . . + (p k + q k )x )
1
1 k = c(p + q ) + c(p 1 + q 1 )x + . . . + c(p k + q k )x k
= (cp + cq ) + (cp 1 + cq 1 )x + . . . + (cp k + cq k )x k k x x x x
= (cp + cp 1 + . . . + cp k ) + (cq + cq 1 + . . . + cq k ) = cp(x) + cq(x). To check (SM3), note that k x x
(a + b)p(x) = (a + b)p + (a + b)p + . . . + (a + b)p k
1 k = (ap + bp ) + (ap + bp )x + . . . + (ap k + bp k )x
1
1 k k x x x x
= (ap + ap 1 + . . . + ap k ) + (bp + bp 1 + . . . + bp k ) = ap(x) + bp(x). To check (SM4), note that k k x x
(ab)p(x) = (ab)p + (ab)p + . . . + (ab)p k = a(bp ) + a(bp )x + . . . + a(bp k )x
1
1 k x x = a(bp + bp + . . . + bp k ) = a(bp(x)).
1 Finally, to check (SM5), note that k k x x x x 1p(x) = 1p + 1p + . . . + 1p k = p + p + . . . + p k = p(x).
1
1 It follows that P k is a vector space over R. Note also that the vectors are the polynomials.
There are a few simple properties of vector spaces that we can deduce easily from the definition. PROPOSITION 5A. Suppose that V is a vector space over R, and that u ∈ V and c ∈ R.
(a) We have 0u = 0. (b) We have c0 = 0. (c) We have (−1)u = −u. (d) If cu = 0, then c = 0 or u = 0.
Proof.
(a) By (SM1), we have 0u ∈ V . Hence 0u + 0u = (0 + 0)u (by (SM3)), = 0u (since 0 ∈ R).
It follows that 0u = 0u + 0 (by (VA3)),
= 0u + (0u + (−(0u))) (by (VA4)), = (0u + 0u) + (−(0u)) (by (VA2)), = 0u + (−(0u)) (from above), = 0 (by (VA4)).
(b) By (SM1), we have c0 ∈ V . Hence c0
- c0 = c(0 + 0) (by (SM2)), = c0 (by (VA3)).
It follows that c0 = c0 + 0 (by (VA3)), = c0 + (c0 + (−(c0))) (by (VA4)), = (c0 + c0) + (−(c0)) (by (VA2)), = c0 + (−(c0)) (from above), = 0 (by (VA4)).
(c) We have (−1)u = (−1)u + 0 (by (VA3)),
= (−1)u + (u + (−u)) (by (VA4)), = ((−1)u + u) + (−u) (by (VA2)), = ((−1)u + 1u) + (−u) (by (SM5)), = ((−1) + 1)u + (−u) (by (SM3)), = 0u + (−u) (since 1 ∈ R), = 0 + (−u) (from (a)), = −u (by (VA3)). −1
(d) Suppose that cu = 0 and c = 0. Then c ∈ R and u = 1u (by (SM5)), −1 = (c c )u (since c ∈ R \ {0}), −1 = c (cu) (by (SM4)), −1 = c (assumption), = 0 (from (b)), as required.
5.2. Subspaces
2 Example 5.2.1.
Consider the vector space R of all points (x, y), where x, y ∈ R. Let L be a line through the origin 0 = (0, 0). Suppose that L is represented by the equation αx + βy = 0; in other words,
2 L = {(x, y) ∈ R : αx + βy = 0}.
Note first of all that 0 = (0, 0) ∈ L, so that (VA3) and (VA4) clearly hold in L. Also (VA2) and (VA5) clearly hold in L. To check (VA1), note that if (x, y), (u, v) ∈ L, then αx + βy = 0 and αu + βv = 0, so that α(x + u) + β(y + v) = 0, whence (x, y) + (u, v) = (x + u, y + v) ∈ L. Next, note that (SM2)–(SM5) clearly hold in L. To check (SM1), note that if (x, y) ∈ L, then αx + βy = 0, so that α(cx) + β(cy) = 0, whence c(x, y) = (cx, cy) ∈ L. It follows that L forms a vector space over R. In fact, we have shown
2 that every line in R through the origin is a vector space over R. Definition.
Suppose that V is a vector space over R, and that W is a subset of V . Then we say that W is a subspace of V if W forms a vector space over R under the vector addition and scalar multiplication defined in V .
2 Example 5.2.2.
We have just shown in Example 5.2.1 that every line in R through the origin is a
2 subspace of R . On the other hand, if we work through the example again, then it is clear that we have really only checked conditions (VA1) and (SM1) for L, and that 0 = (0, 0) ∈ L.
PROPOSITION 5B.
Suppose that V is a vector space over R, and that W is a non-empty subset of
V
. Then W is a subspace of V if the following conditions are satisfied: (SP1) For every u, v ∈ W , we have u + v ∈ W . (SP2) For every c ∈ R and u ∈ W , we have cu ∈ W .
Proof.
To show that W is a vector space over R, it remains to check that W satisfies (VA2)–(VA5) and (SM2)–(SM5). To check (VA3) and (VA4) for W , it clearly suffices to check that 0 ∈ W . Since W is non-empty, there exists u ∈ W . Then it follows from (SP2) and Proposition 5A(a) that 0 = 0u ∈ W . The remaining conditions (VA2), (VA5) and (SM2)–(SM5) hold for all vectors in V , and hence also for all vectors in W .
3 Example 5.2.3.
Consider the vector space R of all points (x, y, z), where x, y, z ∈ R. Let P be a plane through the origin 0 = (0, 0, 0). Suppose that P is represented by the equation αx + βy + γz = 0; in other words,
2 P = {(x, y, z) ∈ R : αx + βy + γz = 0}.
To check (SP1), note that if (x, y, z), (u, v, w) ∈ P , then αx + βy + γz = 0 and αu + βv + γw = 0, so that α(x + u) + β(y + v) + γ(z + w) = 0, whence (x, y, z) + (u, v, w) = (x + u, y + v, z + w) ∈ P . To check (SP2), note that if (x, y, z) ∈ P , then αx + βy + γz = 0, so that α(cx) + β(cy) + γ(cz) = 0, whence
3 c
(x, y, z) = (cx, cy, cz) ∈ P . It follows that P is a subspace of R . Next, let L be a line through the
3 origin 0 = (0, 0, 0). Suppose that (α, β, γ) ∈ R is a non-zero point on L. Then we can write
L = {t(α, β, γ) : t ∈ R}. Suppose that u = t(α, β, γ) ∈ L and v = s(α, β, γ) ∈ L, and that c ∈ R. Then u + v = t(α, β, γ) + s
(α, β, γ) = (t + s)(α, β, γ) ∈ L, giving (SP1). Also, cu = c(t(α, β, γ)) = (ct)(α, β, γ) ∈ L, giving (SP2).
3
3 It follows that L is a subspace of R . Finally, it is not difficult to see that both {0} and R are subspaces
3 of R .
2
3
2
3 Example 5.2.4.
Note that R is not a subspace of R . First of all, R is not a subset of R . Note also
2
3 that vector addition and scalar multiplication are different in R and R .
Example 5.2.5.
Suppose that A is an m × n matrix and 0 is the m × 1 zero column matrix. Consider the system Ax = 0 of m homogeneous linear equations in the n unknowns x , . . . , x n , where
1 x
1 x ..
= . x n n is interpreted as an element of the vector space R , with usual vector addition and scalar multiplication. Let S denote the set of all solutions of the system. Suppose that x, y ∈ S and c ∈ R. Then A(x + y) = Ax + Ay = 0 + 0 = 0, giving (SP1). Also, A(cx) = c(Ax) = c0 = 0, giving (SP2). It follows that S is n a subspace of R . To summarize, the space of solutions of a system of m homogeneous linear equations n in n unknowns is a subspace of R .
3 Example 5.2.6.
As a special case of Example 5.2.5, note that if we take two non-parallel planes in R through the origin 0 = (0, 0, 0), then the intersection of these two planes is clearly a line through the origin. However, each plane is a homogeneous equation in the three unknowns x, y, z ∈ R. It follows that
3 the intersection of the two planes is the collection of all solutions (x, y, z) ∈ R of the system formed by the two homogeneous equations in the three unknowns x, y, z representing these two planes. We have
3 already shown in Example 5.2.3 that the line representing all these solutions is a subspace of R .
Example 5.2.7.
We showed in Example 5.1.3 that the set M (R) of all 2 × 2 matrices with entries in 2,2
R forms a vector space over R. Consider the subset a a
11
12 W = : a , a , a ∈ R
11
12
21 a
21 of M 2,2 (R). Since a a b b a a a a ca ca
- b + b
11
12
11
12
11
11
12
12
11
12
11
12 c ,
- a b a a ca
- b
= and =
21
21
21
21
21
21 it follows that (SP1) and (SP2) are satisfied. Hence W is a subspace of M (R). 2,2 Example 5.2.8.
We showed in Example 5.1.4 that the set A of all functions of the form f : R → R forms a vector space over R. Let C denote the set of all functions of the form f : R → R which are continuous at x = 2, and let C 1 denote the set of all functions of the form f : R → R which are differentiable at x
= 2. Then it follows from the arithmetic of limits and the arithmetic of derivatives that C and C 1 are both subspaces of A. Furthermore, C 1 is a subspace of C (why?). On the other hand, let k ∈ N. Recall from Example 5.1.6 the vector space P of all polynomials of the form k k p x x , , p , . . . , p
(x) = p + p 1 + . . . + p k where p 1 k ∈ R. In other words, P k is the set of all polynomials of degree at most k and with coefficients in R. Clearly P k is a subspace of C 1 .
5.3. Linear Combination
In this section and the next two, we shall study ways of describing the vectors in a vector space V . Our ultimate goal is to be able to determine a subset B of vectors in V and describe every element of V in terms of elements of B in a unique way. The first step in this direction is summarized below. Definition.
, . . . , v Suppose that v 1 r are vectors in a vector space V over R. By a linear combination
, . . . , v of the vectors v 1 r , we mean an expression of the type c v v ,
1 1 + . . . + c r r , . . . , c where c r ∈ R.
1
2 Example 5.3.1.
In R , every vector (x, y) is a linear combination of the two vectors i = (1, 0) and j = (0, 1), for clearly (x, y) = xi + yj.
3 Example 5.3.2.
In R , every vector (x, y, z) is a linear combination of the three vectors i = (1, 0, 0), j = (0, 1, 0) and k = (0, 0, 1), for clearly (x, y, z) = xi + yj + zk.
4 Example 5.3.3.
In R , the vector (1, 4, −2, 6) is a linear combination of the two vectors (1, 2, 0, 4) and (1, 1, 1, 3), for we have (1, 4, −2, 6) = 3(1, 2, 0, 4) − 2(1, 1, 1, 3). On the other hand, the vector (2, 6, 0, 9) is not a linear combination of the two vectors (1, 2, 0, 4) and (1, 1, 1, 3), for
(2, 6, 0, 9) = c 1 (1, 2, 0, 4) + c 2 (1, 1, 1, 3) would lead to the system of four equations c 1 + c 2 = 2,
2c + c = 6,
1
2 c = 0,
2 4c + 3c = 9.
1
2 It is easily checked that this system has no solutions.
Example 5.3.4.
2
2
2 5.1.4, the function cos 2x is a linear combination of the three functions cos x , cosh x and sinh x . It is not too difficult to check that
2
2
2 cos 2x = 2 cos x + sinh x − cosh x,
2
2
2 noting that cos 2x = 2 cos x − 1 and cosh x − sinh x = 1.
2 We observe that in Example 5.3.1, every vector in R is a linear combination of the two vectors i and
3 j . Similarly, in Example 5.3.2, every vector in R is a linear combination of the three vectors i, j and k.
4 On the other hand, we observe that in Example 5.3.3, not every vector in R is a linear combination of the two vectors (1, 2, 0, 4) and (1, 1, 1, 3).
Let us therefore investigate the collection of all vectors in a vector space that can be represented as linear combinations of a given set of vectors in V . Definition.
Suppose that v , . . . , v r are vectors in a vector space V over R. The set
1 span{v , . . . , v r } = {c v + . . . + c r v r : c , . . . , c r ∈ R}
1
1
1
1 is called the span of the vectors v , . . . , v r . We also say that the vectors v , . . . , v r span V if
1
1 span{v , . . . , v r } = V ;
1 in other words, if every vector in V can be expressed as a linear combination of the vectors v , . . . , v .
1 r
2 Example 5.3.5.
The two vectors i = (1, 0) and j = (0, 1) span R .
3 Example 5.3.6.
The three vectors i = (1, 0, 0), j = (0, 1, 0) and k = (0, 0, 1) span R .
4 Example 5.3.7.
The two vectors (1, 2, 0, 4) and (1, 1, 1, 3) do not span R . PROPOSITION 5C. , . . . , v
Suppose that v 1 r are vectors in a vector space V over R.
, . . . , v
(a) Then span{v 1 r } is a subspace of V .
, . . . , v , . . . , v
(b) Suppose further that W is a subspace of V and v
1 r ∈ W . Then span{v 1 r } ⊆ W .
Proof.
, . . . , v , . . . , a , b , . . . , b (a) Suppose that u, w ∈ span{v 1 r } and c ∈ R. There exist a 1 r 1 r ∈ R such that u v v w v v .
= a
1 1 + . . . + a r r and = b
1 1 + . . . + b r r Then u v v v v
- w = (a
1 1 + . . . + a r r ) + (b
1 1 + . . . + b r r ) , . . . , v
= (a 1 + b 1 )v 1 + . . . + (a r + b r )v r ∈ span{v 1 r } and cu = c(a v + . . . + a v ) = (ca )v + . . . + (ca )v ∈ span{v , . . . , v }.
1 1 r r
1 1 r r 1 r , . . . , v
It follows from Proposition 5B that span{v 1 r } is a subspace of V .
(b) Suppose that c , . . . , c r ∈ R and u = c v + . . . + c r v r ∈ span{v , . . . , v r }. If v , . . . , v r ∈ W ,
1
1
1
1
1 then it follows from (SM1) for W that c v , . . . , c r v r ∈ W . It then follows from (VA1) for W that
1
1 u = c v + . . . + c r v r ∈ W .
1
1 Example 5.3.8.
In R , any non-zero vector v spans the subspace {cv : c ∈ R}. This is clearly a line through the origin. Also, try to draw a picture to convince yourself that any two non-zero vectors that 2 are not on the same line span R .
3 Example 5.3.9.
In R , try to draw pictures to convince yourself that any non-zero vector spans a subspace which is a line through the origin; any two non-zero vectors that are not on the same line span a subspace which is a plane through the origin; and any three non-zero vectors that do not lie on the
3 same plane span R .
5.4. Linear Independence We first study two simple examples.
3 Example 5.4.1.
Consider the three vectors v = (1, 2, 3), v = (3, 2, 1) and v = (3, 3, 3) in R . Then
1
2
3 span{v , v , v } = {c (1, 2, 3) + c (3, 2, 1) + c (3, 3, 3) : c , c , c ∈ R}
1
2
3
1
2
3
1
2
3 = {(c + 3c + 3c , 2c + 2c + 3c , 3c + c + 3c ) : c , c , c ∈ R}.
1
2
3
1
2
3
1
2
3
1
2
3 , ,
Write (x, y, z) = (c 1 + 3c 2 + 3c 3 2c 1 + 2c 2 + 3c 3 3c 1 + c 2 + 3c 3 ). Then it is not difficult to see that x c
1
3
3
1 y c ,
=
2
2
3
2 z c
3
1
3
3 and so (do not worry if you cannot understand why we take this next step)
x
1
3 3 c c
1
1
( 1 −2 1 ) y = ( 1 −2 1 )
2
2 3 c = ( 0 0 ) c = ( 0 ) ,
2
2 z
3
1 3 c c
3
3
3 so that x − 2y + z = 0. It follows that span{v , v , v } is a plane through the origin and not R . Note,
1
2
3 in fact, that 3v + 3v − 4v = 0. Note also that
1
2
3
1
3
3 det
2
2 3 = 0.
3
1
3
3 Example 5.4.2.
Consider the three vectors v = (1, 1, 0), v = (5, 1, −3) and v = (2, 7, 4) in R . Then
1
2
3 , v , v , c , c span{v
1
2 3 } = {c 1 (1, 1, 0) + c 2 (5, 1, −3) + c 3 (2, 7, 4) : c
1
2 3 ∈ R} , c , , c , c
= {(c 1 + 5c 2 + 2c
3 1 + c 2 + 7c 3 −3c 2 + 4c 3 ) : c
1
2 3 ∈ R}.
, c , Write (x, y, z) = (c 1 + 5c 2 + 2c
3 1 + c 2 + 7c 3 −3c 2 + 4c 3 ). Then it is not difficult to see that x c
1
5
2
1 y c ,
=
1
1
7
2 z c 0 −3
4
3 so that
x c −25 26 −33 −25 26 −33
1
5
2
1 y c 4 −4
5 = 4 −4
5
1
1
7
2 z c 3 −3
4 3 −3 4 0 −3
4
3 1 c c
1
1
= 1 c = c .
2
2 1 c c
3
3
3 It follows that for every (x, y, z) ∈ R , we can find c , c , c ∈ R such that (x, y, z) = c v + c v + c v .
1
2
3
1
1
2
2
3
3
3 Hence span{v , v , v } = R . Note that
1
2
3
1
5
2 det
1
1 7 = 0, 0 −3 4 and that the only solution for
(0, 0, 0) = c v + c v + c v
1
1
2
2
3
3 is c = c = c = 0.
1
2
3 Definition.
, . . . , v Suppose that v 1 r are vectors in a vector space V over R.
, . . . , v , . . . , c (LD) We say that v 1 r are linearly dependent if there exist c 1 r ∈ R, not all zero, such that c v v
1 1 + . . . + c r r = 0.
, . . . , v (LI) We say that v 1 r are linearly independent if they are not linearly dependent; in other words, if the only solution of c v + . . . + c v = 0 in c , . . . , c ∈ R is given by c = . . . = c = 0.
1 1 r r 1 r 1 r Example 5.4.3.
Let us return to Example 5.4.1 and consider again the three vectors v = (1, 2, 3),
1
3 v = (3, 2, 1) and v = (3, 3, 3) in R . Consider the equation c v + c v + c v = 0. This can be
2
3
1
1
2
2
3
3 rewritten in matrix form as
c
1
3
3
1 c .
2
2
3 2 = c
3
1
3
3 Since
1
3
3 det
2
2 3 = 0,
3
1
3 , c , c the system has non-trivial solutions; for example, (c ) = (3, 3, −4), so that 3v + 3v − 4v = 0.
1
2
3
1
2
3 , v , v Hence v are linearly dependent.
1
2
3 Example 5.4.4.
Let us return to Example 5.4.2 and consider again the three vectors v 1 = (1, 1, 0),
3 v v v v
2 = (5, 1, −3) and v 3 = (2, 7, 4) in R . Consider the equation c
1 1 + c
2 2 + c
3 3 = 0. This can be rewritten in matrix form as
c
1
5
2
1 c .
1
1 7 =
2 c
−3
4
3 Since
1
5
2 det
1
1 7 = 0, −3
4 , v , v the only solution is c
1 = c 2 = c 3 = 0. Hence v
1
2 3 are linearly independent.
Example 5.4.5.
In the vector space A of all functions of the form f : R → R described in Example 5.1.4,
2 , c , c the functions x, x and sin x are linearly independent. To see this, note that for every c ∈ R, the
1
2
3
2 linear combination c + c + c sin x is never identically zero unless c = c = c = 0.
1
2
3
1
2
3 n Example 5.4.6.
, . . . , e In R , the vectors e 1 n , where
, e j = (0, . . . , 0 1, 0, . . . , 0 ) for every j = 1, . . . , n, j−1 n−j are linearly independent (why?).
3 We observe in Examples 5.4.3–5.4.4 that the determination of whether a collection of vectors in R are linearly dependent is based on whether a system of homogeneous linear equations has non-trivial n solutions. The same idea can be used to prove the following result concerning R . n
PROPOSITION 5D. , . . . , v
Suppose that v
1 r are vectors in the vector space R . If r > n, then v , . . . , v 1 r are linearly dependent.
Proof.
For every j = 1, . . . , r, write v , . . . , a j = (a 1j nj ). v v
Then the equation c + . . . + c r r = 0 can be rewritten in matrix form as
1
1 a . . . a c
11 1r
1 ..
.. .. .. = .
. . . . a . . . a c n nr r
1 If r > n, then there are more variables than equations. It follows that there must be non-trivial solutions
c , . . . , c , . . . , v r ∈ R. Hence v r are linearly dependent.
1
1
2 Remarks.
(1) Consider two vectors v = (a , a ) and v = (a , a ) in R . To study linear indepen-
1
11
21
2
12
22 dence, we consider the equation c v + c v = 0, which can be written in matrix form as
1
1
2
2 a a c
11
12
1 .
= a a c
21
22
2 The vectors v and v are linearly independent precisely when
1
2 a a
11
12 det = 0. a a
21
22 This can be interpreted geometrically in the following way: The area of the parallelogram formed by the two vectors v and v is in fact equal to the absolute value of the determinant of the matrix formed
1
2 with v and v as the columns; in other words,
1
2 a a
11
12 . det a a
21
22 It follows that the two vectors are linearly dependent precisely when the parallelogram has zero area; in other words, when the two vectors lie on the same line. On the other hand, if the parallelogram has positive area, then the two vectors are linearly independent.
3 , a , a , a , a , a , a
(2) Consider three vectors v 1 = (a
11
21 31 ), v 2 = (a
12
22 32 ), and v 3 = (a
13
23 33 ) in R . To v v v study linear independence, we consider the equation c
1 1 + c
2 2 + c
3 3 = 0, which can be written in matrix form as
a a a c
11
12
13
1 a a a c .
=
21
22
23
2 a a a c
31
32
33
3 The vectors v , v and v are linearly independent precisely when
1
2
3 a a a
11
12
13 a a a det = 0.
21
22
23 a a a
31
32
33 This can be interpreted geometrically in the following way: The volume of the parallelepiped formed by the three vectors v 1 , v 2 and v 3 is in fact equal to the absolute value of the determinant of the matrix formed with v
1 , v 2 and v 3 as the columns; in other words, a a a
11
12
13 a a a . det
21
22
23 a a a
31
32
33 It follows that the three vectors are linearly dependent precisely when the parallelepiped has zero volume; in other words, when the three vectors lie on the same plane. On the other hand, if the parallelepiped has positive volume, then the three vectors are linearly independent.
3 (3) What is the geometric interpretation of two linearly independent vectors in R ? Well, note that
1 2 ∈ R, not both zero, such that c v v
, c if v 1 and v 2 are non-zero and linearly dependent, then there exist c
1 1 + c
2 2 = 0. This forces the two vectors to be multiples of each other, so that they lie on the
3 same line, whence the parallelogram they form has zero area. It follows that if two vectors in R form a parallelogram with positive area, then they are linearly independent.
5.5. Basis and Dimension
In this section, we complete the task of describing uniquely every element of a vector space V in terms of the elements of a suitable subset B. To motivate the ideas, we first consider an example.
3 Example 5.5.1.
Let us consider the three vectors v = (1, 1, 0), v = (5, 1, −3) and v = (2, 7, 4) in R ,
1
2
3
3 , v , v as in Examples 5.4.2 and 5.4.4. We have already shown that span{v } = R , and that the vectors
1
2
3
3 v , v , v
1
2 3 are linearly independent. Furthermore, we have shown that for every u = (x, y, z) ∈ R , we v v v , c , c can write u = c
1 1 + c
2 2 + c
3 3 , where c
1
2 3 ∈ R are determined uniquely by c −25
26 −33 x
1 c =
4 −4 5 y .
2 c 3 −3 4 z
3 Definition.
, . . . , v , . . . , v
Suppose that v 1 r are vectors in a vector space V over R. We say that {v 1 r } is a basis for V if the following two conditions are satisfied: , . . . , v
(B1) We have span{v 1 r } = V .
, . . . , v (B2) The vectors v 1 r are linearly independent.
2 Example 5.5.2.
, a , a Consider two vectors v 1 = (a
11 21 ) and v 2 = (a
12 22 ) in R . Suppose that a a
11
12 det = 0; a a
21
22 in other words, suppose that the parallelogram formed by the two vectors has non-zero area. Then it follows from Remark (1) in Section 5.4 that v
1 and v 2 are linearly independent. Furthermore, for every
2 u , c v v
= (x, y) ∈ R , there exist c
1 2 ∈ R such that u = c
1 1 + c
2 2 . Indeed, c 1 and c 2 are determined as the unique solution of the system
11
12
1 = . a a c y
21
22
2
2
2 Hence span{v , v } = R . It follows that {v , v } is a basis for R .
1
2
1
2 Example 5.5.3.
, a , a , a , a Consider three vectors of the type v = (a ), v = (a ) and v =
1
11
21
31
2
12
22
32
3
3 , a , a
(a ) in R . Suppose that
13
23
33 a a a
11
12
13 det a a a = 0;
21
22
23 a a a
31
32
33 in other words, suppose that the parallelepiped formed by the three vectors has non-zero volume. Then it follows from Remark (2) in Section 5.4 that v , v and v are linearly independent. Furthermore, for
1
2
3
3 , c , c v v v every u = (x, y, z) ∈ R , there exist c ∈ R such that u = c + c + c . Indeed, c , c and
1
2
3
1
1
2
2
3
3
1
2 c are determined as the unique solution of the system
3 a a a c x
11
12
13
1 a a a c y .
21
22
23 2 = a a a c z
31
32
33
3
3
3 , v , v , v , v
Hence span{v
1
2 3 } = R . It follows that {v
1
2 3 } is a basis for R . n Example 5.5.4.
In R , the vectors e , . . . , e n , where
1 e , j = (0, . . . , 0 1, 0, . . . , 0 ) for every j = 1, . . . , n, j−1 n−j n n
, . . . , e are linearly independent and span R . Hence {e n } is a basis for R . This is known as the 1 n standard basis for R .
Example 5.5.5.
In the vector space M (R) of all 2 × 2 matrices with entries in R as discussed in 2,2
Example 5.1.3, the set
1
1 , , ,
1
1 is a basis. Example 5.5.6.
In the vector space P k of polynomials of degree at most k and with coefficients in R 2 k
, . . . , x as discussed in Example 5.1.6, the set {1, x, x } is a basis. PROPOSITION 5E. Suppose that {v , . . . , v r } is a basis for a vector space V over R. Then every
1
element u ∈ V can be expressed uniquely in the form
u v v , , . . . , c = c
1 1 + . . . + c r r where c 1 r ∈ R. Proof.
, . . . , v , . . . , c v v Since u ∈ V = span{v 1 r }, there exist c 1 r ∈ R such that u = c
1 1 + . . . + c r r .
Suppose now that b , . . . , b ∈ R such that 1 r c v v v v .
1 1 + . . . + c r r = b
1 1 + . . . + b r r Then (c − b )v + . . . + (c r − b r )v r = 0.
1
1
1 , . . . , v
, . . . , c Since v r are linearly independent, it follows that c − b = . . . = c r − b r = 0. Hence c r
1
1
1
1 are uniquely determined.
We have shown earlier that a vector space can have many bases. For example, any collection of three
3 vectors not on the same plane is a basis for R . In the following discussion, we attempt to find out some properties of bases. However, we shall restrict our discussion to the following simple case.
Definition.
A vector space V over R is said to be finite-dimensional if it has a basis containing only finitely many elements. n Example 5.5.7.
The vector spaces R , M (R) and P k that we have discussed earlier are all finite- 2,2 dimensional. n
Recall that in R , the standard basis has exactly n elements. On the other hand, it follows from n
Proposition 5D that any basis for R cannot contain more than n elements. However, can a basis for n
R contain fewer than n elements? We shall answer this question by showing that all bases for a given vector space have the same number of elements. As a first step, we establish the following generalization of Proposition 5D.
PROPOSITION 5F. , . . . , v
Suppose that {v n } is a basis for a vector space V over R. Suppose further
1 , . . . , u , . . . , u that r > n, and that the vectors u r ∈ V . Then the vectors u r are linearly dependent.
1
1 Proof.
Since {v , . . . , v n } is a basis for the vector space V , we can write
1 u v v ,
1 = a
11 1 + . . . + a n 1 n ..
. u r = a v + . . . + a nr v n ,
1r