LINEAR MODELS AND MATRIX ALGEBRA
Chapter 4 LINEAR MODELS AND MATRIX ALGEBRA As more and more commodities are included in models, solution formulas become cumbersome. Matrix algebra enables to do us many things: 1. provides a compact way of writing an equation system 2. leads to a way of testing the existence of a solution by evaluation of a determinant 3. gives a method of finding solution (if it exists)
Catch: matrix algebra is only applicable to linear equation systems. Some transformation can be done to obtain a linear relation.
b
y = ax log y = log a + b log x Matrices and Vectors c P + c P = -c
1
1
2
2 P P
1 2 +
2 2 = -
In general,
a x + a x +…+ a X = d
11
1
12 2 1n n
1 a x + a x +…+ a X = d
21
1
22 2 2n n
2 ……………………………… a m1 x
1 + a m2 x 2 +…+ a mn X n = d m
coefficients a
ij
variables x x
1, …, n
constants d
1 , …,d m
Matrices as Arrays
a a a x d
11
12 1 n
1
1
a a a x d
21
22 2 n
2
2
A x d
a a a x dm
m
1 m 2 mn n
Example: 6x + 3x + x = 22
1
2
3
x + 4x +-2x =12
1
2
3
4x -x +5x = 10
1
2
3
6
3 1 x
22
1
A
1
4 2 x x d
12
2
4
1 5 x
10
3
A matrix is defined as a rectangular array of numbers, parameters, or variables. Members of the array are termed elements of the matrix.
i 1, 2,..., m
j 1, 2,..., n
Coefficient matrix A=[a ]
ij Vectors as Special Matrices
Dimension of a matrix = number of rows x number of columns, m x n
m rows n columns
Note: row number always precedes the column number. this is in line with way the two subscripts are in a are ordered.
ij
Special case:
m = n, square matrix
one column : column vector one row: row vector usually distinguished from a column vector by the use of a primed symbol:
x ' x x x
1 2 n
Note that a vector is merely an ordered n-tuple and as such it may be interpreted as a point in an n-dimensional space. Matrix Notation
Ax = d
Questions: How do we multiply A and x? What is the meaning of equality? Exercise (p60): Q d = Q s Q d = a - bP
- –1Q
- bP =a 0 +1Q s +-dP =-c Coefficient matrix
- must have the same dimension
32
12
12
21
22
21
22
21
21
22
22
31
31
11
32
31
31
32
32
a a b b a b a b a a b b a b a b a a b b a b a b
In general
ij ij ij ij ij ij a b c where c a b
Note that the sum matrix must have the same dimension as the component matrices.
Subtraction:
ij ij ij ij ij ij a b d where d a b
11
12
can be rewritten as
Constant vector
1Q
d
s
=0
1Q
d
1
1
1
1
b d
a c
11
Matrix Operations
Addition and Subtraction
Example 1
4 9 2 0 4 2 9 0 6 9 2 1 0 7 2 0 1 7 2 8
Example 2
11
12
Example 19 3 6 8 19 6 3 8
13
5
2 0 1 3 2 1 0 3
1
3
Scalar Multiplication:
To multiply a matrix by a number – by a scalar – is to multiply every element of that matrix by the given scalar.
3
1
21
7
7
5
35
1
1 a a a a
11 12
2
11
2
12
1
2
1 1 a a a a
21
22
2
21
2
22
Note that the rational for the name scalar is that it scales up or down the matrix by a certain multiple. It can also be a negative number.
a a d a a d
11
12 1
11
12 1
1
a a d a a d
21
22
2
21
22
2
Multiplication:
Given 2 matrices A and B, we want to find the product AB. The conformability
condition for multiplication is that the column dimension of A (the lead matrix) must
be equal to the row dimension of B ( the lag matrix). b b b
11
12
13
A a a B
11
12
b b b
21
22
23
(1x2) (2x3)
BA is not defined since the conformability condition for multiplication is not satisfied.
In general, if A is of dimension m x n and B is of dimension p x q, the matrix product AB will be defined only if n = p. If defined the product matrix AB will have the dimension m x q, the same number of rows as the lead matrix A and the same number of columns as the lag matrix B. b
12
12
c
12
= a
11
b
12
b
11 = a 11 b 11 + a 12 b
22
c
13
= a
11
b
13
21
] c
Exact Procedure
22
11
12
13
11
12
21
23 b b b A a a B b b b
13
AB = C = [c
11
c
12
c
- a
- a
23 Examples:
Example 7
10 3 1 2 1/ 5 3 /10 1 0 0 1 0 6 3 0 3
1 3 1 1/ 5 7 /10 0 0 0 0 1 0
5
10
4 2 2 / 5 1/10 0 0 1 4 0 4 12 0 2
0 0 0
5
10 A B AB
3x3, 3x3, 3x3 3 1 4 9 7 2 0 1 0
Note, the last matrix is a square matrix with 1s in its principal diagonal and 0s everywhere else, is known as identity matrix.
5
Example 9
3 4 1 0 5 6
1 3 1(5) 3(9)
4
7 17 35 20 42
A and B AB
fr 4.4, p56
2x2,2x2,2x2
Example 8
3x2, 2x1, 3x1
(3 2) (2 1)
32
5 2 8 2(5) 8(9)
82
9 4 0 4(5) 0(9)
20
x x A b Ab
6
3 1 x
22
1
A
1
4 2 x x d
12
2
4
1 5 x
10
3
6
3 1 x 6 x 3 x x
1
1
2 3
Ax
1
4 2 x x 4 x 2 x
2
1
2
3
4
1 5 x 4 x x 5 x
3
1
2
3
The product on the right is a column vector When we write Ax= d, we have 6 x
3 x x
22
1
2 3
x
4 x 2 x
12
1
2
3
4 x x 5 x
10
1
2
3
Example 11: simple national income model with two endogenous variables, Y and C
Y = C + Io + Go C = a + bY can be rearranged into the standard format Y – C = Io – Go
- bY + C = a Coefficient matrix, vector of variables, vector of constants
1
1 Y
I G
A x d
b
1 C a
To express it in terms of Ax=d,
1
1 Y 1( ) ( 1)( ) Y C Y C
Ax
b
1 C bY 1( ) C bY C
Y C
I G
bY C a
The equation Ax=d precisely representx the original equation system.
Study the digression on notation (pp.64-65) Exercises: 4.2 # 1, 2, 3, 4
BTW = NOT POSSIBLE TO DIVIDE ONE MATRIX BY ANOTHER !!!
Digression on notation:
Subcripted symbols helps in designating the locations of parameters and variables but also lends itself to a flexible shorthand for denoting sums of terms, such as those which arose during the process of matrix multiplication.
3 x x x x
1
2 3 j j
1
j: summation index x : summand
j
7 x x x x x x
i
3
4
5
6
7 i
3 n x x x x
k 1 n k
The application of notation can be readily extended to cases in which the x term is
prefixed with a coefficient or in which each term in the sum is raised to some integer power.
3
3 ax ax ax ax a x ( x x ) a x
j
1
2
3
1
2 3 j j 1 j
1
3 a x a x a x a x
j j 1 1 2 2 2 3 j
1 n j
1 2 n a x a x a x a x a x
i
1 2 n i
2 n a a x a x a x
1 2 n
- the last example can be used as a shorthand of the general polynomial function
- whenever the context of the discussion leave no ambiguity as to the range of the summation, the symbol can be used alone, without an index attached.
2 c a b a b a b
11 11 11 12 21 1 k k
1 k
1
2 c a b a b a b
12 11 12 12 22 1 k k
2 k
1
2 c a b a b a b
13 11 13 12 23 1 k k
3 k
1 Extending to an m x n matrix, A=[aik] and an n x p matrix B=[bkj], we may now
write the elements of the m x p matrix AB=C=[cij] as
n n c a b c a b
11 1 k k
1
12 1 k k
2 k 1 k
1
or more generally,
n i 1, 2,..., m
c a b
ij 1 k kj
j 1, 2,..., p
k
1
4.3 NOTES ON VECTOR OPERATIONS
- vectors are considered as special types of matrix - has dimensional peculiarities.
Multiplication of vectors An m x 1 column vector u, and a 1 x n row vector v’, yield a product uv’ of dimension m x n. On the other hand, a 1 x n row vector u’ and an n x 1 column vector v, the product u’v will be of dimension 1 x 1.
3
u
' 1 4 5
2 v
Example 1. Given and , we can get3(1) 3(4) 3(5) 3 12 15
uv '
2(1) 2(4) 2(5) 2 8 10
9
v
7 Example 2: Given u’ [3 4] and , we have As written, u’v is a matrix, despite the fact that only a single element is present.
- 1 x 1 matrices behave exactly like scalars with respect o addition and multiplication: [4] + [8] =[12], [3][7]=[21]
- a scalar product Example 3.
Given a row vector u’ = [3 6 9], find u’u. Since u is merely a column vector, with elements of u’ arranged vertically, we have,
3
2
2
2
u u ' 3 6 9 6 (3) (6) (9)
9 Note that the product u’u gives the sum of squares of the elements of u.
In general, if u’ = [ u u … u ], then u’u will be the sum of squares (a scalar) of the
1 2 n elements of u j. n
2
2
2
2 u u u ' u u u
1 1 n j j
1 Linear Dependence
A set of vectors v , …,v is linearly dependent if and only if any one of them can be
1 n
expressed as a linear combination of the remaining vectors; otherwise, they are linearly independent.
Example 4. The three vectors
2
1
4
v , v , and v
1
2
3
7
8
5 are linear dependent because v3 is a linear combination of v1 and v2:
6
2
4 3 v
2 v v
1
2
3
21
16
5 or 3v –2v -v =0
1
2
3
where represents a null vector (also called zero vector)
Example 5. v
1 ’ =[5 12] and v 2’ = [10 24] are linearly dependent because
2v ’= 2[5 12] = [10 24] = v ’
1
2
or 2v
1 ’-v 2 ’ = A set of m-vectors v , …,v is linearly dependent if and only if there exists a set of
1 n
scalars k
1 , …, k n (not all zero) such that n k v
i i i
1
4.4 COMMUTATIVE, ASSOCIATIVE, AND DISTRIBUTIVE LAWS In ordinary scalar algebra, additive and multiplicative operations obey the commutative, associative, and distributive laws: Commutative law of addition a + b = b + a Commutative law of multiplication ab = ba Associative law of addition (a+b) + c = a+ (b+c) Associative law of multiplication ab (c) = a(bc) Distributive law a (b+c) = ab + ac
Matrix Addition: commutative and associative
Commutative law : A+B=B+A 3 1 6 2
Given A and B , we find that
0 2 3 4 9 3
A B B A
3 6`
Associative law: (A+B) + C = A + (B+C) Given
3
9
2
v , v , and v ,
1
2
3
4
1
5
12
2
10
( v v ) v
1
2
3
5
5
3
7
10
( v ( v v )
1
2
3
4
4
Matrix Multiplication: not commutative
If the products are defined:
Example:
AB BA
1 2
1
Let A B then
3 4
6
7 12 13
3
4
AB , BA
24 25 27 40
Example: Let u’ be a 1x3 (a row vector); then the corresponding column vector u must be 3x1. The product u’u will be 1x1 but the product uu’ will be 3x3. Thus obviously, u’u uu’. Exceptions:
- A is a square matrix and B is an identity ma
- 1
A is the inverse of B, A = B -
- scalar multiplication: kA=Ak
Associative Law: (AB)C=A(BC)=ABC
Conformability condition: A is mxn, B is n xp, C is pxq Distributive Law: A(B+C) = AB + AC [premultiplication by A]
(B+C)A = BA + CA [postmultiplication by A]
4.5 IDENTITY MATRICES AND NULL MATRICES 1 0 0 1 0
I I 0 1 0
2
3
0 1
0 0 1
Characteristics: like number 1: a(1) = (1)a=a
IA = AI = A 1 2 3
A
2 0 3
Example: Let
Special Nature of I Matrices it is possible to insert or delete an I w/o affecting the matrix product. -
A I B ( AI B AB )
mxn nxn nxp
Interesting case: if: A I
n
2 AI ( )
I I
n n n k
( )
I I
n n Inverses and their Properties:
For a given matrix A, The transpose is always derivable. Its inverse matrix (another type of - derived matrix) may or may not exist.
- 1
The inverse of matrix A is denoted by A is defined only if A is a square matrix.
1
1 AA A A I 1
1
( A ) A
1
1
1
( AB ) B A
1
1
( ') A ( A ) '
1
1 AA A A I
- 1
Whether A is pre-multiplied or postmultiplied by A , the product will be the same identity matrix. This is an exception to the rule that matrix multiplication is not commutative. Points to note:
1. Not every square matrix has an inverse. Squareness is a necessary but not a sufficient condition for the existence of a matrix.
a. Non-singular matrix : if an inverse exists
b. Singular : if no inverse exists
- 1 -1 -1
2. If A exists, then A is the inverse of A just as A is the inverse of A. A and
- 1 A are inverses of each other.
- 1 3. If A is nxn, then A must also be nxn.
4. If an inverse exists, it is unique.
AB=BA=I Define C such that AC=CA=I Premultiplying both sides of AB=I by C,
CAB=CI=C Since CA=I, IB=C or B=C
1
1
( A ) A
1 1
1
( AB ) B A
1
1
( ') A ( A ) '
Proof: Suppose that the inverse of AB is C. We know that CAB=I
- 1 -1 -1 -1 -1 -1
CABB A =IB A (=B A ) But the left side is reducible to
- 1 -1 -1
CA(BB )A =CAIA
- 1
=CAA =CI=C Substituting to the original equation,
- 1 -1
C= B A
- 1 -1 Or in other words, the inverse of AB is equal to B A .
Given A’, what is the inverse D? By definition, DA’=I.
- 1 But (AA )’=I’=I.
- 1
Thus, DA’=(AA )’
- 1
=(A )’A’
- 1
Postmultiplying both sides by (A )’, we obtain
’ –1 -1 -1
DA’(A ) = (A )’A’(A’)
- 1
Or D=(A )’
- 1 Thus the inverse of A’ is equal to (A )’.
Inverse Matrix and Solution of Linear Equation System
The application of the concept of inverse matrix to the solution of simultaneous equation system is immediate and direct.
6
3 1 x
22 Ax d
1 1
1
4 2 x x d
12
1 A
A Ax A d
2
1
4
1 5 x
10
3 x A d
4.4, p.56 d will then give the solution values of the variables.
by the constant vector d. The product A
52
, and then post multiply A
One way of finding the solution of a linear equation system Ax=d, where the coefficient matrix is nonsingular, first find A
x x x
1
21
18
17
3
1
13
26
13
1
2
10
16
18
3
2
- 1
- 1
- 1