Linear Vector Spaces
A.1.1 Linear Vector Spaces
In this appendix we recall basic elements of finite-dimensional linear vector spaces and related matrix algebra, and introduce some notations to be used in the book. The exposition is brief and meant as a convenient reference.
We will be concerned with the vector spaces R n and C n , that is, the set of real or
complex n-tuples with 1 ≤ n < ∞. Let v 1 ,v 2 ,...,v k
be vectors and α 1 ,α 2 ,...,α k
be scalars. The vectors are said to be linearly independent if none of them is a linear combination of the others, that is,
k α i v i =0⇒α i = 0, i = 1 : k.
i =1
Otherwise, if a nontrivial linear combination of v 1 ,...,v k is zero, the vectors are said to be linearly dependent. Then at least one vector v i will be a linear combination of the rest.
A basis in a vector space V is a set of linearly independent vectors v 1 ,v 2 ,...,v n ∈V such that all vectors v ∈ V can be expressed as a linear combination:
i =1
The scalars ξ i are called the components or coordinates of v with respect to the basis {v i }. If the vector space V has a basis of n vectors, then every system of linearly independent vectors of V has at most k elements and any other basis of V has the same number k of elements. The number k is called the dimension of V and denoted by dim(V).
The linear space of column vectors x = (x 1 ,x 2 ,...,x n ) T , where x i ∈ R is denoted R n ; if x i
∈ C, then it is denoted C n . The dimension of this space is n, and the unit vectors
e 1 ,e 2 ,...,e n , where
1 T = (1, 0, . . . , 0) , e 2 = (0, 1, . . . , 0) ,...,e n = (0, 0, . . . , 1) ,
A-2 Online Appendix A. Introduction to Matrix Computations constitute the standard basis. Note that the components x 1 ,x 2 ,...,x n are the coordinates
when the vector x is expressed as a linear combination of the standard basis. We shall use the same name for a vector as for its coordinate representation by a column vector with respect to the standard basis.
An arbitrary basis can be characterized by the nonsingular matrix V = (v 1 ,v 2 ,...,v n ) composed of the basis vectors. The coordinate transformation reads x = V ξ. The standard basis itself is characterized by the unit matrix
I = (e 1 ,e 2 ,...,e n ).
If W ⊂ V is a vector space, then W is called a vector subspace of V. The set of all linear combinations of v 1 ,...,v k ∈ V form a vector subspace denoted by
span {v 1 ,...,v k }=
α i v i , i = 1 : k,
i =1
where α i are real or complex scalars. If S 1 ,...,S k are vector subspaces of V, then their sum defined by
S = {v 1 +···+v k |v i ∈S i ,i = 1 : k}
is also a vector subspace. The intersection T of a set of vector subspaces is also a subspace,
T =S 1 ∩S 2 ···∩S k .
(The union of vector spaces is generally not a vector space.) If the intersections of the subspaces are empty, S i ∩S j direct sum and denoted by
S=S 1 ⊕S 2 ···⊕S k .
A function F from one linear space to another (or the same) linear space is said to be linear if
F (αu + βv) = αF (u) + βF (v)
for all vectors u, v ∈ V and all scalars α, β. Note that this terminology excludes nonho- mogeneous functions like αu + β, which are called affine functions. Linear functions are often expressed in the form Au, where A is called a linear operator.
A vector space for which an inner product is defined is called an inner product space. For the vector space R n the Euclidean inner product is
Similarly C n is an inner product space with the inner product
where ¯x k denotes the complex conjugate of x k .
A.1. Vectors and Matrices A-3 Two vectors v and w in R n
are said to be orthogonal if (v, w) = 0. A set of vectors
v ,...,v in R n
is called orthogonal with respect to the Euclidean inner product if
(v i ,v j ) = 0, i
and orthonormal if also (v i ,v i ) = 1, i = 1 : k. An orthogonal set of vectors is linearly independent.
The orthogonal complement S ⊥ of a subspace S ∈ R n is the subspace defined by
= {y ∈ R n | (y, x) = 0, x ∈ S}.
More generally, the subspaces S
1 n ,...,S k of R are mutually orthogonal if, for all 1 ≤ i, j ≤ k
x ∈S i , y ∈S j , ⇒ (x, y) = 0.
The vectors q
1 n ,...,q k form an orthonormal basis for a subspace S ⊂ R if they are
orthonormal and span {q 1 ,...,q k } = S.
Parts
» Numerical Methods in Scientific Computing
» Solving Linear Systems by LU Factorization
» Sparse Matrices and Iterative Methods
» Software for Matrix Computations
» Characterization of Least Squares Solutions
» The Singular Value Decomposition
» The Numerical Rank of a Matrix
» Second Order Accurate Methods
» Adaptive Choice of Step Size
» Origin of Monte Carlo Methods
» Generating and Testing Pseudorandom Numbers
» Random Deviates for Other Distributions
» Absolute and Relative Errors
» Fixed- and Floating-Point Representation
» IEEE Floating-Point Standard
» Multiple Precision Arithmetic
» Basic Rounding Error Results
» Statistical Models for Rounding Errors
» Avoiding Overflowand Cancellation
» Numerical Problems, Methods, and Algorithms
» Propagation of Errors and Condition Numbers
» Perturbation Analysis for Linear Systems
» Error Analysis and Stability of Algorithms
» Interval Matrix Computations
» Taylor’s Formula and Power Series
» Divergent or Semiconvergent Series
» Properties of Difference Operators
» Approximation Formulas by Operator Methods
» Single Linear Difference Equations
» Comparison Series and Aitken Acceleration
» Complete Monotonicity and Related Concepts
» Repeated Richardson Extrapolation
» Algebraic Continued Fractions
» Analytic Continued Fractions
» Bases for Polynomial Interpolation
» Conditioning of Polynomial Interpolation
» Newton’s Interpolation Formula
» Barycentric Lagrange Interpolation
» Iterative Linear Interpolation
» Fast Algorithms for Vandermonde Systems
» Complex Analysis in Polynomial Interpolation
» Multidimensional Interpolation
» Analysis of a Generalized Runge Phenomenon
» Bernštein Polynomials and Bézier Curves
» Least Squares Splines Approximation
» Operator Norms and the Distance Formula
» Inner Product Spaces and Orthogonal Systems
» Solution of the Approximation Problem
» Mathematical Properties of Orthogonal Polynomials
» Expansions in Orthogonal Polynomials
» Approximation in the Maximum Norm
» Convergence Acceleration of Fourier Series
» The Fourier Integral Theorem
» Fast Trigonometric Transforms
» Superconvergence of the Trapezoidal Rule
» Higher-Order Newton–Cotes’ Formulas
» Fejér and Clenshaw–Curtis Rules
» Method of Undetermined Coefficients
» Gauss–Christoffel Quadrature Rules
» Gauss Quadrature with Preassigned Nodes
» Matrices, Moments, and Gauss Quadrature
» Jacobi Matrices and Gauss Quadrature
» Multidimensional Integration
» Limiting Accuracy and Termination Criteria
» Convergence Order and Efficiency
» Higher-Order Interpolation Methods
» Newton’s Method for Complex Roots
» Unimodal Functions and Golden Section Search
» Minimization by Interpolation
» Ill-Conditioned Algebraic Equations
» Deflation and Simultaneous Determination of Roots
» Finding Greatest Common Divisors
» Permutations and Determinants
» Eigenvalues and Norms of Matrices
» Function and Vector Algorithms
» Textbooks in Numerical Analysis
» Encyclopedias, Tables, and Formulas
Show more