Numerical Problems, Methods, and Algorithms

2.4.1 Numerical Problems, Methods, and Algorithms

By a numerical problem we mean here a clear and unambiguous description of the functional connection between input data—that is, the “independent variables” in the problem—and output data—that is, the desired results. Input data and output data consist of a finite number of real (or complex) quantities and are thus representable by finite di- mensional vectors. The functional connection can be expressed in either explicit or implicit form. We require for the following discussion also that the output data should be uniquely determined and depend continuously on the input data .

By an algorithm 39 for a given numerical problem we mean a complete description of well-defined operations through which each permissible input data vector is transformed into an output data vector. By “operations” we mean here arithmetic and logical operations, which a computer can perform, together with references to previously defined algorithms. It should be noted that, as the field of computing has developed, more and more complex functions (for example, square root, circular, and hyperbolic functions) are built into the hardware. In many programming environments operations such as matrix multiplication, solution of linear systems, etc. are considered as “elementary operations” and for the user appear as black boxes.

(The concept algorithm can be analogously defined for problems completely different from numerical problems, with other types of input data and fundamental operations— for example, inflection, merging of words, and other transformations of words in a given language.)

Example 2.4.1.

To determine the largest real root of the cubic equation

p(z) =a 0 z 3 +a 2 1 z +a 2 z +a 3 = 0,

with real coefficients a 0 ,a 1 ,a 2 ,a 3 , is a numerical problem. The input data vector is (a 0 ,a 1 ,a 2 ,a 3 ) . The output is the desired root x; it is an implicitly defined function of the input data. An algorithm for this problem can be based on Newton’s method, supplemented with rules for how the initial approximation should be chosen and how the iteration process is to be terminated. One could also use other iterative methods, or algorithms based on the formula by Cardano–Tartaglia for the exact solution of the cubic equation (see Problem 2.3.8). Since this uses square roots and cube roots, one needs to assume that algorithms for the computation of these functions have been specified previously.

39 The term “algorithm” is a Latinization of the name of the Arabic ninth century mathematician Al-Khowârizmî. He also introduced the word “algebra” (Al-jabr). Western Europe became acquainted with the Hindu positional

number system from a Latin translation of his book entitled Algorithmi de Numero Indorum.

2.4. Error Propagation 127 One often begins the construction of an algorithm for a given problem by breaking

down the problem into subproblems in such a way that the output data from one subproblem are the input data to the next subproblem. Thus the distinction between problem and algorithm is not always so clear-cut. The essential point is that, in the formulation of the problem, one is only concerned with the initial state and the final state. In an algorithm, however, one should clearly define each step along the way, from start to finish.

We use the term numerical method in this book to mean a procedure either to approx- imate a mathematical problem with a numerical problem or to solve a numerical problem (or at least to transform it to a simpler problem). A numerical method should be more generally applicable than an algorithm, and set lesser emphasis on the completeness of the computational details. The transformation of a differential equation problem to a system of nonlinear equations can be called a numerical method—even without instructions as to how to solve the system of nonlinear equations. Newton’s method is a numerical method for determining a root of a large class of nonlinear equations. In order to call it an algorithm, conditions for starting and stopping the iteration process should be added.

For a given numerical problem one can consider many different algorithms. As we have seen in Sec. 2.3 these can, in floating-point arithmetic, give approximations of widely varying accuracy to the exact solution.

Example 2.4.2.

The problem of solving the differential equation

2 =x 2 +y

dx 2

with boundary conditions y(0) = 0, y(5) = 1 is not a numerical problem according to the definition stated above. This is because the output is the function y, which cannot in any

conspicuous way be specified by a finite number of parameters. The above mathematical problem can be approximated with a numerical problem if one specifies the output data to be the values of y for x = h, 2h, 3h, . . . , 5−h. Also, the domain of variation of the unknowns must be restricted in order to show that the problem has a unique solution. This can be done, however, and there are a number of different algorithms for solving the problem approximately, which have different properties with respect to the number of arithmetic operations needed and the accuracy obtained.

Before an algorithm can be used it has to be implemented in an algorithmic program language in a reliable and efficient manner. We leave these aspects aside for the moment, but this is far from a trivial task. Most amateur algorithm writers seem to think that an algorithm is ready at the point where a professional realizes that the hard and tedious work is just beginning (George E. Forsythe [120]).