Rounding and Chopping

2.1.3 Rounding and Chopping

When one counts the number of digits in a numerical value one should not include zeros in the beginning of the number, as these zeros only help to denote where the decimal point should be. For example, the number 0.00147 has five decimals but is given with three digits. The number 12.34 has two decimals but is given with four digits.

2 ·10 −t , then ˜a is said to have t correct decimals. The digits in ˜a which occupy positions where the unit is greater than or equal to 10 −t are then called significant digits (any initial zeros are

If the magnitude of the error in a given numerical value ˜a does not exceed 1

92 Chapter 2. How to Obtain and Estimate Accuracy not counted). Thus, the number 0.001234 ± 0.000004 has five correct decimals and three

significant digits, while 0.001234 ± 0.000006 has four correct decimals and two significant digits. The number of correct decimals gives one an idea of the magnitude of the absolute

error , while the number of significant digits gives a rough idea of the magnitude of the relative error .

We distinguish here between two ways of rounding off a number x to a given number t of decimals. In chopping (or round toward zero) one simply leaves off all the decimals to the right of the tth. That way is generally not recommended since the rounding error has, systematically, the opposite sign of the number itself. Also, the magnitude of the error can

be as large as 10 −t . In rounding to nearest (sometimes called “correct” or “optimal” rounding), one chooses a number with s decimals which is nearest to x. Hence if p is the part of the number which stands to the right of the sth decimal, one leaves the tth decimal unchanged if and only if |p| < 0.5 ·10 −s . Otherwise one raises the sth decimal by 1. In the case of a tie, when x is equidistant to two s decimal digit numbers, then one raises the sth decimal if it is odd or leaves it unchanged if it is even (round to even). In this way, the error is positive or negative about equally often. The error in rounding a decimal number to s decimals will always lie in the interval [− 1

2 10 −s , 1 2 10 −s ].

Example 2.1.1.

Shortening to three decimals, 0.2397

rounds to 0.240

(is chopped to 0.239),

−0.2397 rounds to −0.240 (is chopped to −0.239),

(is chopped to 0.237), 0.23650

rounds to 0.238

(is chopped to 0.236), 0.23652

rounds to 0.236

(is chopped to 0.236). Observe that when one rounds off a numerical value one produces an error; thus it is

rounds to 0.237

occasionally wise to give more decimals than those which are correct. Take a = 0.1237 ± 0.0004, which has three correct decimals according to the definition given previously. If

one rounds to three decimals, one gets 0.124; here the third decimal may not be correct, since the least possible value for a is 0.1233.

Suppose that you are tabulating a transcendental function and that a particular entry has been evaluated as 1.2845 correct to the digits given. You want to round the value to three decimals. Should the final digit be 4 or 5? The answer depends on whether there is a nonzero trailing digit. You compute the entry more accurately and find 1.28450, then 1.284500, then 1.2845000, etc. Since the function is transcendental, there clearly is no bound on the number of digits that has to be computed before distinguishing if to round to 1.284 or 1.285. This is called the tablemaker’s dilemma. 30

Example 2.1.2.

The difference between chopping and rounding can be important, as is illustrated by the following story. The index of the Vancouver Stock Exchange, founded at the initial value

30 This can be used to advantage in order to protect mathematical tables from illegal copying by rounding a few entries incorrectly, where the error in doing so is insignificant due to several trailing zeros. An illegal copy could

then be exposed simply by looking up these entries!

93 1000.000 in 1982, was hitting lows in the 500s at the end of 1983 even though the exchange

2.2. Computer Number Systems

apparently performed well. It was discovered (The Wall Street Journal, Nov. 8, 1983, p. 37) that the discrepancy was caused by a computer program which updated the index thousands of times a day and used chopping instead of rounding to nearest! The rounded calculation gave a value of 1098.892.

ReviewQuestions

2.1.1 Clarify with examples the various types of error sources which occur in numerical work.

2.1.2 (a) Define “absolute error” and “relative error” for an approximation ¯x to a scalar quantity x. What is meant by an error bound?

(b) Generalize the definitions in (a) to a vector x.

2.1.3 How is “rounding to nearest” performed?

2.1.4 Give π to four decimals using (a) chopping; (b) rounding.

2.1.5 What is meant by the “tablemaker’s dilemma”?