4
CHAPTER 1. INTRODUCTION
At this point, it is important to realize that the distinction between the function which is to be optimized and the functions which describe the constraints, although convenient for presenting the
mathematical theory, may be quite artificial in practice. For instance, suppose we have to choose the durations of various traffic lights in a section of a city so as to achieve optimum traffic flow.
Let us suppose that we know the transportation needs of all the people in this section. Before we can begin to suggest a design, we need a criterion to determine what is meant by “optimum traffic
flow.” More abstractly, we need a criterion by which we can compare different decisions, which in this case are different patterns of traffic-light durations. One way of doing this is to assign as cost to
each decision the total amount of time taken to make all the trips within this section. An alternative and equally plausible goal may be to minimize the maximum waiting time that is the total time
spent at stop lights in each trip. Now it may happen that these two objective functions may be inconsistent in the sense that they may give rise to different orderings of the permissible decisions.
Indeed, it may be the case that the optimum decision according to the first criterion may be lead to very long waiting times for a few trips, so that this decision is far from optimum according to the
second criterion. We can then redefine the problem as minimizing the first cost function total time for trips subject to the constraint that the waiting time for any trip is less than some reasonable
bound say one minute. In this way, the second goal minimum waiting time has been modified and reintroduced as a constraint. This interchangeability of goal and constraints also appears at a
deeper level in much of the mathematical theory. We will see that in most of the results the objective function and the functions describing the constraints are treated in the same manner.
1.2 Some Other Models of Decision Problems
Our model of a single decision-maker with complete information can be generalized along two very important directions. In the first place, the hypothesis of complete information can be relaxed
by allowing that decision-making occurs in an uncertain environment. In the second place, we can replace the single decision-maker by a group of two or more agents whose collective decision
determines the outcome. Since we cannot study these more general models in these Notes, we merely point out here some situations where such models arise naturally and give some references.
1.2.1 Optimization under uncertainty.
A person wants to invest 1,000 in the stock market. He wants to maximize his capital gains, and at the same time minimize the risk of losing his money. The two objectives are incompatible, since
the stock which is likely to have higher gains is also likely to involve greater risk. The situation is different from our previous examples in that the outcome future stock prices is uncertain. It is
customary to model this uncertainty stochastically. Thus, the investor may assign probability 0.5 to the event that the price of shares in Glamor company increases by 100, probability 0.25 that the
price is unchanged, and probability 0.25 that it drops by 100. A similar model is made for all the other stocks that the investor is willing to consider, and a decision problem can be formulated as
follows. How should 1,000 be invested so as to maximize the expected value of the capital gains subject to the constraint that the probability of losing more than 100 is less than 0.1?
As another example, consider the design of a controller for a chemical process where the decision variable are temperature, input rates of various chemicals, etc. Usually there are impurities in the
chemicals and disturbances in the heating process which may be regarded as additional inputs of a
1.2.
SOME OTHER MODELS OF DECISION PROBLEMS 5
random nature and modeled as stochastic processes. After this, just as in the case of the portfolio- selection problem, we can formulate a decision problem in such a way as to take into account these
random disturbances.
If the uncertainties are modelled stochastically as in the example above, then in many cases the techniques presented in these Notes can be usefully applied to the resulting optimal decision
problem. To do justice to these decision-making situations, however, it is necessary to give great attention to the various ways in which the uncertainties can be modelled mathematically. We also
need to worry about finding equivalent but simpler formulations. For instance, it is of great signif- icance to know that, given appropriate conditions, an optimal decision problem under uncertainty
is equivalent to another optimal decision problem under complete information. This result, known as the Certainty-Equivalence principle in economics has been extended and baptized the Separation
Theorem in the control literature. See Wonham [1968]. Unfortunately, to be able to deal with these models, we need a good background in Statistics and Probability Theory besides the material
presented in these Notes. We can only refer the reader to the extensive literature on Statistical De- cision Theory Savage [1954], Blackwell and Girshick [1954] and on Stochastic Optimal Control
Meditch [1969], Kushner [1971].
1.2.2 The case of more than one decision-maker.
