1.2.
SOME OTHER MODELS OF DECISION PROBLEMS 5
random nature and modeled as stochastic processes. After this, just as in the case of the portfolio- selection problem, we can formulate a decision problem in such a way as to take into account these
random disturbances.
If the uncertainties are modelled stochastically as in the example above, then in many cases the techniques presented in these Notes can be usefully applied to the resulting optimal decision
problem. To do justice to these decision-making situations, however, it is necessary to give great attention to the various ways in which the uncertainties can be modelled mathematically. We also
need to worry about finding equivalent but simpler formulations. For instance, it is of great signif- icance to know that, given appropriate conditions, an optimal decision problem under uncertainty
is equivalent to another optimal decision problem under complete information. This result, known as the Certainty-Equivalence principle in economics has been extended and baptized the Separation
Theorem in the control literature. See Wonham [1968]. Unfortunately, to be able to deal with these models, we need a good background in Statistics and Probability Theory besides the material
presented in these Notes. We can only refer the reader to the extensive literature on Statistical De- cision Theory Savage [1954], Blackwell and Girshick [1954] and on Stochastic Optimal Control
Meditch [1969], Kushner [1971].
1.2.2 The case of more than one decision-maker.
Agent Alpha is chasing agent Beta. The place is a large circular field. Alpha is driving a fast, heavy car which does not maneuver easily, whereas Beta is riding a motor scooter, slow but with good
maneuverability. What should Alpha do to get as close to Beta as possible? What should Beta do to stay out of Alpha’s reach? This situation is fundamentally different from those discussed so
far. Here there are two decision-makers with opposing objectives. Each agent does not know what the other is planning to do, yet the effectiveness of his decision depends crucially upon the other’s
decision, so that optimality cannot be defined as we did earlier. We need a new concept of rational optimal decision-making. Situations such as these have been studied extensively and an elaborate
structure, known as the Theory of Games, exists which describes and prescribes behavior in these situations. Although the practical impact of this theory is not great, it has proved to be among the
most fruitful sources of unifying analytical concepts in the social sciences, notably economics and political science. The best single source for Game Theory is still Luce and Raiffa [1957], whereas
the mathematical content of the theory is concisely displayed in Owen [1968]. The control theorist will probably be most interested in Isaacs [1965], and Blaquiere, et al., [1969].
The difficulty caused by the lack of knowledge of the actions of the other decision-making agents arises even if all the agents have the same objective, since a particular decision taken by our agent
may be better or worse than another decision depending upon the unknown decisions taken by the other agents. It is of crucial importance to invent schemes to coordinate the actions of the individual
decision-makers in a consistent manner. Although problems involving many decision-makers are present in any system of large size, the number of results available is pitifully small. See Mesarovic,
et al.
, [1970] and Marschak and Radner [1971]. In the author’s opinion, these problems represent one of the most important and challenging areas of research in decision theory.
6
CHAPTER 1. INTRODUCTION
Chapter 2
OPTIMIZATION OVER AN OPEN SET
In this chapter we study in detail the first example of Chapter 1. We first establish some notation which will be in force throughout these Notes. Then we study our example. This will generalize
to a canonical problem, the properties of whose solution are stated as a theorem. Some additional properties are mentioned in the last section.
2.1 Notation
