The way to go depends on the coefficient of variation CV of the distribution of job runtimes. The coefficient of variation is the standard deviation divided by the mean.
This is a sort of normalized version of the standard deviation, and measures how wide the distribution is. “Narrow” distributions have a small CV, while very wide or fat
tailed distributions have a large CV. The exponential distribution has CV = 1.
Returning to jobs in computer systems, if the CV is smaller than 1 than we can expect new jobs to be similar to the running job. In this case it is best to leave the
running job alone and schedule additional jobs FCFS. If the CV is larger than 1, on the other hand, then we can expect new jobs to be shorter than the current job. Therefore
it is best to preempt the current job and run the new job instead.
Measurements from several different systems show that the distribution of job runtimes is heavy tailed. There are many very short jobs, some “middle” jobs, and
few long jobs, but some of the long jobs are very long. The CV is always larger than 1 values from about 3 to about 70 have been reported. Therefore responsiveness is
improved by using preemption and time slicing, and the above examples are correct.
To read more: The benefit of using preemption when the CV of service times is greater than 1 was established by Regis [13].
Details: the distribution of job runtimes
There is surprisingly little published data about real measurements of job runtimes and their distributions. Given the observation that the CV should be greater than 1, a com-
mon procedure is to choose a simple distribution that matches the first two moments, and thus has the correct mean and CV. The chosen distribution is usually a two-stage
hyper-exponential, i.e. the probabilistic combination of two exponentials. However, this procedure fails to actually create a distribution with the right shape, and might lead to
erroneous performance evaluations, as demonstrated by Lazowska [9].
An interesting model for interactive systems was given by Leland and Ott [10], and later verified by Harchol-Balter and Downey [7]. This model holds for processes that are longer
than a couple of seconds, on Unix systems. For such processes, the observed distribution is
Prr t = 1t where
r denotes the process runtime. In other words, the tail of the distribution of runtimes has a Pareto distribution.
3.2.2 Multiprogramming and Utilization
The second reason for multiprogramming is to improve hardware utilization.
Applications use one system component at a time
Consider a simple example of a system with a CPU and disk. When an application reads data from the disk, the CPU is idle because it has to wait for the data to arrive.
61
Thus the application uses either the CPU or the disk at any given moment, but not both:
CPU disk
1st IO operation
ends 3rd IO
operation IO
ends IO
operation 2nd IO
ends IO
time
idle idle
idle idle
idle idle
Note: The time to perform IO
An important issue concerning the use of different devices is the time scale involved. It is important to note that the time scales of the CPU and IO devices are typically
very different. The cycle time of a modern microprocessor is on the order of part of a nanosecond. The time to perform a disk operation is on the order of several milliseconds.
Thus an IO operation takes the same time as millions of CPU instructions.
Multiprogramming allows for simultaneous use of several components
If more than one job is being serviced, then instead of waiting for an IO operation to complete, the CPU can switch to another job. This does not guarantee that the CPU
never has to wait, nor that the disk will always be kept busy. However it is in general possible to keep several systems components busy serving different jobs.
CPU disk
time job1 IO
operation ends
IO operation
job1 IO ends
IO ends
IO operation
job2 IO
idle idle
idle idle
job1 job1
job1 job1
job2 job2
job2
Improved utilization can also lead to improved responsiveness
Improved utilization is good in itself, because it means that the expensive hardware you paid for is being used, rather than sitting idle. You are getting more for your
money.
In addition, allowing one job to use resources left idle by another helps the first job to make progress. With luck, it will also terminate sooner. This is similar to
the processor sharing idea described above, except that here we are sharing all the system resources, not just the CPU.
62
Finally, by removing the constraint that jobs have to wait for each other, and al- lowing resources to be utilized instead of being left idle, more jobs can be serviced.
This is expressed as a potential increase in throughput. Realization of this potential depends on the arrival of more jobs.
Exercise 49 In the MM1 analysis from Chapter 2, we saw that the average response time grows monotonically with utilization. Does this contradict the claims made here?
All this depends on an appropriate job mix
The degree to which multiprogramming improves system utilization depends on the requirements of the different jobs.
If all the jobs are compute-bound, meaning they need a lot of CPU cycles and do not perform much IO, the CPU will be the bottleneck. If all the jobs are IO-bound,
meaning that they only compute for a little while and then perform IO operations, the disk will become a bottleneck. In either case, multiprogramming will not help
much.
In order to use all the system components effectively, a suitable job mix is re- quired. For example, there could be one compute-bound application, and a few IO-
bound ones. Some of the applications may require a lot of memory space, while others require only little memory.
Exercise 50 Under what conditions is it reasonable to have only one compute-bound job, but multiple IO-bound jobs? What about the other way around?
The operating system can create a suitable job mix by judicious long-term schedul- ing
. Jobs that complement each other will be loaded into memory and executed. Jobs that contend for the same resources as other jobs will be swapped out and have to
wait. The question remains of how to classify the jobs: is a new job going to be compute-
bound or IO bound? An estimate can be derived from the job’s history. If it has already performed multiple IO operations, it will probably continue to do so. If it has
not performed any IO, it probably will not do much in the future, but rather continue to just use the CPU.
3.2.3 Multitasking for Concurrency
