Characterizing the Workload

12.5 Characterizing the Workload

Selecting an appropriate workload is decisive for a performance evaluation study to be successful and supply meaningful results. Only a representative workload model can help to make representative statements about a system’s performance behavior. When selecting the workload model, we have to consider two dimensions (see Figure 12-4). The first differentiation concerns

12.5 Characterizing the Workload 253 the executability of the workload model, i.e., when measuring techniques should be used for

performance evaluation, or the non-executability of techniques that model applications. The second differentiation concerns the restriction to the actual workload (that is executable by definition), or the use of an artificial workload, which is generated by load generators rather than by real users.

Load executable

Load not executable

real workload

users / applications Load generated by real

synthetic workload

·Application suite ·Benchmark

·Instruction mix

·Analyt. models

·Kernels

·Event-driven models

·Synthet. scripts

generators Load generated by load

artificial workload

Figure 12-4 Taxonomy of workload models.

The use of a real workload is actually less significant for performance prediction. Though we could study the system under real load, i.e., in real-time operation, simply by measuring it, it would only show us a posteriori whether and when performance problems have occurred. Extracting characteristics yielded from studying the real load (see Williamson et al. 2005), and combining typical elements of the real workload (e.g., selecting typical requests), and so-called synthetic workload in experiments are more useful methods. Synthetic workload means that requests identical to those occurring in the real world are used, but with an artificially created frequency and sequence. For example, it is possible to study how the system would behave under a changed workload (i.e., the same types of requests, but with different arrival rates). Stress tests also require the creation of a synthetic workload model. If we do entirely without the kind of components that typically occur in the real workload we speak of artificial workload models. Executable artificial workload models could consist of requests used to knowingly try to represent “meaningless” user input, for example, to find security flaws in a system. The class of non-executable workload models is another artificial workload model; in this case, we would

254 Performance of Web Applications have to find a suitable abstraction level in the modeling effort. A model used in the analysis of

queuing networks (see section 12.6) describes the workload by the frequency in which requests arrive at the system (e.g., as a stochastic process) and the expected consumption of resources, the so-called service demand (D), measured in the time units a request needs to have a system component processed.

The workload cannot be meaningfully described by one single type of request, nor is it meaningful to model it as an average value from the set of all requests, especially for complex Web applications. Such a single-class workload model is easy to handle (with regard to both its modeling and measurements) and can be used for initial estimates of a system’s performance. In addition, we need a differentiated description of the workload to yield specific findings on the system performance behavior. This means that we have to use a multi-class workload model. The basic idea of multi-class workload models is to group similar requests into a load class, and then describe all requests occurring within a load class by a representative from that class. This approach uses clustering methods that recognize similarities in requests and determine representative load parameters. Clustering first defines those characteristics of the workload by which the similarity should be investigated (for example, this can be the arrival frequency or the service demanded from specific resources). The clustering principle is particularly easy to visualize when two parameters are used, but the method can also be used for an arbitrary number of parameters (which may have to be transformed before the analysis, and should be studied as to mutual independence).

The heterogeneity of documents and their multimediality require a detailed description of the workload. Only in a very simplistic model can a HTTP request be seen as a unit of the workload;

a performance analysis requires a classification and representation of a wide range of different request types. The field of workload characterization can be even more difficult because user behavior is often hard to predict and changes continually. To solve these problems, we could use generative or learning models for workload characterization (see Haring et al. 2001).