System Definition and Indicators

12.4 System Definition and Indicators

Every performance evaluation should begin with a definition and delimitation of the system under test (commonly abbreviated to SUT ). When analyzing Web applications, the analysis will typically include the (set of) Web applications themselves, the Web servers and application servers (hardware and software) to be used, the network connections between servers, between clients and servers, and the clients themselves. The SUT analysis usually studies the performance of a specific component under study (CUS) more closely. For example, a CUS can be a software component of a Web application, or the CPU of the Web server. We could, therefore, abstractly describe an SUT as a set of components, of which a subset has been identified as a CUS (see Figure 12-3). Within an SUT, we distinguish between load-creating components, i.e., clients,

252 Performance of Web Applications

Arriving Completed requests

Memory

requests

Disk Network

Figure 12-3 Component-oriented view of a system as a black box.

and load-processing components, i.e., servers, which are generally referred to as stations in performance analysis lingo. In this context, a station can not only be the server in its entirety, but also components of a server, i.e., a station can be an arbitrary system part that provides services for clients and requires time to process requests. This view allows, for example, to make assumptions on the distribution of requests within the system, or to recognize bottlenecks in the system.

In addition to this detailed component-oriented view of a system, there is a simplified representation on a higher abstraction level. This level groups all load-creating components into one single “source”, and all load-processing components into one single aggregated station. This view of an SUT is also known as the black-box model for performance analysis. This model merely looks at the characteristics of the incoming stream of requests, and the outgoing stream of completed jobs, and derivable sizes, such as the mean system response time.

We can understand from the above discussion that the (component-oriented or black-box) view of the SUT determines the measurable metrics in the PA. We have already mentioned throughput, response time, and utilization as the most important metrics, which can be stated either for the entire system or for single components. The so-called job flow analysis (see, for example, Jain 1991 for an explanation) allows to relate system-specific parameters to component-specific parameters. For example, we could use so-called “one-class” models and analytical techniques, which are based on the assumption that all requests behave equally within a system, i.e., that there is only one “class” of requests. In practice, however, this is generally not the case. It is, therefore, necessary for the work load characterization to find a representative description of the work load, which additionally allows to analyze different types of requests. The methods used for the latter analysis are called “multi-class” models, and the analytical techniques are generally rather complex, but they provide more detailed assumptions about the performance behavior of

a Web application. For example, these models can distinguish between less computing-intensive requests and (presumably more costly) insert operations in Web applications with database connections.