PERFORMANCE-TESTING TOOLS

15.4 PERFORMANCE-TESTING TOOLS

15.4.1 LoadRunner

LoadRunner is a testing tool for performance and load testing of software applica- tions; it proceeds by simulating arbitrary levels of workload (in the form of concurrent users) on an application under test and observing its behavior.

LoadRunner

Source Hewlett Packard Web page

http://www8.hp.com/us/en/software-solutions/loadrunner-load- testing/index.html?#.Udr82zs9-td

Access Commercial Target language(s)

Web apps, mail, databases, and so on Life cycle phase

Performance testing Test phase

Test management

Sample example of use : The main artifact that LoadRunner deals with is that of a load scenario; a load scenario is defined by a set of virtual users, along with charac- teristics of their demands on the system under test. LoadRunner enables the tester to create, edit, save, and invoke load scenarios. When a load scenario is invoked on an application, it enables the tester to observe and analyze the behavior of the system under the workload imposed by the load scenario. Once the scenario has completed its execution, Load Runner delivers a table of statistics, which may include such details as the following:

• Tables showing the number of active virtual users, the number of processed transactions, and the number of failed transactions. • Graphs plotting a variety of parameters as a function of time, including number of virtual users, response time, number of hits per second, and system resources.

15.4 PERFORMANCE-TESTING TOOLS 339

• Statistics (such as Max, Min, Average, and Standard deviation) of relevant para- meters, such as rate of occupancy of processor time, file data operations per sec- ond, processor queue length, and page faults per second.

The main strength of LoadRunner is clearly its ability to simulate arbitrary levels of workload, along with its ability to capture detailed statistical information on the behavior of the program under test; this information is a great resource to a tester who wants to analyze the operational properties of a software application.

15.4.2 Grinder

Grinder is a desktop application that enables the tester to test a software application under controlled workload conditions; to this effect, Grinder uses a specialized script- ing language called Jython, which is derived from Python.

Grinder

Source Freeware Web page

http://grinder.sourceforge.net/ Access

Open source Target language(s)

Web applications Life cycle phase

Performance testing, load testing Test phase

Test management

Sample example of use : When Grinder is called, a window opens with four tabs, labeled Graphs, Results, Processes, and Scripts. Under the Processes tab, we config- ure the agent processes that act upon the application under test. Under the Script tab, we are given the opportunity to edit test scripts or to invoke test scripts that are already written in Jython. We start the test codified in the selected script by pressing the button labeled “start process.” Upon completion of the test, we obtain statistics about the per- formance of the target application. For a transaction processing application, these may include such details as the following:

• Number of tests, divided into number of successful tests and number of failures. • Mean number of transactions per second and peak number of transactions per

second. • Mean response time, divided into mean time to resolve host, mean time to estab- lish connection, and mean time to access the first byte. • Mean response length and number of response bytes per second.

340 SOFTWARE TESTING TOOLS

Among the strengths of the Grinder, we can cite its ability to load test any appli- cation that has a Java API, its ability to support detailed test scripts written in Jython, and its ability to deploy a wide range of distributed load injectors (i.e., processes that simulate the workload on the system).

15.4.3 QF-Test

QF-Test is a tool for the creation, execution, and management of automated system and load tests for Java and Web applications with a GUI. It is compatible with a wide range of development technologies and development environments, including Swing, JavaFX, and SWT and supports cross-browser testing of web GUI. It can test static html web pages as well as dynamic web applications developed in Ajax technologies.

QF-Test

Source Quality first software Web page

http://www.qfs.de/ Access

Commercial Target language(s)

Java, web applications Life cycle phase

Performance testing, reliability testing Test phase

Test management

Sample example of use : QF-Test has a capture-and-replay function that enables it to run individual tests or complete test suites by merely starting the recording function then manually executing whatever test script the tester wants. Whenever the test script is invoked, it executes the test on the application and produces a configurable report of the execution including statistics about possible errors. Because of its scalability, this tool can be used for system-level testing.

15.4.4 Appvance PerformanceCloud

Appvance PerformanceCloud is a tool that specializes in testing the performance and load capabilities of web applications. It is compatible with Ajax development envir- onments, supports end-to-end testing to a great level of detail through all the steps of the application, can simulate millions of simultaneous users using a wide variety of web browsers, down to fine detail such as every keystroke and every message transfer, and supports pervasive agile integration. In addition, it requires little or no coding and supports customizable report generation, thereby providing faster test development.

15.4 PERFORMANCE-TESTING TOOLS 341

Appvance performanceCloud

Source

Appvance

Web page http://appvance.com/products/#sthash.ihZqzsRq.dpbs Access

Commercial, free demo, free trial Target language(s)

Web applications

Life cycle phase Performance testing, load testing Test phase

Test scripts, test execution

For a simulated user load, the tool provides a detailed analysis of system perfor- mance, including information about the performance of individual transactions, their rate of success, their resource usage, their execution path, and so on. It also offers a large menu of measures one can collect on the application under test, such as its overall resource usage, its failure history, its performance scalability, or its throughput.

15.4.5 JMeter

Apache JMeter enables the tester to analyze the performance of an application under controllable workload conditions; it can be used to evaluate system performance on static and dynamic resources and to simulate a transaction load on a server, a group of servers, or a network. It provides performance data under a wide variety of textual and graphic formats, under arbitrarily controllable heavy workloads.

Apache JMeter

Source Apache software foundation Web page

http://jmeter.apache.org/ Access

Open source Target language(s)

SQL, http, Java Life cycle phase

System-level Test phase

Test driver design and execution, test outcome analysis, performance analysis

Sample example of use : The basic artifact in JMeter testing is the test plan, which is a sequence of steps that JMeter executes whenever it is invoked. JMeter supports the tester in editing a test plan, which may contain any combination of the following elements:

• One or more thread groups, where each thread group is a collection of concurrent processes submitting simultaneous queries to the system under test.

342 SOFTWARE TESTING TOOLS

• Logic controllers, which coordinate the deployment of the thread groups by

means of control constructs (similar to programming language constructs). • Sample-generating controllers, which generate requests (e.g., http requests) that each thread group will produce when they are activated. • Listeners, which are JMeter functions that collect performance data and store it as part of the application performance analysis. • Assertions, which are test scripts that enable the tester to check whether the appli- cation under test is returning the expected results.

The output of JMeter depends on the format the tester has specified in the Listener and may include detailed information about the execution of the system under test (functions invoked in the system, number of http queries, number of database queries, number of ftp requests, number of active and sleeping tasks, etc.) as well as the resource usage of the system (CPU usage, memory usage, buffer usage, communica- tion infrastructure bandwidth, etc.).

Among the strengths of JMeter, we may mention the following: • Because JMeter is an open-source tool, a user may adapt to her/his needs and

customize it as needed. • One may build a relatively sophisticated test plan without prior knowledge of test scripts or scripting languages; also, the process of constructing test plans is fairly simple, thanks to a user-friendly GUI.