Lab Management

8 Lab Management

210 Chapter 8: Lab Management

WHAT THIS CHAPTER DOESN’T COVER This chapter does not cover the set up of an environment to support Lab Management nor does it cover configuring TFS to use Lab Management because these are administrative tasks and play no part in the actual test- ing process.

Managing Virtual Environments Through MTM

Before getting into the usage of Lab Management, you need to understand how to interact with the Virtual Environments through MTM.

Managing Virtual Environments Through MTM 211

4. Find an appropriate template or stored VM, and click Add (see Figure 8-3).

5. Enter the Name, Description (optional), and Default role, and click Next.

6. Set the amount of memory to use, and click the OS profile tab.

7. Fill in any relevant information. (For this chapter, the machines are joined to the external domain already.)

8. Click Finish.

212 Chapter 8: Lab Management

Figure 8-3: Import step 3

Managing Virtual Environments Through MTM 213

Figure 8-4: A stored VM

214 Chapter 8: Lab Management not be joined to an external domain. Network Isolation enables you to either

create your own virtual domain (this requires that one of the machines in the environment be a domain controller) or the VMs are just in a workgroup. If the machines are joined to a domain, this option is not be available to you. This capability and general control of the VMs and environment as it relates to testing is provided by the Lab Agent.

A completed two-tier environment (the environment used for the rest of the examples in this chapter) is shown in Figure 8-5.

Managing Virtual Environments Through MTM 215

216 Chapter 8: Lab Management Table 8-1: Continued

Option

Description

Shut Down Shuts down each machine in the Virtual Environment. This is the equivalent of selecting Start, Shut Down from each machine.

Power Off Shuts down each machine in the Virtual Environment. This is the equivalent of pulling the power cord out of the back of each machine; in general, do not use this method to shut down an environment.

Pause Suspends each machine in the Virtual Environment. The state of the environment is stored, and resources the environment was using are released back to the operating system.

Because you may have created certain environments for certain uses on

Finishing Virtual Environment Configuration 217

Finishing Virtual Environment Configuration

Before starting any types of builds, you need to finish the configuration of the Virtual Environment. This is actually a post-setup configuration done by the test team. To that end following are three critical tips:

1. Be sure to install any software needed for the deployment of applica- tions before taking any snapshots.

2. Always take a snapshot of the Virtual Environment in its base state— that is, before you have made any changes to it and while it is in the Off state.

3. Always take a snapshot of the Virtual Environment in the running state with the account running the Test Agent as the logged on user.

218 Chapter 8: Lab Management The third recommendation seems a bit odd because it violates one of the

reasons for taking the first snapshot. This is an issue of configuration and speed. To run Coded UI Tests, you must be logged on as the account under which the TFS Test service runs. Taking a snapshot at this point means you don’t need to log on. Another benefit is that when the environment runs and you roll back to that running snapshot, the environment does not need to spool up again; it simply resumes where it last was. This makes your testing process that much faster.

The first step is to configure the machine that will be executing the tests, for this example the Webserver (App Tier) server. Connect to the Virtual Envi- ronment which will open the Environment Viewer (see Figure 8-8).

Finishing Virtual Environment Configuration 219

in Chapter 7. Then shut down the environment and take two snapshots: the base and a running snapshot. Exercise 8-2 walks you through this process.

EXERCISE 8-2 Snapshot of an Environment

To take a base and running snapshot of your environment, follow these steps:

1. In the Environment Viewer, click the Shut Down Environment button above the Machines and Snapshot tabs.

2. When all the machines in the environment have shut down, select the Snapshots tab.

3. Click the Take Snapshot button and name the snapshot <environ-

220 Chapter 8: Lab Management On the Snapshot tab, you have a number of options to manage your snap-

shots. You can take a new snapshot, rename a snapshot, delete a snapshot, revert to the previous snapshot, or restore to a specifically selected snapshot. Another option discussed in the section “Manual Tests in a Virtual Environ- ment” is sharing a snapshot that enables some easy troubleshooting of failed tests. If you are not familiar with Hyper-V, the Now node indicates the snap- shot that the current state of the VM started from. In other words, the current state is the point in time at which the snapshot was taken plus any additional actions up to now. And the tree structure is not by accident; you can have a complex tree with different branches to meet a variety of needs. For exam- ple, looking at Figure 8-9, you could select the Two Tier (Base–Off) node and select Restore to Selected Snapshot, and you would end up with the situation shown in Figure 8-10.

Automated Te st Settings 221

• Try to keep only three levels deep to the tree: the base (off), the base (running), and a snapshot for each build, and deploy to the Virtual Environment. There is a performance impact when you have large numbers of snapshots.

• Perform maintenance on the snapshots as you would for any environ- ment, and remove unnecessary snapshots.

For the purposes of this chapter, everything depends on the Two Tier (Base–Running) snapshot.

Automated Test Settings

Before creating the lab build, you need to create the automated test settings

222 Chapter 8: Lab Management Here you can select the type of roles to execute the tests. Roles are things

such as a database server or client or application server, or some other type of environment. The environments that match the required roles display. (Phys- ical environments are also shown.) After you select the type of role for the automated settings, you have the option to choose any environment that matches the role (in the Test Plan properties, Automated Settings, Environ- ment selection). The other key item, which was briefly noted in Chapter 7, is that you must select which role will execute the tests. In this case the tests will

be executed from the Web Server role. The last difference, which was also noted in Chapter 7, is the Advanced page that enables you to execute scripts and select other items related to automated tests.

The last thing to take care of is to set the automated test settings for the Test Plan. In the situation in which you use the lab build template, you must

Automated Te st Settings 223

in Visual Studio. (Workflows cannot be edited in MTM.) This folder contains three build templates: DefaultTemplate.xaml, LabDefaultTemplate.xaml, and UpgradeTemplate.xaml. These are the Workflow (WF) 4.0 build definitions. You can reuse workflows as needed and simply supply different data. Double-click- ing any file shows you the workflow (see Figure 8-12).

224 Chapter 8: Lab Management http://blogs.msdn.com/b/jimlamb/archive/2009/11/18/how-to-create-a-

custom-workflow-activity-for-tfs-build-2010.aspx.) For the purposes of this section, the BlogEngine.NET already exists. The actual creation of the Lab build is shown next. Open Visual Studio, and go to Team Explorer. Expand the Team Project, and right-click the Build node. Select New Build definition. Creating the build settings is the same for any build until you change the process template. On the Process tab, click the Show Details button next to DefaultTemplate.xaml, and select the LabDe- faultTemplate.xaml from the list.

USING THE BUILT-IN TEMPLATES The built-in templates have been thoroughly tested, so you should use them.

Automated Te st Settings 225

Clicking in the Lab Process Settings gives you access to an ellipsis that brings up a lab-specific set of build options. These screens are covered in detail here. Figure 8-14 shows the Environment screen.

226 Chapter 8: Lab Management

Automated Te st Settings 227

After you select the option to Deploy the build, you can provide pointers to scripts that execute to perform this deployment. Deploying applications via scripts is actually simple in Visual Studio 2010. In Figure 8-16 you can see three listed scripts: Deploy.cmd, SetupWebServer.cmd, and DeployData- base.cmd. Two good options for creating these scripts are PowerShell or the Windows Shell command line. PowerShell provides the most options. The command line is used for this walkthrough. Also, although this deployment has been broken up into three scripts, these can all be in one script. You will probably like the granularity and reusability of scripts. Look at each of these scripts, starting with the Deploy.cmd script in Listing 8-1. (Line numbers are added for clarity.)

Listing 8-1: Deploy.cmd Deployment Script

228 Chapter 8: Lab Management Table 8-2: Macros and Their Descriptions

Macro

Description

$(BuildLocation) The location of the build output or the specific directory selected if you chose to use an exist- ing build.

$(InternalComputerName_ The name of the machine that you are referring to. <VMName>

In this chapter you have been using two systems: DataServer and WebServer. To reference them you use $(InternalComputerName_WebServer) or $(InternalComputerName_Dataserver).

$(ComputerName_<VMName> The fully qualified domain name (FQDN) of the machine (for example webserver.demo.local). The reason for a second computer name macro is to support isolated environments. Isolated

Automated Te st Settings 229

Line 1 of the SetupWebServer script invokes the appcmd command-line tool (this is an IIS-specific command-line tool) to actually create the Blo- gEngineWeb site, set the appropriate bindings, and create the physical path where this site points to. You can find more information on this tool at http://technet.microsoft.com/en-us/library/cc772200(WS.10).aspx. There are many more options to appcmd, and if you haven’t used it before, it can be

a big help. Line 2 performs a neat trick; it executes the MSDeploy engine. You learned in the section on configuring the VM environment that you need to install any applications that are a prerequisite to installing your application. In this case MSDeploy is installed on the web server. The end result of this deployment is that the website content has been deployed to the physical directory, and all virtual directory settings have been set based on the Blo- gEngineWeb project. You can find more information on MSDeploy at

230 Chapter 8: Lab Management need to be installed separately. The first command-line argument provides

the path to the files, and the second argument provides the name of the data- base server to connect to.

One option you have is running different scripts on different machines. The first column of the script list enables you to select which machine the script is executed on, so many different scripts can be executed on many dif- ferent machines. The limit is only your imagination.

So where do these scripts come from? They are part of the actual project and placed in a Scripts folder with the action set to Copy Always. These scripts can be part of the test project as well or even a separate project included as part of the build—it’s up to you. The recommended method for doing this though is to version them as part of the actual project because they reference information contained in the project (such as the database project

Exe cuting a Lab Build 231

This screen enables you to select your test plan, tests to execute, test con- figuration, and the automated settings. The biggest gotcha here is making sure the automated test settings are created first. If you forget to create them, you will not have any options available to you in the final drop-down, and the tests will not execute. (You can save this build definition, create the test settings, and then come back and update the definition.) The settings here are self-explanatory. When you finish, click Finish and save the build.

Executing a Lab Build

When you execute the build, assuming everything is set up correctly, it just runs and executes the tests specified. That’s it. No drama, no fuss, no muss. Okay, the reality is that the first time you go to execute any lab build you may

232 Chapter 8: Lab Management Figure 8-18 shows a completed lab build.

Running Automated Te sts Through MTM 233

SERVER NAMES Pay special attention to server names, especially when dealing with web ap- plications. To make life easier, record the manual tests against a web server in the virtual environment against which the tests will be executed. This way the correct server name is recorded and output as part of the Coded UI test. If you forget to do this, do a search and replace in the .uitest file to replace the server name with the correct name.

Running Automated Tests Through MTM

After a build deploys to a Virtual Environment, running automated tests against the build is trivial. At this point, you have already created the auto-

234 Chapter 8: Lab Management tests in the Virtual Environment, enabling you to work on other items. At this

point, as with executing the build, it is all fairly well done for you.

SPEED UP YOUR TESTING Adding lots of diagnostic data adapters to gather information takes time. It is worthwhile to keep multiple sets of automated (and maybe even manual) settings that you can use as needed. For example, one set may collect virtu- ally no information (no IntelliTrace, no video recording, nothing) and another may gather detailed diagnostics because a bug was found in an earlier test run, and you need more data for the developer.

When the tests finish running, you can view them in the Analyze Test

Manual Te sts in a Virtual Environment 235

To set up manual testing in a Virtual Environment from outside of the envi- ronment, create a new set of test settings, and on the Roles tab, select the Vir- tual Environment configuration you want (see Figure 8-19).

236 Chapter 8: Lab Management

Manual Te sts in a Virtual Environment 237

238 Chapter 8: Lab Management Second, data was captured from both agents.

The ability to execute manual tests in a controlled environment according to a set of scripts opens up several possibilities for managing software accept- ance. Users can test the latest builds of software without impacting their day- to-day work and provide fast feedback to the team. Acceptance testing can also be performed in a controlled environment so that when a customer finds

a problem it can be easily diagnosed.

SUMMARY Whether you need a single machine or an entire farm of VMs, Lab Manage-

ment provides an infrastructure to meet your testing and development needs. Regardless of whether you do only manual testing or a mix of manual and