- 227 -
Basically, we just ask the public server for a request. The releaseRequest method blocks until one is available.
Then we process it and start again. for;;
{ request = queueServer.releaseRequest ;
isAvailable = false; result = requestProcessor.processRequestrequest;
returnResultresult; isAvailable = true;
} }
public boolean isAvailable { return isAvailable;} public void returnResultRequestResult r {}
}
Note that the server classes as they stand can be tested with the following minimal implementations for support classes:
class FIFO_Queue { java.util.Stack v = new java.util.Stack ;
public void addObject o{v.pusho;} public Object pop {return v.pop ;}
public boolean isEmpty {return v.isEmpty ;} }
class RequestProcessor { public RequestResult processRequestRequest r
{ System.out.printlnProcessing request: + r;
try{Thread.sleep2000;}catchInterruptedException e{} return new RequestResult ;
} }
class RequestResult {} class Request {}
10.8.3 A Load-Balancing Example
It may help to look at a concrete implementation of load balancing. Ill consider the task of downloading many pages from a web server as quickly as possible.
It is impolite to batch download at high speeds from a single web server. Automated programs that download multiple pages from web servers have a voluntary protocol
they should adhere to. More information can be found at
http:info.webcrawler.commakprojectsrobotsrobots.html and
http:web.nexor.co.ukmakdocrobotsguidelines.html . One item of the protocol is to
avoid overloading web servers by downloading many pages at a high access rate. Automated download programs that are polite specifically stagger downloads over a
long period in order to minimize the load on the web server.
The individual page download code is quite simple. Open a URL , read the data, and dump it into a local file :
- 228 -
Two args, the local file to put the downloaded page into, and the URL where the page to download is.
public static void dowloadString file, String url throws IOException
{ URL u = new URLurl;
InputStream in = null; Try repeatedly to get the page opened. Note that catching
all exceptions is not such a good idea here. It would be much better to catch individual execption types and handle
them separately. Some exceptions should not lead to a repeated attempt to access the page. But this definition is okay for testing.
whilein == null try{in = u.openStream ;}
catchException e{try {Thread.sleep500;}catchException e2{}} FileOutputStream out = new FileOutputStreamfile;
byte[] buffer = new byte[8192]; read until the connection terminates this is not a
keep-alive connection, and write to the file. int len = in.readbuffer;
whilelen = -1 {
out.writebuffer, 0, len; len = in.readbuffer;
} out.close ;
in.close ; }
All our tests use this same
download
method. The most straightforward test implementation is extremely simple. Simply take a list of URLs and corresponding data files, and loop calling
download
for each URLfile pair:
Use one array to hold alternate file and URL elements public static void iterativeTestString[] files
throws IOException {
for int i = 0; i files.length; i+=2 downloadfiles[i], files[i+1];
}
The opposite to downloading pages one by one is to try to download everything at the same time. Once again, the code is quite straightforward apart from timing issues: see
Section 10.5 . You
simply define a
Runnable
class, and loop starting a thread for every download:
public class LoadBalancing implements Runnable
{ String url;
String localfilename; public static void massivelyParallelTestString[] files
throws IOException {
for int i = 0; i files.length; i+=2 new Threadnew LoadBalancingfiles[i], files[i+1].start ;
} public LoadBalancingString f, String u
- 229 -
{ localfilename = f;
url = s; }
public void run {
try {
downloadlocalfilename, filename; }
catchException e {e.printStackTrace ;} }
The earlier iterative test takes seven times longer than the latter multithreaded test.
[8]
However, the latter test suffers from significant resource problems. Creating so many threads simultaneously can
seriously strain a system. In fact, every system has a limit to the number of threads it can create. If the download requires more threads than the system is capable of supporting, this multithreaded test
fails to download many pages. In addition, with so many threads running simultaneously, you are using more of the systems resources than is optimal.
[8]
For my tests, I downloaded a large number of pages. I validated the tests over the Internet, but not surprisingly, Internet access times are extremely variable. For detailed repeatable tests, I used a small local HTTP server that allowed me to control all the parameters to the tests very precisely. The full test class,
tuning.threads.LoadBalancing
, is available with the other classes from this book.
Lets look at a more balanced approach. In fact, you can create a very simple load-balanced test with one small variation to the last test. Simply add a delay between each thread creation to stagger the
system load from threads and downloading. This new version of the
massivelyParallelTest
method is simple:
public static void roughlyParallelTestString[] files, int delay throws IOException
{ for int i = 0; i files.length; i+=2
{ new Threadnew LoadBalancingfiles[i], files[i+1].start ;
try{Thread.sleepdelay;}catchInterruptedException e{} }
}
Now you have a tuning parameter that needs to be optimized. Obviously, a delay of zero is the same test as the previous test, and a very large delay means that the test is spending most of its time
simply waiting to create the next thread. Somewhere in between is an optimal range that creates threads fast enough to fully use the system resources, but not so fast that the system is overloaded.
This range is different depending on the full environment, and probably needs to be experimentally determined. But you can make a decent first guess by considering the bottlenecks of the system. In
this case, the bottlenecks are CPU, system memory, disk throughput, network-connection latency, server-download rates, and network throughput. In my tests, system memory and CPU limit the
number of threads and download speed for the massively parallel case, but you are using a delay specifically to reduce the load on those resources. System memory constrains the number of threads
you can use, but again, the delay avoids overloading this resource provided that the delay is not too short. Disk throughput can be significant, but network and server throughput are far more likely to
limit data-transfer rates. So we are left with network-transfer rates and network-connection latency to consider.
- 230 - Now you can make a good guess as to a starting point for the delay. You can evaluate the average
number of bytes to transfer for each download, and work out the amount of time this takes based on the available network throughput. You can also estimate the average time taken to make a
connection by measuring some real connections. A straightforward guess is to set the delay at a value below the higher of these two averages. In my tests, the files being downloaded are not large,
and the average connection time is the larger time. I started with a delay of about half the average connection time and ran tests increasing and decreasing the delay in steps of about 10 at a time.
Figure 10-3 shows the results of varying the delay times. An optimum choice for the delay in this
particular test environment is approximately 70 of the average connection time. The flat line in the middle of the graph shows the relative time taken for the massively parallel test.
Figure 10-3. The results of varying the delay
The results show that for this environment there are several advantages to running with a delay. A decisive advantage is that you never run out of system resources. There are never so many threads
running simultaneously that you run out of memory and completely lose URLs, as occurred with the massively parallel test. In fact, the system doesnt even need to run to full capacity for most of
the test.
Another significant advantage is that by tuning the delay, you can run the test faster. The optimum value for the delay, at 70 of the average connection time, executes the full test in 90 of the
massively parallel time.
What about our nice load-balancing architecture classes? Lets test these to see how they compare to the last simple optimization you made. You need to add a few support classes so that your load-
balancing architecture is running the same download test. Basically, there are three classes to define:
Request
,
RequestProcessor
, and
RequestResult
. They are fairly simple to implement.
Request
needs to hold only a URL and a local file for storing the downloaded page.
RequestProcessor
simply needs to call the
download
method.
RequestResult
does not need any extra behavior for the test.
[9]
The classes are as follows:
[9]
RequestResult
does need extra state and behavior in order to make timing measurments, and
RequestProcessor
is similarly a bit more complicated for timing purposes. For full details, see the test class,
tuning.threads.LoadBalancing
, which is available with the other classes from this book.
class RequestProcessor { public RequestResult processRequestRequest r
{ try
{ LoadBalancing.dowloadr.localfilename, r.url;
} catchException e {e.printStackTrace ;}
return new RequestResult ;
- 231 -
} }
class Request {
String localfilename; String url;
public RequestString f, String u {
localfilename = f; url = u;
} }
class RequestResult {}
In addition, of course, you need to define the method that kicks off the test itself:
public static void loadBalancedTestString[] files, int numThreads throws IOException
{ ActiveRequestQueue server = new ActiveRequestQueuenumThreads;
for int i = 0; i files.length; i+=2 server.acceptRequestnew Requestfiles[i], files[i+1];
}
I have included a variable to determine the optimum number of threads. As with the earlier test that used a variable delay, the optimum number of threads for this test needs to be experimentally
determined. For my test environment, the bottleneck is likely to be my small network throughput. This is easy to see: each thread corresponds to one download. So for n threads to be working at full
capacity, they need to be downloading n files, which amounts to a throughput of n times the average file size. This means that for my test environment, about 10 threads reach capacity. In fact, since
files are not all the same size and there are some overheads in the architecture, I would expect the optimum number of threads to be slightly larger than 10.
Running the test with different numbers of threads shows that for 12 or more threads, the time taken is essentially the same see
Figure 10-4 . This time is also the same as that achieved with the
previous most optimal test. This is not surprising. Both tests optimized the downloads enough that they have reached the same network-throughput bottleneck . This bottleneck cannot be optimized
any further by either test.
Figure 10-4. Time taken for the load-balanced download versus number of threads
The load-balancing architecture is more complex than adding a delay between threads, but it is much more flexible and far more controlled. If you want to vary your download in a number of
ways, such as prioritizing URLs or repeating certain failed ones, it is much easier to do so with the
- 232 - load-balancing architecture. By looking at the CPU utilization graphs for the load-balancing
architecture compared to the other tests in Figure 10-5
, you can easily see how much more controlled it is and how it uses resources in a far more consistent manner.
Figure 10-5. CPU utilization for various download tests
10.9 Threaded Problem-Solving Strategies
