Free Load Balancing from TCPIP

- 224 - response time gets worse. The operating-system thread scheduler or Java-system thread scheduler, if youre not using OS threads spends more and more time managing threads, and this overhead takes up the CPU time rather than allowing the threads to run. You also need to consider whether the queue object handles the responses collecting them from the request processes and handing them back to the clients or whether the request-processor objects can hand the responses back directly. The former design has the advantage that the client cannot get any direct access to the request-processor objects, but the disadvantage that you are introducing an unnecessary bottleneck in processing terms. The latter option handing responses back directly, of course, has the opposite characteristics: no extra bottleneck, but access to client objects is enabled.

10.8.1 Free Load Balancing from TCPIP

If you use sockets to handle incoming requests within one process, the operating system provides some load-balancing support. If you want, the operating system will provide the queue for free. TCP sockets can have multiple threads reading or accepting on them. A connectionless TCP server such as a web server performs the following process: 1. Opens a server socket. 2. Starts however many threads you want. 3. Each thread sits on an ServerSocket.accept call, waiting for the call to return all threads call accept on the identical ServerSocket object. 4. Whenever a client connects to the server socket, the operating-system TCP stack hands the connection off to only one thread that is blocked on the accept call. This is guaranteed behavior for TCP. 5. The thread that returns from the accept call gets the client connection Socket object, reads the request, processes it, and writes the request back directly to the client. 6. The thread goes back into the accept call, waiting for the next connection. At any time, you can start further threads to scale up the server as long as each thread has access to the previously created ServerSocket object. TCP does not allow more than one ServerSocket object to be bound to the same port on any machine actually, any one network interface. It is therefore not possible to have multiple separate processes i.e., independent operating-system processes, rather than threads within one operating-system process serving on the same server socket. Strictly speaking, it is possible to fork a process into multiple system processes after the socket has been opened. This is a standard practice on Unix servers. Multiprocess TCP servers have some small disadvantages over multithreaded TCP servers, mainly when they need to communicate between themselves or use other expensive resources. However, multiprocess TCP servers do have one big advantage over multithreaded servers, which is that if one server process crashes, the others can continue running independently, unaffected by the crash. Win32 does not support a fork procedure. With UDP sockets, the architecture can be slightly different, as you can open a UDP server socket on a port that already has a server socket bound to it. A UDP socket is not connection-oriented but packet-oriented, so there is no accept call to wait on. Instead, all the threads from potentially multiple system processes sit on a read call on the UDP socket, and the UDP stack hands off each incoming packet to just one of the threads that is waiting on the read . The server then has to use the information from the packet either at the application level or the protocol level to determine the return address to send the result of the processed request again, directly back to the client. - 225 -

10.8.2 Load-Balancing Classes