Non-blocking IO IO on SSL Connections

132 Typically, using blocking IO alleviates the complications of retrying failed calls. With SSL, this is not the case. There are times when a call to SSL_read or SSL_write on a blocking connection requires a retry.

5.2.2.2 Blocking IO

Blocking IO means that an operation will wait until it can be completed or until an error occurs. Thus, with blocking IO in general, we shouldnt have to retry an IO call, since a single calls failure indicates an error with the communication channel, such that the channel is no longer usable; with blocking IO using OpenSSL, this is not true. The SSL connection is based upon an underlying IO layer that handles the actual transfer of data from one side to the other. The blocking property of an SSL connection is based solely on the underlying communication layer. For instance, if we create an SSL connection from a socket, the SSL connection will be blocking if and only if the socket is blocking. It is also possible to change this property in a connection weve already established; we change the property on the underlying layer and then handle all SSL IO functions appropriate for the new paradigm. Conceptually, SSL_read and SSL_write read and write data from the peer. Actually, a call to SSL_read may write data to the underlying IO layer, and a call to SSL_write may read. This usually occurs during a renegotiation. Since a renegotiation may occur at any time, this behavior can cause unexpected results when using a blocking IO layer; namely, an IO call may require a retry. Thus, our implementation must handle this. In order to handle a blocking call correctly, we need to retry the SSL IO operation if we receive SSL_ERROR_WANT_READ or SSL_ERROR_WANT_WRITE . Dont let the names of these errors confuse you. Even though they tell us the SSL connection needs to wait until we can read or write, respectively, we just need to retry whatever operation caused the condition. For instance, if an SSL_read caused the SSL_ERROR_WANT_WRITE error, we must retry the SSL_read rather than making the call to SSL_write . It is worth taking a moment to understand the potential errors for a single IO call. Though nonintuitive, a call to SSL_read may indeed return the error SSL_ERROR_WANT_WRITE due to the possibility of renegotiations at any point. In many ways, implementing a blocking call is similar to implementing a non-blocking one. Its similar in the sense that we must loop for retries, but it differs in that we dont need to check for IO availability on the underlying layer, as with poll or select . In the end, we will not show an example of looping to accomplish the blocking call; there is an easier way. Using the SSL_CTX_set_mode function or its counterpart for SSL objects, SSL_set_mode , we can set some IO behaviors of SSL connections. The second parameter is a set of defined properties joined with the logical OR operator. SSL_MODE_AUTO_RETRY is one such mode. Setting this on a blocking SSL object or on the context that will create a object will cause all IO operations to automatically retry all reads and complete all negotiations before returning. Using this option allows us to implement IO as simply as normal blocking IO with the system calls read and write . In general, we should set this option on the SSL_CTX object before creating the SSL object. We can set the option later on the SSL object itself, but its best to do so before any calls to IO routines on that object. If we elect not to use this option and instead implement our blocking IO with our own loops, we might fall into a few traps. This is due to some special requirements for function call retries, which are detailed with our discussion of non-blocking IO.

5.2.2.3 Non-blocking IO

133 This paradigm causes all of our IO calls never to block. If the underlying layer is unable to handle a request, it reports its requirement immediately, without waiting. As weve hinted, this adds complexity to our IO routines. A non-blocking SSL IO call returns the reason of failure, but only the application can check to see if that status has been cleared. This is the source of the complexity in implementation. For instance, a call to SSL_read may return SSL_ERROR_WANT_READ , which tells the application that once the underlying IO layer is ready to fulfill a read request, the SSL call may be retried. Generally, the applications IO loop will need to serve both read and write requests, however. The problem we need to solve in the IO loop is that once weve made a call to an SSL IO function, and it requires a retry, we should not call other IO functions until the original call has succeeded. Since the logic for correctly implementing IO routines for the application can have several subtleties, especially with multiple input and output sources, well look at a detailed example. Example 5-16 provides the code for the function data_transfer . This function takes two SSL objects as parameters, which are expected to have connections to two different peers. The data_transfer function will read data from one connection A and write it to the other B and at the same time read data from B and write it to A. Example 5-16. A sample non-blocking IO loop 1 include opensslssl.h 2 include opensslerr.h 3 include string.h 4 5 define BUF_SIZE 80 6 7 void data_transferSSL A, SSL B 8 { 9 the buffers and the size variables 10 unsigned char A2B[BUF_SIZE]; 11 unsigned char B2A[BUF_SIZE]; 12 unsigned int A2B_len = 0; 13 unsigned int B2A_len = 0; 14 flags to mark that we have some data to write 15 unsigned int have_data_A2B = 0; 16 unsigned int have_data_B2A = 0; 17 flags set by check_availability that poll for IO status 18 unsigned int can_read_A = 0; 19 unsigned int can_read_B = 0; 20 unsigned int can_write_A = 0; 21 unsigned int can_write_B = 0; 22 flags to mark all the combinations of why were blocking 23 unsigned int read_waiton_write_A = 0; 24 unsigned int read_waiton_write_B = 0; 25 unsigned int read_waiton_read_A = 0; 26 unsigned int read_waiton_read_B = 0; 27 unsigned int write_waiton_write_A = 0; 28 unsigned int write_waiton_write_B = 0; 29 unsigned int write_waiton_read_A = 0; 30 unsigned int write_waiton_read_B = 0; 31 variable to hold return value of an IO operation 32 int code; 33 34 make the underlying IO layer behind each SSL object non- blocking 35 set_nonblockingA; 36 set_nonblockingB; 37 SSL_set_modeA, SSL_MODE_ENABLE_PARTIAL_WRITE| 38 SSL_MODE_ACCEPT_MOVING_WRITE_BUFFER; 39 SSL_set_modeB, SSL_MODE_ENABLE_PARTIAL_WRITE| 40 SSL_MODE_ACCEPT_MOVING_WRITE_BUFFER; 41 134 42 for ;; 43 { 44 check IO availability and set flags 45 check_availabilityA, can_read_A, can_write_A, 46 B, can_read_B, can_write_B; 47 48 this if statement reads data from A. it will only be 49 entered if the following conditions are all true: 50 1. were not in the middle of a write on A 51 2. theres space left in the A to B buffer 52 3. either we need to write to complete a previously blocked 53 read and now A is available to write, or we can read from 54 A regardless of whether were blocking for availability to read. 55 56 if write_waiton_read_A || write_waiton_write_A 57 A2B_len = BUF_SIZE 58 can_read_A || can_write_A read_waiton_write_A 59 { 60 clear the flags since well set them based on the IO 61 calls return 62 63 read_waiton_read_A = 0; 64 read_waiton_write_A = 0; 65 66 read into the buffer after the current position 67 code = SSL_readA, A2B + A2B_len, BUF_SIZE - A2B_len; 68 switch SSL_get_errorA, code 69 { 70 case SSL_ERROR_NONE: 71 no errors occured. update the new length and 72 make sure the have data flag is set. 73 74 A2B_len += code; 75 have_data_A2B = 1; 76 break; 77 case SSL_ERROR_ZERO_RETURN: 78 connection closed 79 goto end; 80 case SSL_ERROR_WANT_READ: 81 we need to retry the read after A is available for 82 reading 83 84 read_waiton_read_A = 1; 85 break; 86 case SSL_ERROR_WANT_WRITE: 87 we need to retry the read after A is available for 88 writing 89 90 read_waiton_write_A = 1; 91 break; 92 default: 93 ERROR 94 goto err; 95 } 96 } 97 98 this if statement is roughly the same as the previous if 99 statement with A and B switched 100 101 if write_waiton_read_B || write_waiton_write_B 102 B2A_len = BUF_SIZE 103 can_read_B || can_write_B read_waiton_write_B 104 { 105 read_waiton_read_B = 0; 106 read_waiton_write_B = 0; 135 107 108 code = SSL_readB, B2A + B2A_len, BUF_SIZE - B2A_len; 109 switch SSL_get_errorB, code 110 { 111 case SSL_ERROR_NONE: 112 B2A_len += code; 113 have_data_B2A = 1; 114 break; 115 case SSL_ERROR_ZERO_RETURN: 116 goto end; 117 case SSL_ERROR_WANT_READ: 118 read_waiton_read_B = 1; 119 break; 120 case SSL_ERROR_WANT_WRITE: 121 read_waiton_write_B = 1; 122 break; 123 default: 124 goto err; 125 } 126 } 127 128 this if statement writes data to A. it will only be entered if 129 the following conditions are all true: 130 1. were not in the middle of a read on A 131 2. theres data in the A to B buffer 132 3. either we need to read to complete a previously blocked write 133 and now A is available to read, or we can write to A 134 regardless of whether were blocking for availability to write 135 136 if read_waiton_write_A || read_waiton_read_A 137 have_data_B2A 138 can_write_A || can_read_A write_waiton_read_A 139 { 140 clear the flags 141 write_waiton_read_A = 0; 142 write_waiton_write_A = 0; 143 144 perform the write from the start of the buffer 145 code = SSL_writeA, B2A, B2A_len; 146 switch SSL_get_errorA, code 147 { 148 case SSL_ERROR_NONE: 149 no error occured. adjust the length of the B to A 150 buffer to be smaller by the number bytes written. If 151 the buffer is empty, set the have data flags to 0, 152 or else, move the data from the middle of the buffer 153 to the front. 154 155 B2A_len -= code; 156 if B2A_len 157 have_data_B2A = 0; 158 else 159 memmoveB2A, B2A + code, B2A_len; 160 break; 161 case SSL_ERROR_ZERO_RETURN: 162 connection closed 163 goto end; 164 case SSL_ERROR_WANT_READ: 165 we need to retry the write after A is available for 166 reading 136 167 168 write_waiton_read_A = 1; 169 break; 170 case SSL_ERROR_WANT_WRITE: 171 we need to retry the write after A is available for 172 writing 173 174 write_waiton_write_A = 1; 175 break; 176 default: 177 ERROR 178 goto err; 179 } 180 } 181 182 this if statement is roughly the same as the previous if 183 statement with A and B switched 184 185 if read_waiton_write_B || read_waiton_read_B 186 have_data_A2B 187 can_write_B || can_read_B write_waiton_read_B 188 { 189 write_waiton_read_B = 0; 190 write_waiton_write_B = 0; 191 192 code = SSL_writeB, A2B, A2B_len; 193 switch SSL_get_errorB, code 194 { 195 case SSL_ERROR_NONE: 196 A2B_len -= code; 197 if A2B_len 198 have_data_A2B = 0; 199 else 200 memmoveA2B, A2B + code, A2B_len; 201 break; 202 case SSL_ERROR_ZERO_RETURN: 203 connection closed 204 goto end; 205 case SSL_ERROR_WANT_READ: 206 write_waiton_read_B = 1; 207 break; 208 case SSL_ERROR_WANT_WRITE: 209 write_waiton_write_B = 1; 210 break; 211 default: 212 ERROR 213 goto err; 214 } 215 } 216 } 217 218 err: 219 if we errored, print then before exiting 220 fprintfstderr, Errors occured\n; 221 ERR_print_errors_fpstderr; 222 end: 223 close down the connections. set them back to blocking to simplify. 224 set_blockingA; 225 set_blockingB; 226 SSL_shutdownA; 227 SSL_shutdownB; 228 } 137 Since the code sample is rather large, well dissect it and explain the reasons behind the implementation decisions. We use two buffers, A2B and B2A , to hold the data read from A to be written to B, and the data read from B to be written to A, respectively. The length variables corresponding to each buffer A2B_len and B2A_len are initialized to zero; throughout our function, they will hold the number of bytes of data in their counterpart buffer. An important observation to make at this point is that we do not use an offset variable for pointing into our buffer; the data we want to write will always be at the front of our buffers. We also declare three sets of flags. The first set have_data_A2B and have_data_B2A indicates whether there is any data in our buffers. We could have left these two variables out, since throughout the function they will be zero only if the corresponding buffer lengths variable is also zero. We opted to use them for code readability. The next set of variables is availability flags, which are of the form can_read_A , can_write_B , etc. These flags are set to indicate that the named operation on the named object can be performed, i.e., the connection is available to perform the operation without needing to block. The last set of flags is the blocking flags. When one of these is set, it tells us that a particular kind of IO operation requires the availability of the connection to perform another kind of IO operation. For instance, if write_waiton_read_B is set, it means that the last write operation performed on B must be retried after B is available to read. We use three functions in this example that are not explicitly defined. The first is set_nonblocking . This function takes an SSL object as its only argument and must be implemented in a platform-specific way to set the underlying IO layer of the SSL object to be non-blocking. Likewise, set_blocking needs to set the connection to a blocking state. The last platform-specific function is check_availability . This functions obligation is to check the IO status of the underlying layers of both A and B and set the variables appropriately. Additionally, this function should wait until at least one variable is set before returning. We cannot perform any IO operations if nothing is available for either connection. These omitted functions can be implemented easily. For instance, on a Unix system with SSL objects based on sockets, set_nonblocking and set_blocking can be implemented using the fcntl system call, and the check_availability function can use fd_set data structures along with the select system call. The calls to SSL_set_mode set two mode variables weve not discussed. Normally, a call to SSL_write will not return success until all of the data in the buffer has been written, even with non-blocking IO. Thus, a call that requests a large amount of data to be written can return many retry requests before succeeding. This behavior is undesirable in our function because we wish to interlace the reading routines with those that write, and to do this effectively, we cannot retry a single call for too long because it will prevent us from reading more data from that connection. To alleviate this problem, the SSL_MODE_ENABLE_PARTIAL_WRITE mode flag instructs the library to allow partially complete writes to count as successes. This allows our function to perform several write operations successfully, and between those calls, we can read more data. Because the SSL protocol is built around sending complete messages on the channel, sometimes well be required to retry a call because only a part of the requested data was actually sent. As a result, all retried operations must be called with the exact same arguments as the original call that caused the error. This behavior can be cumbersome since it does not allow us to add to the write buffer after making a call to SSL_write that needed a retry. To change this property partially, we use the SSL_MODE_ACCEPT_MOVING_WRITE_BUFFER flag. This flags allows us to retry the write operation with different parameters, i.e., a different buffer and size, so long as the original data is still contained in the new buffer. In our example, we use the same buffer, but enable this mode so that we can keep reading into the end of the buffer without causing errors on attempts to retry writes. The function has one main loop, beginning on line 42 and ending on line 216. In this loop, we have four separated subsections, lines 56-96, lines 101-126, lines 136-180 and lines 185-215. Each of these conditional blocks corresponds respectively to reading data from A, reading data from B, 138 writing data to A, and writing data to B. Looking carefully, we can see that the first and second section mirror each other just as the third and fourth do. We must be sure it is safe to read. The entry conditions on the if statements do this. They assert we are not in the middle of a read operation by checking the write blocking flags for the connection write_waiton_read_... || write_waiton_write_... . Additionally, it needs to be sure there is space in the buffers ..._len = BUF_SIZE . In addition to these conditions, we must be sure this is the right time to call the read function, i.e., either were waiting to retry a failed read because it was waiting for the connection to be available for writing can_write_... read_waiton_write_... , or we simply have data to read can_read_... . If all of these conditions are met, we can attempt a read. Before we do, though, we reset the blocking flags for the read operation, and then perform the IO call and check the error code. The IO call itself instructs the SSL library to write the data into the buffer offset by the number of bytes already stored within it. If the call succeeds, well update the length counter and make sure that the have_data_... flag is set. If we are instructed to retry through either SSL_ERROR_WANT_READ or SSL_ERROR_WANT_WRITE , we will set the appropriate blocking flag and carry on. If an error occurs, or the end of the SSL connection is detected, we will break out of the IO loop. As with read operations, write operations are protected by an if statement to assure entry conditions. These statements ensure that we are not in the middle of a read operation read_waiton_write_... || read_waiton_read_... . Additionally, there must be data in the buffer have_data_... . Lastly, the write operation must make sure that writing is possible, i.e., either weve been waiting for read availability to retry a write operation can_read_... write_waiton_read_... , or there is write availability can_write_... . Before attempting the write, we must zero the blocking flags. The write operation always tries to write from the beginning of the buffer. If we are successful, we move the data in the buffer forward so that data already written is pushed out. If the buffer is empty after writing, we zero the have_data_... flag. As with the read operation, if we are instructed to retry, we set the corresponding blocking flag and continue. If errors occur, or if the SSL connection has been closed, we break out of the IO loop. These paradigms for performing reading and writing enable us to do effective non-blocking communication when combined in the loop. For instance, if B cannot be written to, we will continue to buffer data from A until the buffers fill and we are forced to wait for writing on B. We can extend the general form presented in this function to allow for many different non-blocking IO needs that applications may have. 5.2.3 SSL Renegotiations An SSL renegotiation is essentially an SSL handshake during a connection. This causes client credentials to be reevaluated and a new session to be created. Weve discussed the effects of renegotiation on the IO and other aspects of a programs implementation, but havent talked about why renegotiations are important. Since renegotiations cause the creation of a new session, the session key is replaced. For long- lasting SSL connections or for those that transfer a high quantity of data, it is a good idea to replace the session key periodically. In general, the longer a session key exists, the more the likelihood of key compromise increases. Using renegotiations, we can replace the session key so that we dont encrypt too much data with just one key. SSL renegotiations can occur during normal application data transfer. When the new handshake is completed, both parties switch to using the new key. The function to request renegotiation on an SSL connection is SSL_renegotiate . 139 This function does not actually perform the new handshake when called; however, it sets a flag that causes a renegotiation request to be sent to the peer. The request is sent out on the next call to an IO function on the SSL connection. An important point to consider is that the peer has the option of not renegotiating the connection. If the peer chooses not to respond to the request and to continue transferring data instead, no renegotiation will occur; unless the requestor checks for negotiation success, it may not have happened. This is especially important when making applications that will connect with other non-OpenSSL SSL implementations. A renegotiation can also be done explicitly. In other words, we will send a request and not send or receive application data until the new handshake has completed successfully. While this will sometimes be the appropriate way to refresh session keys for long-term connections, it serves another important purpose. It allows us to upgrade client authentication from the server side. For instance, our example server above requires a valid client certificate to be presented in order for the initial connection to take place. This is overly restrictive since at connection time, we do not know the clients intentions. As an example, consider a simple protocol that allows clients to connect via SSL and send commands to the server. A subset of these commands will be reserved for administrators. Our goal is to allow anyone to connect to the server to run the general commands and allow only administrators to run reserved commands. If we require all users to present a certificate at connection time, we must issue a certificate to every possible client and special certificates to administrators. This can become cumbersome rather quickly. Alternatively, we can run two servers, one for normal users and another for administrators. This is also a suboptimal solution since it requires extra consumption of resources. We can use renegotiations to make our job simpler. First, we will allow anyone to connect without a certificate. When a client connects, it sends its command. If its a reserved command, we require a renegotiation with the more stringent requirements on the client in place. If the client renegotiates successfully, we can accept the command, or otherwise discard it. Thus, we need to issue certificates only to administrators. This option is clearly better than the previous two since it allows us to determine what the client is attempting before putting stronger authentication requirements in place.

5.2.3.1 Implementing renegotiations