SQL Server Service Broker

30.12 SQL Server Service Broker

Service Broker helps developers create loosely coupled distributed applications by providing support for queued, reliable messaging in SQL S erver. Many database applications use asynchronous processing to improve scalability and response times for interactive sessions. One common approach to asynchronous processing is to use work tables. Instead of performing all of the work for a business process in a single database transaction, an application makes a change indicating that outstanding work is present and then inserts a record of the work to be performed into a work table. As resources permit, the application processes the work table and completes the business process. Service Broker is a part of the database server that directly supports this approach for application development. The Transact- SQL language includes DDL and DML statements for Service Broker. In addition, SQL Server Management Objects (SMO) for Service Broker are provided in SQL S erver. These allow programmatic access to Service Broker objects from managed code.

Previous message-queuing technologies concentrated on individual mes- sages. With Service Broker, the basic unit of communication is the conversation —

a persistent, reliable, full-duplex stream of messages. SQL S erver guarantees that the messages within a conversation are delivered to an application exactly once, in order. It is also possible to assign a priority from 1 to 10 to a conversation. Mes- sages from conversations with higher priority are sent and received faster than messages from conversations with a lower priority. Conversations occur between two services. A service is a named endpoint for a conversation. Each conversation

1262 Chapter 30 Microsoft SQL Server

is part of a conversation group. Related conversations can be associated with the same conversation group.

Messages are strongly typed, i.e., each message has a specific type. SQL S erver can optionally validate that messages are well-formed XML , that messages are empty, or that messages conform to an XML schema. A contract defines the mes- sage types that are allowable for a conversation, and which participant in the conversation may send messages of that type. SQL S erver provides a default con- tract and message type for applications that only need a reliable stream.

SQL S erver stores messages in internal tables. These tables are not directly accessible; instead, SQL S erver exposes queues as views of those internal tables. Applications receive messages from a queue. A receive operation returns one or more messages from the same conversation group. By controlling access to the un-

derlying table, SQL S erver can efficiently enforce message ordering, correlation of related messages, and locking. Because queues are internal tables, queues require no special treatment for backup, restore, failover, or database mirroring. Both application tables and the associated, queued messages are backed up, restored, and failed-over with the database. Broker conversations that exist in mirrored databases continue where they left off when the mirrored failover is complete — even if the conversation was between two services that live in separate databases.

The locking granularity for Service Broker operations is the conversation group rather than a specific conversation or individual messages. By enforcing locking on the conversation group, Service Broker automatically helps applica- tions avoid concurrency issues while processing messages. When a queue con- tains multiple conversations, SQL S erver guarantees that only one queue reader at a time can process messages that belong to a given conversation group. This eliminates the need for the application itself to include deadlock-avoidance logic — a common source of errors in many messaging applications. Another nice side effect of this locking semantic is that applications may choose to use the conversation group as a key for storing and retrieving application state. These programming-model benefits are just two examples of the advantages that derive from the decision to formalize the conversation as the communication primitive versus the atomic message primitive found in traditional message-queuing sys- tems.

SQL S erver can automatically activate stored procedures when a queue con- tains messages to be processed. To scale the number of running stored procedures to the incoming traffic, the activation logic monitors the queue to see if there is useful work for another queue reader. SQL S erver considers both the rate at which existing readers receive messages and the number of conversation groups avail- able to decide when to start another queue reader. The stored procedure to be activated, the security context of the stored procedure, and the maximum number of instances to be started are configured for an individual queue. SQL S erver also provides an External Activator. This feature allows an application outside of SQL S erver to be activated when new messages are inserted into a queue. The applica- tion can then receive and process the messages. By doing this, CPU -intensive work can be offloaded out of SQL S erver to an application, possibly in a different com- puter. Also, long-duration tasks, e.g., invoking a Web service, can be executed

30.13 Business Intelligence 1263

without tying up database resources. The External Activator follows the same logic as internal activation, and can be configured to activate multiple instances of an application when messages accumulate in a queue.

As a logical extension to asynchronous messaging within the instance, Service Broker also provides reliable messaging between SQL S erver instances to allow de- velopers to easily build distributed applications. Conversations can occur within

a single instance of SQL S erver or between two instances of SQL S erver. Local and remote conversations use the same programming model. Security and routing are configured declaratively, without requiring changes to the queue readers. SQL S erver uses routes to map a service name to the network address of the other participant in the conversation. SQL S erver can also perform message forwarding and simple load balancing for conversations. SQL S erver provides reliable, exactly once in-order delivery regardless of the number of instances that a message travels through. A conversation that spans instances of SQL S erver can be secured both at the networking level (point to point) and at the conversation level (end to end). When end-to-end security is used, the contents of the message remain encrypted until the message reaches the final destination, while the headers are available to each SQL S erver instance that the message travels through. Standard SQL S erver permissions apply within an instance. Encryption occurs when messages leave an instance.

SQL S erver uses a binary protocol for sending messages between instances. The protocol fragments large messages and permits interleaved fragments from multiple messages. Fragmentation allows SQL S erver to quickly transmit smaller messages even in cases where a large message is in the process of being trans- mitted. The binary protocol does not use distributed transactions or two-phase commit. Instead, the protocol requires that a recipient acknowledge message fragments. SQL S erver simply retries message fragments periodically until the fragment is acknowledged by the recipient. Acknowledgments are most often included as part of the headers of a return message, although dedicated return messages are used if no return message is available.

SQL S erver includes a command line diagnostics tool (ssbdiagnose) to help analyze a Service Broker deployment and investigate problems. The tool can run in either configuration or runtime mode. In configuration mode, the tool checks

whether a pair of services can exchange messages and returns any configuration errors. Examples of these errors are disabled queues and missing return routes. In the second mode, the tool connects to two or more SQL S erver instances and monitors SQL Profiler events to discover Service Broker problems at runtime. The tool output can be sent into a file for automated processing.