05-Middleware Web Services.pdf

  Lecture 2: Middleware Cesare Pautasso Computer Science Department Swiss Federal Institute of Technology (ETHZ) pautasso@inf.ethz.ch http://www.iks.inf.ethz.ch/

  Contents – Lecture 2 - Middleware †

  Understanding middleware ƒ Middleware as a programming abstraction ƒ Middleware as infrastructure for integration

  †

  A quick overview of conventional middleware platforms ƒ RPC ƒ TP Monitors ƒ Object brokers

  †

  Middleware convergence Middleware: Programming Abstractions Integration Infrastructure

  Programming abstractions †

  Programming languages and almost any form of software system evolve always towards higher levels of abstraction ƒ hiding hardware and platform details ƒ more powerful primitives and interfaces ƒ leaving difficult task to intermediaries (compilers, optimizers, automatic load balancing, automatic data partitioning and allocation, etc.)

  ƒ reducing the number of programming errors ƒ reducing the development and maintenance cost of the applications developed by facilitating their portability

  †

  Middleware is primarily a set of programming abstractions developed to facilitate the development of complex distributed systems ƒ to understand a middleware platform one needs to understand its programming model ƒ from the programming model the limitations, general performance, and applicability of a given type of middleware can be determined in a first approximation

  ƒ the underlying programming model also determines how the platform will evolve and fare when new technologies appear

  ©IKS, ETH Zürich.

  5 The genealogy of middleware Remote Procedure Call sockets TCP, UDP Internet Protocol (IP) Remote Procedure Call: hides communication details behind a procedure call and helps bridge heterogeneous platforms sockets: operating system level interface to the underlying communication protocols TCP, UDP: User Datagram Protocol (UDP) transports data packets without guarantees Transmission Control Protocol (TCP) verifies correct delivery of data streams Internet Protocol (IP): moves a packet of data from one node to another Transactional RPC Object oriented RPC (RMI) Asynchronous messaging TP-Monitors Object brokers Message brokers Application servers Specialized forms of RPC, typically with additional functionality or properties but almost always running on RPC platforms And Web Services? And Java?

  †

  Programming abstractions are a key part of middleware but there’s more: ƒ a programming abstraction requires a good supporting infrastructure

  (i.e., a good implementation and support system underneath)

  †

  Programming abstractions, in fact, appear in many cases as a consequence of changes in the underlying hardware or the nature of the systems being integrated

  †

  Java is a programming language that abstracts the underlying hardware: programmers see only the Java Virtual Machine regardless of what hardware platform they use

  ƒ code portability (not the same as code mobility) ƒ the first step towards standardizing middleware abstractions (since now the can be based on a virtual platform everybody agrees upon)

  †

  Web services apply existing abstractions to a specialized kind of network: the Internet.

  ƒ The Simple Object Access Protocol (SOAP) of Web services can be seen as a form of RPC wrapped in XML and mapped to HTTP for easy transport through the Internet Middleware as infrastructure

DCE

client process server process

development

environment

client server

  

IDL

code code

  

IDL

sources

language specific language specific

call interface call interface

client stub server stub

  

IDL compiler

RPC API

RPC API

  

interface

RPC run time RPC run time

headers

service library service library

  

RPC security cell distributed thread

protocols service service file service service

DCE runtime environment ©IKS, ETH Zürich.

  7 Infrastructure for integration †

  As the programming abstractions reach higher and higher levels, the underlying infrastructure implementing the abstractions must grow accordingly

  ƒ Additional functionality is almost always implemented through additional software layers ƒ The additional software layers increase the size and complexity of the infrastructure necessary to use the new abstractions

  †

  The infrastructure is also intended to support additional functionality that makes development, maintenance, and monitoring easier and less costly

  ƒ RPC => transactional RPC => logging, recovery, advanced transaction models, language primitives for transactional demarcation, transactional file system, etc. ƒ The infrastructure is also there to take care of all the non- functional properties typically ignored by data models, programming models, and programming languages: performance, availability, recovery, instrumentation, maintenance, resource management, etc.

INTEGRATION INFRASTRUCTURE

  Trend is towards service oriented architectures at a global scale and standardization of interfaces

  Middleware Examples

  To understand middleware, one needs to understand its dual role as programming abstraction and as infrastructure for integration

  Evolution is towards integration of platforms and flexibility in the configuration (plus autonomic behavior)

  †

  Another important trend is towards single vendor software stacks to minimize complexity and streamline interaction

  †

  ©IKS, ETH Zürich.

  9 Understanding middleware PROGRAMMING ABSTRACTION †

  Intended to provide a comprehensive platform for developing and running complex distributed systems

  †

  Evolution and appearance to the programmer is dictated by the trends in programming languages (RPC and C, CORBA and C++, RMI and Java, Web services and XML)

  †

  Trend is towards increasingly more powerful primitives that, without changing the basic concept of RPC, have additional properties or allow more flexibility in the use of the concept

  †

  Intended to hide low level details of hardware, networks, and distribution

  † Basic middleware: RPC †

  One cannot expect the

  CLIENT Client process

  programmer to implement a

  call to remote procedure

  complete infrastructure for every distributed application. Instead, one can use an RPC system (our

  CLIENT stub procedure

  first example of low level

  Bind

  middleware)

  Marshalling †

  Communication

  What does an RPC system do? Send

  module

  ƒ Hides distribution behind procedure calls ƒ Provides an interface definition language (IDL) to

  Communication

  describe the services SERVER stub procedure

  module Unmarshalling

  ƒ Generates all the additional

  Return

  code necessary to make a procedure call remote and to Dispatcher deal with all the

  (select

  communication aspects stub)

  SERVER

  ƒ Provides a binder in case it

  remote procedure Server process

  has a distributed name and directory service system

  ©IKS, ETH Zürich.

  11 INVENTORY What can go wrong here?

CONTROL CLIENT

  †

  RPC is a point to point protocol in the sense

  Lookup_product

  that it supports the interaction between

  Check_inventory

  two entities: the client and the server

  IF supplies_low †

  THEN

  When there are more entities interacting

  Place_order

  with each other (a client with two servers,

  Update_inventory

  a client with a server and the server with ... a database), RPC treats the calls as independent of each other.

  However, the calls are not

  Server 2 (products) Server 3 (inventory)

  independent

  New_product Place_order †

  Recovering from partial system

  Lookup_product Cancel_order

  failures is very complex. For

  Delete_product Update_inventory

  instance, the order was placed but

  Update_product Check_inventory

  the inventory was not updated, or payment was made but the order was not recorded …

  S S Inventory

  †

  Avoiding these problems using M M

  Products B B and order

  plain RPC systems is very

  D database D database

  cumbersome Transactional RPC

† † Simplifying things quite a bit, one can

  The solution to this limitation is to

  say that, historically, TP-Monitors are

  make RPC calls transactional, that

  RPC based systems with transactional

  is, instead of providing plain RPC,

  support. We have already seen an

  the system should provide TRPC

  example of this: Encina †

  What is TRPC? ƒ same concept as RPC plus …

  Distributed Applications

  ƒ additional language constructs and run time support (additional services) to bundle several RPC calls into an atomic

  Encina Monitor

  unit ƒ usually, it also includes an

  Structured Peer to Reliable

  interface to databases for

  File Peer Queuing

  making end-to-end

  Service Comm Service

  transactions using the XA standard (implementing 2 Phase Commit)

  Encina Toolkit Encina

  ƒ and anything else the vendor may find useful (transactional callbacks, high level locking, etc.)

  OSF DCE ©IKS, ETH Zürich.

  13 TP-Monitors

†

  The design cycle with a TP-Monitor is

  INVENTORY CONTROL very similar to that of RPC:

  IF supplies_low ƒ define the services to implement THEN and describe them in IDL

  BOT ƒ specify which services are Place_order transactional Update_inventory

  ƒ use an IDL compiler to generate EOT the client and server stubs

  

†

Execution requires a bit more control Server 2 (products) Server 3 (inventory) since now interaction is no longer point to point:

  New_product Place_order

Lookup_product Cancel_order ƒ transactional services maintain

context information and call Delete_product Update_inventory records in order to guarantee Update_product Check_inventory atomicity ƒ stubs also need to support more information like transaction id

  S S Inventory

  M M Products and call context

  B B and order D database D

†

  Complex call hierarchies are typically database implemented with a TP-Monitor and TP-Monitor Example Interfaces to user defined services Programs implementing the services

  Yearly balance ? Monthly average revenue ? Control (load balancing, TP-Monitor

  Front end cc and rec., replication, environment distribution, scheduling, priorities, monitoring …)

  1’

  2

  1 r r r recoverable ve ve ve _{ram ram er ram ram er} queue er s s s user user user user p p p _{prog prog p prog prog p} p a a a app server 3 wrappers

  Branch 1 Branch 2 Finance Dept.

  ©IKS, ETH Zürich.

  15 TP-Heavy vs. TP-Light = 2 tier vs. 3 tier † †

  

A TP-heavy monitor provides: A TP-Light is a database extension:

ƒ a full development environment ƒ it is implemented as threads,

(programming tools, services, instead of processes, libraries, etc.),

  ƒ it is based on stored procedures ƒ additional services (persistent ("methods" stored in the queues, communication tools, database that perform an specific

transactional services, priority set of operations) and triggers,

scheduling, buffering), ƒ it does not provide a

  ƒ support for authentication (of development environment. users and access rights to

  

†

Light Monitors are appearing as different services), databases become more ƒ its own solutions for sophisticated and provide more communication, replication, load services, such as integrating part of balancing, storage management the functionality of a TP-Monitor ... (similar to an operating within the database. system).

  

†

Instead of writing a complex query,

  † Its main purpose is to provide an the query is implemented as a stored execution environment for resource procedure. A client, instead of managers (applications), with running the query, invokes the stored guaranteed reasonable performance procedure.

  † This is the traditional monitor: CICS,

  

†

Stored procedure languages: Sybase's Encina, Tuxedo.

  ©IKS, ETH Zürich.

  17 Databases and the 2 tier approach †

  Databases are traditionally used to manage data.

  † However, simply managing data is not an end in itself. One manages data because it has some concrete application logic in mind. This is often forgotten when considering databases.

  † But if the application logic is what matters, why not move the application logic into the database? These is what many vendors are advocating. By doing this, they propose a 2 tier model with the database providing the tools necessary to implement complex application logic.

  † These tools include triggers, replication, stored procedures, queuing systems, standard access interfaces (ODBC, JDBC). user defined application logic database resource manager external application Database developing environment client database management system

  The Web as software layer (N-tier) Branch 1 Branch 2 ^{ap p s er v er 1 ap p s er v er 1 ’ wrappers}

  Front end _{u ser} progra ^{m u ser progra m u ser progra m u ser progra m}

  MIDDLEWARE Web Server Browser

  

†

  N-tier architectures result from connecting several three tier systems to each other and/or by adding an additional layer to allow clients to access the system through a Web server

  

†

  The Web layer was initially external to the system (a true additional layer); today, it is slowly being incorporated into a presentation layer that resides on the server side (part of the middleware infrastructure in a three tier system, or part of the server directly in a two tier system)

  

†

  The addition of the Web layer led to the notion of “application servers”, which was used to refer to middleware platforms supporting access through the Web CORBA †

  The Common Object Request

  Client Server

  Broker Architecture (CORBA) is part

  (CORBA (CORBA

  of the Object Management

  object) object)

  Architecture (OMA) standard, a reference architecture for

  client server

  component based systems

  interface stub stub to remote calls

  †

  The key parts of CORBA are: (proxy) (skeleton) ƒ Object Request Broker (ORB): in charge of the interaction

  CORBA CORBA

  between components

  Marshalling Basic library Object serialization

  ƒ CORBA services: standard

  Adaptor

  definitions of system services ƒ A standardized IDL language for the publication of interfaces Object Request Broker (ORB)

  ƒ Protocols for allowing ORBs to talk to each other

  †

  CORBA was an attempt to

  CORBA

  modernize RPC by making it object

  services

  oriented and providing a standard ©IKS, ETH Zürich.

  19 CORBA follows the RPC model † † Development is similar to RPC:

  CORBA follows the same model as RPC :

  ƒ define the services provided by the server using IDL ƒ they are trying to solve the

  (define the server object) same problem ƒ compile the definition using

  ƒ CORBA is often implemented an IDL compiler. This produces on top of RPC the client stub (proxy, server

  †

  Unlike RPC, however, CORBA proxy, proxy object) and the proposes a complete architecture server stub (skeleton). The and identifies parts of the system method signatures (services to much more detail than RPC that can be invoked) are ever did (RPC is an inter-process stored in an interface repository communication mechanism,

  CORBA is a reference architecture ƒ Program the client and link it with its stub that includes an inter-process

  ƒ Program the server and link it communication mechanism) with its stub

  †

  CORBA standardized component

  †

  Unlike in RPC, the stubs make based architectures but many of client and server independent of the concepts behind were already the operating system and in place long before programming language Objects everywhere: IIOP and GIOP †

  In order for ORBs to be a truly universal

  Client

  component architecture, there has to be

  Server (CORBA

  a way to allow ORBs to communicate (CORBA

  object)

  with each other (one cannot have all object) components in the world under a single ORB)

  ORB 1 ORB 2 †

  For this purpose, CORBA provides a General Inter-ORB Protocol (GIOP) that specifies how to forward calls from one

GIOP GIOP

  ORB to another and get the requests back

  †

  The Internet Inter-ORB Protocol (IIOP) specifies how GIOP messages are translated into TCP/IP

  IIOP

  IIOP †

  There are additional protocols to allow ORBs to communicate with other systems

  †

  The idea was sound but came too late and

  Internet (TCP/IP)

  was soon superseded by Web services ©IKS, ETH Zürich.

  21 The best of two worlds: Object Monitors

  Middleware technology should be interpreted as different stages of evolution of an “ideal” system. Current systems do not compete with each other per se, they complement each other. The competition arises as the underlying infrastructures converge towards a single platform:

  † OBJECT REQUEST BROKERS (ORBs):

  Reuse and distribution of components via a standard, object oriented interface and a number of services that add semantics to the interaction between components.

  †

TRANSACTION PROCESSING MONITORS:

  An environment to develop components capable of interacting transactionally and the tools necessary to maintain transactional consistency

  

What about Object Transaction Monitors?

Object Monitor = ORB + TP-Monitor Middleware Convergence Conventional middleware today

  †

  RPC and the model behind RPC are at the core of any middleware platform, even those using asynchronous interaction

  †

  RPC, however, has become part of the low level infrastructure and it is rarely used directly by application developers

  †

  TP-Monitors are still as important as they have been in the past decades but they have become components in larger systems and hidden behind additional layers intended for enterprise application integration and Web services. Like RPC, the functionality of TP- Monitors is starting to migrate to the low levels of the infrastructure and becoming invisible to the developer

  †

  CORBA is being replaced by other standards although its ideas are still being used and copied in new systems. CORBA suffered from three developments that changed the technology landscape: the quick adoption of Java and the Java Virtual Machine, the Internet and the emergence of the Web, the appearance of J2EE and related technologies to an almost de-facto standard for middleware

  ©IKS, ETH Zürich.

  Property

  †

  Trading/Discovery

  †

  Transaction

  CORBA services and facilities http://www.omg.org/technology/documents/spec_catalog.htm

  †

  Query

  †

  Relationship

  †

  †

  †

  Notification

  †

  Event Domains

  †

  Internationalization

  †

  Mobile Agent Facility

  †

  Persistent State

  †

  Structuring

  Security

  Naming

  25 † In practice, one always needs more than one type of middleware.

  Concurrency

  The question is what is offered by each product.

  †

  Existing systems implement a great deal of overlapping functionality: what in CORBA are called services and facilities

  †

  Because of this overlapping functionality, there are many possible combinations. Some of them work, some don’t. In many cases the focus is on the overlapping functionality, not on the key aspects of a system.

  Middleware Convergence RPC Name services repository

  App. wrappers platform support

Runtime

engine

  †

  Collection

  †

  †

  †

  Time

  †

  Event

  †

  Externalization

  †

  Licencing

  †

  Life Cycle

  †

  Logging

  MOM TP M on RPC J2EE Interchangeable Functionality

WF WF

  App. wrappers App. wrappers

runtime runtime

engine engine

engine platform engine platform CORBA TP-Monitor support support Name Name

  RPC repository RPC repository services services TP

  App. wrappers App. wrappers

runtime runtime

CORBA

engine engine

monitor platform platform CORBA TP-Monitor support support Name Name

  RPC repository RPC repository services services †

  Warning: the fact that all of these combinations are possible does not mean that they all make sense

  †

  In an integrated environment, this functionality should be incorporated not by plugging heavy, stand-alone components but by designing a coherent system from the beginning. This is not always feasible nowadays.

  ©IKS, ETH Zürich.

  27 System design nowadays s er pp ra w rm t fo runtime p. itory or at App. wrappers rappers

  Ap pl y pp

engine

or support su platform

e platform

sit repos pp. w im po

  Name

RPC A

nt ne re support ru gi services en repository e s m e ice

  Na engine rv runtime

  Nam se

  App. wrappers services runtime platform

engine

  C support

  RP Name repository

  RPC services RPC

  ©IKS, ETH Zürich.

  29 “Ideal” Middleware System for EAI

  process management data management message management object management transaction management