Anand Iyer, Glenn Smith, Paul Roe, and Joel Pobar
Queensland University of Technology,
Faculty of Information Technology,
Brisbane, Australia
Web services are touted to turn the internet into a general platform for distributed computing as required for B2B eCommerce. Web services are the glue that will link together web deployed components to form web applications. Just as there are issues with traditional component adaptation due to versioning and independent development so there are with web services. In this paper we discuss how interoperability between different SOAP web services may be achieved in a platform independent manner using XML scripts and XSL.
Today the internet is dominated by two applications: web browsers and web servers. Initially these supported the simple presentation, browsing, and searching of HTML documents. This has now matured to the point of being able to do personal banking over the internet. XML (Extensible Markup Language) is now set to revolutionise the internet. XML is a platform neutral language for describing data. This together with proposed standards such as SOAP (Simple Object Access Protocol) will enable web services - services designed not for human, but program consumption. Finally the internet will become a general platform for distributed computing enabling business to business (B2B) eCommerce.
The essential idea of web services is to move beyond HTML and to support arbitrary program to program interaction across the internet. Web services allow a broad range of internet applications to be constructed from web components which offer web service interfaces. These applications range from "classic" B2B applications such as supply chain automation, through to collaborative applications such as telemedicine; some even think that web services might wire our homes [2].
The building of applications from web services is analogous to the building of traditional applications from components: web services are web deployed components. As with traditional components [1] there are certain issues which arise from deploying independently developed web services. In particular web service mismatch problems can arise from versioning and from independent development of services - perhaps utilising different standards. Note here we are not concerned with protocol mis-match issues but higher level interface mismatch. In this paper we show how for a particular example, stock market share trading, we are able to adapt web services to overcome impedance mismatch problems, and thereby achieve business integration across the web. The adaptation is achieved using a platform independent adaptation architecture. An XML script is used to drive multiple XSL (extensible stylesheet language)[HREF1] transformations of web services; this enables the interoperability of different web services. The contribution of this paper is to show an example of how web service adaptation may be achieved through an adaptation architecture; this enables B2B integration.
The remainder of this paper is organised as follows, the next section overviews XML and web services. Section 3 describes an architecture for web service interoperability. Section 4 describes an example of web service adaptation for share trading; the final two sections cover related work and conclusions.
XML is a data description language. It supports the description of semi-structured hierarchical data. XML comprises elements and attributes. Elements can contain text; they may also be composite i.e. contain further elements and attributes. Attributes are atomic and represent atomic data. XML may be typed through XML schemas. These enable guarantees to be made about data attributes and elements being present, in the right number, and having the right form. Thus we may guarantee that, for example, any XML data claiming to represent a person has a name, an address and optionally an email address. XML is often compared with HTML; essentially XML describes content and HTML describes presentation (although not originally intended so). XML must be syntactically well formed and may require conformance to a schema; it is precise; there is no concept of trying to recover from badly formed XML, unlike browsers' interpretation of HTML. The most important attributes of XML are that it has been standardised and that it has widespread adoption.
Web services represent the next generation of the internet. Much simpler than CORBA, Java RMI and DCOM they represent a simple form of messaging or RPC designed for communication between heterogeneous systems across the internet. Typically they operate on top of the HTTP protocol, and are encoded using XML. A popular web service protocol is SOAP - Simple Object Access Protocol[HREF3].
Figure 1: High level adapter architecture
The high level architecture of the adaptor is illustrated in Figure 1. SOAP messages from the client are intercepted by the adapter. The adaptor then applies an XML specification to select appropriate XSL documents for the required transformation based on the incoming SOAP message. Once the adaptor has applied the transformations using the selected XSL documents on the message, it then forwards the resulting message to the destination server identified in the XML specification. The reply from the server may also be transformed before being sent back to the client. Where the adaptor may be located is illustrated in the Figure 2. There are distinct motivations for choosing to locate the adaptor at the client or server, these are discussed in the following sub-sections.
Figure 2: Potential positioning of adapter relative to client and server
The motivation for applying adaptation on the client side arises when the client side administration is aware of a service provided that has similar functionality to what is required, but does not exactly match the required service. However the client would like to use this service. For example a company using web based purchasing would like to be able to access as many providers as possible, even if the purchasing mechanisms are slightly different. This would allow for a wider choice of product.
In this work applying adaptations to SOAP messages on the client side has used SOAP Extensions. SOAP Extensions is a facility provided to manipulate SOAP messages during transit. SOAP Extensions are used here to trap the outgoing SOAP message and provides the opportunity to manipulate the message before the message is passed into the internet. Using SOAP Extensions in this manner allows arbitrary transformations to be applied to the SOAP message. However, it is not possible to redirect the resulting message to a different web server using only SOAP Extensions. To do this the enclosing transport protocol details need to be modified and SOAP Extensions do not provide access to this information. This also restricts the ability to refactor single client calls into multiple calls as the enclosing transport details would need to be replicated to send the additional messages created. Hence, client side adaptation is not considered further for the purpose of this paper.
The motivation for applying adaptation on the server side arises when the server side administration is aware of clients requiring web services which differ from the one provided. However, the server side administration would like to allow as many clients as possible to use their service by presenting it in multiple forms. For example a company that provides a service, such as stockmarket settlements, and charges for clients to use this service would like to enable as many customers as possible to use their service.
To perform server side adaptation the adaptor is provided as a proxy for the requested web service. As such it receives the message in the same way as any other web service, hence there is no need to use SOAP Extensions to trap the SOAP message. In this case the adaptor simply becomes a client of the web services that will be used to produce the required functionality. Hence, the server side adaptor is not restricted in redirecting or refactoring messages in the same way as client side adaptors.
The high level architecture of both client side and server side adaptors are the same, except for the way the message is trapped.
For an adaptation to be successful it must transform a message (intended for the required service) into a message acceptable by the services that are actually available. The transformations applied by the adaptor can range from the null transformation when the required and provided services match, to arbitrarily complex transformation that implements the required service if the provided services do not provide any common functionality. However, the adaptations of interest are those which bridge the gap between required and provided services which have a significant overlap in functionality.
In many cases there is a direct correspondence between the methods of the required service, and the methods of the provided services. In these cases the adaptation provides for incompatibilities such as different method names, different ordering of parameters, different parameter names, and structure of parameters. However, even in simple cases it is unlikely that a one-to-one mapping exists between the methods of the required and provided services. This refactoring of the functionality is performed within the adaptor. This includes the composition of multiple provided methods to achieve the require functionality.
Other simple requirements can make adaptation complex. Refactoring becomes even more complex when the result of one constituent call needs to be used as input to another constituent call. To address this requires the ability to syncronise the constituent calls. What has been achieved in this work is to provide an easy to use framework for creating adapters in the face of these complexities. It is essential that the construction of the adapater is easier than creating a custom coded solution, we believe this has been achieved.
The following section describes an example of web service interoperability achieved using the adaptation architecture previously described.
Current stock market shares trading is predominantly settled electronically. This has lead to the extensive use of HTTP for the transport of messages. From this has evolved some message formats suitable for use in shares trading. Currently the Financial Information eXchange Markup Language (FIXML) [HREF7] and Straight Through Processing Markup Language (STPML) [HREF8] are two of the most popular financial messaging formats. The demonstative example presented in this paper will look at the adaptation of services that use FIXML and STPML messages.
Two web services, based on existing share trading applications, have been defined. There is the web service required by the client, and the web service actually provided. The generally functionality of these web services is to enable the buying and selling of shares, providing transaction histories, listing of executed, canceled, and outstanding orders, and to provide current price quotes. The interfaces to these services are defined as follows.
struct snapquote
{
string securitycode
string securityname
int buyrate
int sellrate
}
enum enordertype (executed, cancelled, outstanding)
struct order
{
string securitycode
string securityname
string type (buy,sell)
enordertype ordertype
int rate
int howmany
}
struct stportfolio
{
string securitycode
int rate
int howmany
}
snapquote[] quote(string securitycode[])
order[] outstanding(int clientid)
order[] executed(int clientid)
order[] cancelled(int clientid)
order[] history(int clientid)
stportfolio[] portfolio(int clientid)
struct snapquote
{
string securitycode
int buyrate
int sellrate
}
struct order
{
string securitycode
string type (buy,sell)
enordertype ordertype
int rate
int howmany
}
snapquote quote(string securitycode)
order[] outstanding(int clientid)
order[] executed(int clientid)
order[] cancelled(int clientid)
stportfolio[] portfolio(int clientid)
The difference between client and server is quite subtle. The server does not support the history method required by the client. It also does not support multiple quote requests within one call. The structures used to pass the messages are also different. The client requires the securityname field in the return value of the snapequote and order methods.
Perhaps the most interesting feature of the adaptation is the parsing of the securitycode array found in the client's quote web service request and the construction of an array result by using multiple calls to the servers actual single quote web service. Also, the notion of the adapter being able to construct new functionality is interesting. This is shown in the client's history request. The web service can use the server's outstanding, executed and cancelled web services to create the history result for the client.
The first action that is taken when a message is received is to determine how it should be transformed, if at all. This task is guided by a specification which is written in XML. The XML specification guides the adapter in the choice of transformations to apply, if any, to incoming and outgoing messages. The extract shown below is from the XML specification for this example. This extract is used for the detection of the quote message.
<ClientInterface SoapAction="http://tempuri.org/quote">
<ServerInterface name="snapquote">
<SoapAction>http://tempuri.org/quote</SoapAction>
<PreTransformation>c:\\work\\ShareTrader\\pretransformation.xsl</PreTransformation>
<PostTransformation>c:\\work\\ShareTrader\\posttransformation.xsl</PostTransformation>
</ServerInterface>
</ClientInterface>
This is only one element of many in the adapter's XML specification. If
message matches the SoapAction named quote then this element
of the specification is invoked. It indicates which transformation should
be applied to the incoming and return message, it does this using the
PreTransformation and PostTransformation keys. A very
similar piece of XML is used for the other methods. However, in this particular
case the transformation is to create multiple calls to the server so that
multiple results can be returned, to do this the XSL documents used
reference a second XSL document to do the transformation of each individual
item. This is explained in the following section.
<xsl:stylesheet xmlns:xsl="http://www.w3.org/1999/XSL/Transform" xmlns:tempuri="http://tempuri.org/" version="1.0">
...
<soap:Envelope ... >
<soap:Body>
<xsl:apply-templates />
</soap:Body>
</soap:Envelope>
...
<xsl:template match="tempuri:string">
<DSCRIPT function="inject">
<url>http://plas2000.fit.qut.edu.au:8080/ShareTradingServer/ShareServer.asmx</url>
<SoapAction>http://tempuri.org/quote</SoapAction>
<PreTransformation>c:\\work\\ShareTrader\\quotepretransformation.xsl</PreTransformation>
<PostTransformation>c:\\work\\ShareTrader\\quoteposttransformation.xsl</PostTransformation>
<input>
<securitycode>
<xsl:value-of select="." />
</securitycode>
</input>
</DSCRIPT>
</xsl:template>
</xsl:stylesheet>
Recall that the client call to the quote method passes an array of security codes, where the quote method provided by the server accepts only a single security code as input. To overcome this differnce the above XSL document matches each one of the security codes passed in the array, and indicates that another XSL document be applied to each securtity code string individually. This allows multiple messages to be generated and passed to the provided quote service. Currently this cannot be done using a single XSL document as the application of a single XSL document can only produce a single document as output.
In this example a simple scripting language is used to effect the multiple applications of the defined XSL document. This is declared by the DSCRIPT key. With the required transformation documents again being specified in the pretransformation and posttransformation keys. The pretransformation document creates the XML message to be sent in the web service request. The posttransformation extracts the result from each individual call, and this result replaces the DSCRIPT element. Further processing of the result is required to generate the securityname element of the return structure. To do this the quoteposttransformation XSL document inserts a further DSCRIPT call to generate another web service request. This request is to a supporting web service which is provide to perform the translation of securitycode to securityname. The end result forms the structure required by the client. The supporting web service in this case was an additional service implemented to do the require code translation. Creating supporting web services in this way allows the adaptor to provide arbitrary computation.
There has been much work in the area of traditional component interoperability and adaptation for example [1,4]. Such work has concentrated on bridges to adapt between different component technologies and the adaptation of traditional components written using CORBA, COM, and Java.
Other more recent work concerns the translation of XML documents for example using XSL. One such example of this occurs in the Microsoft Biztalk server [HREF4,HREF6] which supports the adaptation of XML documents through a graphical language which is translated to XSL. Our work differs from this in focusing on the adaptation of web services rather than simple XML documents. This requires more sophisticated adaptation than is possible in Biztalk.
We have shown how web service adaptation can be achieved through an adaptation architecture. A particular example of interoperability between different stock trading services has been described. This demonstrates the applicability of the adaptation architecture and the necessaity for web service interoperability. The resulting interoperability is reasonably efficient and platform independent, in particular it does not require the programming of custom adapters.
At present our adaptation is functional. We do not take into account issues such as transactions and distributed transaction management. For example if one web service is refactored into two the two resulting calls may need to be made into an atomic transaction. Another important issue is security and the adaptation of security. Adapting one web service to another may require the use of different security mechanisms. Indeed we may need to adapt the actual security mechanisms.
Finally in our current system the construction of composite adaptors is manual. That is composite adaptors must be built from other adaptors by hand. What we would really like is to have a collection of adaptors which a system can dynamically use, perhaps even discover, and automatically build composite adaptors as required.
[1] C.Szyperski. Component Software: Beyond Object-Oriented Programming. Addison-Wesley, 1998
[2] Special Issue on "Embedding the Internet" CACM May 2000
[3] G.Smith, J.Gough, and C.Szyperski. A Case for Meta-Interworking: Projecting CORBA Meta-data into COM. Technology of Object-Oriented Languages and Systems, TOOLS 28. Melbourne, Australia.
[4] G.Smith. Conciliation: Adaptation of independently developed software components. Queensland University of Technology, 1999, Brisbane, Australia
[HREF1] W3C The Extensible Style Sheet Language. http://www.w3.org/Style/XSL/
[HREF2] W3C. Web Services Activity. http://www.w3.org/2002/ws/
[HREF3] W3C. SOAP 1.1. http://www.w3.org/TR/SOAP/
[HREF4] Microsoft. Biztalk. http://www.microsoft.com/biztalk/
[HREF5] Microsoft .NET. http://msdn.microsoft.com/net/
[HREF6] Microsoft. Biztalk Adapters. http://www.microsoft.com/biztalk/evaluation/adapters/adapterslist.asp
[HREF7] Robin Cover. FIXML - A Markup Language for the FIX Application Message Layer http://www.oasis-open.org/cover/fixml.html
[HREF8] Robin Cover. Straight Through Processing Markup Language (STPML) http://www.oasis-open.org/cover/stpml.html