Jin Yu, School of Computer Science and Engineering, University of New South Wales, Sydney, NSW 2052. Email: jyu@cse.unsw.edu.au
Boualem Benatallah, School of Computer Science and Engineering, University of New South Wales, Sydney, NSW 2052, Email: boualem@cse.unsw.edu.au
The Extensible User Interface Protocol (XUP) [8] is a SOAP [4, 5] based protocol for communicating events and incremental user interface updates on the web. It lays a foundation for developing and consuming highly interactive web applications and services. XUP is designed to serve as the UI front end for applications based on the Web Services architecture [2]. XUP may be used with any user interface model or any event model with an XML based representation.
The current generation of web applications is built with HTML pages. This page-based programming model lacks true user interactivity, since HTML was designed explicitly for presenting passive documents and therefore lacks many useful user interface controls. Often extensive JavaScript code is added to improve interactivity, but this results in high development costs due to the lack of standardization of JavaScript support in different web browsers and even in different versions of the same browser.
Furthermore, in HTML-based web applications, web pages are always reloaded after each user click. This is very annoying to end users, since it takes time to reload a web page even if only a very small portion of the user interface needs to be updated. Also, this wastes network resources since a full web page is always downloaded after each user interaction. With the advent of Web Services based applications, it is obvious that a new web UI framework is needed to support Web Services requiring complex user interactions.
The Extensible User Interface Protocol (XUP) is an SOAP-based protocol for communicating events and user interface changes on the web. Its goal is to provide a framework for developing and using highly interactive web applications. In addition, XUP may serve as the UI front end for applications based on the Web Services architecture, and it is inherently cross-language, cross-platform, and cross-device.
XUP provides a model that bridges between the traditional desktop (client / server) based and web page (browser) based user interface paradigms. With XUP, events are delivered from client to server as SOAP messages. Programmers implement event handlers on the server side. They no longer need to process form data as URL-encoded strings or name / value pairs. User interface changes are delivered from server to client as incremental updates, so end users will no longer experience slow page refreshes, and network bandwidth is conserved.
The traditional desktop-based model offers rich and powerful user interface components, but it requires client-side software administration. The web page based model has no client software administration cost, as it has a universal client - the web browser, but it lacks sophisticated user interface components required by highly interactive web applications. XUP provides a foundation for a user interface programming model that combines the advantages of both the desktop and web page based models. XUP offers the benefit of zero client administration cost as well as better, richer, and more powerful user interactivities. The XUP based model complements, rather than replaces, the traditional desktop or web page based models. The desktop model is more appropriate for graphics-intensive applications, such as movie players and video games, and the web page based model is more suitable for information browsing, such as browsing news web sites.
The next section describes the concept of user interface model. Section 3 details the design of the protocol. We illustrate two simple examples in Section 4. And in Section 5, we describe our current prototype implementation. Finally, we outline some related work in Section 6 and discuss conclusion and future work in Section 7.
A user interface model (UI model) is a representation of the user interface which the end user perceives and interacts with. Common UI models include .NET Windows Forms, Java Swing, XUL [HREF14], UIML [1], etc.
Some UI models have XML-based representations, such as XUL and UIML. We call them declarative UI modeling languages. That is, instances of the UI models are declared, not programmed. A declarative UI model is described by a tree of XML elements, with UI components (e.g. panels, buttons) mapping to XML elements and the properties (e.g. color, size) of the components mapping to XML attributes. Throughout this paper, we will use the terms UI element and UI attribute to refer to UI components and their properties. One major advantage of XML-based UI models is that they can be edited with existing XML authoring tools. It is also easy to develop UI designing tools to further simplify XML-based UI model creation.
XHTML and WML can be also regarded as UI modeling languages, although they are more suited for presenting information rather than rich user interfaces.
XUP is a protocol for delivering events and user interface updates. It is independent of the actual UI model. It places no restriction on the UI component set, or the attributes or events associated with each component. XUP can work with any UI models with XML-based representations.
An event model is a representation of events triggered by user interactions with the UI model. The event model defines how events are fired, what types of events are fired, and from which UI elements the events are fired in the UI model. The event model is often part of the UI model.
XUP supports both delegation (e.g. Java Swing) [HREF9] and capturing/bubbling (e.g. DOM) [7] event models, as long as they have XML-based representations. The protocol itself does not specify any event details, such as what types of events may occur on which UI elements, or the syntax and semantics of any particular event.
This section provides an overview of the various elements of XUP and the interactions among them.
An XUP client is a program that sends event requests and processes server responses on behalf of the end user. The client renders user interfaces and interacts with the end user.
An XUP server is a program that processes event requests from the client and sends back UI update responses.
An XUP application is a program that resides in the server and provides desired functionalities to the end user through the client. The application uses an API provided by the server to process events and update UI.
An event handler is part of the XUP application. The server invokes the application by calling registered event handlers upon receiving an event request from the client.
An event listener defines event selection criteria in the client. Not all client side UI events are delivered to the server; only those matched by an event listener are sent.
An XUP client communicates with an XUP server by sending UI events encapsulated in XUP requests. The server then handles the events and returns UI and event listener updates in XUP responses.
An end user interacts with an XUP application by operating a client which sends events to the XUP server hosting the XUP application. XUP applications are developed by programmers to provide desired application functionalities for end users. An XUP server may host one or more XUP applications which may be running concurrently. Similarly, an end user may use a client to interact with more than one application from one or more servers at the same time.
Each XUP application is identified by a URL. The format of the URL depends on a particular server installation. Query strings can be used to identify different XUP applications within a server.
Diagram 1 illustrates XUP message exchanges between a client and a server. The client starts an application by sending a request to the URL identifying the application. This request includes client software information and a list of UI model namespaces supported by the client. The server then responds by sending a session ID used for identifying the client in subsequent request, the initial UI model, and an initial list of event listeners. The server side maintains a user session for each client, identified by a session ID. The initial UI model's namespace must be in the list of UI model namespaces supported by the client. The client interprets this UI model and renders its initial UI accordingly. The event listeners are used by the client to deliver events to the server in subsequent requests.
After the startup phase, the end user interacts with the application by manipulating the UI model, which triggers events to be sent to the server. An event is sent to the server only when it matches with an event listener in the client. Together with the event, a list of UI data is also sent. Those UI data are referenced in event listener specifications in prior server responses. They often reflect changes made by the end user to the UI model, such as text entered in a text box and a check box been unchecked.
Upon receiving an event request from the client, the server locates the XUP application by the request URL and calls the event handlers registered by the application for the event. The UI data sent with the event are also available to the event handlers. Note that there is only one event in each request, but multiple event handlers within an application may be invoked to handle that event. Event handlers execute the necessary application logics and generate UI model updates and event listener updates to be returned to the client. If some errors occurred, a SOAP fault is returned in place of the UI model and event listener updates.
The client processes XUP response by rendering UI changes and updating its list of event listeners. The UI model changes include both UI element level and attribute level changes, and the UI elements could be simple UI components or complex UI containers. This approach allows applications to perform both macro and micro UI updates, eliminating end users' visual discomfort with frequent page refreshes as in HTML-based applications.
XUP does not define a specific mechanism for the client to terminate the session with an XUP application. The client may simply close the transport connection and expect the server or application to purge the user session after some time interval. Alternatively, a particular event on a UI element may be interpreted by the application as a termination notice. For example, a click event on a button labeled “Quit” may be used by the application to terminate the user session.
In XUP, events are triggered by user interactions on the UI model in the client. Once an event is fired, the client sends a request encapsulating the full event detail and its associated UI data to the server for further processing. Server sends back UI changes and listener updates after handling the event.
The client maintains a list of event listeners which are updated by server through XUP responses. An event is sent to the server when a listener matches with the event in the following manner:
Together with the event, a list of UI data is sent, as referenced by the event listeners that match the event. Within each event listener specification, there are a list of UI element and attribute references which indicate the UI data to be sent to the server when an event is fired. Typically, the list of references points to the UI elements or attributes changeable by the end user, such as text boxes and check boxes. If multiple event listeners match an event, the client will send the event to the server only once, with the UI data referenced by the multiple listeners merged together. The client will send the event without any UI data if the matching listener does not reference any UI data. The event will not be sent if there are no listeners matching the event.
After the server receives the event, it calls one or more event handlers registered by the XUP application. The event handlers process the event and perform any necessary computation for the application. For capturing/bubbling event models, the application may also indicate in the response whether to stop the propagation of the event in the client. By default, events propagate in the client according to their types (e.g. some types of events bubble, others do not) defined in the event model.
Because events are dispatched over the network, to conserve network bandwidth, the client does not send certain types of high-frequency events, such as mouse movement. For medium-frequency events, such as key press, we suggest that the XUP application should created event listeners matching a particular key press (e.g. F5) to avoid all keystrokes being sent over the network.
This section illustrates how XUP works by showing two examples. In order to conserve space, we deliberately made both examples very simple, but much richer user interfaces can be built with the same techniques. The examples use XUL as the UI modeling language, with the namespace “http://www.mozilla.org/keymaster/gatekeeper/there.is.only.xul”. We shortened the namespace URL to “http://mozilla.org/...” in the code listings since there is not enough horizontal space. The examples use a simple delegation-based event model, with the namespace URL “http://www.xup.org/event/delegation”. The UI elements in the examples all have XML IDs, which are used by XUP to uniquely identify the UI elements and their attributes. Note that the examples can be replicated in other UI models and event models in a similar fashion.
Figure 1.1 Before button click
In Example 1, the user clicks on “Button1” which has an XML ID of “btn1” (Figure 1.1). The XUP request in Listing 1.1 describes the button click event. The XUP event element specifies a client side UI event.
Listing 1.1 XUP request for Example 1
<env:Envelope xmlns:env="http://www.w3.org/2003/05/soap-envelope" xmlns:xup="http://www.w3.org/2002/03/xup"> <env:Header> <xup:sessionID env:mustUnderstand="1">4124</xup:sessionID> </env:Header> <env:Body> <xup:event type="ev:click" element="btn1" xmlns:ev="http://www.xup.org/event/delegation"> <detail> <ev:mouseClick button="0" clientX="20" clientY="7"/> </detail> </xup:event> </env:Body> </env:Envelope>
In the server response, the XUP application instructs the client to update the text of the label element with ID “lbl1”. This is achieved by using the XUP protocol element updateUIAttr, as in Listing 1.2. The content of the updateUIAttr element specifies the value of the attribute “value” of the label element “lbl1”. Figure 1.2 shows the result after the client processed the response and updated the text of “lbl1” to “Button1 clicked.”
Figure 1.2 After button click
Listing 1.2 XUP response for Example 1
<env:Envelope xmlns:env="http://www.w3.org/2003/05/soap-envelope" xmlns:xup="http://www.w3.org/2002/03/xup"> <env:Body> <xup:uiModel> <updateUIAttr element="lbl1" attr="value">Button1 clicked.</updateUIAttr> </xup:uiModel> </env:Body> </env:Envelope>
In Example 2, the user types in “Chocolate Chip” into the text box with ID “text1” and then clicks the button “Add to list” with XML ID “addBtn” (Figure 2.1). The XUP request in Listing 2.1 describes the button click event. Note that this request includes a piece of UI data: the content of the text box “text1”, which is the string “Chocolate Chip”. This is achieved by using the XUP protocol element uiElement, which describes a single UI element. In our example, the UI element is a simple text box, but it may contain nested children in more complex scenarios.
Figure 2.1 Before clicking “Add to list” button
Listing 2.1 XUP request for Example 2 (button click)
<env:Envelope xmlns:env="http://www.w3.org/2003/05/soap-envelope" xmlns:xup="http://www.w3.org/2002/03/xup"> <env:Header> <xup:sessionID env:mustUnderstand="1">3876</xup:sessionID> </env:Header> <env:Body> <xup:event type="ev:click" element="addBtn" xmlns:ev="http://www.xup.org/event/delegation"> <detail> <ev:mouseClick button="0" clientX="18" clientY="6"/> </detail> <uiElement element="text1"> <xul:textbox xmlns:xul="http://mozilla.org/..." id="text1" value="Chocolate Chip"/> </uiElement> </xup:event> </env:Body> </env:Envelope>
The XUP application processes the event and creates a list item based on the string “Chocolate Chip”, which is the value of the text box “text1”. It then inserts the list item before the second item in the list box, which has an XML ID “list1”. This is achieved by the XUP element addUIElement, as in Listing 2.2. The XUP addUIElement element describes a single UI element to be added to the client UI. In our example, the UI element is a simple list item, but it may contain nested children in more complex scenarios. In addition, the application also clears the text box “text1”, so that the user may input further text. This is accomplished via the XUP updateUIAttr element. Note that in Listing 2.2, the updateUIAttr element is an empty element; this essentially clears out the value of the “value” attribute of the UI element “text1”. The resulting UI is shown in Figure 2.2.
Figure 2.2 After clicking “Add to list” button
Listing 2.2 XUP response for Example 2 (after button click)
<env:Envelope
xmlns:env="http://www.w3.org/2003/05/soap-envelope"
xmlns:xup="http://www.w3.org/2002/03/xup">
<env:Body>
<xup:uiModel>
<!-- insert before the second list item -->
<addUIElement parent="list1" beforeIndex="1">
<xul:listitem xmlns:xul="http://mozilla.org/..."
label="Chocolate Chip"/>
</addUIElement>
<!-- clear the text box -->
<updateUIAttr element="text1" attr="value"/>
</xup:uiModel>
<xup:updateListeners>
<add>
<listener id="listListener1"
event="ev:select" element="list1"
xmlns:ev="http://www.xup.org/event/delegation">
<uiRef>id("list1")/@selectionIndex</uiRef>
</listener>
</add>
</xup:updateListeners>
</env:Body>
</env:Envelope>
In the XUP response in Listing 2.2, the application also adds an event listener “listListener1” to the list box “list1”. That means all list item selection event will be sent to the server. By default, no list selection event is sent to the server, since an XUP client will not send any event unless there is a matching listener. Within the event listener in Listing 2.2, the uiRef element specifies the UI data to be sent to the server when event occurs. The content of the uiRef element is an XPath expression and may point to any UI elements and/or UI attributes. In this case, the expression “id("list1")/@selectionIndex” refers to the “selectionIndex” attribute of the list box “list1”.
Figure 2.3 Selecting list item “Chocolate Chip”
In Figure 2.3, the user clicks the list item “Chocolate Chip”, which triggers the list selection event to be sent to the server. Listing 2.3 describes the XUP event request. Note that the XUP element uiAttr specifies the value of the “selectionIndex” attribute of the list box “list1”. This value (“1”) is the index of selected list item (“Chocolate Chip”).
Listing 2.3 XUP request for Example 2 (selecting list item)
<env:Envelope xmlns:env="http://www.w3.org/2003/05/soap-envelope" xmlns:xup="http://www.w3.org/2002/03/xup"> <env:Header> <xup:sessionID env:mustUnderstand="1">3876</xup:sessionID> </env:Header> <env:Body> <xup:event type="ev:select" element="list1" xmlns:ev="http://www.xup.org/event/delegation"> <!-- the index of the selected list item "Chocolate Chip" --> <uiAttr element="list1" attr="selectionIndex">1</uiAttr> </xup:event> </env:Body> </env:Envelope>
It should be noted that XUP is ideal for applications with complex user interactions involving many UI controls and screens, such as ERP and CRM products. However, we can only show the above simple examples since descriptions for more complicated UI scenarios will not fit into the length of this paper. We hope from these simple examples, it should be obvious to see the advantage of XUP over the current web page (browser) based UI framework.
We have implemented a proof-of-concept prototype which supports a limited subset of UI controls in XUL. The client is a standalone Windows application built with .NET Windows Forms, and the server toolkit runs in IIS (ASP.NET Web Services).
All of user's application code resides on the server side. Our server toolkit provides two sets of .NET APIs: one for XUP messages, and the other for XUL UI controls. Of course, the protocol (XUP) API the UI model (XUL) API are independent of each other, so we could add different UI model APIs in the future. Since the APIs are in .NET, application developers can use any .NET languages, such as C#, VB, C++, etc.
During runtime, both the client and server maintains a copy of the UI model tree (for each end user) in the memory, and XUP messages are passed between the client and server to synchronized their UI trees. An application developer uses the server side APIs to handle events from the client and make modifications to the UI tree; in turn, those UI modifications will be passed back to the client for rendering. The data structure representing the UI model tree is very compact, so we have not seen any memory-related scalability issues at the server side.
Our current implementation is about 20,000 lines of code. About three quarters of that is in the client, since UI rendering and layout computation are all done at the client side.
XUL (Extensible User Interface Language) is an XML-based UI modeling language. It was originally designed to model Mozilla's menus and windows. Later more UI components were added to provide more functionality. Its major drawback is the lack of standardization; there is no single specification that clearly describes the language. XUL may serve as an UI model for XUP, as shown in the examples in the previous section.
UIML (User Interface Markup Language) is also an XML-based UI modeling language. Its goal is to describe device independent user interfaces and events. It may also serve as an UI model for XUP.
RemoteJFC [6] is a distributed user interface toolkit based on the JFC (Java Swing) API. It has two components: the client is a viewer with Java Swing look and feel; the server contains application code, which uses an API that resembles JFC. The client side does not contain any application logic. Through the server API, the application code processes UI events from the client and sends UI updates to the client.
RemoteJFC uses RMI for network communication, which is Sun proprietary and firewall unfriendly. In addition, it requires two-way RMI servers. When the client sends UI events to the server, the server behaves as a RMI server; when the server sends UI updates to the client, the client also becomes an RMI server. This architecture poses serious security problems.
XWeb [HREF15] is a system for supporting multi-modal user interactions on the web. It supports multiple modalities (speech, gesture, etc.) as well as multiple devices. However, its protocol, XWeb Transfer Protocol, modifies HTTP. This makes XWeb incompatible with common HTTP servers such as IIS and Apache and therefore severely limits XWeb's usefulness.
Droplets [HREF8] is a thin client based application development framework. Similar to RemoteJFC, the client is a viewer with no application logic, and the server side contains all the application code. Droplets uses a proprietary communication protocol called User Event Protocol (UEP). Its UI model is also proprietary. Currently the Droplets API only supports Java and C++.
Comparing to RemoteJFC, XWeb, and Droplets, XUP uses a standard protocol (HTTP/SOAP), and has no language, platform, or UI model limitation. XUP may use any XML-based UI model and is inherently cross-language, cross-platform, and cross-device.
ASP.NET Web Forms [HREF6] provides an event-driven programming model to page-based applications. It is one step toward XUP's approach. However, it is still limited by the UI controls offered by HTML / browser. Moreover, in every protocol response, a full page is always returned.
JavaServer Faces (JSF) [HREF10] offers a Java-based event-driven API for developing web applications. Its default HTML / JSP based implementation is very similar to ASP.NET Web Forms, and therefore suffering similar limitations. However, JavaServer Faces only standardizes on the API, not the UI model or the underlining protocol. Therefore, XUP may become an ideal protocol implementation for JSF. Together with an UI model such as XUL, JSF and XUP will provide an ideal framework for developing and consuming highly interactive web applications.
There are many mature remote display technologies, such as X11 [HREF13], NeWS [3], DPS [HREF7], VNC [HREF12], RDP [HREF11], etc. Comparing to XUP, these are lower level UI protocols, transporting graphics and low-level controls. For example, in X11, everything is “window” based; the protocol doesn't offer higher level abstractions such as buttons or scrollbars. Furthermore, these protocols have very high network bandwidth requirement, since typically every keystroke and mouse movement is transported over the network. Therefore, these protocols only work well in a local area network environment, where network resource is plenty. XUP works with high-level UI controls, and it filters out high-frequency events such as mouse movements. Therefore, XUP is ideal for developing interactive web-based applications in local and wide area networks.
XUP leverages SOAP as the transport protocol; this makes it open and interoperable. In addition, the design of XUP makes it an ideal UI front end for any backend infrastructure, including Web Services, EJB, COM, CORBA, etc.
XUP provides a model that bridges between the traditional desktop (client / server) based and the current web page (browser) based user interface paradigms. It combines the advantages of both UI paradigms: rich and powerful UI components (from the desktop paradigm) and zero client administration (from the web page based paradigm).
With XUP, end users will no longer experience frequent “page refreshes” as commonly seen in today's web applications. Users shall perceive much faster response time since UI updates are delivered to them incrementally, rather than one full page at a time. Furthermore, since XUP provides much richer UI components, the users will have a more pleasant experience when using highly interactive web applications with complex user interfaces.
For developers, XUP offers a much more efficient UI development environment, achieving faster time to market for their products. With XUP, developers no longer need to process unstructured form data and regenerate web pages for each user interaction, and they no longer need to constantly tweak HTML and JavaScript to support complex user interfaces. This allows them to rapidly develop web applications with rich user interactions and complex UI components.
Our current proof-of-concept prototype uses XUL as the presentation language. However, due to its lack of standardization, we will be working on a new XML-based UI language, Simple User Interface Language (SUL). In addition, we will also be working on more sophisticated example applications in order to collect performance statistics (comparing with the same applications implemented in HTML browser environment).
Due to the nature of HTTP/SOAP request/response model, it is hard for the server to send asynchronous notification to the client. This facility would be needed when reporting server-side events and the status of long time server side operations. Therefore, we are investigating several different approaches to solve this problem. One possible approach is for the client to send periodical status requests to the server. This is fine for reporting progress of long running jobs, but insufficient for reporting server-side events, such as device events.
Finally, we are planning to implement a server toolkit that runs in any Java Servlet containers, therefore providing an XUP Java API. We are also exploring the possibility of porting the client to the Pocket PC platform, which will enable rich user interactions in mobile devices. Of course, all the client implementations will be fully interoperable with all the server toolkit implementations.
[1] Abrams, M., Phanouriou, C., Batongbacal, A.L., Williams, S.M., and Shuster, J.E. UIML: An Appliance-Independent XML User Interface Language. In Proceedings of the Eighth International World Wide Web Conference (Toronto, Canada, May 11-14, 1999).
[2] Booth, D. et al. Web Services Architecture. W3C Working Draft, August 8, 2003. Available online [HREF1].
[3] Gosling, J., Rosenthal, D.S.H., Arden, M.J. NeWS Book: An Introduction to the Network / Extensible Window System. Springer Verlag, March 1990.
[4] Gudgin, M., Hadley, M., Mendelsohn, N., Moreau, J.J., and Nielsen, H.F. SOAP Version 1.2 Part 1: Messaging Framework. W3C Recommendation, June 24, 2003. Available online [HREF2].
[5] Gudgin, M., Hadley, M., Mendelsohn, N., Moreau, J.J., and Nielsen, H.F. SOAP Version 1.2 Part 2: Adjuncts. W3C Recommendation, June 24, 2003. Available online [HREF3].
[6] Lok, S., Feiner, S.K., Chiong, W.M., and Hirsch, Y.J. A Graphical User Interface Toolkit Approach to Thin-Client Computing. In Proceedings of the Eleventh International World Wide Web Conference ( Honolulu, Hawaii, May 7-11, 2002).
[7] Pixley, T. Document Object Model (DOM) Level 2 Events Specification. W3C Recommendation, November 13, 2000. Available online [HREF4].
[8] Yu, J. and Chen, J. XUP - Extensible User Interface Protocol. W3C Note, May 28, 2002. Available online [HREF5].