Issues in Supporting Real-Time Retrieval, Transmission and Presentation of Multiple Media Streams in the WWW

Guojun Lu, GSCIT , Monash University, Churchill 3842, Australia.
Phone +61 3 99026857 Fax: +61 3 99026842 Email: guojunl@fcit.monash.edu.au Home Page: Guojun Lu
Keywords: WorldWideWeb Extension, Multimedia Communication

Introduction

The current World Wide Web (WWW) cannot support real-time retrieval, transmission and presentation of multiple media streams. To provide this support, all major components of the WWW - HTML, HTTP, servers, browsers - need to be extended and/or redesigned. In this paper we discuss research issues in extending and redesigning these components.

Limitations of Current WWW

A multimedia document may contain multiple media types. Some of these media are static such as text, graphics and image. Some are dynamic or time continuous, such as audio and video. Media in a multimedia document have specific temporal and spatial relationships which must be maintained during presentation in order to achieve desired effects. For organizational and storage efficiency reasons, these media data may be stored in different servers on the Internet. It is desirable that when a multimedia document is requested, all media are retrieved from appropriate servers, transmitted and presented synchronously (according to the temporal and spatial specifications) to the user after a short latency.

The World Wide Web is a networked multimedia system based on the client and server paradigm. Currently it has the following limitations.

  1. Audio and video cannot be displayed on-the-fly, leading to long delay and large storage requirements.
  2. It is not possible to retrieve and display multiple media simultaneously and synchronously (except text with inline image). So what users get is either pictures without sound or sound without pictures.
  3. The external viewer plays back video at a rate determined by the workload and CPU power of the client machine, instead of the natural frame rate of video (25 or 30 frames per second depending on the video system).

Because of these limitations, the potential of multimedia documents with simultaneous sound, picture, graphics, text etc. is not fully utilized. Some important requirements for multimedia document retrieval, transmission and presentation are:

Research Issues

In order to make the WWW to support multimedia document retrieval, transmission and presentation, the following issues must be addressed.

  1. The current HTML must be extended and a script is used so that one link can refer to multiple media streams and their temporal and spatial relationships can be specified.
  2. The servers should retrieve and deliver data at the same rate as the playback rate at the client.
  3. Current HTTP is based on TCP/IP which is not suitable for supporting continuous media communications. A real time or multimedia transport system should be used to replace TCP/IP.
  4. The browser should be extended to support simultaneous display of multiple media streams. The display should be scheduled according to the temporal and spatial relationships specified for each multimedia document.

Extension to HTML and browser

In current HTML, a link refers to a static web page which could be a text file (possibly with inlined images), an image, an audio file or a video file. It is not possible for a link to refer to both an audio file and a video file. We proposed to extend HTML so that a link refers to a meta-file of a multimedia document. The meta-file contains details of media streams of the document, their URLs and temporal and spatial relationships. With this information the extended client should be able to send request for each stream to appropriate servers at appropriate time. When media data arrive at the client, they are presented to the user according to the temporal and spatial specifications. An appropriate process scheduling discipline should be used in the client operating system in order to achieve real- time synchronous presentation (Hyden 94) . The proper operation of the browser also depends on the proper data delivery by the following modules.

Multimedia server

The current web server will accept a request no matter how busy it is if the user is authorized and the requested document is stored in the server. The server then serves the request at a rate determined by the workload and its power. The rate varies all the time. This is unacceptable for continuous media which require that the server retrieves and delivers the data at the same rate as its natural playback rate. There are a lot research and development in multimedia servers with special disk scheduling and organization techniques (Anderson et al 92, Hanko el at 91, Lougher 93, and Ragan et al 92). These technologies should be evaluated and used in the web servers.

Multimedia transport system

The transport system (including transport protocol and underlying networks) should transmit data from the server to the client with acceptable delay, delay jitter and error rate. In current HTTP based on TCP/IP, delay and delay jitter are not considered. Data will suffer variable amount of delay depending on the network load and capability of the networks. In the past few years, a number of multimedia transport systems have been proposed (Banerjea et al 94, Casner and Dearing 92, Clark et al 92, Cruz 95, Delgrossi 92, Ferrari et al 95, Ferrari and Verma 90, Macedonia and Brutzman 94, Schulzrinne at al 94, Topolcic 90, and Zhang et al 94). HTTP should be based on one of or a combination of these systems in order to support real-time multimedia transmission.

Multimedia synchronization

Multimedia synchronization refers to the maintenance of temporal and spatial relationships within a multimedia document. There are two aspects to multimedia synchronization. The first is to specify desired temporal and spatial relationships. The second is to provide a mechanism to guarantee these relationships. The task of the first aspect is carried out off line and the specification is stored in the document meta-file. The synchronization mechanism is implemented in servers and clients to coordinate transmission, buffering and presentation time.

Quality of service paradigm and system integration

The critical issue for supporting multimedia communication is how to provide end-to-end performance guarantee and at the same time how to use system resources (network bandwidth, CPU cycles, storage etc.) efficiently by taking advantage of bursty and error tolerance nature of continuous media. It has reached consensus in the research community that end-to-end performance guarantee should be achieved using quality of service (QOS) paradigm (Campbell et al 94, Nahrstedt and Smith 95, Nahrstedt and Steinmetz 95, and Vogel et al 95). In QOS paradigm, an application specifies its QOS requirements which are submitted to the system. The system determines whether it has sufficient resources to meet the requirements. If yes, it will accept the application and allocate the necessary resources to serve the application so that its requirements are satisfied. If it has insufficient resources to meet the application's requirement, it may either reject the application or suggest a lower QOS requirements which it can satisfy to the application. In the latter case, if the application accepts the new set of QOS parameters, the application will be accepted and executed at the lower QOS. Otherwise the application will be rejected, and it may try some time later in the hope that some resources may have been released by other applications. Based on this operation model there should be the following elements in order to provide a QOS guarantee:

Server, transport, synchronization and client support to multimedia document retrieval is at different levels of the QOS paradigm. In most reported work, ad hoc QOS management is carried out at different levels or components of a system. So far there are no systems which successfully integrate these components to provide general multimedia applications.

Related Work

There are attempts to provide video service using the WWW based on TCP/IP. These systems cannot provide QOS guarantees to continuous media as they don't reserve necessary resources for applications. Chen et al. worked on supporting continuous media in the WWW using an augmented Real Time Transport Protocol (RTP) called VDP . But their main focus is on increasing the transmission speed instead of trying to provide QOS guarantees to applications. Their work also does not address the simultaneous retrieval, transmission and presentation of multiple media as required for multimedia documents. Recently, Researchers at Berkeley has developed a continuous media toolkit which can display audio and video on-the-fly. But it does not provide QOS guarantees.

Conclusion

It is desirable to be able to retrieve and present multimedia documents in real time in the WWW. To do this, many WWW components need to be extended. HTML should be extended so that can interpret meta-files of multimedia documents. The format of the meta-file should be defined and techniques to specify spatial and temporal relationships among related media streams should be developed. A suitable real-time transport protocol needs to be developed and incorporated into HTTP. The server needs to store large amount of multimedia data and provides QOS guarantees to each media stream. The display should be able to display multiple media streams synchronously according to the temporal and spatial specifications.

The most important issue in achieving the above extension is how to accurately specify QOS requirements for each stream and each multimedia document. Only when QOS requirements are specified, can we carry out resource reservation, scheduling and admission control to provide required QOS to users.

We are currently researching into issues described in this paper.

References


Anderson D. P., Osawa Y. and Govindam R., "A file system for continuous media", ACM Transactions on Computer Systems, Vol.10, No.4, November 1992, pp. 311-337.
Banerjea A. et al, "The Tenet real-time protocol suite: design, implementation, and experiences", Technical Report TR-94-059, International Computer Science Institute, Berkeley, California, Nov. 1994.
Campbell A., Coulson G. and Hutchison D., "A Quality of Service Architecture", Computer Communication Review, July, 1994, also available from ftp: ftp.comp.lancs.ac.uk/pub/mpg/MPG-94-08.ps.Z.
Casner S. and Deering S., "First IETF Internet Audiocast," ACM SIGCOMM Computer Communications Review, San Diego California, July 1992, pp.92-97.
Clark D. D., Shenker S. and Zhang L., "Supporting real-time applications in an integrated services packet network: architecture and mechanism", Proc. ACM SIGCOMM'92, Baltimore, August 1992, pp.14-26.
Cruz R. L., "Quality of service guarantees in virtual circuit switched networks", IEEE Journal on Selected Areas in Communications, Vol.13, No.6. August 1995, pp.1048-1056
Delgrossi L. et al., "HeiTP - A transport protocol for ST-II", Globecom'92, pp. 1369-1373.
Ferrari D. and Verma D. C., "A scheme of real time channel establishment in wide-area networks", IEEE Journal of Selected Areas in Communications, Vol.8, April 1990, pp368-379.
Ferrari D., Gupta A. and Ventre G., "Distributed advance reservation of real-time connections", Proc. 5th Intl. Workshop on Network and Operating System Support for Digital Audio and Video, Durham, New Hampshire, April 18-21, 1995, pp.15-26.
Hanko J. G. et al., "Workstation support for time-critical applications", Proceedings of Second International Workshop on network and Operating System Support for Digital Audio and Video, Heidelberg, Germany, November 18-19, 1991, pp 4-9
Hyden E. A., Operating System Support for Quality of Service, PhD Dissertation, University of Cambridge, 1994.
Lougher P. K., The Design of a Storage Server for Continuous Media, PhD thesis, Department of Computing, Lancaster University, september 1993.
Macedonia M. R. and Brutzman D. P., "MBone Provides Audio and Video Across the Internet," IEEE Computer, Vol.27 No.4, April 1994, pp. 30-36.
Nahrstedt K. and Steinmetz R., "Resource management in networked multimedia systems", Computer, May 1995, pp. 52-63.
Nahrstedt K. and Smith J. M., "The QOS broker", IEEE Multimedia, Vol.2, No.1 Spring 1995, pp.53-67.
Ragan P. V., Vin H. M. and Ramanathan S., "Designing an on-demand multimedia service", IEEE Communications Magazine, Vol.30, No.7, July 1992, pp. 56-64.
Schulzrinne H., Casner S., Frederick R. and Jacobson V., "RTP: A Transport Protocol for Real-Time Applications", work in progress, Internet Draft , July 1994.
Topolcic C., "Experimental Internet Stream Protocol, Version 2 (ST-II)", Internet Request for Comments No. 1190, October 1990.
Vogel A. et al., "Distributed multimedia and QOS: a survey", IEEE Multimedia, Vol.2, No.2, Summer 1995, pp. 10-18
Zhang L. et al, "ReSource ReserVation protocol (RSVP) - Functional Specification", Internet Draft, March 1994. .

Copyright

Guojun Lu ©, 1996. The author assigns to Southern Cross University and other educational and non-profit institutions a non-exclusive licence to use this document for personal use and in courses of instruction provided that the article is used in full and this copyright statement is reproduced. The authors also grant a non-exclusive licence to Southern Cross University to publish this document in full on the World Wide Web and on CD-ROM, and for the document to be published on mirrors on the World Wide Web. Any other usage is prohibited without the express permission of the author.
Pointers to other Papers
Papers & posters in this theme All Papers & posters AusWeb96 Home Page

AusWeb96 The Second Australian WorldWideWeb Conference "ausweb96@scu.edu.au"