Network Performance - Impact On Networked Educational Software

Jan Newmarch, University of Canberra, PO Box 1, Belconnen, ACT 2616, Australia. Ph: +61-6-2012422, Fax: +61-6-2015041 Email: jan@pandonia.canberra.edu.au, Home Page: Jan Newmarch [HREF3]

John Baird, University College, School of Aerospace and Mechanical Engineering, University of New South Wales (ADFA), Campbell ACT 2601, Australia. Ph: +61-6-2688279, Fax: +61-6-2688276, Email: j-baird@pop.adfa.oz.au

Introduction

Background

The Cross-platform Multimedia Courseware Project

The NSFNET Backbone dataset :

The NSFNET (National Science Foundation Network) study was an empirical research carried out to measure the behaviour of the deployed network. NSFNET performance statistics have been collected, processed, stored, and reported by the Merit Network, Inc. since 1988, in the early stages of the NSFNET project. Note that in December 1994 the numbers started decreasing as the traffic began to migrate to the new NSF network architecture. On April 30, 1995, the NSFNET Backbone Service was successfully transitioned to the new network architecture and this data was no longer collected [NSFNET95].

• As was true two decades ago in the ARPANET environment, traffic favouritism is high. For example 0.28% of the (customer/campus/site) network pairs generate 46.9% of the traffic [HREF5], and extended delays happen mainly around a few favourite nodes (see tables 1.1 a & b). This signifies an uneven distribution of network traffic, which can make the network seems worse than it actually is. Also delivering a courseware in an area which attracts a lot of traffic will likely have more problems than in oher areas.
• Link delays seem to have increased substantially. Take node 141 in tables 1.1 for example, in January 1993 (table 1.1a) the delay from previous month was increased only by 1 ms and at only one link, which is node 136. In April 1994 (table 1.1b) almost all links to node 141 have their link delays increased from previous month, with a magnitude of generally 7 ms. A general comparison between the two filtered delay matrices shows that the median changes between the 2 months 3/94 and 4/94 are generally 7 times greater in magnitude and 3.6 times higher in frequency compared to the median changes between 12/92 and 1/93. All of these mount to a longer delay at the user end.

• WWW has made a giant leap from zero to the highest volume application within only one year, pushing FTP down to the second rank (figure 1.1). This highlights the increasing popularity of the WWW and predicts a new era for authorship. The implication on network performance however, is that Web-based courseware will have to “compete” more rigorously with other Web-based services as the Web is becoming more and more popular for both users and content providers.
• Traffic volume in bytes seems to increase quadratically, while the number of networks appear to increase linearly [HREF5] and see also figure 1.1.

In short the NSFNET statistics reveals a tremendous growth of network traffic, which heavily affects the performance of the network and hence courseware delivery. It also shows the opening of a new “Web age” with the migration of network volume from FTP and other services to the WWW. These facts suggest the use of the WWW as a very good courseware delivery platform, but with some performance penalty probably at least in the next few years.

Fig 1.1 Extrapolated total traffic in byte count
Source : ftp://ftp.nic.merit.edu/nsfnet/statistics, Graph by : James Pitkow, GVU Centre

(a) ANSNET T3 Delay Matrix Report for January 1993
Filtered Median Delay Times (ms)

	128	129	130	131	132	133	134	135	136	137	138	139	140	141	142	143	144	145	146
128
129
130
131									14,1							33,1
132
133																43,1
134										5,1	16,1
135				33,1		43,1					30,1
136				14,1										26,1					2,1
137							5,1
138
139						28,1
140																			24,1
141																		26,1
142					21,1
143				33,1		43,1
144
145				14,1											30,1
146

(b) ANSNET T3 Delay Matrix Report for April 1994
Filtered Median Delay Times (ms)

	128	129	130	131	132	133	134	135	136	137	138	139	140	141	142	143	144	145	146
128		32,7												37,7
129	32,7													18,7	22,7	24,7	32,7
130														19,7	23,7
131														25,7	29,7
132														24,7	28,7
133														33,7	37,7
134														31,7	35,7
135
136														46,7	50,7	49,3
137														29,7	34,7
138														38,7	42,7	44,7
139														25,7	29,7	31,7
140														23,7	27,7	29,7	37,7
141																		46,7	45,7
142																		50,7	50,7
143																			48,3
144
145														46,7	50,7
146														45,7	50,7	48,3

Table 1.1a & b ANSNET T3 Delay Matrix Report
Data source : ftp://ftp.nic.merit.edu/nsfnet/statistics

Note : Results are one-way delays in milliseconds using 100 byte packets. For each cell, first number shows median, second number shows changes in median from previous reporting period (Dec 1992 and March 1994 for (a) and (b) respectively). Results are shown for changes > 1 ms.
Note : The numbers at the top row and first column represent the following nodes :

128 Palo Alto	129 Urbana-Champaign	130 Chicago	131 Ann Arbor	132 Pittsburgh
133 Ithaca	134 Boston	135 San Diego	136 College Park	137 Princeton
138 Georgia Tech	139 Houston	140 Lincoln	141 Boulder	142 Salt Lake City
143 Seatle	144 FIX-West NASA Ames	145 FIX-East College Park		145 DARPA

The GVU Center's Third WWW User Survey data (10/4/95 - 10/5/95) - Georgia Institute of Technology

The Graphics, Visualisation and Usability (GVU) Centre is a research centre affiliated with the Georgia Institute of Technology, who has pioneered the use of the Web as a surveying tool in January 1994, and has since then been periodically conducting surveys on the development and usage of the Web. Data are collected, analysed and released every 10th April and 10th October. This third survey has received over 13,000 unique responses from 10/4/95 to 10/5/95, more than 10 times the response rate to any other online survey. The surveying techniques are pioneering and require conservative interpretation of collected data due to the absence of time-tested validation and correction metrics. The survey uses non-probabilistic sampling, which does not employ randomization techniques to get respondents. This reduces the ability of the gathered data to generalise to the entire population, and data presented here should be taken only as a snap shot [HREF8].

• Overall people spend a lot of time on the Web (41% users spend 6 - 10 hours/week, 21% 11 - 20 hours/week, an increase of 5% and 6% respectively since 1st October, and 72% use their Web browser at least once a day). The most common use of the Web however, is simply for browsing (82.6%) and entertainment (56.6%) (figure 1.2). Of the browsing usage, the most regular activity is to replace other browsers such as FTP, Gopher, etc (6.7), closely followed by finding reference information (6.2), reading electronic news (5.7) and reading product information (5.1) (figure 1.3). These findings predict a continuously increasing trend of heavy traffic load for bulk transfer (FTP and FTP from within the WWW). The numbers are also very encouraging for services such as product information, so it might be expected that the network will be loaded with a considerable amount of high volume graphics and probably other media for business and advertisement purposes. This prediction is confirmed further by the fact that most users prefer visually oriented pages (46.06%) (figure 1.4), which will serve as a driving force for Web-based content designers. All these facts suggest a heavily loaded network, which will certainly impair the general network performance unless the network links can be upgraded.

• Looking at the network links, at the servers end, the most popular servers connections are now 10 Mb/sec (32.254%), and the next most common are 1 Mb/sec (18.07%), an increase of 32.254% and decrease of 36.03% respectively since 1st October, 1995. Only 1.813% users have servers connections over 100 Mb/sec as was in the previous survey. At the other end, users are mainly connected to the network through 14.4 Kb/sec links (43.8%). Only 13.1% have 10Mb/sec connections via workplace or educational institutions (figures 1.5 and 1.6). There is no previous data about users connections, but about one-third of the servers connections have been upgraded from 1 Mb/sec or below to 10 Mb/sec.

  This is a considerable improvement of the overall network bandwidth, however it still does not keep up with the quadratic growth of the network traffic.

In general the GVU survey data consolidates the results from the NSFNET research, giving more details about how the network traffic is increased (by bulk transfer and images, and the increasing usage of the network via the WWW), and the limited development of the network capacity (users and servers connections). These facts confirm the users preferences of the WWW but predict performance problems for delivering Web-based courseware. They can be alleviated however, if more control over the generation of network traffic is exercised and/or the network bandwidth can be increased. These issues give rise to other research topics.

Project statistics (31/8/95 - 31/11/95)

The Web-based courseware incorporates two kinds of instructional materials: lessons and self-assessment tests; and three major media types: text, graphics and video. The graphics are both static and dynamic. The dynamic graphics, which are mathematical plots illustrating various fluid dynamics concepts, are generated in realtime via CGI (Common Gateway Inteface) scripts, using either the server-push-client-pull mechanism [HREF4] (non-intereactive) or the MATLAB program (interactive). Most attention is given to the MATLAB plots to measure the network performance, since this is the more interesting use of the Web, and also the one which likely has more delivery problems as it requires a download, a server program execution, and more communication between the client and the server.

The courseware is authored in HTML, ISMAP, FORM, CGI scripts and MATLAB, and was published together with the release of other versions of the courseware in Visual Basic and Macromedia Director on 31 August, 1995. Statistical data have been collected since 31 August, 1995 until 31 November, 1995 via questionaire, emails, and scripts. Data is collated, analysed and summarised every week.

1. Results of the questionaire

   There have been 8 completed questionaires sent back, 3 of which are from the USA, the rest are from Australia. This is a very poor response rate compared to the total number of accesses to the courseware (434 times). The server connection is 64Kb. All users have 10 Mb or 28.8 Kb links. The results are summarised as follows:

   • Average time for loading up the Course Overview Page (33K) is 22.8 seconds from the USA and 21 seconds in Australia
   • Average time for downloading a QuickTime movie (250-350K) is 3 minutes from the USA and 2.3 minutes in Australia
   • Average response time for generating a realtime plot is 2.2 minutes from the USA (on 10 Mb link), 0.67 minutes (on 10 Mb link) and 1.8 minutes (on 28.8 Kb link) in Australia.
   • 83.4% users can generate a realtime plot, 33.2% of whom have problem displaying the graphs in Ghostscript.
   • 62.5% users are connected to the network through 10Mb/sec links, 37.5% through 28.8Kb/sec links.
   • 75% users think the response time is very good, good or satisfactory and the realtime application is of practical use for teaching and learning on the Web; 12.5% disagree, and 12.5% do not know.

2. Results from log files

   There are three MATLAB functions in the courseware: Flow Past a Circular Cylinder and an Aerofoil, Kutta Condition, and Joukowski Transformation. 48 attempts have been recorded to generate plots for the first function (30K postscript file). Of these, 18 are from the US, 1 from Canada, and the rest from within Australia. The other two functions, Kutta Condition (generates 57K postscript file) and Joukowski Transformation (generates 12K postscript file) are tried 17 times each, with 2 attempts from the US and 1 from Norway. Most users hosts are from educational institutions or NASA.

   The results are summarised in Table 1.2.

   File size	Mean (sec)	Median (sec)
   13K	29.59	23
   30K	60.65	53
   57K	115.35	102

   Table 1.2 Matlab runtime on the network

   Note that of these 3 datasets, only the 30K is most representative, since it has much more data than the other 2 datasets. The differences between the mean and the median are quite large, which reflect a large range between the upper and lower bounds of the dataset. This might be attributed to the workload at the server site at the time the query is executed. There does not seem an association between the runtime and the time of the day.

   Note also that the time recorded here is the runtime, ie. the MATLAB execution time only, not the response time, ie. the time since the query is generated until the output is sent back to the user. To find out more information about the response time, these datasets are cross-referenced with the one generated by the server log file to extract the time one particular host spent on a particular page. This shows the response time to generate a 30K file from the USA appears to range from 2'36'' to 3'22'', and from within Australia from 1'02'' to 1'39''. These numbers might not precisely reflect the response time, but can give the upper bound of the response time range, since the actual response time might only be shorter than the time estimated by this method (users might spend more time on that page without generating a realtime plot). Another remark is this result generally agrees with the feedback collected from the questionaire.

3. Discussion

   The questionaire results are not statistically significant due to the poor response rate. When combined with the log file data however, they appear to agree with each other and both provide useful information.

   Despite the delay factor, the Web seems still of practical use for delivering courseware. As the time required for realtime activities are still within an acceptable range, we can be quite optimistic about the response time for other simpler activities. The download time for multimedia elements is also generally acceptable, provided of course that they should not be too large. This is a consideration that all content providers should take into account anyway.

   An unsolved problem in this courseware is when users cannot display the postscript file due to inappropriate setup of the viewer or helper application on their machine (ghostscript or ghostview in this case). This is a problem at the client site and out of control of the courseware provider. However the recent Java and Shockwave technology offers the same or higher level of interactivities with more or complete control to the courseware provider. This has made the Web even a more attractive courseware delivery platform !

Conclusion

References

[ARCHEE95] R Archee. The World Wide Web and Distance Education: Convergence or Cacophony?, Proceedings of AUUG'95 and Asian-Pacific WWW'95, pp 350-4

Hypertext References

HREF1

http://www.adfa.oz.au/~tln - Thuy-Linh Nguyen's Home Page

HREF2

http://www.adfa.oz.au/MECH/ - The Fluid Mechanics Courseware Home Page

HREF3

http://pandonia.canberra.edu.au/ - Jan Newmarch's Home Page

HREF4

http://home.netscape.com/assist/net_sites/dynamics_doc.html - Dynamic Document.

HREF5

ftp://ftp.nic.merit.edu/nsfnet/statistics/nsfnet-t.ps - H-W Braun. Traffic Characteristics of The T1 NFSNET Backbone pages.

HREF6

ftp://ftp.nic.merit.edu/nsfnet/statistics/ - NSFNET Backbone Statistics

HREF7

http://web.mit.edu/afs/sipb/user/mKgray/ht/wow-its-big.html - M Gray. Wow, The Web Is Big

HREF8

http://www.cc.gatech.edu/gvu/user_surveys/ - The GVU Center's 3rd WWW User Survey

Copyright