This paper describes the use of WWW as a support medium for an undergraduate Information Systems unit taken by some 1000 students over a 15 week semester. It describes the unit content and context, the traditional manner of running and supporting it, the implementation process and the instruments used to determine the effectiveness or otherwise of using this medium. The results obtained and conclusions drawn are discussed, as well as suggestions for further research in the use on new technology to support education where education is required.
The School of Information Systems [HREF2] is one of seven Schools comprising the Curtin Business School [HREF3] (CBS) at Curtin University of Technology [HREF4] . All CBS students have to complete five common core first-year units (designated as "100" units), irrespective of their choice of major [HREF5] within the Bachelor of Commerce program. These units are designed to provide the foundation for the remainder of CBS undergraduate program. The School of Information Systems offers Information Systems 100 (IS100 [HREF6] ) as a common core unit with 3 class contact hours per week.
The IS100 unit syllabus [HREF7] covers information systems and technology in an organisational context, and has a number of objectives: to provide students with an introduction to the discipline of IS, to arm students with the IT skills they will need for the remainder of their undergraduate education and to act as a marketing vehicle for the School.
IS100 to date has been conducted along traditional delivery methods with each student attending one one-hour lecture and one two-hour tutorial/workshop per week. The lectures cover the IS theory component, whereas the tutorials reinforce the theory as well as provide the IT skills training: to date these have been the effective use of a WP package (MS Word 6.0) and a spreadsheet (MS Excel 5.0) running under Windows 3.11..
This unit is compulsory for all first year CBS students, irrespective of the major(s) selected - students must pass this unit in order to graduate.
IS100 is undertaken by some 1000 students in Semester one each year (February-June), and some 250 students in Semester 2 (July-November). The cohort is in the main students coming straight from secondary education system, though a significant proportion are part-time students seeking career development. Less than 20% of the study cohort had enrolled in IS or IT majors: the overwhelming majority came from Accounting, Economics and Finance, Business Law, Management and Marketing - the general and thus non-IT areas.
During Semester 1, the lecture is repeated twice to accommodate part-time students and existing lecture theatres. Some fifty tutorials are conducted by 16-20 tutors, the overwhelming majority of those are part-time staff who have no office space or dedicated support facilities.
The student thus needs to be provided will all the information necessary for him or her to complete the unit, otherwise they can have difficulty in contacting their tutor outside class time. Students in the past have resorted to contacting the lecturer in charge (LIC) who sets aside a couple of hours per week for student consultation - with 1000 students this time allocation is often not enough when the administrative support systems are not performing as planned.
The student is normally provided with a unit outline at the first lecture during the semester, which as a minimum contains the following
This document has traditionally been paper based, comprised of anything up to 23 A5 pages and given to students at the beginning of the first lecture of the semester. The student must physically carry this booklet around during the semester for on-campus reference - it contains all of the reference information the LIC believes the student needs to undertake the unit.
Preparation of the unit outline can be problematic, in that the LIC would like to get it finalised as soon as possible so that it can be sent to the printers and be ready for the start of the semester, but each semester there are generally changes to the general Information section due to policy changes at either CBS or University level.
Each of the three assignments were handed out on paper to each student, with assessment taking place in class.
Lectures were delivered via the use of Powerpoint presentations. If a student missed a lecture, he/she could obtain a printed copy of the presentation notes from the LIC's office.
Each tutorial is run in one of 9 CBS PC laboratories, each contained 20 machines. Two of the laboratories are equipped with 486 machines, the remainder with Pentium 100's. Each lab is a self-contained LAN, linked into CBS divisional LAN (currently a Banyan/Novel hybrid). The application software used by the students is installed on the hard disk on each PC (to reduce network load), and utilises a RAM drive for workspace.
In the laboratories, students are provided access to Word 6.0, Excel 5.0, Powerpoint 4.0 and Netscape Navigator 2.02 running under Windows 3.11, as well remote access to a VAX cluster at Curtin University's Computing Centre.
Students are provided with an E-mail account on the VAX, and this is accessed via either PINE or Netscape Navigator. Neither of these facilitate easy E-Mail communication with staff on the Vines network above .
Due to the administrative overhead and resulting costs, it became quite apparent that IS100 would be an ideal candidate for the implementation of WWW-based support systems. The catalyst was the last minute unavailability of the original LIC for Semester 1 1996: some 1000 unit outlines each of 24 pages had already been printed, and significant policy and staffing changes meant that the unit outlines needed to be modified accordingly. In addition, the opportunity existed to reduce the cost of operating the unit by reducing the amount of paper consumed during the semester and the overhead placed on IT Services personnel in decollating and distributing student printouts.
The proposal that students have on-line access to their support material from any machine on campus or from home was deemed a significant contribution to improving the quality of service delivery at CBS. This was especially relevant to the review of the quality of services provided to external students, some of whom are over 2000km away from the Bentley campus of Curtin.
It was also deemed desirable that the skill set training provided by IS100 should be expanded to include new technologies such as E-mail, voice-mail, newsgroups and WWW to assist in student research and their familiarisation with tools that were becoming common place in the commercial environment in Perth.
It was decided that, due to the short lead time available, a limited implementation would be attempted and measured. It would only be made available to internal students. Much has been said about the potential benefits of utilising WWW in education, but there was little in the way of quantitative data on student use and behaviour in the WWW environment.
As there existed a window of opportunity of putting the technology we taught students about into practice, it was decided to undertake some research on the activity. We wished to find out
It was decided to place the following information on the School's WWW server for student access
The author was interested not only in the page development process, but of more importance was the utilisation of the medium and how the students felt about the use of the WWW. An ancillary area to be investigated was system performance, as the initial presence was to be housed on a comparatively small machine.
Visit counters were placed on each of the WWW pages, so that some quantitative picture could be gained of student usage of the pages - were they looking at the pages they were supposed to, were they downloading assignments and review question answers. The counter readings were taken at the same time each week - Monday mornings between 10:00 and 11:00 am.
An informal survey of 143 students (some 17% of the enrolled cohort) was taken to obtain a profile of the student population, looking in particular at the IT facilities students had access to off-campus.
Finally, sample groups of the student undertaking IS100 were surveyed at the end of the semester to determine the phenomenon from the users' perspective, including suggested improvements. Snapshots would be taken during peak periods to determine system response and completeness of page delivery.
The design of the IS WWW presence was undertaken from the students' perspective. It should be noted that the overwhelming majority of students were not undertaking or planning to undertake an IS or IT major - they were enrolled in accounting, business law, economics and finance, management and marketing.
There would be a limited number of options available on the home page
The welcome page gave the student an informal introduction to the site and how to navigate around it.
The unit outline was re-engineered and restructured to better utilise the medium.
Lecture notes were in the form of Powerpoint (.ppt) files which could be downloaded and viewed via the Powerpoint Viewer installed in the laboratories.
Assignments were in the form of HTML pages for viewing, and Word 6 (.doc) and Excel 5.0 (.xls) files for downloading.
Answers to review questions were placed on WWW pages, released weekly as appropriate.
Any significant modification or announcement would be placed in the latest news pages.
The existing paper-based unit outline was provided to the author in Word 6.0 format. It was then converted to HTML 2 format via the Internet Assistant add-on to Word. A standardised look and feel was designed and implemented, based the need for the presence to reflect both the School of IS and CBS. Due to the layout of the unit outline information (in particular the Timetable of Classes [HREF13] and Semester at a Glance [HREF14] sections), some considerable effort was required in converting the text into tables. The version of Internet Assistant used was only HTML 2 compliant and thus would not handle tables, and as such HotDog Pro [HREF15] and Live Markup [HREF16] were used to generate the appropriate pages. Beta versions of Netscape Gold [HREF17] were evaluated, but again did not include table processing.
The pages and graphic files were developed off-site, and installed on a small server - a 486-DX66 machine with 32Mb RAM & 1Gb SCSI HDD, running EMWAC HTTPS Version 0.96 under Windows NT 3. 1. This machine, housed in the School of IS postgraduate laboratory, was connected by Curtin's own internal WAN to the Internet via AARNET.
The IS100 Web Site can be accessed from three access points
Requests were forwarded for the site to be indexed by InfoSeek, CNet, Yahoo, Lycos, AltaVista and WebCrawler.
The presence was available for student access from the first day of semester. Students were offered the obsolete paper version of the unit outline in the first lecture, and were then told that it was not longer current and that they would see the current outline in the labs. The first tutorial/laboratory was dedicated to making students comfortable with the technology (Netscape and Eudora), navigating around the site and going through the Welcome (introductory) section of the presence. They were then told that they should start planning their next two weeks, using the week by week planner provided, and to check the WWW site at least once a week for news and assignment details.
Assignment 1 was available to students via the Web site from week 4, and Assignment 2 in week 11. It was decided that assessment of assignments would take place in the class, so that students could get immediate feedback from their tutors. Tutors would then update their class spreadsheets and forward them to the LIC for consolidation.
Students were encouraged to share their experiences via the unit specific newsgroup curtin.is100.newsgroup.
The analysis of this use of WWW in tertiary education was undertaken from two perspectives - site utilisation and user perception.
It should be remembered that over 80% of the cohort were not enrolled in IT disciplines and thus a representative sample of business students. This gives the author some confidence that the results would be skewed due to user bias towards technology.
To undertake comparative analysis of site visits, a unit of measure is proposed - the number of site visits per student per week (abbreviated to svsw).
At the root level across the semester, there was an average 1.26 svsw, though early in the semester the readings were to a maximum of 3.81 svsw.
During the sixteen weeks of the study, the visits reached a maximum of .93 svsw, though across the semester the average came down to .28 svsw with a std dev of .23. Anecdotal evidence suggests that students felt that they needed to read it only the once.
The mean of svsw for the week-by-week planner / semester at a glance pages was only .19 svsw (std dev .19). Though the svsw rating during the first few weeks was around 1.5, the number of weekly visits reduced dramatically as the semester proceeded. This may be explained by a combination of students being familiar with the weekly requirements, students finding out the information through informal information systems and students not wanting to know what was to be undertaken each week.
It was also noted that there were as many visits in total during the semester to the week by week planner as there were to the Semester at a Glance section of the unit outline. These contained the same information in differing forms. The week by week planner had an index page, where the student would select a week number and a page would be returned containing the topics and requirements for that week. The Semester at a Glance page was a 18x7 matrix that the student would scroll down to the week required.
When the assignments were first released, the access ratings were quite high (e.g. 1.21, 1.25 svsw), then rapidly reduced down to 0.02-0.08 svsw. Anecdotal evidence indicates many students printed hardcopy of the specification to take with them. The combination of 52% of students having compatible hardware and software at home, only 12% having Internet access from home and laboratories being in high demand with an 88% occupancy rate would support this hypothesis.
The mean weekly access by students of lecture material across the semester was .48 svsw after peak of 1.13 svsw in the first few weeks of semester, and an increasing access rate late in the semester when pre-examination studies were being undertaken.
Answers to review questions [HREF12]
This was divided into two areas. Review questions at the end of each chapter in the theory text were to be done each week as a means of students determining their recall ability of the text material read and covered in the lecture. Access rates varied from 1.59 svsw at beginning of semester to .63 svsw at the end.
Students who were not familiar with WP or SS software were encouraged to go though the relevant chapters in the workbooks and answer the review questions as a means of re-enforcing the materials covered. The access rates on these workbook answer pages varied from .68 svsw to .07 svsw at the end the semester, which is explained by the number of students who were familiar with the packages and thus who did not feel that they needed to either go through the workbook chapters and/or answer the review questions.
Apart from the week during which the newsgroup was made operational (.47 svsw), access to the newsgroup page was limited (mean of .08 svsw). An preliminary investigation of the newsgroup traffic indicated that students needed some encouragement and leadership in the use of the newsgroup. An additional factor is the ability for students to access the newsgroup directly via Netscape without accessing the WWW page and thus triggering the counter increment.
Remote Access
The cohort profile indicated some 12.6% of the enrolled students had remote access to the site. However, the counters used in this experiment did not provide a breakdown of the sources of the accesses. A further analysis of the server logs is required to determine when students access the site from home, and when they accessed on campus, as well as when the site was accessed by people not enrolled in the unit.
At the end of the semester, a sample of the current student cohort were asked their opinion of the WWW as a means of accessing information. The quantitative data provided indicated that 94% found the WWW site useful, and only 18% found the paper version of the unit outline more useful for obtaining information. The most interesting data coming from the feedback was that 58% of the sample would like to be able to do more of their studies from home via WWW and E-Mail. When students asked "what three things worked best for you?", some 86% of student responses included the WWW site.
One suggested improvement was in site navigation. For a visitor
to return to the central index required an understanding of the
function of a given icon (
) was required, and the visitor then had to click at least twice
to proceed to the next area of interest. It was suggested that
if the index were on each page, visitors would be able to immediately
determine if there were any other sectors of the presence related
the visitor's particular interest.
Lecture notes were perceived to be of benefit to students, though one difficulty encountered was for those students accessing remotely who did not have the appropriate release of Powerpoint loaded on the remote machine. It is proposed that this facility would be more effective if the IMM material were somehow incorporated in the WWW page and not dependent upon a separate software package for access.
Comments were made that though it was a plus that contemporary electronic means were used for information disbursement to students, assignment submission was still via paper and diskette, and assessment feedback was on paper. A possible extension of the use of contemporary technologies would be the submission of assignment material via E-mail or Lotus Notes. However, to effectively implement this would necessitate a common platform for both staff and students.
User Profile
A survey was undertaken of the students to determine their access to IT while taking the unit. Some 52% of students surveyed had access to machines running the software packages used in the unit, and of the remainder some 49% had access to PC's or Macintoshes currently incapable of running the selected packages - thus only 23% of students did not have access to IT. More than 10% of students surveyed purchased the appropriate software during the semester.
As mentioned previously, at semesters end some 12.6% of the surveyed students had Internet access.
Cost Factors
Anecdotal evidence gathered from the computer operators and support staff indicated that the volume of paper produced by the students was significantly lower during the period of the study. Additionally, of the 1000 printed copies of the unit outline produced, only some 40% were taken by students. This appears to reinforce student perceptions that WWW is a preferred means of accessing information.
Demand on Staff Time
The LIC found students quick to adapt to E-mail as a communication medium. Indeed, meetings were organised and questions answered in this manner. Anecdotal evidence indicates that students felt they were able to ask the question of the staff while it was fresh in their minds. The LIC, who was telecommuting during the period of this study, was able to respond to students at any time.
By July 1996, the new unit outline would be implemented at which time qualitative data could be generated reflecting the comparative costs of production in both media - paper and WWW. It is proposed to copy and paste the changes into the presence, and reduce the volume of printed unit outlines available to students.
System Performance
The hardware/software used for this study ran remarkably well during the semester. Response times were respectable, as there were no more than 3 tutorials running concurrently, each of a maximum of 20 students. It is hoped that upgrades to both the operating system and WWW server software should improve performance and the level of services offered to students.
From the preliminary results above, the following conclusions are drawn
Further analysis of the data from the server logs is proposed to determine from where the visits took place, and thus determine the level of off-campus access as well as access by those not enrolled in the unit in question.
For second semester, it is proposed monitor the effectiveness of the following enhancements
Once a validation link to the student record system is implemented, it is proposed to trial open access to the unit material, with an appropriate student security access to downloadable and assessable material.
| Pointers to Abstract and Conference Presentation | |||||
|---|---|---|---|---|---|
| Abstract | Conference Presentation | Interactive Version | Papers & posters in this theme | All Papers & posters | AusWeb96 Home Page |
AusWeb96 The Second Australian WorldWideWeb Conference ausweb96@scu.edu.au