Assessment via the Web: A Snapshot of University Practice

Rod Byrnes rbyrnes@scu.edu.au and Allan Ellis aellis@scu.edu.au

School of Social Sciences, Southern Cross University

Abstract

An exploratory study of assessment practices in Web-based educational delivery was undertaken at a regional Australian university. It involved the collection of interview data from academics, administrative staff and students, statistical data on course provision, as well as a review of the relevant literature. The University has a relatively long history of involvement with Web-based educational delivery and is already working with its second university-wide course management system.

While the basic concept of assessment via various Web-based tools is well understood by academic staff the reality is that very little use is being made of the available tools. There is clear evidence that staff development has been crucial to stimulating and supporting academics to teach via the Web therefore it is reasonable to assume that assessment practices could also be improved and extended if specifically targeted by training and staff development intitiatives.

It is suggested that several factors contribute to these low levels of use, these include: the absence of a holistic approach to assessment across the university; increased workload concerns; concerns for equity; and concerns about the quality of Web-based assessment tools and practices.

Historical Developments

Southern Cross University [HREF1] was an early adoptor of Web technology having its first server up in late 1992 and a university home page online in early 1993 (Ellis & Sawkins, 2000). Earliest Web-based units were developed in 1995 and offered in 1996 (Ellis, Wildman & O'Reilly, 1996) and several university staff were advocating that the University develop a Networked Learning environment (Ellis, 1995; Chua, Debreceny & Ellis, 1995). These early units were "hand stitched" mixes of text, images and some audio and only catered for a small number of postgraduate students. Assessment items were still conventional essay responses but these were submitted online. In the early 90's browser technology was incapable of supporting the submission of Web based assessments. Prototype product development and pilot research conducted by Byrnes (1994) revealed that issues such as security and the difficulty of marking assignments on-screen were of major concern.

In 1997 course materials from a number of existing paper-based distance education programs were Web-mounted and along with email lists, used to support existing units and programs. In mid 1998 the decision was taken by the University's Executive to develop two full undergraduate degree programs online and this lead to the adoption of the commercial IBM software LearningSpace as the course management platform. Many problems were encountered with this system (Patterson, Ellis and Brice, 2000) which lead to the decision being taken in late 1999 to phase it out over the next year and replace it with a new course management shell, Blackboard's platform CourseInfo. A plan was developed for this transition process which was completed by the end of 2000 (Sawkins, Ellis and Gardner, 2001).

The years 2001 to 2003 have been a period of consolidation with Blackboard being firmly accepted as an approrpriate course management platform for the University's current needs. During this period staff received adequate training and support. The newly released version 6 is currently being trialed and will be phased in at the beginning of 2004.

Assessment as a Component of Web-based Education

There are several pedagogical uses of Web-based assessment [HREF2]:

The vast majority of Web-based assessment products available focus on the use of quizzes. The quiz itself can be comprised of a number of different question types which are easy for computers to automatically mark [HREF3] including:

Significant implementations of Web-based assessment programmes, for example [HREF4] also focus on the use of quiz tools for assessment. These tools, however, suffer from certain limitations [HREF5] such as "claims by test-makers that the scores of computerised and pencil-and-paper tests are equivalent are inadequately supported". With no other options available, these considerations are usually overlooked.

The Assessment Process from Different Perspectives

The task of assessment can be viewed from a variety of perspectives. Each is important and an understanding of the complete suite of stakeholder issues is crucial if assessment practices are to be both effective and optimal in a given situation. In relation to the online environment, there are five major perspectives: teacher, student, instructional designer, technical infrastructure provider and staff developer.

Teacher - The traditional role of the teacher involved all aspects of teaching, development, delivery and assessment. The teacher was traditionally the person who: set objectives, assembled appropriate content, devised delivery strategies, conducted the tests, and assessed the learning outcomes. Each step involved hands-on work by the teacher. For the teacher assessment was just one of the steps in a process of creating and managing the learning environment.

Student - For the student assessment is often seen as the task undertaken at the end of a learning session. Ideally it has to be relevant, authentic and motivational. In addition assignment submission processes and grading and scaling practices must be secure and accurate if the student is to be confident about the overall assessment process.

When teaching is provided via an Internet connection and a Web browser some other stakeholders appear.

Instructional designer - The technical issues involved in Web-based delivery mean that the traditional teachers' role frequently becomes more of a subject matter expert (SME). The teacher then puts forward the content based on course objectives but may leave it to someone else to design the instruction, that is, how the material will be presented and accessed online and to varying degrees what style of assessment items are used. The latter are usually restricted to what is provided by the course management shell software that is being used.

Technical infrastructure provider - The people who run the course management system and makes sure it does what it is supposed to do, like stay running, remain secure etc. The infrastructure provider also has a perspective on assessment. They are likely to be concerned with things like being able to receive and read various file formats, handle the submission of multiple linked file types (eg. text, audio, video), date receipt and dispatch of files, etc.

Staff developer - This persons' perspective is likely to be related to identifying asessment tools that are easy to understand and learn to use so that appropriate staff training can be provided. As staff computer literacy varies across discipline areas there is a need for a range of assessment tools that suit a range of user capabilities. An attractive feature of an assessment tool might be that online help and training is available.

Accessibility and Equity

The use of Web-based education and assessment has implications for individuals with disabilities. Accessibility issues have been important to the World Wide Web Consortium (W3C) [HREF6] as it has been developing the Web standards we use today. For many disabled people making the Web accessible has enabled them to interact in ways they may never have otherwise been able to. However, with the benefits of the technology come drawbacks. Despite all our efforts, Web-based education is still difficult to use or even unusable for certain individuals. In the interest of equity, it is essential that we provide alternatives for those individuals that require them. In moving education into a Web-based environment, we must not lose sight of the bigger picture, that is, providing education.

The issue of accessibility is also a problem when considering socio-economic sub-classes. It is a reality that in our society, certain portions of the population have better access to technology, and are better able to use it, than others. The reasons for this are as diverse as the people involved, but some of the more apparent reasons are economic (for example, inability to afford a computer, Internet access, etc.), cultural (maybe possessing little computer literacy, or having language barriers), and geographic (especially in Australia, access to technology and bandwidth in remote areas continues to be a problem).

Issues such as these are too important to be ignored, and to some extent have contributed to the slow uptake of Web-based assessment that has occurred to date.

Moving to a Web-based Assessment Environment for Assessment

In a way that is reminiscent of the paperless office ideal of the business world, Web-based education, but more particularly the assessment component, appears to have failed to gain use and acceptance at the levels that were initially predicted. Nevertheless some progress has been made and most acedemic staff, even if not actually engaging in some form of Web-based education or assessment, are at least aware that these facilities exist. Of those actually engaged in Web-based delivery most are only providing supplementary course materials; even fewer are providing a direct online component that can be used instead of more traditional face-to-face or paper-based methods (as used for traditional distance education). Only a tiny minority provide complete units of study that can be undertaken entirely over the Web and not even all of these employ Web-based assessment.

One thing that has become clear to teachers attempting to move assessable coursework to the Web is that it's a lot more difficult than it initially seems. Some very interesting approaches to online assessment have been devised, such as Virtual ChemLab [HREF7] which simulates an entire chemical laboratory online. However, the amount of work involved in developing a software tool of this nature and quality is daunting, and the market for it relatively specialised and small. There are thousands of such educational markets ready to be tapped, but the realities of economics may mean they are never adequately catered to.

With few exceptions, the majority of Web-based assessment tools in actual use at Southern Cross University fall into three broad categories:

Quizzes, such as those implemented by course management software such as Blackboard [HREF8] or WebCT [HREF9]. These are used for conducting surverys, teaching and assessment.

Participation Lists, (including such things as discussion forums, mailing lists, and real-time chat), which are primarily used to facilitate interaction and discussion between peers. As participation in these forums usually accounts for a small percentage of a students' grade, they are considered to be a form of assessment, though this is not generally considered to be the case when discussing forms of Web-based assessment.

Assignment Submission, which is hardly online assessment at all, but instead uses technology to facilitate the delivery of assessment items that are created offline. However, online submission does allow the creation of richer assessment items, incorporating different media such as audio and video.

Some of the more interesting developments in the field of Web-based assessment include role plays and simulations, portfolio creation, peer ranking, and problem solving activities. The potential value of these newer forms of innovative Web-based assessments will become more apparent as further research and studies are conducted.

The reason that the more common forms of Web-based assessment are used at Southern Cross University more than others is likely related to several factors. These include:

Web-based Course Delivery at Southern Cross University

A snapshot of the Blackboard CourseInfo database of April 12, 2002 (covering the Semester 1/Trimester A 2002 period) illustrates the level of activity in relation to Web-based unit provision at Southern Cross University. It is expected that more recent data will be available in time for the conference, where it will then be presented.

In the following figures, MySCU is the term used for the Web environment built around the Blackboard platform.


Figure 1: Degree of Utilisation of the MySCU (Blackboard) Environment (from the April 12, 2002 snapshot)


Figure 2: Summary data from April 12, 2002 snapshot of Usage of MySCU (Blackboard) Environment

Some explanation of the terms used in the preceeding figures is necessary. For all units offered at the University, the following summary categories were specified, into which the units were grouped:

Fully Online - unit contains no face-to-face component. The data used does not contain sufficient information to confirm that the complete assessment process occurs online, that is, submission, marking, returning, and collation of grades. Further research will be undertaken to determine the extent to which Web-based assessment is being used in units across the university. For the purposes of this study, the data will be taken at face value - that is, all assessment occurs online.

Web Dependent - a requirement to use the Web existed; however it is not clear whether this included assessable content. Further breakdowns of this category were related to whether the unit provided educational content, interactive content, or both. The granularity of this information precludes us from determining either the use or extent of Web-based assessment in these units. For the purposes of this discussion, the assumption is made that the the use of interactive facilities may contribute to assessment, but the provision of educational content does not.

Web Supplementary - no requirement to use the Web exists; supplementary learning material only is provided. Units in this category, were considered to have not adopted the use of Web-based assessment adequately.

No Online - units with no online component.

From the snapshot summaries, it is apparent that Web-based assessment makes only a small contribution to the University's overall assessment requirements. Of the 589 units offered in semester 1, 2002, 73 (12%) were either fully Web-based or contained a required, interactive, Web-based component. Only 16 units (3%) were conducted fully online, with no face-to-face interaction required to complete the unit. It is unclear whether all these units are conducted entirely online, due to the prevalence of certain factors, such as teachers printing out online submissions for offline marking.

Details of the nature of assessment in a few of the units with a fully online or dependent Web-based assessment component are described below (this is not intended to be a definitive list).

It is interesting that note that not only are the number of units being offered fully on the Web few (3%), but the range of assessments provided in those units are fairly trivial - with most being straight transitions from traditional equivalents. The use of quizzes - currently a fundamental aspect of Web-based assessment - is virtually non-existant. There are, however, a number of other assessment methods being employed. For the most part these require more direct involvement by the teacher in the assessment process.

Distribution of Web-based units between the schools is also of interest. Of the 73 (12%) of units with the necessary level of Web-based assessment to be considered for this study, their distrubution between schools is as follows:


Figure 3: Distribution of units employing Web-based assessment between schools

Note: The School of Social and Workplace Development (SaWD) and the School of Human Services (HumServ) merged on Jan 1, 2003 to form the School of Social Sciences.

There is a widespread discrepancy here between the number of online units available within the different schools. Further research will be required to assess the reasons for this, though the presence of individual proponents and groups promoting Web-based assessment within their schools, as well as the suitability of the course material to easy online adoption, appear to be two significant factors.

It is interesting to note that both the School of Law & Justice (L&J) and SaWD each received major funding in 1998/1999 to develop online units. The School of Law & Justice spent much of the allocated funds on external contractors who developed instructional video tapes. SaWD spent the funds on staff development and professional support for creating Web pages (Ellis, O'Reilly & Debreceny, 1998; Ellis & Phelps, 1999). The results, in terms of number of online units that have survived the test of a few years operation, between the two schools, speak for themselves.

Web-based Assessment In Use

According to instructional designers who have examined the use of quizzes within the university, the majority of Web-based quizzes fail to use feedback effectively. Effective feedback involves more than just providing "right" and "wrong" responses. It can be used to encourage further thinking, correct misconceptions and provide additional knowlegde when used well. Quiz questions are predominantly taken directly from textbooks or from online question banks provided in conjunction with a textbook, or are self written. Instructors are not renoun for their ability to create good quality questions, nor do they tend to analyse the results and use this feedback to improve the quality of their quizzes. Quality control is lacking both from an individual teacher standpoint, and from the institution as a whole.

Generally, Web-based assessment accounts for only a very small percentage of a student's final grade (ie., 5%), although as has been mentioned there are some exceptions. This low level of significance of Web-based assessment is indicative of the issues that have been raised in this discussion.

Participation lists generally account for a few percent as an incentive for students to actually participate. Little real analysis of participation lists is done, and the enormous potential of this medium for self-teaching and the diagnosis of students that need more direct attention are largely unexplored. It has been found that students will use participation lists for coursework related discussion even if not being assessed on it (O'Reilly, M. & Newton, D. 2002). [HREF11]

Assignment submission is used predominantly as a facilitatory tool, and is not properly utilised due to common problems such as the difficulty of marking assignments on-screen. Teachers are also dissatisfied with the facilities of existing software to provide feedback on online submissions. The ability to attach a voice annotation or scribble some feedback is largely unavailable. For these reasons, in practice many online submissions are printed by teachers, who then take the printed work home for marking.

There is a perception that online has reverted to "shovelware", as is evidenced by the fact that fully some 48% of units have some form of online presence - but this is largely lacking in quality control nor does it attempt to incorporate current pedagogical thinking on the issues involved in moving courseware online.

The Teacher

What is the appeal of Web-based assessment to the teacher? Reduced workload is often touted as a benefit of assessment that is online, though this is not currently the case. The potential benefits of computers being able to mark assignments and respond intelligently to students' learning needs still appears to be some way from mainstream . In many cases it represents a level of complexity in terms of programming and implementation that is beyond the reach of all but a few teachers.

This type of functionality, perhaps difficult to set-up, should reduce the amount of time a teacher needs to allocate to each student, especially in non-interactive marking tasks, enabling more efficient and beneficial utilisation of a teacher's time. Also of benefit to teachers is the potential of Web-based assessment to track students, help highlight those who need extra assistance, and identify those who can participate in other forms of peer to peer learning. The benefits of this are better and more quantifiable learning outcomes.

On the down side, the effort needed to construct and keep up-to-date an online course in parallel with traditional delivery approaches, and possibly also support distance education requirements, can be prohibitive. Another drawback that was noted, is that with certain types of Web-based assessment activities, particularly those involving groups or group discussions, is that it can be more difficult to teach a small Web-based class than a traditional class of the same size. The need to motivate students more and become more involved in online discussions represented a significant investment of time. As class sizes grow, the advantages of Web-based delivery and assessment practices becomes more apparent.

There is not as yet adequate support for Web-based assessment, making it somewhat necessary for teachers to "go it alone". This is unappealing and unmanageable to most. Teachers are already finding it difficult to keep up with burgeoning teaching and research loads. Furthermore, equity issues are a major bone of contention, as is uncertainty surrounding the possibility of it being easier for students to cheat when using online assessment.

So to many teachers Web-based assessment is great in theory, but in practice it just isn't worth their time; and in plenty of cases, it is beyond their abilities to effectively implement and beyond the resources of the institution to support.

The Student

Web-based assessment can be of great benefit to students. It potentially enables them to take a test at any time from any location. Rather than one or two assessment items during a semester, a student might take eight to ten smaller tests as sections are completed as there would be no need to organise a single major test. It is possible in a Web-based environment to allow a student to take a trial test, perhaps even several times, in order to provide feedback to the student as to how well they have grasped the material; as well as providing experience with the actual test environment. This type of activity is not practical in face-to-face learning where the overheads of providing testing facilities are much greater.

However, the drawbacks for students are significant. The reality is that most units don't provide the required level of Web-based assessment to create a complete online learning experience. Furthermore, most students have a relatively low income, making access to technology difficult. If access to campus computers are necessary, the difficulties of gaining computer access in overcrowded laboratories is off-putting and annoying. This can make online activities more difficult for students to engage in than their traditional counterparts. Students are also worried about the need to learn how to use and navigate the technology as well as the need to learn the course material. With the disparate nature of online technologies between units, there can be a significant additional learning hurdle with each unit undertaken.

Students also fear the technology failing. Sending an electronic assignment into the ether is not as comforting as dropping a printed copy into a box. This fear is not entirely unfounded, as these electronic systems do suffer from outages and failures from time to time. The mantra for students in regard to assignment submission should always be - keep a copy of what you submit.

Instructional Designer

The principles of good assessment transfer from traditional to online assessment. However, the range of skills and knowledge needed by an Instructional Designer is somewhat different for traditional and Web-based formats. Instructional Designers must continually enhance their own education in order to keep up with developments in the field of Web-based assessment as they occur; in order to know what the technology can do.

With the increased spread of the use of Web-based assessment, concepts that were common within certain technical disciplines, such as using the computer as the primary workspace to complete assessments, are becoming more common within a number of other disciplines. Instructional Designers see the move to Web-based assessment as an opportunity to leverage a wide variety of academics into improving the quality of their assessments - something that may have been difficult to achieve otherwise.

Technical Infrastructure Provider

The Technical Insfrastructure Provider is concerned with trying to satisfy other stakeholders' technology requirements. It is an important role as the richness of assessment options available is determined by the course shell they implement. The Technical Infrastructure Provider's concerns arise from the need to provide the latest technology as soon as it's available, and the conflicting need of providing a solid, reliable platform that is easy to use and not continually changing.

Staff Developer

A Staff Developer is concerned with the training of staff members in adequately using the available technology. For example, training in how to use a quiz management tool would enable individual teachers with little knowledge of computers or the Web to create their own Web-based assessment tasks. The Staff Developer is concerned with the ease of use and quality of the technology product, and may be closely aligned with the Technical Infrastructure Provider, but they leave issues of pedagogy to the Instructional Designer.

Support

Over the last decade various organisational support groups have come and gone within the University in an attempt to provide a form of order to what is still a rather ad-hoc experience. At the time of writing there are two University wide groups that facilitate online education and assessment within the University. They are:

Southern Cross University On-Line (SCUOL) [HREF12]. SCUOL provide technical and infrastructure support for the online environment at the University. Another key facility that SCUOL provides is regular training courses for teachers on how to use the various features of the online environment for their courses. They are not, however, responsible for the educational content that goes into that environment, and provide no comment on issues of pedagogy.

Teaching and Learning Centre (TLC) [HREF13]. The goal of the Teaching and Learning Centre is to provide "quality of teaching and learning at the University through the provision of research-based advice and experienced professional support to staff and students." They specialise in educational design and are committed to quality in education. Although the TLC provides advice for Web-based delivery, their primary focus is on existing teaching methods.

Another group that has revelance to this discussion is the newly formed Flexible Learning Centre (FLC) within the Division of Business. Previously a project of The School of Social and Workplace Development (SaWD), they are currently responsible for assisting the School of Social Sciences (SoSS) and the School of Commerce and Management (ComMgmt) in converting new or existing units to a Web-based format. It is inarguably the presence of this group that is largely responsible for the quantity of units available online within these schools. The FLC is also responsible for maintaining the schools' web sites.

It appears that no group has a direct mission to actively promote Web-based assessment university-wide; to facilitate the migration and redevelopment of existing assessment tasks to an appropriate Web-based format; and to monitor and feedback on teaching and learning outcomes.

Issues

A number of issues have so far come to light.

Workload issues. While Web-based assessment has promised us the holy grail of greater student numbers per teacher, with better learning outcomes, the reality is the opposite. The increased workload of supporting traditional teaching methods, while at the same time trying to create and support Web-based methods, is proving tiring and demoralising for teachers. Because Web-based assessment is more demanding to develop than simply providing Web-based learning materials, it is often left behind.

Due to the lack of centralised momentum, the move to Web-based assessment is spearheaded largely by enthusiastic individuals, or "early adopters". While this is a good thing for the schools in which those individuals teach - as they often end up providing momentum and support for the entire school - other schools less fortunate miss out on these opportunities. Due to the lack of quality control over Web-based assessments, important pedagogical and equity issues may be overlooked.

Closely linked with the lack of centralised support, is the fact that there are no clear strategies or policies from University administration relating to Web-based assessment. Not only does this lead to further isolation of teachers, the added fact that there is little to no marketing provided for the online support groups that do exist often means teachers are unaware of the few resources that do exist for them.

Lack of appropriate resources are another difficulty. Existing groups involved with Web-based education generally operate on low budgets, making proactive approaches to facilitating Web-based assessment difficult to implement.

One aspect that has been only touched on in this discussion is the perception of students. Several interviews have revealed that students are not at all satisfied with the state of online education they experience. They are confused by inconsistencies and what they can expect. Units differ vastly in the quantity and quality of Web-based information and assessment that is available to students. Delivery and learning may be online, but not assessment. This appears to be the case in 74% of the units offering a Web-based component. Between schools, the issue is even more pronounced.

Conclusion

There is a remarkably low uptake of Web-based assessment practices at Southern Cross University. Although the university purports to be a leader in educational technology, this is not evident when considering the utilisation of Web-based assessment. However, even though there are low levels of activity in relation to assessment, the reasons for this are relatively clear, and the challenges easy to identify. Due to the complexity of the issues involved, Web-based coursework delivery receives much attention, but Web-based assessment falls through the cracks or is placed in the too hard basked, and assessments revert to more traditional means.

Web-based assessment has implications for a number of stakeholders across the university. The current workloads involved in moving assessment to the Web impede many teachers wishing to move in this direction. Were momentum to increase in this area, existing staff developers and instructional designers would be pressed to cope with the demand. If all units were conducted fully online, there would be implications for the technical infrastructure provider to ensure that systems could adequately meet the demand. Extra access for students in the form of laboratory computers and dial-up access would also need to be provided.

There are several support groups in existance at the university. These groups provide invaluable services, but they have a narrow focus and no close integration. For Web-based assessment methods to gain ground, specific focus must be given to this issue by the various support groups already in place.

A climate of innovation needs to be fostered if the university is to fully reap the value of its investment in new technology. Ways in which this type of climate can be achieved need to be investigated. Innovation is essential for Web-based assessment to become more relevant, and for the University to remain competitive.

Furthermore, there needs to be more definitive research into the instructional benefits of Web-based assessment methods. Known deficiencies in existing methods need to be overcome before widespread acceptance can be justified. Variations on existing techniques and new innovations that explore the ways in which assessment can be provided in the networked environment that are not possible in the face-to-face environment need to be considered.

References

Byrnes, R. (1994) The Development and Evaluation of a Computer-Aided Assignment Management System. Unpublished B.App.Sc. thesis, Southern Cross University.

Chua, K., Debreceny, R. & Ellis, A. (1995) Implementing networked learning: Gender and access issues, Conference on Higher Education and the National Information Infrastructure - From Vision to Reality, Monterey, California, (ed) E. Barnhart, EDUCOM, pp. 87-96.

Ellis, A. (1995) Networked Learning: Developments at Southern Cross University, Proceedings of the 1st North American World Wide Web Conference, University of New Brunswick, Fredericton, Canada, (eds) A. Ellis & R. Hall, University of New Brunswick, Fredericton, pp. 194-206.

Ellis, A., O'Reilly, M. & Debreceny, R. (1998) Staff Development Responses to the Demand for Online Teaching and Learning, Proceedings of the 15th Annual Conference of the Australian Society for Computers in Learning in Tertiary Education, Nov, University of Woolongong, pp.191-202.

Ellis, A., O'Reilly, M. & Debreceny, R. (1998) Teaching Using Online Technologies: A Review of Approaches to Staff Development in Australian Universities, Proceedings of the 10th World Conference on Educational Multimedia and Hypermedia, ED-MEDIA98, (eds) T. Ottmann and I. Tomek, Freiburg, Germany, 20-25th June, Association for the Advancement of Computing in Education, Charlottesville, p. 1932.

Ellis, A. & Phelps, R. (1999) Staff Development for Online Delivery; A Collaborative Team-based Action Learning Model, in J. Winn (ed) ASCILITE 99 - Responding to Diversity, 16th Annual Conference of the Australian Society for Computers in Learning in Tertiary Education, Brisbane: QUT, pp.71-82.

Ellis, A. & Sawkins. S, (2000) From Exploration to Consolidation: Ten Years of the Web at Southern Cross University, in R.Hall, J. Li., & E. Rubin (eds) NAWeb2000, Proceedings of the Sixth International North American Web-based Learning Conference, University of New Brunswick, Fredericton,pp143-148.

Ellis, A., Wildman, P. & O'Reilly, M. (1996) Migrating from Paper-based Course Materials to Interactive Web-based Multimedia Courseware, 1996 IEEE International Conference on Multi Media in Engineering Education, University of Melbourne, 3rd-5th July, (ed) M. Aldeen, IEEE, New Jersey, pp. 345-354.

O'Reilly, M. & Newton, D. 2002, Interaction online: Above and beyond requirements of assessment, Australian Journal of Educational Technology, vol. 18, no. 1, pp.57-70. Available online [HREF11].

Patterson, K., Ellis, A. & Brice, D. (2000) Client Versus Browser: A Case Study from Southern Cross University, in M. Wallace, A. Ellis & D.Newton (eds) Proceedings of the Moving Online Conference, pp190-202, Southern Cross University Press, Lismore.

Sawkins, S., Ellis, A. & Gardner, S. (2001) Changing Cars in Mid-race: A Case History of a Web-based course Management System Transition at Southern Cross University, in A.Treloar & A. Ellis. "The Pervasive Web", AusWeb01, Proceedings of the 7th Australian World Wide Web Conference, Southern Cross University Press, Lismore, pp269-283. Available online [HREF14].

Hypertext References

HREF1
http://www.scu.edu.au/
HREF2
http://www.uoregon.edu/~jqj/edtech/testing.html
HREF3
http://www2.warwick.ac.uk/services/its/elab/consultancy/assessment/web/
HREF4
http://ausweb.scu.edu.au/aw2k/papers/whittington/paper.html
HREF5
http://www.fairtest.org/facts/computer.htm
HREF6
http://www.w3.org/WAI/
HREF7
http://vchemlab.chem.byu.edu/home/index.htm
HREF8
http://www.blackboard.com/
HREF9
http://www.webct.com/
HREF10
http://www.doit.gmu.edu/Archives/fall00/jdrake_1.htm
HREF11
http://www.ascilite.org.au/ajet/ajet18/oreilly.html
HREF12
http://www.scu.edu.au/online/
HREF13
http://www.scu.edu.au/services/tl/
HREF14
http://ausweb.scu.edu.au/aw01/papers/refereed/sawkins/

Copyright

Rod Byrnes and Allan Ellis © 2003. The authors assign to Southern Cross University and other educational and non-profit institutions a non-exclusive licence to use this document for personal use and in courses of instruction provided that the article is used in full and this copyright statement is reproduced. The authors also grant a non-exclusive licence to Southern Cross University to publish this document in full on the World Wide Web and on CD-ROM and in printed form with the conference papers and for the document to be published on mirrors on the World Wide Web.