Solving some problems with University Education: Part II

David Jones [HREF1], Faculty of Informatics and Communication, Central Queensland University, Rockhampton, Australia, 4702. d.jones@cqu.edu.au

Abstract

This paper describes the experiences gained during four years of using online learning since the original paper (Jones, 1996a). It discusses what approaches have worked and which have been less than successful and offers one explanation of why this may be the case. It closes by describing the fourth generation, online offering.

Introduction

The first paper in this series was presented at Ausweb'96 (Jones, 1996a). Since then the increasing pressure to "be online" has seen the conversion of almost all of the original skeptics into people with a keen interest in the possibilities of online learning. By the end of 1998 even Vice-Chancellors were finally getting "it". Almost every new convert observed by the author, whether an individual or an organisation, has repeated some of the mistakes of made by early adopters of online learning.

This paper aims to share some of the experiences gained during four years of designing and teaching online offerings. It is hoped that this will help others avoid some of the mistakes that have been made. Hopefully, the paper will also increase awareness that simply because an innovation offers significant improvements over previous practice this alone does not guarantee success. The successful implementation of innovations in teaching and learning is a difficult task which requires significant consideration of the context in which the innovations will be used.

The paper is divided into four sections

1. Setting and background
   Describes the context in which the last four years of development has occurred.
2. Mistakes and successes
   Discusses online learning approaches which worked and those which were less than successful.
3. Explanations
   Attempts to offer on explanation of these success and failures by drawing on the some theories and literature.
4. The fourth generation approach
   Describes the design and implementation of the fourth generation of the online offering of the unit 85321, Systems Administration [HREF2].

Setting and Background

The context of these developments are advanced, undergraduate computing units offered by Central Queensland University at a number of branch and commercial campuses and via distance education. The first applications of online learning started in 1992 with the use of mailing lists. The first totally online offering began in 1996 and used both the Web and mailing lists (Jones 1996a). The use of the Web and other technology, in conjunction with changes in pedagogy and assessment have continued since 1996. This work has led to a number of publications (Jones and Jamieson, 1997; McCormack and Jones, 1997; Jones and McCormack, 1997, Cardnell et al, 1998) and the design of a tool to aid in the development of online learning (Jones and Buchanan, 1996).

The developments discussed in this and the previous paper are examples of "lone ranger" innovation in teaching and learning. "Lone ranger" developments are performed by energetic, early adopters with a lack of institutional interest and with the main aim of improving the accessibility and quality of teaching (Taylor 1998). Jones (1996a and 1996b) describe many of the problems which these developments have attempted to address.

Since 1996 the context in which these developments have taken place has changed considerably. In particular, the variety and complexity of tasks involved in managing a unit has increased considerably due to a number of imposed, institutional changes. Table 1 summarises these changes.

Table 1
Changes in context for 85321

Component	1996	1999
"semesters"	2 x 13 weeks	4 per year: 2 x 12, 2 x 6, or 3 per year: 3 x 12
Students	77: 20 at 1 campus, 57 distance	168: 47 at 5 campuses, 121 distance
Staff	"Lecturer", marker	"Lecturer", 2 "campus lecturers", 3 tutors, marker
Offerings	Once a year	Twice a year
Assessment	3 assignments and final exam	6 assignments

These changes in context have introduced a number of new problems which have influenced recent approaches. The major problems these changes have created include

• increasing complexity
  Increases in student numbers, number of teaching staff, variety of modes and offerings, variety of student backgrounds and cultures and forced changes in assessment practice has significantly increased the effort required to co-ordinate a unit.
• increasing duplication
  The multi-campus, multi-offering trend with CQU units has increased the duplication of procedures and services. Duplication of procedures, such as the submission and return of assignments and the distribution of learning materials, contribute to uncertainty about how, when and if tasks are completed. Duplication of services, such as lectures, wastes resources and further increases the problem of consistency (Jones 1996a). For example, in some CQU units individual lectures are being repeated up to sixteen times a year (4 terms x 4 campuses = 16 lectures) often by 16 different lecturers.
• decreasing time and resources
  Increasing pressure to research, the absence of assistance in teaching innovations, little recognition of changing needs and decreasing funding have all contributed to teaching staff having significantly less time and motivation for teaching. While at the same time staff are expected to work harder, especially in reducing attrition.

Other changes in context which have influenced recent approaches include

• increasing access to technology
  In 1996, 91% of distance students had access to CD-ROM and only 74% had access to the Internet (Jones, 1996a). Almost all students now have access to these technologies, particularly CD-ROM. The quality of Web publishing tools have also improved since 1996. The 1996 85321 web site [HREF3] was constructed "by hand" using the vi editor while the 1997 [HREF4], 1998 [HREF5] and 1999 [HREF2] sites were constructed using Webfuse (Jones and Buchanan, 1996). Table 2 provides a comparison of the 1996 and 1999 sites and shows the increase in size new tools have enabled.
• increasing student experience with online learning
  Most CQU computing units make use of some form of online learning. The majority of students in the current online offering completed a pre-requisite unit in 1998 which made use of online learning. This experience with online learning, the web and mailing lists reduces the need for student training. However, some of that previous experience has been less than positive and has created in some students negative feelings about the use of online learning. For example, a number of students will not subscribe to unit mailing lists due to indifferent or absent moderation in the mailing lists of previous units.
• decreasing preparedness of students entering the unit
  Decreasing entry requirements, broadening student backgrounds, the changing nature of computer interfaces and problems with pre-requisite units means that students entering advanced units have significantly different skills and knowledge.

Table 2
A comparison of Web sites: 1996 and 1999

Year	# of Files	Size
1996 [HREF3]	5430	60Mb
1999 [HREF2]	11097	1024

Mistakes and Successes

In four years of online learning a number of different online learning approaches have been tried. Each approach has its positive and negative aspects and appeals to different people. Evaluations and discussions with students have identified those approaches which were well received and those that weren't. This section briefly describes what worked (the successes) and what didn't (the mistakes) and illustrates the experience using information gathered during student evaluations. The "Explanations" section attempts to draw on theory to offer one explanation of these successes and failures.

The quotes from students are taken from student evaluations from the 1996, 1997 and 1998 offerings of 85321 and have not been edited. All of the evaluations were administered as anonymous, Web-based forms. Table 3 summarises the percentage of students who responded for each year while Table 4 breaks those responses down by the student's mode of study. The overall response to the unit is demonstrated Table 5 which summarises student responses to the statement "The new approach in 85321 is better than the normal teaching method".

Table 3
Survey responses: 1996, 1997 and 1998

	1996	1997	1998
Total # of students	77	116	107
Survey Respondents	32 (41%)	34 (29%)	40 (37%)

Table 4
Responses by study mode: 1996, 1997 and 1998

Student type	1996	1997	1998
NO ANSWER		6%
Distance student	78%	63%	66%
Rockhampton	21%	21%	17%
Sydney	N/A	6%	7.5%
Singapore	N/A	3%	2.5%
Mackay	N/A	N/A	5%

Table 5
The new approach in 85321 is better than the normal teaching method.

Response	1996	1997	1998
Strongly Agree	31.25%	24%	20%
Agree	31.25%	36%	42.5%
No opinion	12.5%	9%	10%
Disagree	15.6%	15%	17.5%
Strongly Disagree	9.3%	15%	7.5%

Mistakes

Some of online learning approaches, or perhaps more correctly their implementation in this context, suffered from a number of problems and were less than successful. These less than successful applications include

• the distribution of print material via the Web,
• the absence of on-campus lectures,
• assessable group work using a mailing list,
• the use and management of mailing lists,
• the late arrival of study material,
• and unchanging practice.

Distributing print via the Web

In 1996 and 1997 the 85321 text, at least 300 pages, was distributed via the Web to both on-campus and distance students. The major benefit of this approach was the flexibility it provided the lecturer in being able to prepare and adapt the material. In fact, it was necessary to use this approach due the inability to meet deadlines for the production of distance education material. From a purely financial point of view this approach provided CQU with significant savings on the cost of producing and distributing print based materials.

While the advantages of this approach were enjoyed by CQU and teaching staff it was the students who were burdened with the major cost of this approach. Students were required to expend time and resources in downloading and printing the material. This mismatch of cost and benefits did not contribute to positive feelings about the approach. The perception was that they were being forced to take on additional costs due to a "failure" on CQU's part.

Student comments included

I have no printer, so another student with access from work prints and photocopies notes and sends to me.

The major problem I have is not having access to the study material with out having access to my computer or spending considerable time and money to down load and print all of the study material available of the subject home pages.

Having to read the study material from the screen because I didn't want to spend 24 hours a week printing them out on a 9-pin dot matrix printer.

No on-campus lectures

During the 1996, 1997 and 1998 offerings of 85321 there were no on-campus lectures for students at CQU's branch campuses. On-campus students were expected to make use of the study material and attend weekly tutorials. This approach was motivated by perceived problems with lectures and to reduce the variety in delivery modes which was creating problems with consistency (Jones 1996a).

There was a strong perception amongst students that they needed lectures in order to learn.

Need lectures

It would be nice to have lectures, but if this is not possible then the course it structured fairly well. The use of perhaps 3 tutorials/workshops at critical times in the course would be the next best ting!

There were other student perspectives

I like the idea of not attending lectures everything can be read off the computer screen and having a discussion group is far better than a lecture

Group work

40% of the assessment for the 1998 offering of 85321 was allocated to group work. All students were placed into a group of approximately 10 to 12 students. Where possible all students were from the same geographical area. Each group communicated via their own mailing list and there were a number of set tasks the groups had to complete including

• weekly summaries
  Each week one student was responsible for summarising the group's discussion and posting that summary to the main 85321 mailing list which included the entire class.
• discussion questions
  The study material for most weeks included a discussion question designed to get the groups discussing a particular topic.
• documentation assignment
  Towards the end of the semester each student had to write a piece of documentation explaining some technical task. The documentation was first distributed to their group for comments before being marked by the lecturer. Comments from the group members formed part of the final mark for the documentation.

While some students enjoyed this approach.

Have more "Group" assignments.

on-line/group method is a great way to learn. Being part of a group and knowing people are studying, having the same problems, helping each other, etc, all made the unit more enjoyable particularly when your a distance enducation student. The interaction was great... now I wish I was full-time and able to experience the same for my other subjects.

There was a significant disquiet amongst a number of students

enjoyed the subject but fell behind when I had to dedicate a lot of time to the assignments. I don't like having to rely on group members with assignment work, prefer working by myself, that way I can only bring my marks down.

Group participation was a problem in my group but that isn't something lecturers can fix.

felt too many marks were allocated to group discussion and that more marks should have been allocated to assignments. It is not possible to come up with answers to review question as a group using email. These should have been submitted individually. Those srudents connected to the email permently (we had one of those in our group) were able to submit a greater volume of email and as volume was taken into account when marking group discussion then these people had a distinct advantage.

However the unrealistic work load and the non relevant item (group work) destroyed any enjoyment that should have occurred.

there was a high proportion of marks on group work. I understand why this was so and agree whole heartily with the motivation, but the implimentation does not coalasce with the difficulties of merging the conflicting interests of external students. Our demands are great ... and it is impossible for us to consistently allocate the required number of hours EACH week. .... there is always someone who is being waited on to continue with the assignment.

On-line communication for the purposes of assignment co-ordination (group activities) just doesn't work (our office came to this decision 12 months ago!)

I was REALLY p****d off with one of the members of my group. 3 of us went GREAT and one left us in the lurch TWICE!!!!!

Mailing Lists

Associated with the problems with the group work were problems with mailing lists. Mailing list problems included

• getting the right size
  

  Experiments with list size in different offerings of 85321 have seen the use of one unmoderated mailing list for all students, individual lists for small groups groups and a combination of both. No one approach has been completely successful with a number of students always asking for some alternative implementation.

  • Group mailing list was excellent.

    I feel that combining all the students in one large group would be a much better idea. This is because the problems would then be answered faster by more people.

    Our group size was to small so the group size need to be look at so that they stay at 10 members. The main mail list needs to remain in operation.

    Only have access to your own group list, with David sending on anything else that is relevant.

• encouraging good practice
  Moderation of mailing lists in previous units has been limited. This means that many students are not familiar with good list behaviour which results in a number of problems. The 1998 mailing list for the entire was so bad it required moderation
  • Main mailing list was hopeless. Initially to much garbage appeared (which was later rectified) which should have appearred in the individual group lists.
• fear about participation
  Posting to a list with a large population can be frightening for some students. In the 1995 offering of 85321 there was one mailing list for the entire class. One of the 1995 students made the following comment
  • Most on the list would be scared to take part.

  In the 1996 offering of 85321 a number of the students were actually professional Systems Administrators studying to get their "piece of paper". The presence of professionals effected at least one student who made the following comment

  • The mailing lists are a good idea. When I found out there were SA's doing the subject, I was reluctant to ask questions. Silly I know !

Unchanging practice

The innovative approaches discussed here have not been aided by appropriate changes in institutional policy and procedure. Infrastructure, funding and workload allocation policies for teaching and learning are still targeted at traditional teaching and learning methods. With no support or recognition of new approaches.

For example, calculations for part-time staffing are based on the number of on-campus and distance students enrolled in a unit. On-campus students are "worth" considerably more than distance students since they are assumed to consume more staff time in face-to-face lectures and tutorials which aren't available to distance students. An approach using the Internet to enable group work and interaction for all students receives no additional funding. This leaves staff having to perform additional work especially when distance students are three quarters of the class.

Other examples of institutional policy and practice not changing to meet the needs of new approaches include

• materials production
  Still targetted at producing print-based distance education the current infrastructure provides no support for CD-ROM or Web site production. The lack of support places the burden for all aspects of materials production onto the teaching staff and has contributed to the late arrival of material such as CD-ROMs and textbooks.
• dialup Internet access
  CQU provides limited dialup access to students at each of its branch campuses which is significantly less than required resulting in frustrations for students.
  • It is quite often very difficult to connect to the Uni via the bundaberg campus modem bank due to the following reasons. a. insufficient lines. Most times it is busy and you cannot get a line. b. lines not working properly. Sometimes one of the modems in the hunt group would lock up or be turned off which causes the phone line to ring out rather than pass the call to the next modem in the line. c. Once you got a modem to answer, more often than not the connection would hang and you would need to redial several times befor it would work properly.
• computer labs
  The same funding restrictions and associated problems which prevent the provision of better Internet access limit the quantity and quality of computers available in student computer labs. For example, the absence of sound cards in most computer labs makes innovations such as online lectures less useful.

Unchanging practice within the management of the unit has also caused problems. The main example of this is assignment marking. The adoption of online assignment submission drastically changes the nature of assignment marking. It is necessary for markers to change practice in order to take advantage of the medium. Problems in changing existing marking practice has created a number of problems including limited comments and slow turnaround times.

NEED BETTER FEEDBACK ON ASSIGNMENT RESULTS.When we get assignments back, there's next to no comments/explanation of why we got the mark we did. Need more info on this - VERY IMPORTANT.

Return of assignment results could be quicker.

Successes

Online learning approaches which consistently receive the most positive student comments include

• online lectures
• online assignment submission
• CD-ROM mirror of web sites
• increased interaction

CD-ROM Mirrors of Web sites

The 1998 offering of 85321 was the first year in which a mirror of the web site was distributed to students on a CD-ROM. The advantages of distributing Web materials on CD-ROM include high speed delivery of graphics and other large files, no online connection charges and materials can be used even when there are no Internet connections (Cardnell et al, 1998). This is especially important for students who pay for Internet access by the hour, have less than good connections or are unable to access the Internet.

It also allows staff to concentrate on continual development of the Web site as the primary multimedia interface to a unit. Once a new semester approaches a simple program mirrors the current state of the Web site and allows it to be placed onto CD-ROM without any change in web authoring practice.

I think the CDRom is a good idea as I could not have used the resources to the full extent if I had to do it online.

Table 6 summarises 1998 student responses to the statement "The mirror of the 85321 Web site on CD-ROM was very useful".

Table 6
The mirror of the 85321 web site on CD-ROM was very useful.

Response	Percentage
NO ANSWER	2.5%
Strongly Agree	23%
Agree	51%
No opinion	10%
Disagree	12%
Strong disagree	1.5%

Online lectures

Online lectures are the most recently adopted online learning approach, first being used in the second half of 1998, and are one of the best received approaches. Online lectures are lectures recorded and digitised, usually before the start of semester, and distributed via the Web (and CD-ROM mirror) along with slides and in some cases animations. This approach is similar to that described by Smeaton and Crimmins (1997).

This approach offers a number of advantages including

• increased flexibility
  Students are able to listen to and revise the online lectures when and where they please. They are no longer tied to a single location and time each week.
  • the online lectures with audio are the best way to understand the material being able to listen to David go over the concepts is very helpful infact if you don't get it first time you can go over it again

    I do attend lectures occasionally at the Rockhampton campus they are not a patch on being able to sit at home in comfort and keep going over the material until you do understand it

• increased accessibility
  Previously lectures were only available to on-campus students (about one quarter of the class in 85321 in 1999). Online lectures are now also available to all students including distance students.
• increased variety
  Previously distance students were limited to text as the primary source of learning. Online lectures provide an alternative medium for distance students.
  • Additional study material is always helpful fordistance ed students. Audio provides way more info than purely written stuff can.

    Listening to a voice was excellent - hearing the lecture instead of just reading it on the screen was the best part.

• increased consistency
  Traditional practice requires that students at the Sydney and Melbourne campuses (managed by a commercial partner) receive lectures. This leads to lectures being given by numerous different people often with little or no consistency. Online lectures offer an approach where lectures are given by a single individual of known "quality".
• meeting expectations
  Regardless of what educators say about the educational value of lectures most students expect them and some believe it is the best way for them to learn. Online lectures, while removing the face-to-face contact for on-campus students, do meet student expectations of having lectures. Even distance students, who previously had no access to lectures, believe they now have a large advantage because they have access to lectures.
• an opportunity for change
  The most exciting advantage is the potential that online lectures can enable the adoption of more effective teaching approaches. Once created, the online lectures free up staff resources traditionally used to deliver lectures. Those resources can now be utilisied to increase student/staff contact.

  I definately used them instead of Face-Face lecturers, and started off reading the entire textbook reading and then listening to the lectures. I then moved onto to using the two at the same time - and skipping over a little of the text. I found that I was taking enormous amounts of notes from the online lectures which really helped

Online assignment submission

1999 sees the development and use of the fourth generation of online assignment submission system in 85321. The third (Jones and Jamieson, 1997) and fourth generations offer considerable improvements over the approach used in the 1996 offering (Jones, 1996a). In 1997 and 1998 over 80% of students responding to surveys considered the online assignment submission process to be better than traditional methods. With appropriate computing resources markers report that online marking takes between 20-30% less time than marking paper based assignments (Jones and Jamieson, 1997).

An important lesson from the on-going development of online assignment submission is to reduce the amount of "meaningless freedom" available to students. Early systems relied on students submitting assignments via email attachments. The freedom to choose file formats, mail programs and types of attachments significantly increased the amount of work required to mark assignments. Moving to a Web-based system where student freedom is reduced to choosing which file to upload was a significant improvement.

Student comments about the online assignment submission system have included

No concerns about postal delays or loss of time to complete due to the same. As a distance education student I loose approximately 2 weeks a semester in submission time and response time on assignments.

Faster (no matter where you are) and cheaper.

Feedback is much quicker than the traditional paper-based submission.

Easier to submit.Less worries about the post.

Instant acknowledgement, quicker turnaround when resources are available for quick marking. Also when results are posted to a web site you can compare your progress with others and see how you are going.

Far, far, FAR more convenient, and fast.

MUCH easier, cheaper, reliable

online assignment submission is great, I usually send assignments express post so the money saved pays several times over for the paper used printing study guide chapters

Table 7 summarises students responses to the statement "Web-based assignment submission is better than paper based submission".

Table 7
Web-based assignment submission is better than paper based submission

Response	1997	1998
Strongly Agree	48%	55%
Agree	33%	32.5%
No opinion	6%	7.5%
Disagree	3%	2.5%
Strongly Disagree	9%	2.5%

Increased interaction

While causing a number of problems and at times being very frustrating one of the most successful aspects of the use of online learning is the increasing interaction (both student/student and student/teacher) it enables, especially for distance students.

I think I have gained a lot more from this subject than just UNIX, espec. with the interaction for first time in 8 years of external study!

Definitley nice (and helpful) to have contact with others for discussion. Studying externally can be difficult without this. Archives fantastic as it saved having to waste memory by storing thwem on my machine, and format made it easy to locate a particular message when needed.

The group camaraderie and the dissipation of isolation that resulted.

Questions and responses from other students. This gave a good indication as to where my standard should be in relation to others. This is where normal distance students miss out.

...through your study groups last semester I gained an Email buddy (What do you call someone you do not even know, but communicate with on Email on a friendly basis?) ... who has been a God-send in helping point me in the right direction at times and lending me books!

Email contact with lecturer is great - usually immediate feedback when contact is made.

I have found the mailing list very important for progressing in this subject. If anything sometimes there is information overload - but this is not a problem.

Explanations

This section attempts to use some of the available literature and theories as a framework to explain why some approaches described here have worked better than others. This framework is also being used to guide the ongoing development of online learning. There is limited space so a full discussion of the theories and literature which have been useful is not possible. Instead this section will concentrate on a particular theory, diffusion theory (Rogers, 1995), which seems to offer one possible explanation.

Other theories and literature which are not discussed here but have been influential include

• constructivist related theories of learning (Jonassen et al, 1995),
• the literature on computer-mediated communication (Harasim, 1990)
• the no significant difference phenomenon
  Both the for (Russell, 1999) and against (Cater, 1996) cases.
• organisational computing (Kling 1996)

Diffusion Theory

Most of the literature and work in online learning deals with the justification, description or reflection on the implementation of innovations in teaching and learning. There is a growing realisation that many of these innovative products and practices, which appear to provide numerous advantages, still suffer from a lack of utilisation (Surry and Farquahar, 1997). The burning question is why do people reject an innovation which is educationally sound and improves the teaching/learning experience?

The process of trying to answer this question has seen the author move from being a

• technological determinist
  One who assumes that others will see the benefits of a particular innovation, whether it be the use of the web or of constructivist approaches, and based solely on those benefits be persuaded to adopt the innovation.
• to a technological instrumentalist (Surry and Farquahar, 1997)
  Finally coming to a realisation that the diffusion of innovations depends strongly on the context in which a potential innovation is to be used. That people with different backgrounds and needs will perceive an innovation differently and these perceptions, in conjunction with other factors, will influence whether or not they adopt the innovation.

A theory which has contributed to this change and which seems to provide an answer to the adoption question is diffusion theory (Rogers 1995). Diffusion theory grew out of work in rural sociology and in particular a 1943 study by Ryan and Gross at Iowa State University (Rogers 1995). Since that time over 3800 papers on diffusion theory have been published in a range of fields including education, marketing, anthropology, public health and rural sociology (Rogers, 1995, pp 43). Possibly the most important publication in diffusion theory is the book "Diffusion of Innovations" (Rogers 1995), currently in its fourth edition. This book synthesises findings from many of these studies to arrive at a theoretical framework for looking at the diffusion of innovations.

That framework defines diffusion as "the process by which an innovation is communicated through certain channels over time among the members of a social system" (Rogers, 1995, p5). The diffusion theory framework involves four main elements: innovation, communication channels, time and the social system. Each of these elements influence the adoption or rejection of an innovation.

Given current constraints on space a full exposition of diffusion theory as it applies to online learning is not possible. Instead the following concentrates on one element of diffusion theory, the perceived characteristics of an innovation, and how it could be used to explain some of the success and failures discussed above.

Perceived characteristics

Concentrating on the characteristics of an innovation may seem to be falling back into the technological determinist mind set. The major difference is that diffusion theory is interested in how each individual in a social system perceives the innovation not the "actual" characteristics of the innovation. It recognises that individuals will perceive an innovation differently and that these perceptions will influence the likelihood of adoption. Rogers (1995, p15) identifies five perceived characteristics of an innovation which influence adoption

1. Compatibility
   The degree to which an innovation is perceived as being consistent with the existing values, needs and past experiences of potential adopters.
2. Relative advantage
   The extent to which a potential adopter views the innovation as offering an advantage over previous methods.
3. Complexity
   The degree of difficulty potential adopters believe they will have in understanding or using the innovation.
4. Trialability
   The ease with which potential adopters can experiment with the innovation on a limited basis.
5. Observability
   The degree to which the results of an innovation is visible to others.

Research studies confirm that an innovation which is perceived as having high compatibility, relative advantage, trialability and observability while having low complexity is more likely to be adopted. Tornatzkey and Klein (1982) found that relative advantage, compatibility and complexity are the most significant factors in explaining relationships across abroad range of innovations. Surry and Farquhar (1997) report on a number of studies which confirm the links between relative advantage, complexity and compatibility and the adoption of innovations in education.

Table 8 summarises how teaching staff, on-campus students and distance students perceive the characteristics of some of the innovations discussed in this paper. It focuses on the most significant characteristics of relative advantage, compatibility and complexity. The perceptions of assigned to students in Table 8 are not based on specific research but on observations drawn from student evaluations and discussions.

Table 8
Perceived characteristics of innovations in 85321

Innovations	Relative Advantage	Compatability	Complexity
Mistakes	Staff campus students Distance students	Staff campus students Distance students	Staff campus students Distance students
Distributing print via Web	High Low Low	High Low Low	Low High High
No on-campus lectures	High Low No change	High Low No change	Low Highish No change
Group work	Low Low Low	Low Low Low	High High High
Successes	Staff campus students Distance students	Staff campus students Distance students	Staff campus students Distance students
Online lectures	Highish Highish High	Highish Highish Highish	Highish Lowish Lowish
Online assignment submission	High Highish High	High Highish Highish	Low to High Lowish Lowish
Increased response	Low/High Highish High	Highish Low/High Low	Low Low Low

It should be noted that "bad scores" on these criteria does not mean that an innovation is by some measurement "bad" or destined to fail. A "bad score" indicates that an innovation is not likely to be adopted without significant effort on the part of a change agent.

It appears from the combination of the experience reported above and the perceptions summarised in Table 8 that there is some match between perceived characteristics and the likelihood of adoption. Those innovations which are perceived to be successful generally show high relative advantage and compatibility in combination with low complexity for all participants. While innovations which are perceived to be failures tend to have perceived characteristics which reverse that trend.

Innovations, adoption and effort

Group work, as implemented in the 1998 offering of 85321, is a good example of the problems with the adoption of innovations in teaching and learning. There is agreement in the literature that collaborative activities among students provide clear educational advantages (Oliver, Herrington and Omari, 1996). There is also considerable literature providing advice on how to implement such collaborative activities in an online environment (Harasim, 1990) of which the author is familiar. In fact, while designing the 1998 approach the author was writing a book chapter on enabling communication in a Web-based classroom (McCormack and Jones, 1997). If this is the case then why was the 1998 approach less than a success?

Given that group work, in this context, had the following perceived characteristics

• low compatibility,
  Even thought collaborative, group work is a major requirement of the computing profession existing teaching practice in computing units at CQU includes very little collaborative group work (Jones 1996a). This means it is unfamiliar to students and staff. This is particularly true for distance students, over three-quarters of the class, who are used to significant independence, isolation and flexibility in their study.
• high complexity,
  Successful implementation of collaborative, group work in an online environment is a difficult and time-consuming task (McCormack and Jones, 1997) which leads to staff perceptions of high complexity. Students have a similar perception given the difficulties of group work, especially when using mailing lists as the main medium.
• and low relative advantage
  While collaborative, group work does provide advantages in improving the learning it is suggested that this advantage is over shadowed by other difficulties. A significant proportion of students wish to achieve good grades with a minimum of effort while collaborative, group work requires significantly greater work from students. For staff finding themselves pressured to research and reduce the amount of effort involved in teaching collaborative, group work provides little advantage.

It is not difficult to see that feelings towards collaborative, group work from both students and staff are not likely to be positive. Changing those feelings would require a significant amount of effort. Given the context of increasing complexity, decreasing funding and unchanging practice that effort is unlikely. On reflection, while the group work was somewhat successful it did not receive sufficient effort to ensure widespread success in this context.

The lessons

The social context in which innovations take place are more important considerations than the advantages an particular innovation offers. Innovations should not be promoted or accepted simply because they offer theoretical advantages but because they are likely to be useful in a particular context. It is suggested that innovations should be evaluated based on the characteristics of each individual class. What works well in one situation will not work well in all situations.

Revolutionary change requires significant effort to ensure success and adoption. Such effort is typically not available for most University based implementations of online learning. Evolutionary change, concentrating on the development of innovations which aim for high compatibility, high relative advantage and low complexity, is more likely to be successful and require less effort. Such evolutionary change can be used as the foundation for eventually achieving more radical change.

This is the philosophy driving the continued development of online learning approaches in 85321.

The Fourth Generation

This following section describes the approach being used in the fourth generation of online offering for 85321 in 1999. It starts with a brief discussion of the aims which have driven the design of the approach before giving an overview of the major components of the approach.

Aims

The main aim of the unit is to allow students to gain a sound understanding of the tasks and responsibilities of Systems Administration. It is intended that students achieve this understanding through performing realistic tasks.

In keeping with the framework offered by diffusion theory all new approaches should, where possible, be perceived by students and other teaching staff as being high in relative advantage, compatibility and low in complexity. It is hoped that this will encourage adoption with a minimum of work from the author. The 1999 approach to 85321 is evolutionary. It attempts to reuse and slightly improve upon previous successful approaches and address some of the previous problems.

Approaches

The major components of the 1999 online learning approach include

• Web-based assignment submission [HREF6]
  This is the fourth generation, online, assignment submission system and is tied in with the University's student database to provide personalised access.
• Online lectures [HREF7]
  The first five weeks of lectures were produced prior to the semester and made available on the Web site and CD-ROM. The lectures are designed to supplement existing material, provide alternatives and encourage activities.
• Continual, flexible assessment [HREF8]
  The assessment for 1999 is summarised in table 9. It is designed so that students can complete at their own pace, independent of any one else. To discourage cheating randomly selected students will be asked to defend a piece of their assessment by 85321 teaching staff. The student's final mark will be modified based on that defence.
• CD-ROM mirror of the web site
  Students are able to purchase the CD-ROM mirror of the web site for use at home while disconnected from the Internet.

Table 9
85321 Assessment for 1999

Assignment	Worth	Due	Description
Log book	20%	Weeks: 2, 4, 6, 8, 10	Reflective log book with set sections including a record of work completed
Submission Questions	30%	Weeks: 2, 3, 4, 5, 6, 7, 8, 9, 10, 11	Selected exercises from the text. Exercises from 3 randomly selected weeks are marked.
Shell programming	15%	End of week 7	Traditional "solve a problem" with a program assignment
System Emergencies	15%	Weeks 8, 10 and 12	A program is executed by the student. It creates a problem with their Linux system which the student must diagnose and solve.
Laboratory	20%	Week 14	Students are set a realistic Sys Admin task which they must implement. The tasks draw on material from throughout the semester.

The particular problems the 1999 approach seeks to address includes

• unchanging administrative procedures
  All assessment is due by the close of business on Friday. It is intended that the marker will mark assignments over the weekend and return them by monday or tuesday. Administrative procedures at the organisational level have not changed.
• poor Internet access
  The provision of the CD-ROM mirror and no group work significantly reduces the need for students to have Internet access.
• duplication and complexity due to variety
  The provision of the online lectures, which are available to all students, reduces duplication of lectures and further reduces the differences between students. Using the online assignment submission system for all students removes the need for different processes for different types of students.
• surface learning
  Continual assessment, the requirement to defend assessment and the use of problem-based learning are all attempts to address this particular problem.
• expectations of students
  On-campus students expect to have lectures, to see their lecturer. It is hoped that the provision of online lectures and four scheduled "face-to-face" lectures will fulfill part of this expectation. The "face-to-face lectures" are question/answer sessions scheduled and held in a normal lecture theatre once every few weeks.

Conclusions

The literature shows that innovations in online learning and pedagogy offer a range of advantages which can offer a number of significant improvements to teaching and learning. Experience over the last four years confirms this observation. However, just because an innovation is "good" its successful implementation is not guaranteed. Innovations, particularly those that require significant departures from previous practice, are difficult and require significant effort to guarantee success.

In the author's experience evolutionary change, which concentrates on the context of the innovation and attempts to provide high relative advantage, high compatibility and low complexity for all participants, is more likely to succeed, will require less effort and provide a foundation for future developments. This is particularly true in contexts, such as Universities, where there are insufficient resources available for the implementation of online learning.

References

Cardnell, D., Jones, D., Stewart, S. and Aldred, S. (1998), Providing alternatives for distance education students, In W. Au, R. Geer & B. White (Eds.), Where is IT&T; at? Proceedings of the Australian Computers in Education Conference 1998

Carter, V. (1996). Do media influence learning? Revisiting the debate in the context of distance education. Open Learning, Feb. 1996, 31-40.

Harasim, L. (Ed.). (1990). Online Education: Perspectives on a New Environment. New York: Praeger Publishers.

Jonassen, D., Davidson, M., Collins, C., Campbell, J., and Haag, B.B. (1995). Constructivism and Computer-Mediated Communication in Distance Education. The American Journal of Distance Education, 9(2), 7-26.

Jones, D. (1996a). Solving some Problems of University Education: A Case Study. In R. Debreceny & A. Ellis (Eds.), Proceedings of AusWeb'96 (pp 243-252). Lismore, NSW: Norsearch.

Jones, D. (1996b), Computing by Distance Education: Problems and Solutions, In G. Davies (Ed.), Proceedings of the 1st Conference on Integrating Technology into Computer Science Education (pp 139-146). ACM Press.

Jones, D. and Buchanan, R. (1996). The Design of an Integrated Online Learning Environment, Making New Connections, In A. Christie, P. James & B. Vaughan (Eds.), Proceedings of ASCILITE'96 (pp 331-345).

Jones, D., Jamieson, B. (1997). Three Generations of Online Assignment Management. In R. Kevill, R. Oliver & R. Phillips (Eds.), What Works and Why, Proceedings of ASCILITE'97 (pp 317-323).

Jones, D., and McCormack, C., (1997). Class Management: The Forgotten Task. In H. Greenberg & R. Hall (Eds.), Shortening the Distance to Education, Proceedings of the 3rd International North American Web Conference (pp 109-125).

Kling, R. (1996). Computerization and controversy: value conflicts and social choices (2nd Ed.). San Diego: Academic Press.

McCormack, C., Jones, D. (1997). Building a Web-based Education System. New York: John Wiley & Sons.

Oliver, R., Herrington, J., Omari, A. (1996). Creating Effective Instructional Materials for the World Wide Web. In R. Debreceny & A. Ellis (Eds.), Proceedings of AusWeb'96. Lismore, NSW: Norsearch.

Rogers, E., (1995). Diffusion of Innovations. (4th ed.). New York: The Free Press.

Russell, T. (1999). The No Significant Difference Phenomenon. North Carolina State University. [HREF9]

Smeaton, A.F. and Crimmins, F. (1997). Virtual Lectures for Online Lectures: Delivery Using RealAudio and the WWW". in Proceedings of ED-Media/ED-Telecom (pp 990-995).

Surry, D.W., & Farquhar, J.D. (1997). Diffusion Theory and Instructional Technology. Journal of Instructional Science and Technology. 2(1).

Taylor, P. (1998). Institutional Change in Uncertain Times: lone ranging is not enough. Studies in Higher Education. 23(3). pp269-278.

Tornatzky, L.G., & Klein, K.J. (1982). Innovation Characteristics and Innovation Adoption-Implementation: A Meta-Analysis of Findings. IEEE Transactions on Engineering Management. 29(1). pp 28-45

Hypertext References

HREF1

http://cq-pan.cqu.edu.au/david-jones/

HREF2

http://www.infocom.cqu.edu.au/85321/

HREF3

http://www.infocom.cqu.edu.au/Units/aut98/85321/Old_Stuff/1996_Website/85321/

HREF4

http://www.infocom.cqu.edu.au/Units/aut98/85321/Old_Stuff/1997_Website/index.html

HREF5

"http://www.infocom.cqu.edu.au/Units/aut98/85321/

HREF6

http://www.infocom.cqu.edu.au/85321/Assessment/Submit/

HREF7

http://www.infocom.cqu.edu.au/85321/Resources/Online_Resources/Lectures/

HREF8

http://www.infocom.cqu.edu.au/85321/Assessment/

HREF9

http://teleeducation.nb.ca/nosignificantdifference/

Copyright

David Jones, 1999. The author assigns to Southern Cross University and other educational and non-profit institutions a non-exclusive licence to use this document for personal use and in courses of instruction provided that the article is used in full and this copyright statement is reproduced. The author also grants a non-exclusive licence to Southern Cross University to publish this document in full on the World Wide Web and on CD-ROM and in printed form with the conference papers and for the document to be published on mirrors on the World Wide Web.