Jim Woulfe, Instructional Designer, aclEnglish, acl, 157 - 161 Gloucester Street, The Rocks NSW 2000. Email: jimw@acl.edu.au
Finding a learning management system that matches the needs of an educational content provider is often seen as a difficulty. In the collaboration between acl, a provider of English language courses, and NextEd, a provider of educational infrastructure, much effort has been made to align the delivery system and the content. This enquiry examined the changes which were made by both partners to improve the delivery of the content, and asked key participants about their satisfaction with the outcomes. Overall the level of satisfaction was high, though more so for educational and delivery outcomes than for organisational ones. The close collaboration of the two organisations has resulted in significant progress in improving the delivery of the educational content.
It is possible that, faced with the task of re-configuring a set of learning materials for a new delivery mode, every educator thinks that the chosen delivery system was designed to meet different learning challenges to their own. Many learning content providers report difficulties finding a learning management system that matches their needs (Clouston & Wenmoth, 2003; Online Learning News and Reviews, 2004 [HREF1]). Hedberg (2003) describes the purchase of an LMS as "the first of many decisions educational managers must take," and then goes on to point out that a mismatch between educational aims and educational software can have costs in terms of educational outcomes. Hampel (2002) claims that much educational software is more supportive of content authors and designers than it is of student learning, and others (cf. Farmer, 2002 [HREF2]) explore ways to enhance the educational value of existing learning management systems. Possibly, for many institutions the relationship between the educational content and the delivery platform is an uneasy one.
This is partly borne out in the relationship between NextEd and acl, the author's employer: there have definitely been some challenges in their collaboration.
It is difficult to find accounts of the process of resolving such challenges. The purpose of this paper is to provide an account of how the collaboration between delivery partner and content provider enabled ongoing improvement in the effective delivery of a suite of language learning materials.
acl is Australia's largest private language teaching organisation. It teaches English language and communication skills to migrants and refugees, as well as General English and English for Academic Preparation (EAP) to overseas students. In addition, acl has offshore language teaching centres in Vietnam and Thailand, and participates as a joint venture partner in several centres in China. aclEnglish is currently used by clients in ten countries.
In 2000, the decision was taken that acl's most popular courses for overseas students would be re-built for flexible delivery both offshore and onshore. NextEd was chosen as the partner for delivery of this content: it was able to offer a robust delivery platform (the NextEd Delivery System) and it is a participant in the Global University Alliance.
acl is a unique client for NextEd, in that the content and delivery model is radically different to a university-based model of delivery. Some of the relevant characteristics of the aclEnglish language courses are:
According to the respondents, this contrasted greatly with the typical course that NextEd was delivering for its other, mainly university-based clients.
In 2001 and 2002 fifteen separate courses were written and implemented, a total of 1320 hours of instruction. The author is one of the two instructional designers involved in this project.
The courseware is designed for online, face-to-face or mixed-mode delivery:
Figure 1: aclEnglish resources
The material includes extensive teachers' notes, intended to support native- or non-native-speaker teachers in delivering the materials in the mode chosen by their institution.
aclEnglish is provided in a business-to-business model between acl and the client organisation, with NextEd as the host. Client organisations can use one course, or the whole suite, offering students a pathway (see Figure 2) from beginner level to further tertiary study at an English-speaking institution.
Figure 2: aclEnglish Pathways
The NextEd Delivery System has four main areas (see Figure 3). This paper is concerned with the two innermost areas, the LMS, and the User Information System:
Figure 3: NextEd Global Knowledge Extranet
The core is Blackboard 4.08, in which all the aclEnglish quiz content is stored. Approximately 30% of the content consists of Blackboard quizzes, discussions, and tasks related to the student web page feature. The remaining 70% of the content consists of static html pages containing images, sound files, Flash files, and RealPresenter presentations. The interface will be familiar to many:
Figure 4: aclEnglish course content
The User Information System provides administrators with tools for enrolling and tracking students. It also provides students with a space for note-taking and file storage, and tools for tracking their enrolments, changing their personal details, and setting their personal preferences in the system.
Figure 5: User Information System - this screen shows the enrolled courses
The information for this account was gathered by putting some open-ended questions to six key acl and NextEd participants about the changes they had seen in the delivery system during the association of the two organisations. Initially these questions were put by email. Clarification was sought by telephone, by email and in person.
Open-ended questions were used in order to elicit as wide-ranging a set of responses as possible (see appendix).
There were three respondents from NextEd, and three from acl. All responses were useful. The author was a key participant in the development process, and in the collaboration between NextEd and acl. Being outside the purpose of this investigation, the views of student users were not sought.
The remainder of this paper is an account of problems which were identified in the delivery of aclEnglish, and the steps that were taken to resolve them. The issues are described roughly in chronological order, as they arose.
During the development phase, when courseware was being written and implemented in the NextEd Delivery System, efforts were made to become as familiar as possible with the system, and to design and write materials which achieved the educational outcomes within it. Nevertheless, the earliest problem which arose not only related to the instructional design of the courseware, it has also proved to be unsolvable in the current system.
After the first courses were uploaded, it was discovered that feedback on quiz items in the Blackboard core of the materials was limited to 256 characters. It became necessary to go back and re-write the feedback for all the quiz items in the first few courses, so that they were under 256 characters.
In order for users to click directly from a quiz to the next activity, it was necessary to put the link into the feedback field, further reducing the number of characters available for actual feedback.
Figure 6: The feedback for the last item in each quiz is very limited
In fact, in many cases, because the URL of the link is so long, there was only space to indicate to the student if the response was correct or incorrect (the feedback for question 9, shown in Figure 6, is 251 characters). This only applied to the last item in a quiz, so it was necessary to re-arrange quizzes to ensure that items requiring more than absolutely minimal feedback didn't appear last.
The early problems were annoying, but not too difficult to work around. A much larger problem that arose very quickly was related to student enrolments. At the outset, it was expected that students would register themselves in the NextEd Delivery System, and enrol themselves in the correct courses. acl faced several difficulties with this. First, students with limited English-language skills were unable to complete the registration and enrolment. Second, students needed to do a Placement Test to determine which course they should participate in, but couldn't do this test until they were registered in the system.
NextEd resolved this problem by initiating a batch-enrolment procedure, whereby client institutions could send a spreadsheet file with the student's identifier and a generic password. Initially this batch registration and enrolment was done by NextEd, but later this was refined so that it became part of the User Information System, and the batch registration and enrolment could be done by the aclEnglish Administrator.
Where a typical course at a university might be published once or twice a year, aclEnglish courses needed to be re-published each time a client commenced a new class (this is referred to as an 'offering'). Each offering has a unique URL, which includes a client code and the course date, and this URL had to be hard-coded into every internal link in the course. Initially this meant that each of the links to the next activity had to be manually edited every time a new offering was uploaded.
In the Certificate IV in English for Academic Purposes (EAP) Part 1 course there are 14 such links, and in General English Level 2 there are 25. Although this might seem to be small amount of work, in the two years since the course was first uploaded, there have been 86 offerings of Certificate IV Part 1, so the cumulative effect would have been considerable.
NextEd solved this problem by creating a multiple-file find-and-replace tool which could be used to update the links in each new offering. Later they introduced an automated LMS redirect script which enabled links in the aclEnglish content to point to the name of the Blackboard quiz, rather than to the alphanumeric ID generated by the LMS database. This reduced the time required at acl to upload a new course from 3 days to half a day, which (given the number of courses being uploaded) represented considerable savings.
The first client of aclEnglish requested that the courseware carry their own branding, which meant that from the start a method was needed to replace the aclEnglish button set with a client's own button set. NextEd enabled a process to swap the button set during the publishing process.
The responses from NextEd personnel all mentioned the volume of acl's content, both in terms of the number of courses (15 in total), and the amount of data they contain, particularly as audio files (see following).
At the core of the problem with the volume of content is the amount of audio in aclEnglish courses. In total there are approximately forty hours of listening materials in the courseware. Currently, most of this audio content is streaming RealAudio, though there is also a considerable amount delivered in Flash files. The English for Academic Purposes (EAP) courses make extensive use of RealPresenter files, which synchronise audio with presentation slides.
The audio content caused several problems:
The difficulties with audio delivery became a focus for clients who were having problems with the courseware, so a solution needed to be found urgently. A number of steps have been taken to improve audio delivery.
First, NextEd developed a script allowing acl to offer the audio to clients over the internet, as well as on the client's own network server. This required an extra button to be inserted into each page linking to a RealAudio file, so that students could choose to access the audio locally, or over the internet.
Figure 7: Users choose to access audio locally or over the internet
Second, NextEd developed a script allowing each offering of a course to access the same media files. This ended the requirement to upload the entire course and its content each time.
acl's course publishing practices placed considerable strain on NextEd's resources, even after the linking problem had been solved ("course publishing habits" is the term two respondents used, implying some level of difficulty). Courses were being published at up to forty times the rate of other clients, study periods were not standardised, and since new clients were constantly coming on stream, the timing of course offerings was very unpredictable. Resource planning to deal with the workflow involved is likely to have been quite difficult for NextEd.
NextEd initiated a dialogue to establish a new course publishing model. This has involved extensive and on-going changes to both the publishing process and the courseware. The changes to the UIS (User information System) have involved the introduction of new census dates which apply to the student rather than to the course. Previously students enrolled in a course, and their enrolment finished on the course end date. Now it is possible for a student to cease to be enrolled in a course which continues on.
This, in turn, has enabled acl to provide courses to clients which continue beyond the nominal course length, meaning that new courses do not have to be uploaded for each new cohort. This is achieved by placing each cohort in groups. As a result, class activities such as discussion forums are accessed through the Group Pages. If students click on the Discussion Board (see below) they will find that nothing is available there.
This work-around has required considerable adjustments to the courseware, with significant instructional design implications. Content has been re-written to re-direct students from Discussion Boards to the Group Pages. Training materials and teachers' notes have been re-written to ensure that teachers are aware of the changes to the LMS, and how they need to go about ensuring that the appropriate groupings for their classes appear in the course.
Figure 8: Class discussion activities are now accessed through the group pages
A user rights distinction has been created in the LMS between instructors, who have rights to the settings for the whole course (and thereby potentially hundreds of students in different cohorts), and teacher assistants, who have rights to adjust the settings only for their own groups.
While these measures have still further reduced the setup time for a new course, it has involved a lot more administration work for acl. Every time a new cohort starts in a course, the aclEnglish Administrator has to set up the new groups and copy the content of discussions into the new group pages.
A side-effect of the new publishing habits of aclEnglish is that there is now a regular publishing cycle. Courses are published for six months, and each cycle includes the most recent updates to the courseware.
The respondents were asked the how satisfied they were with the outcomes.
Generally the respondents were pleased with the measures that have been taken. The changes to the publishing cycle have greatly reduced the time need to publish courses, with consequent benefits in resources required. All respondents were pleased with the introduction of the local audio option, and it was reported that the feedback from clients about the local audio has also been very positive.
While the response to the technical improvements was generally positive, there was some doubt that these have relieved the financial pressure brought about by the demands of aclEnglish on bandwidth and server capacity.
One respondent observed that ultimately the changes described above have not resolved the problem that this system is not ideal for the content, the business model, and the type of student aclEnglish is targeted at. This respondent felt that ongoing improvements will still be required.
The design of the aclEnglish courseware is being reviewed with the dual aim of making it (a) less dependent on real-time connections with the server, and (b) more re-usable in other course configurations. This involves migrating much of the content from the Blackboard core of the NextEd Delivery System to Flash elements which can operate more independently of the NextEd servers.
Nevertheless there is a continuing need to deliver large amounts of data, requiring further re-consideration of how aclEnglish is delivered. Under consideration are a range of local server options, plus a version of the courseware which is delivered on CD, but which requires the student to connect and authenticate.
A recent proposal (Grech, 2003) illustrates the effort being put into the collaboration between acl and NextEd. The model for hosting NextEd's typical content looks like this:
Figure 9: University delivery model (Grech, 2003)
However a model which would meet the demands on bandwidth and server capacity by aclEnglish looks like this:
Figure 10: Proposed new delivery structure for aclEnglish (Grech 2003)
It is agreed among the respondents that aclEnglish has created challenges and problems for the NextEd Delivery System. The measures taken to deal with these challenges were regarded positively, for the most part. It is clear that through the ongoing dialogue between NextEd and acl, significant improvements in publishing processes and delivery to clients have been achieved.
It is notable that changes occurred in the content, publication and delivery of the courseware. In the content areas, adjustments have been made to item sequences in quizzes, audio buttons, instructions to students and teacher support and training materials. The publication process has been re-worked and improved to allow faster uploading of new courses, and significantly less effort in configuring courseware in each publication cycle. On the delivery side there have been changes to the hosting of audio, course duration, the rights of teachers and coordinators in client institutions, and the method in which students access discussion and group activities.
The process has been an ongoing one of trying to resolve the mismatches in the relationship. It has affected every area of the course delivery process, and both acl and NextEd have had to put considerable effort into achieving these improvements.
Because the respondents were not questioned about how the relationship between acl and NextEd could be improved, it is difficult to draw conclusions about how the experiences described here can be applied to future practice. However it does seem that the effects of some problems could have been lessened through more information gathering at the planning stage. For example, more could have been known in advance about the demands acl's materials would place on the NextEd Delivery System, reducing the need for retrospective changes.
The experience here bears out what is reported in the literature - that there is difficulty finding and maintaining a match between educational content and a learning management system. However the experience of aclEnglish and NextEd also indicates that a commitment to ongoing dialogue and collaboration can minimise the areas of mismatch, and steer the process back towards achieving both educational and organisational goals.
Clouston, J. & Wenmoth, D. (2003) Mailbox to modem: Making eLearning sustainable in a traditional correspondence education environment. 2003 ODLAA Forum Proceedings (16th, Canberra, Australia, October 1-4, 2003)
Farmer, Lesley S. J. (2002) Seven Ways to BlackBoard. Libraries for Life: Democracy, Diversity, Delivery. IFLA Council and General Conference: Conference Programme and Proceedings (68th, Glasgow, Scotland, August 18-24, 2002) Accessed online 21 January 2004 [HREF2] http://www.ifla.org/IV/ifla68/papers/049-093e.pdf
Grech, J. (2003) acl Proposal Jul 03 Presentation to acl Pty Ltd (used with permission)
Hampel, Thorsten (2002) sTeam - Providing Primary Media Functions for Web-Based Computer-Supported Cooperative Learning. ED-MEDIA 2002 World Conference on Educational Multimedia, Hypermedia & Telecommunications. Proceedings (14th, Denver, Colorado, June 24-29, 2002)
Hedberg, J. (2003) Creating, growing, and sustaining learning environments 2003 ODLAA Forum Proceedings (16th, Canberra, Australia, October 1-4, 2002)
Online Learning News and Reviews (2004) LMS VENDOR TROUBLES Online Learning News and Reviews Newsletter Distributed by email, Tuesday, January 20, 2004 [HREF1]
a) Can you give an example of an aspect of aclEnglish content which provided a challenge for the NextEd delivery system? Please describe the issue.
b) How did you respond?
c) How well do you think this problem was addressed in the outcome?
2. Delivery
a) Can you give an example of an aspect of the delivery of aclEnglish which provided a challenge for the NextEd delivery system? Please describe the issue.
b) How did you respond?
c) How well do you think this problem was addressed in the outcome?
a) Can you think of a content problem which using the NextEd Delivery System posed for you? Please describe the problem.
b) How was this resolved?
c) How well do you think this problem was addressed in the outcome?
2. Delivery
a) Can you think of a delivery problem which using the NextEd Delivery System posed for you? Please describe the problem.
b) How did you respond?
c) How well do you think this problem was addressed in the outcome?