Leslie Burr, Manager, CSU Online, Division of Information Technology, Charles Sturt University
Matthew Morton-Allen, Analyst Programmer - Web Development, Division of Information Technology, Charles Sturt University
"Begin at the beginning and go on till you come to the end: then stop"
This paper discusses solutions for the development and implementation of online forums within a very large online teaching and learning environment. The paper provides quantitative analysis of the use of forums and of the impact of use on IT infrastructure.
"When I use a word, it means just what I choose it to mean"
Electronic forums are discussion groups conducted through the exchange of messages via computer networks. In these forums, people exchange information, viewpoints, and ways of doing things, on a variety of topics. Through the exchange, people are able to signal their interests and explore the interests of others. Consequently, electronic forums bring together people with similar interests who otherwise may not have the opportunity to meet (people who do not know each other and people who may be geographically distant), at a comparatively low cost. (Rojo 1996)
Electronic forums have the capability to efficiently link those in need of information with the appropriate sources of expertise. However, some unintentional, negative effects have been reported, such as information overload and content irrelevance, as well as problems in the quality of information and difficulties in stimulating the contribution of messages (Rojo, 1996).
In a narrower educational perspective Auburn (2001) describes a forum merely as an electronic tool that provides for a course-driven electronic discussion group. Members of the class (and others, if allowed by the instructor) participate in the exchange of information and viewpoints on issues related solely to the course. (Auburn 2001).
Rojo and Auburn's rather limited views of discussion groups describe listserves and web based message boards. We have widened the definition of online forums to include:
"The time has come the Walrus said to speak of many things"
In the beginning was the word, and the word was the web (WWW). Actually the word was the Internet, but that would have created the acronym WWI which looks uncomfortably like WW1 which, in turn, was just a little too close to the truth of what lay ahead for the many of us engaged in the large scale provision of online teaching and learning environments.
One early development of the online learning and teaching space began for Charles Sturt University during 1986 where the present Executive Director of Information Technology, then a Senior Lecturer, ran weekly interactive terminal sessions with his students during classes in Engineering. Other sessions were not synchronous (this is on the assumption that VAX mainframe terminal sessions could be considered synchronous – a contention with which, many may well wish to argue) and relied on participants sending email to the lecturer during defined periods when the mainframe could be booked for that purpose. The emails were then queued for the start of the lecture.
It was not unusual for the lecturer to end up with three times the amount of email at the end of the session than he had at the start! Charles Sturt University had learnt its first lesson in scaling online learning and teaching facilities; in the amount of individual responses that a lecturer could educationally and reasonably deal with, the flooding of the network with traffic and the inadequacy of disk space to store the incoming mail and subsequent replies and the ability to archive all the messages over a term.
Fast forwarding to 15 years later, we are currently in the middle of our busiest session ever with over 800 online supported subjects in the Distance Education mode and approximately 100 online supported subjects in the Internal mode. Each subject has its own forum (at least one) individually populated and is fed forum specific information (e.g. subject co-ordinator) by the Banner student administration and other systems.
In addition to this simplistic view of a University session, many subjects have additionally linked sub forums, and indeed there can be up to 11 variations of any one subject, all with separate cohorts. For example, one subject can be offered internally, by distance education as well as to a large number of different overseas student cohorts. Add to this complexity, the ability for students to examine their previous session forums, a range of interest group specific forums, course forums, and forums for staff and the result is a total of 1200 current forums for the session with the potential for 35,000 users.
During the two busiest sessions of year 2000 (i.e. Autumn and Spring), over 59,000 messages were posted to 2,000 forums. Additionally, 1,430,000 message views were recorded by 10,000 distinct online users.
The walrus certainly now speaks of many things. But this activity and interactivity has come at considerable development time and not without pain.
The systemic institutional wide development of online teaching and learning had its genesis at Charles Sturt University in 1995, with the development of the concept of the online supported subject and the practical design and implementation of a subject template and learning environment.
The notion of an online supported subject should be explained in a little more detail.
At CSU we have developed the web as a communication tool first, a service provider second and a content deliverer third.
From its inception, the online supported subject provided the communication tools of email, listserves and bulletin boards. Initial and tentative moves were made to deliver some online services like change of email address for example. Content, was delivered by mailed print packages. In case there is a perception that online has taken over the world, it is worth noting that last year CSU printed 32 million pages for mail packages.
In 1995, a template for each online supported subject was hand coded. That is, each of the subjects were manually programmed in html (we were about to learn the rule of scalability, again!) and after 6 months we had finished only 20 subjects!
The first online discussion board, introduced into the template in 1996 was functionally very primitive. It was freeware, unauthenticated and open to the world. Anyone could access the message board and read and offer comments, whereas we wanted to be able to restrict access to only those students enrolled in a specific subject.
Other than inserting a new header graphic, the application was off the shelf (figure 1). Most communication was still done with email and to some extent listserves.
The next session populated the application by authentication to the front page of the forum which of course could be easily be circumvented by guessing any subsequent URLs. Listserves were manually populated by cutting and pasting class lists. By the second session, listserves and forums were populated via a manually run program.
Figure 1
The next year population was done with cron jobs (i.e. automatically run programs), an absolute necessity, since 200 subjects were now online with a student population of 5,000.
As an historic marker, it is worth noting that, at this time, some online learning environments had started to become available notably Webct and Web in a Box. We were most interested in the communication tools and methods of population used in these applications, rather than their content delivery. It has not been until very recently that the large commercial e-environments have started to auto populate their classes in real time, directly from student admin systems.
At this time, the University developed Version 1 of its own online discussion forum. It is a sad reminder that because of the dynamically generated nature of the application, no images of this version now exist. The driving force was the ability to form finer granularity of learning groups within each forum. An accounting subject, for example, may wish to have all distance education students located in Brisbane, allocated to one sub-forum and all Melbourne students to another. In addition to this, the increasingly sophisticated clientele for the CSU forums was demanding enhancements to the basic functionality provided by the freeware products.
CSU did not ignore large scale development of online content. CSU has been delivering print material to students since 1972. Thus we have a long tradition in the development, production and delivery of materials. For those sections of print ready material that would be provided online, the challenge was to automate the process and delivering these sections to online production at the same time as their delivery to the printery.
This led to an online learning shell which was as much about online administrative services and communication as it was about content.
The automation process was, and still is, based on a web based publishing system (figure 2) (again developed in house due to a lack of a suitable commercial product) that includes a development, QA, and production life cycle, each with their own roles to ensure that developers cannot QA for example. Because the system is database driven, it also includes versioning, allowing for roll backs for challenges and legal purposes together with other features such as page owners and expiry dates.
Figure 2
Out of a student population of approximately 35,000, 25,000 students study by distance education. Understandably, our main priority was to ensure that all distance education subjects were online supported, and this target of 800 – 900 subjects per Autumn and Spring sessions was achieved by the start of 1999.
"Build it, and they will come."
What implications did this target mean for forums? Given that our main focus is on communications, it was important that the discussion board software be integral to each virtual learning community. Thus a forum was automatically created and populated for every subject together with a raft of general interest forums not tied to a specific subject.
Version 2 of the forums (figure 3) was developed again to satisfy both the increasing granularity of the virtual cohorts (for example) and the need for increased functionality. This version was called "Enhanced Forums" and was the first to provide a management tool for the subject co-ordinators (also automatically populated) and was released in June 1999.
Figure 3
By February 2000, use of the forums had increased dramatically, causing embarrassing loads on our network and particularly our online machine. Attempts at increasing CPU, RAM and disk space were to no avail. The more resources we installed on the machine, the more users and use the forums would attract.
As the user load increased, response times to the client slowed as the back office authentication and roles checking started to heavily impact on performance.
Interdependence between the machines running the application, the authentication, and the student database also started to take its toll. It was clear that the fundamental server environment combination was not scaling. An added complication was that in theory, and often in practice, these machines could be located anywhere across our entire network, from Albury to Sydney.
In this environment of high and increasing use, raised user expectations but poor performance, we had built a spectacularly popular product that in market terms we were struggling to supply.
By March 2000, the performance of the forums had slowed to the point where a wholesale review of the application, platform and infrastructure was required.
The review of the infrastructure resulted in an N Tier approach to our infrastructure, and a massive upgrade to the server capacity for forums. This was in addition to a total rewrite of the code in order to take advantage of a more efficient server environment.
A target access time of 2 seconds was set for any message retrieval. This has largely been achieved with the new code however overheads of role checking and authentication can some times extend this time.
"Curiouser and curiouser"
In pursuing the design of the forums application, it was important to examine the criteria of successful online discussion environments. This examination included not only forums, but email, listserve, debating and resource bank tools and research. The present functionality and user requirements were determined from several sources including:
This examination pointed to the main characteristics of successful forum design.
The ability to search, sort, filter and sift large amounts of material within a busy forum is paramount to the successful use of the forum. Many studies, primarily detailed by Rojo (1996), have pointed to the direct relationship between forum membership activity and content relevance.
These factors also include the threat of information overload, time investment, and list content relevance. Rojo (1996) noted that participants quickly learn the need to screen messages and that they had developed individualized screening routines which were not necessarily part of the client software.
Our own evaluations show that students also filter by physically scanning the message index for relevancy of the subject topic, the number of threads associated with a message and the credibility of the person posting the message. This notion compares favourably with both Conner (1992); and McCarty (1992), who also note that the paradigms for subconscious filtering of old media are transferred to new media.
Furthermore, both McCarty (1992) and Conner (1992) have pointed out that the solution lies in getting participants to perceive mailing lists as a new medium with its own characteristics rather than as a variant of other media. We would argue this is not the case, indeed the ability to filter should be entirely compatible with the users previous experiences and expectations.
These indicators led to a heavy design criteria being applied to indexing, threading, searching, sorting and filtering messages. These are achieved by implementing a database model of message storage which not only prepares the application for scalability but has the more important priority of allowing detailed and refined filtering of sender, date, message title and most importantly searching the body of the message. In our application the searching is based on the Oracle 8i platform.
The main difference between a message board and a forum is the ability to manage the content and the population. The critical requirement for organising an online teaching and learning environment is the ability to successfully manage the learning groups.
As with any classroom management, the techniques vary widely depending on both the teacher/lecturer/facilitator and the learner. Thus the management functions need to be flexible enough to support a vast array of teaching strategies, from passive observers to teacher focused discussions, either in the context of the entire student cohort or sub groups of it.
While listserve models empower the manager as moderator and this function is available within the CSU forum software, it has not been widely used within the University teaching and learning context. In fact, the opposite could be argued, that at the tertiary level, the manager should not have the right to moderate, edit or delete student contributions to a forum.
Numerous authors have described in some detail the roles of managers and facilitators in online discussion environments (Berge, 1992; Davie, 1990; Steinfield & Fulk, 1988) and to a lesser or greater extent we have accommodated each of those roles.
The creation of subsets of the discussion cohort presents the ability for small group collaboration and discussion with the ability to report back to the main cohort. The sub groups can be populated by the forum manager using pre-determined pedagogic criteria or by a random assignment of students into sub forums.
Although not identified in the research literature, an additional level of functionality that we have layered over the application is a maintenance role usually directed at systems administration personnel. This also allows for the compliance with the Australian Broadcasting Authority’s Take Down legislation where one institutional contact must be provided for any reported breaches of the law. However, since most forums are within an authenticated environment, it is unlikely for this condition to occur.
Maintenance also allows for the initial auto creation and population of forums at the beginning of each teaching semester. Maintenance also provides corporate statistics for evaluation and planning.
"What is the use of a book without pictures or conversations?"
CSU "in-house" developed forums have the following features which are over and above those currently offered commercially.
Client
Management
Maintenance
Population
While there are several methods of connecting databases to web servers and vice versa in addition to those detailed by Zhao (1999), mod_perl (sic) has become the industry standard. It is also built upon the knowledge base we already have, and it performed and scaled exceptionally well during load testing.
Initial forum design relied on a standard Perl based CGI environment. The current environment continues to utilize Perl, however in order to accomplish the performance goals, it is used within the mod_perl framework. As an indication the present code runs by a factor of 10 times faster under mod_perl than under Perl/CGI.
The new forums run on a dedicated Sun E45000 with 8 x 450 Mhz CPUs, 8 Gb RAM and 144 Gb of disk. The machine is located in our Bathurst machine room connected to Sydney via our Albury to Sydney 32 Meg microwave link.
Both the forums QA environment and the disaster recovery machine are Sun E450, 4 x 450 Mhz CPU’s and 4 Gb RAM. A sorry server and an ISDN back up network to the microwave is in place for worst case scenarios.
This current version of the CSU forums employs Oracle 8i as its database backend. In the initial design where message numbers were relatively low, the flat file structure employed for message content was adequate. However as the number of interactions grew the flat file structure became unmanageable. The prime reasons for moving to an all database design are:
As stated previously the original user interface was no more than reimaging of a shareware program that reflected the basic functions of a message board. Later versions used a customized interface. The look and feel of the interfaces produced the expected amount of discussion over the placement of buttons and of course, colour. Although we continually promote the use of reference groups in our software development, it is always frustrating to note the lack of discussion in regard to functionality and pedagogic modeling. The emphasis always returns to micro-functionality and colours.
In the present version (figure 4), the interface was designed to match the Microsoft Outlook paradigm, the most widely used collaborative tool in our University. The reasons for emulating the Outlook interface was to get immediate user familiarity with the product through established UI elements within the Microsoft family. The interface uses split screens with loading of individual messages into a bottom frame.
Figure 4
"Will you, won't you, will you, won't you, will you join the dance?"
Given the software and infrastructure capable of scaling to very high levels of traffic and interactivity, what then are the predictors for very high forums useage?
There is a number of physical givens within any educational online environment namely:
The larger the size (i.e. population) of a forum the more the benefits are likely to be perceived by each user (Rojo 1996) inferring a higher participation rate.
Rojo (1996) noted that more benefits are obtained from forums with a higher number of regular contributors. Within the teaching and learning environment, the regular contributor, where present, is the subject co-ordinator. It is our experience that forums that have several managers (e.g. in a team teaching situation) exhibit higher online activity than forums with a single subject co-ordinator. This appears equally true for the number of students as regular contributors.
Reciprocal interdependence means that individual users of interactive media (such as electronic forums) cannot achieve the potential benefits offered by the medium on their own. Reciprocal interdependence among users has another referent besides the adoption process and that is the online interaction process. It is online interaction itself that draws people in to contribute messages. (Rojo 1996)
Furthermore Markus (1990) describes the opposite effect for the failure of a forum. Markus (1990) discusses the issue of reciprocal interdependence amongst users in relation to the adoption process. Prospective adopters of interactive media need to know that they will have sufficient and relevant communication partners; if these conditions are absent there is a risk that medium use will not only fail to spread but will also extinguish. (Rojo 1996). Rafaeli (1997) claims that interactivity plays a role in creating networks and in generating their growth patterns.
The growth of communities is not only the province of online discussion groups. Robert Metcalfe, who founded 3Com Corporation and designed the Ethernet protocol for computer networks stated that the usefulness, or utility, of a network equals the square of the number of users. Until a critical mass of users is reached, a change in technology only affects the technology. But once a critical mass is attained, social, political, and economic systems change (Boyd 2000).
Although system performance is a parameter yet to be researched, our contention is that increase in system performance, has a positive benefit to the user and thus increases interactivity. In each case where we have increased system performance traffic has increased. Examples of this include:
Conversely, poor system performance results in lower traffic. Examples include:
"What I tell you three times is true"
Version 2 and 3 of the forums, contain significant logging programs which provide for a range of statistics to be collected. This information is available to the manager of each forum (through the management tool) together with a corporate view of all forums (through the maintenance tool). The use of forums over the last three complete DE sessions (i.e. Spring 1999, Autumn 2000 and Spring 2000) had very similar patterns of use. The session displaying the highest activity, Autumn 2000, was chosen for this study.
Forum access is very high during the working day (figure 5), peaking at 1.00pm, which is not consistent with previous surveys relating to conventional DE study habits. As a result of CSU’s high DE intake, students may be accessing the forums from work during these times, which is supported by the peak occurring during work lunch times. High use also occurs at 8.00pm, which is consistent with previous study patterns for distance education students.
It is interesting to note the traffic occurring between 12 midnight and 6.00am. Presumably this traffic is mostly occurring from our overseas student cohort.
Figure 5
Surveys previously conducted concerning our distance education students and their study habits, have indicated that Saturdays, Sundays and weekday evenings are the most popular study times. An analysis of forum useage shows that forums are being accessed most heavily on Mondays and least on Saturdays (figure 6). Approximately twice the number of messages are viewed on Mondays in comparison with Saturday. Forums are least accessed on the weekends where students may opt to spend a greater time off-line studying the printed resources at home.
Questions, concerns and issues arising from the weekends study can then be posted to the forum. Consequently there is increased forum traffic during the start of the working week.
Figure 6
The ratio between message views and message posts, averaged over the session for the total number of forums, was found to be approximately 25:1. This is consistent over the whole session with the exception of the first 3 weeks of the session where the ratio of message views and message posts was found to be 20:1. Traffic was found to peak in the 3rd week of session with a large dip at the time of the mid session break which coincided with Easter (figure 7).
Again, this is consistent with the conclusion that forums are being frequently accessed from the work place.
Views and posts, understandably, fall toward the end of session, although a decrease in the ratio of views to posts (approx 15:1) occurs in the last 3 weeks of session as students increase their posts of specific questions relating to the examination period.
Figure 7
Over a sample group of 200 forums, it was found that each forum contained between 10 and 50 posts. The number of posts was found to be independent of the size of the learning community, (i.e. an increase in the community size did not increase the total number of posts). It is hypothesized that students require a finite body of information and once this is reached, the number of total posts plateaus.
Examination of the relationship between the number of posts per user vs community size (figure 8) indicates that posts/user decreases as the size of the community increases. This is in accordance with our previous hypothesis of a finite amount of messages for each forum.
This explanation is based on the assumption that the forums are currently used primarily as an information gathering source rather than as an interactive learning tool. This is justified by the low number of postings/user, which is in the range 0 to 2.5.
Figure 8
The relationship between the number of message views and an increase in users or community size revealed that the number of message views/user is again independent of the size of the forum community
Figure 9
The top ten forums containing the most number of messages was identified. Messages posted ranged from 404 to 1,153. The top ten contributors to each of the forums were identified and the percentage of messages posted by those ten contributors analysed as a percentage of the total messages. It was found that the top ten contributors posted 40% (on average) of the messages. Initial indications are that the community size of a forum was not related to the percentage of postings contributed by the top 10 posters.
"I’ve seen the future and it works"
Rojo's (1996) contention that more benefits accrue from larger forum communities is sustained by our study in as much that students have a larger range of postings from which to view.
The hypothesis that a small number of regular contributors constitutes a proportionally large number of postings was sustained. Each forum where the hypothesis held true was a highly interactive forum. Further work needs to be undertaken to examine the role of the forum manager in seeding and facilitating forums.
Reciprocal interdependence was shown to plateau after a finite amount of posted information was reached. In our study this found to be between 10 and 50 posts, independent of the size of the forum population. Further it was found that the number of posts/user declines as the population increases. This contradicts previous evidence which suggests that activity builds on activity.
We have found that forum participants are using both visual methods and software provided filtering methods for scanning the message index for relevance. It was found that on average, a user reads approximately 16 messages in any one forum, regardless of the number of messages posted or community size of the forum.
Patterns of forum useage have also been analysed and indicate that it is likely that the use of online forums complements the traditional study patterns of use of printed material rather than competing with traditional study times.
The history of the development of forum software within CSU’s online teaching and learning environment has been detailed. The rapid increase in load has led to the development of a large scale solution for the application. The relatively low useage rate is an indicator that use of the forums is only at the initial stages of development and that as online pedagogy develops to make higher interactive use of the forums rather than as an additional information dissemination and retrieval channel, we can expect a large increase in traffic. The hardware and infrastructure solution developed for the latest version of the forums should scale to expected loads in the short to medium term.
The authors gratefully acknowledge Shane Dawson, Educational Designer, Charles Sturt University for his help, support and assistance and Bernie O’Donnell, Pro-Vice-Chancellor (Planning and Development) for his review of early versions of the paper. We also wish to acknowledge Mike Rebbechi, Executive Director, Division of Information Technology, for his constant encouragement and the whole team within the Division of Information Technology, Charles Sturt University, without whom there would be no online environment.
| Auburn (2001) | Glossary
of Auburn University Technology Terminology http://www.auburn.edu/helpdesk/glossary/forum.html |
| Berge, Z. (1994) | Electronic discussion groups. Communication Education, 43(2). |
| Boyd, C. (2000) | Why
Strategy Must Change http://www.mgt.smsu.edu/mgt487/mgtissue/newstrat/metcalfe.htm Updated December 16, 2000 |
| Conner, P. (1992) | Networking
in the Humanities: Lessons from ANSAXNET. Computers and the Humanities(26), 195-204. |
| Davie, L. (1988) | Facilitation techniques for the on-line tutor. In R. Mason and A. Kaye (Eds.), Mindweave. London: Pergamon Press. |
| McCarty, W. (1992) | HUMANIST: Lessons from a global electronic seminar. Computers and the Humanities(26), 205-222. |
| Rafaeli, S. & Sudweeks, F. (1997) | Networked
Interactivity. In McLaughlin, M, and Rafaeli, S. (eds)Journal of Computer-Mediated
Communication Volume 2, Number 4: March, 1997 http://www.ascusc.org/jcmc/vol2/issue4/rafaeli.sudweeks.html |
| Rojo, A. (1996) | University
of Toronto Ph.D. thesis. http://www.digitaltempo.com/e-forums/Oframes.html |
| Steinfield, C. &. Fulk, J. (1988) | Computer-mediated communication systems as mass communication media. ED305041. RIEAUG89. |
| Section 1 | Lewis
Carroll, from "Alice’s Adventures in Wonderland" (1865 Chapter 12) |
| Section 2 | Lewis
Carroll, from "Through the Looking-Glass" (1872 Chapter 6) |
| Section 3 | Lewis
Carroll, from "Through the Looking-Glass" (1872 Chapter 4) |
| Section 3.5 | from
"Field of Dreams" (Universal Films)(1989) Director: Phil Alden Robinson Executive Producer: Brian E. Frankish |
| Section 4 | Lewis
Carroll, from "Alice’s Adventures in Wonderland" (1865 Chapter 2) |
| Section 5 | Lewis
Carroll, from "Alice’s Adventures in Wonderland" (1865 Chapter 1) |
| Section 6 | Lewis
Carroll, from "Alice’s Adventures in Wonderland" (1865 Chapter 10) |
| Section 7 | Lewis Carroll, from "The Hunting of the Snark" (1876) |
| Section 8 | Lincoln Steffen, from "Letters" (1938) Vol. 1, p. 463 |
Leslie Burr and Matthew Morton-Allen © 2001. The authors assign to Southern Cross University and other educational and non-profit institutions a non-exclusive licence to use this document for personal use and in courses of instruction provided that the article is used in full and this copyright statement is reproduced. The authors also grant a non-exclusive licence to Southern Cross University to publish this document in full on the World Wide Web and on CD-ROM and in printed form with the conference papers and for the document to be published on mirrors on the World Wide Web.
[ Proceedings ]
AusWeb01 Seventh Australian World Wide Web Conference, Southern Cross University, PO Box 157, Lismore NSW 2480, Australia. Email: "AusWeb01@scu.edu.au"