Pen Savage, Education Dept, Monash University, Clayton 3168, Victoria, Australia. pen@savage.net.au
Ron Savage, Redrock Communications Pty Ltd, 1/6 Riddell Pde, Elsterwick 3185, Victoria, Australia ron@savage.net.au
This paper discusses how production of web sites in an education environment is still in the early stages, and how there is little project management documentation to draw on. Consequently, mistakes are being made and projects are taking longer and are costing more than initially predicted. The paper analyses several web site production problems and offers solutions.
Much web site production for the implementation of flexible learning and off-campus study modes in higher education is in its early stages. Academics and technical staff alike are piloting education courses using Internet technology. There is little literature available that discusses what has been tried and tested, what works and why. Understandably then, those who are working in these early stages do not yet appreciate that web site production projects will take longer and be more expensive if lessons are not learned. This paper is presented in two parts: the first discusses several typical web site production problems, and the second part of the paper discusses how these problems can be addressed. In summary, the paper aims to provide some guidance for those who are embarking along this path.
Web site production problems are mostly the result of poor project planning, which leads to poor understanding of the processes involved and the time required. Furthermore, often because of the pressure from stakeholders to see something and have the web site built quickly and cheaply, the project development stages of Analysis and Design are skipped. Moreover, many people have no training in Analysis and Design and so don't appreciate the need for these stages. This is reinforced by the Internet's ethos of speed and rapid construction and deployment, which is part of the hype and the culture that currently pervades the industry. Consequently the valuable prototype and design testing stages are by-passed, which can necessitate major reworking of the web site after it has been built.
Typical web site production projects supply little pre-project and design documentation. In part this has probably occurred because the early adopters are over-committed to new technology, and have begun experimenting without appreciating that web site production, similarly to book publishing and software development, are individual projects with stages and processes that need to be gone through. Projects need to be thought about in a business-like way. Where projects are funded the builder-client relationship needs to be modelled on one of accountability and delivery. Siegel (1999) maintains that educational organisations need to think through their projects in advance, particularly where web site development is concerned and there are large sums of money involved. He suggests the use of a 'Project Profiler' (Seigel, 1999, p 176) to be developed prior to the tender process. This is a rigorous questionnaire to ascertain aims and objectives, project scope, marketing, audience, content, functionality, time lines, success criteria for a project, and contingency plans.
Another consideration relates to authority and responsibility, and according to Lock (1996) can cause much controversy if not addressed. Authority and responsibility should be given to the project manager before the project commences and be made visible to all members of the team. This authority will ensure that the project manager can take action if project members do not perform or tasks are not delivered on time, thus increasing assurance that milestones and goals are met.
As the development and use of learning technologies is still in its infancy, web sites are often built in isolation, with little reference to overall organisational policies and guidelines. For instance, it is often difficult to determine how a web site fits within the overall direction of the department or faculty, i.e. whether it has been developed in relation to an organisation's strategic plan, or as the result of a process that involved approval by a policy and procedures committee, and whether there is an underlying pedagogical basis. This raises the question of standardisation of policy and guidelines within faculties, and the ownership of them by all who are developing courses, subjects and materials.
Web site projects needs to be set within the context and culture of an organisation. Bennett, Priest, & Macpherson (1999) contend that academic staff with valuable experience and expertise in university teaching and learning are not participating in discussions and decision making about new delivery methods. Organisational debate or serious discussion about the use of the web as a delivery mechanism for education, and what it means philosophically for academic practice, needs to occur before the project begins. Also, the web site needs to be understood and supported by the organisation as a bona fide teaching tool. Pasula (1996) provides empirical research, and demonstrates the barriers to the adoption of learning technologies and the use of multimedia in educational organisations. These barriers are: the management system and lack of organisational support; availability of funding resources; user non-acceptance; and, user knowledge. Rossiter (1997) reports a study that evaluated computer-based education. It had a particular focus on the potential uses of new computer-based technologies to enhance the educational process at the university. The recommendations of the study included greater emphasis on support for students and the requirement of a better understanding of their needs. On an organisational level, staff development was emphasised, and university-wide discussion to manage change and develop cohesive strategies for the future were recommended prior to implementation. Pasula (1996) further proposes that commitment and support are required from the organisational management for funding, workload re-design, organisational structural changes, rewards and incentives, operational support and training. Moore and Kearsley (1996) share the same view, and discuss the need for strategic planning, proper administration, staffing, and preparation of educators.
Documentation is mostly seen as unnecessary or something that's forgotten in the rush to produce. For example, a simple audit of educational web sites at a university revealed that they were very experimental and creative, had achieved some excellent work and results, and those involved with these projects were enthusiastic to share their work. However, these projects in general seemed to have one major flaw - a lack of documentation. Separately from the anecdotal conversations amongst colleagues, there was little or no documented record in the form of a report of the processes used in the development of these projects, nor of items such as: how objectives were achieved; what analysis of the underlying assumptions had been made; and what theories underpinned the development. Further, there was little or no record of either the types of technologies used or the rationale behind the choices. The documentation that was available was in the form of conference papers or journal articles; however the factual information as described above was missing.
When projects are not planned comprehensively prior to their implementation, time is not properly accounted for and so leads to people resources not being properly managed. On one project because this had not been done, the academics involved had not been relieved of their other duties to contribute to the project and so were not available to develop their educational materials and properly contribute to the design of the web site in a timely fashion. The web site production was seen as an add-on to their already over-burdened responsibilities. On another project, holidays and the natural attrition of staff (leaving and moving to other positions) were not accounted for and resources became very reduced because there were no replacements. On both occasions the deadlines were not met.
Web site production requires skills and an understanding of what's involved. On one web site development project an academic in charge of a course was also the same person responsible for assisting in the site design and development of the content. Unfortunately, this same person was somewhat computer illiterate and unfamiliar with web technology and how it worked. Thus graphic design using the web as the medium was new to them, and as a consequence graphical images and the overall design took longer than anticipated. Furthermore, they had little or no sense of what was required and how long each implementation step took in terms of time and so often put other priorities ahead of the project.
Testing of web sites consists of two major areas: usability design testing and expert evaluation or technical testing. Most frequently done is expert evaluation or technical testing, where metrics are collected on items such as checking of links and images, pages, forms, graphics, applets, scripts and plug-ins, to see if they are working correctly. Content is also checked for accuracy and currency. However, when projects are hurried and/or not thought through, testing of the screen interface design doesn't occur. The design is not checked against the user's behaviour and thinking, and information is not collected as qualitative data for redesigning purposes. Questions such as how long does it take for users to perform a task; how do users perform a task; did they enjoy using the site; and whether the tasks were completed efficiently and effectively, need to be asked (Nielsen, 1999). Lastly, in an educational environment further questions need to be asked about whether students are able to learn with the site. From the literature, there appears to be very little information about how to assess a web site for its 'learning ability'.
Project planning needs to occur at the outset of the project. The plan needs to well thought through and should include:
Woolston (1996) suggests that a barrier to the implementation of learning technologies is the level of awareness among academics of what can be accomplished. This might be interpreted as a need for professional development. Further, Bennet et al (1999) argues that successful staff development programs should empower academics and encourage them to be involved in on-line projects in their own disciplines. Teamwork is needed to build a web site. Academics and many educators are used to working on their own - reading, writing, teaching and publishing, and so team-building and professional development would be needed to develop skills to enable team work to occur. Teamwork (to be an effective team member), is not only an art and a science, but also relies on specific technical and development skills, and team chemistry, and in many instances training. (The armed forces use a lot of team building training.) To be an effective team member, according to Wysocki et al (1995) requires personal qualifications and learned skills such as: commitment to the team and the ability to meet the goals of the project; a sense of shared responsibility; ability to be flexible; a task-orientated attitude; ability to work within a schedule and its constraints; trust and mutual support for fellow team members; open mindedness; ability to work across a structure and with authorities; understanding of project management principles; and, being able to use project management tools.
Testing is part of the project's quality assurance process. Testing should occur early in the project, as part of an overall evaluation strategy in the development of a web site, and as a means to continually improving the on-line materials. Testing includes concept testing (trying out ideas before development), prototyping (using a mock-up of the key features), peer review, useability tests, field trials and acceptance testing. Evaluating effectiveness of a site for student learning should be associated with the purpose of the design and the communicative, adaptive or informational approaches used.
Concept test any early designs or paper prototypes with a sample group from your targeted audience, for their comments and feedback. Students are the people who will be using the site, and they can propose ideas that can be developed creatively. Then develop a formal series of usability design test cases. Expert evaluation of the web site should also be tested, where typical user environments are determined in conjunction with a testing site or test bed. Thus the site is tested according to the user's conditions, whereby the most likely web browser, versions thereof, and different login locations can be trialled with the site. Evaluation data can also be obtained from tracking software (which monitors usage of the site) and can be supplemented by a range of other quantitative and qualitative data from users in the form of questionnaires, focus groups and interviews (Webster, Murphy, & Savage, 2000), (Nielsen, 1999). Lastly, the site should be tested to ensure that students with disabilities are catered for and international students have reduced problems with language usage.
All projects have problems and take longer than you think. It is important therefore to predict risks and problems and develop contingency plans. The heuristic for most project problems is to add on another 10 per cent in terms of cost and time. Web site production is no exception.
Web site production in educational settings is still in its early days. Many web sites have been built in isolation and experimentally. However as departments and faculties move towards flexible learning and delivery this will no longer be the case and web sites will be built along project management lines. To ensure web site projects meet goals and expectations it is essential that the project: has a proposal with a documented project plan before implementation; suitable staff are selected or at least have professional training and development as part of the project; time is scheduled for content development and web site development; and, testing occurs early, to avoid any major re-development of the original design.
[ Proceedings ]