Karey Patterson, School of Social Science, Southern Cross University, Lismore, NSW, Australia, 2480. Email: kpatte10@scu.edu.au
Allan Ellis, Senior Lecturer, School of Social Science, Southern Cross University, Lismore, NSW, Australia, 2480. Email: aellis@scu.edu.au
This paper examines Australian university Web Sites in terms of their use as a business information tool. The development of the concept that a university Web Site is an important primary means of contact with potential students and other stakeholders is relatively recent and as such there seems to be little research into and awareness of the management practice of the individuals or teams that manage this important medium.
With a large amount of information to convey and an increasingly competitive educational marketplace, for on campus as well as distance education and online educational markets, the importance of effective Web Site management of a university Web site cannot and should not be underestimated. Much of the hype surrounding educational Web sites and the possibilities or promise of this new technology has to do with potential access to new markets, and course delivery technologies.
The majority of Australian University Web Site managers interviewed for this study perceive the development or implementation of an organisational wide Content Management System (CMS) as a key factor in improving the timeliness of information, consistency of design and accessibility of the university Web Site that they manage. Many respondents reported ad-hoc methods of backing up and archiving data, areas of policy deficiency and severe time management issues due to their consistently large workload.
Web Site evolution is still often viewed in terms of technical advances and changes in the technical environment rather than in the management practice and team structure of those that administer the Web Site.
Commercial and private business Web Sites have become an extremely important element in cultivating and projecting the business identity of the institution, marketing to targeted groups and as an information resource (Patterson, Ellis and Brice, 2000).
University Web Sites are subject to the same general pressures but increasingly have an important role as a tool by which universities gain student interest, enrollment and revenue . For international students this involves fee-paying courses, which have grown in importance to Australian Universities due to changes in the funding structure of the higher education market . An increase in competition and the need for revenue generation combined with high costs of doing business has added to the pressure for Australian universities. Global information provision via effective use of the University Web Site is often seen as a primary way to raise the profile of the institution and to potentially increase market share (Ellis and Patterson, 2002; Patterson, Ellis and Brice, 2000). Clearly university managers must ask themselves what are the ideal structures for the management of Web Sites. This research is a formative step in understanding the current environment for Australian University Web Site managers.
The data in this paper, a series of interviews with the Web Site managers
of Australian degree granting institutions, was collected from February 2002
to April 2003. Each of the managers interviewed in the study was contacted
by email and phone and, by appointment, interviewed by phone with responses
being recorded and then transcribed into the research database for analysis.
Several of the Web site managers had to obtain permission from their supervisors
before agreeing to participate in the research and data collection interview.
The data contained in this report is descriptive in nature covering the position
and opinions of the interviewed Web site administrators. It was agreed that
in order to safeguard the privacy of participants and facilitate openness,
the research would not specifically identify any individual or institution.
To ensure confidentiality, the researchers are able to identify the contributors
to the report but those who contributed to the report are unable to identify
other contributors.
The participants were asked questions relating to their administrative roles
and structure of the Web Site management teams, the role the Web Site management
unit has in their University as well as the types of technology used for managing
the corporate Web Site and what they believe are the expected trends and important
issues affecting them in the foreseeable future in the industry. The data does
provide us with information relating to reporting relationships and responsibilities
among Australian University Web Site managers and possible insight into optimal
structure and reporting relationships for these units.
For the purposes of this research a University “Homepage” is the
first page loaded when the complete URI has been inserted into a browser location
bar. Often the Web Site managers referred to this part of the university Web
Site as the ‘corporate site’.
The Web Site managers (n=41) interviewed were typically contacted by an introductory phone call or via a direct introductory email, after a search of the institution’s Web Site.
After agreeing to participate in the research the Web Site managers were asked a series of questions relating to various elements of the on the job tasks and administrative environment in their workplace, the Web Site design development process, the size and scale of the Web Site they manage, how Web Site content is managed, who determines and how the Web Site design is selected, Web Site Accessibility policy, use of Web Site statistics and level of centralisation/decentralisation of the Web Site. Several of the questions could be scored as either being "yes" or "no" and when that occurred further open questions were asked to elicit more information on the subject matter.
Initial interviews found that several of the participants were interested in what the other institutions were doing with regard to the research areas. As a result of this unexpected outcome all survey participants were asked, at the conclusion of the research questions and interview, whether or not they would be interested in reviewing and/or contributing to a Web Site that discusses the issues raised by the research discussed in this report.
One hundred percent (100%) of interview subjects stated that they would be interested in reviewing the proposed collegial Web site and 79% stated that they would contribute to the Web Site if it was worthwhile. The collegial Web site has been discussed further in a short paper presented at AusWeb03 by the author Patterson and E. Kronen.
Interviews ran for between 50 to 70 minutes per University. The time allocated was spent gathering data and familiarising the respondent with the research. At the conclusion of the interviews several of the participants volunteered more information, kept in email contact and in one instance invited a site visit.
Each Web site manager was contacted by phone and interviewed. They
were asked the following questions:
A majority of respondents have been in their current job position for a period of one to two years as illustrated by Figure 1.
Figure 1 Web Site Managers Time in Current Position
Overall results show that a slight majority of respondents have been employed in their current position for less than 2 years (38%) compared to their colleagues who have been employed over 2 years (34%). Several respondents (n=5) stated that they had held similar positions to their current job position within their institution prior to their current employment showing that some Universities are fostering skills and experience from within. The general feeling, as expressed by a majority of respondents, was that the industry and work environment they work in is dynamic and consistently changing and evolving and that they expected regular change in the work environment and team makeup. Many respondents reported that their unit or workplace had undergone, or was undergoing, organisational change at the time of the interview and they had just changed their unit or team names, reporting relationships, job titles, and responsibilities.
Figure 2 shows the number of team members in the Web site management units.
Figure 2 Number of Team Members in Web site Management Unit
Most teams (72%) had between one and six team members. Several of the smaller teams in the study were supplemented with casual staff on occasion. The larger teams were often very diverse in nature and contained staff from a variety of backgrounds other than Web Site management, including marketing, design (print) and academe. It was noted that 33% of respondents work as part of a team of one to three staff members. For these teams their limited time and large numbers of tasks were seen to be an area of extreme concern and stress. They were unable to take on tasks that required planning and were not technically orientated such as training and policy education and enforcement and information workshops.
Figure 3 Who do Web Site Managers Report to?
Figure 3 shows that most Web Site managers report to an IT related administrative area within the University. Very few University Web Site managers report to an academic or Teaching and Learning area. With one exception every one of the Web Site managers that reported to an academic area at the University was part of a small team of 1-3 members (see both Figure 2 and Figure 3). The managers of these teams described fatigue and burnout as a symptom reflective of the small team coping with large workload.
The fact that 33% of the Web Site managers report to areas in marketing or to a committee is of interest. There were numerous examples of innovative systems Universities are implementing in order to maximise the investment, importance and influence of the Web Site. Several of the Web Site managers (17% of total) reported to a marketing department and others (16%) reported to a committee made up of a variety of stakeholders from a variety of areas (library, marketing, IT, distance learning, student services) often including executive level managers. One university reported that they report to both an IT based area and a marketing area in equal proportion. When questioned the respondent reported satisfaction at the situation and stated that they felt that the technical needs and informational or content needs were well satisfied. They also believed that this arrangement was good for accessing training resources and other like support when needed.
The Web Site managers who were either part of an academic or marketing unit reported frequent frustration in relying on technical tasks to be completed by specialists in IT areas as they were seen to be outside the technical units and therefore had to wait for IT staff to have time to tend to their needs.
The Web Site managers who were part of mixed teams, reported to IT or had executive level sponsorship on the University Web Site committee or a similar structure, reported higher levels of satisfaction with the resourcing and support their team or unit receive within the University.
Figure 4 shows the number of Web Servers used to deliver corporate Web Site data.
Figure 4 Number of Web Servers Used to Deliver Corporate Web Site Data
Thirty seven percent (37%) of the survey respondents reported that the main Web Site was delivered on six or more servers. Some of these respondents stated that this situation was unsatisfactory and that the number of servers was indicative of a decentralised University Web Site environment that was hard to administer. Several of the Universities that delivered data on more than 6 servers had server numbers in double, and in one case, triple figures. These situations were deemed unsatisfactory and counter productive with regard to the need for consistent visual design and policy (usability and accessibility) enforcement.
The reason that the situation evolved was sometimes due to political considerations and internal organisational sensitivities. It was also reported that the respondents believed that the implementation of a Content Management System (CMS) on a single server would empower the Web Site management team and would result in greater adherence to usability and accessibility guidelines. The Web Site managers who did not expect to use a CMS in the short to medium term felt that both the University Web Site and their ability to administer it would have been enhanced by a CMS.
Respondents were asked about the primary roles of their unit or team and the results reflected in Figure 5 attest to the diversity of the responsibilities of the Web Site managers.
Figure 5 What areas of the Web Site is your unit primarily responsible for?
Only one respondent stated that their core Web Site responsibility was marketing and the unit was nested within a marketing area of the university. Thirty five percent (35%) of respondents stated that their primary role included all tasks shown in Figure 5. Twenty one percent (21%) and twenty-four (24%) respectively of respondents reported that their primary roles were either technical in nature or related to Web Site policy development and enforcement.
Some respondents who answered ‘Web Site policy’ to the question shown in Figure 5 mentioned that this policy role was related to an emphasis on the use of improved accessibility and usability of corporate Web Site HTML templates throughout the university. These respondents did not always use CMS in their work.
However respondents overall often perceived use of a CMS as a means of ensuring the delivery of usable and accessible data as the policy compliant templates would be embedded within the CMS.
Figure 6 shows how Australian universities deal with webmaster@youruni emails that are often a significant area of initial student contact.
Figure 6 Who receives the webmaster@youruni emails?
Several respondents stated that the content author would be the rightful recipient of any enquiries through an area of the Web Site that was “owned” by an identifiable area or unit. Often, in these circumstances, the Web Site manager would receive a copy of these emails as well as the content author or member of the Web Site management team. One respondent was very adamant that their university shunned automatically generated email responses and that “we are web humans not webmasters and are custodians and guides of the Web Site” and that that Web Site management team would “personally deal with all generic and other email enquiries depending on the context”. This role was seen by the respondent to be an example of a positive human interaction in an otherwise typically sterile and asynchronous Web Site engagement with a client or potential student. Contradicting these statements other respondents voiced the desire to avoid this level of personal contact due to their particularly hectic work schedule and existing large amounts of work related email correspondence.
Figure 7 shows that those Webmaster emails which were answered tended to depend on the contents.
Figure 7 How are the webmaster@youruni emails responded to?
Fifty nine percent (59%) of the webmaster@youruni emails were filtered according to their contents and can be interpreted as a positive indication that the emails are being read and passed on where relevant. All respondents responded in some way and none admitted to not responding to the webmaster based email enquiries. There was typically no formal policy related to the way such email was to be handled and in general the practice at most institutions was based in habit and not formal procedure.
Most respondents stated a strong desire to hand responsibility for content to the content creator or area that generated it, to decentralise the responsibility for content. This may seem at odds in a sense with the concurrent strong desire to centralise look and feel and design and the overall navigational structure and accessibility soundness. CMSs were again seen to be crucial in solving the issue as a win/win for all parties. There were respondents who felt that they had quite a bit of responsibility for content and it was not always desired a role. Some of the more positive of these respondents however worked closely with content creators in marketing or public relations and felt they could interpret the needs of those particular areas better as a result.
When asked whether or not they archive their older Web Site content a staggering 45% do not. Figure 8 illustrates the divide between those who archive their university Web Site content and those who don’t.
Figure 8 Do you archive your older Web Site content?
Even though one University noted that they were required by statute to backup and archive their Web Sites the fact that 45% do not back up and of the 55% that do many do so informally or in an adhoc (irregularly to CD-ROM instead of following a more structured procedure for example) fashion should be of considerable concern as a legal and technical risk to these Universities. Most of the respondents who back up and use a CMS stated that there was a built in archival ability as part of their CMS.
The respondents detailed different strategies for the phasing in of CMS at their universities or the partial steps they have taken in getting to the situation where they could use a well-developed and resourced CMS. The considerable cost of the various CMS options available on the market was a great concern to respondents and there was a feeling that the limited budgets allocated to the Web Site management teams could easily be entirely spent on the implementation of a CMS. As a result of the concern with large CMS costs several respondents discussed how their institution had developed a CMS in house and, in one instance, how a team of final year computing students had developed as a final project the University CMS. Those who couldn’t afford a CMS often had extensive use of template and Server Side Include (SSI) based systems or ones based on meta data expiry and built in review dates used where software reads the expiry and notifies the person responsible. These types of system were mostly a work in progress or a very low level CMS for page expiration and none of the additional features of full-blown CMS. They were generally viewed as a stepping-stone to the use of an automated CMS and were not very satisfactory in ongoing management of these large Web Sites due to their somewhat manual nature of their upkeep.
Respondents were asked who ultimately determined the style and design of their site (see Figure 9).
Figure 9 Who ultimately determines the style and design of your Web site?
The majority of respondents stated that they used a committee approach where a Web Site management committee, made up of various stakeholders that met regularly, determined the visual design and style of their organisations Web site. One respondent felt that the committee didn’t really work effectively as many members were often too busy to attend every meeting and as a result decisions were hard to implement quickly. They also stated that the committee ultimately made his job easier by taking over the top level decision making and he could devote more attention to his other core job tasks. This respondent was a one person team who expressed relief that some of his time was freed up by the creation of the committee.
The general feeling was that the formation of stakeholder-based committees was a positive development in that Web Site consciousness and interest was permeated throughout the institution and that decisions made by the committee tended to be supported by the university population. The Web Site managers saw strong executive level representation on the committees as a means of protecting the Web Site management unit from the internal politicking found at the institution and for facilitating positive outcomes for the team.
There was an explicit example of what usability expert Jakob Nielson describes as the “Vice-Presidents button”, where a high level executive personally dictates the design, placement and colour scheme of the corporate Web Site. There is ample evidence in the literature that this situation is less than desirable for users (Web Site designers and marketing specialists typically produce more usable designs) and is a poor use of executive time. Other respondents alluded to this situation but only one explicitly stated that their Vice-Chancellor determined the style and design of the University Web site.
The respondents were frustrated, and somewhat amused, at the situation and felt that politically they were compelled to comply no matter how unprofessional the result. Interestingly this situation was more prevalent at universities that were members of the Group of 8 (the older “Sandstone” universities).
External agencies were contacted occasionally for design advice or assistance in developing templates but this was not a typical circumstance.
The respondents that stated that their own Web Site team was responsible for design felt that they had a good level of design experience and that their team was well suited to the task. The next report that examines the usability of the studied population will hopefully shed light on the whether this is true and which are the most usable site designs from the users perspective.
Respondents that worked with the marketing unit at their university to design the Web Site look and feel often felt that they were part of a team and that they were facilitators whose job was “getting the message out there”. This contrasted with the respondents whose team took responsibility for design. The second set of respondents typically stated that they were a more independent team due to their expertise. They protected their Web Site development process in a “secret web business” fashion not particularly willing to see their creative independence compromised by less knowledgeable areas of the university. This group especially feared the “Vice-Presidents button” scenario of Web Site design.
Respondents were asked what types of procedures they undertook when evaluating a new Web Site design (see Figure 10). Thirty eight percent (38%) of respondents said that their unit would brief other stakeholders within the university (most typically marketing and management areas or a Web site management committee) of the design options available and gather feedback on the designs during the briefing session.
Figure 10 What testing procedures do you undertake when evaluating a new Web Site design?
Twenty four (24%) of respondents conducted formal user testing of design options with targeted groups of stakeholders though the users were often internal staff and rarely students or potential students. Of great concern is the finding that seventeen percent (17%) of respondents stated that they did not test their new designs. Given the increasing reliance on Web sites as a key source of information it seems counter productive to fail to undertake some testing of the site design involving its primary users. Some Web Site teams ran focus groups with stakeholders and checked the design against predetermined criteria and others met informally with stakeholders to assess the visual design, which would then be determined by the Web Site team. Several respondents saw user testing as a means of achieving more a usable Web Site (one of their stated goals) and thus were very keen to continue the process. There was one institution in the survey that claimed that their design was tested on over 1000 students across browsers and platforms.
When asked a whether or not their University conducted formal user testing on their Web Site, 52% of respondents stated that they did no formal testing with users.
This result should be of great concern to university managers as there is a considerable amount literature that emphasises the improvements in user experience gained by the conduct of formal user testing. Twenty nine percent (29%) of those that did not conduct formal user testing stated that they desired to do so but for a variety of reasons (mostly budgetary and time constraints) they could not do so at the present time. Those that did conduct user testing did so typically with small user numbers, less than 15 users, (except for the large user, one thousand plus subjects, testing phase at one university). Usability experts such as Jakob Nielson encourage small user groups used in testing due to the personal and fine nature of the data collected (Nielson et al, 2000). Large scale user testing is often prohibitively expensive and blurs the individual user experience so that precision is lost in interpreting the results.
When the Web Site managers were asked what they thought the design priorities of the Web Site was, many of them stated that there was no one cohesive institutionally accepted priority and in fact that there were many (including usability and accessibility). However when pressed to decide upon one option by far the majority (59%) selected marketing (see Figure 11).
Figure 11 What are the design priorities for Web Site?
Course information was determined to be a strong priority by 41% of respondents. No respondents selected research or enrolment information as priorities for the site design. While a very interesting finding it may result from the limited sample of university staff interviewed. Views from Web Site managers may not be reflected in other areas of the institution.
Sixty nine percent (69%) of Web Site managers reported that they follow a formal accessibility policy. The high number of negative responses again should be a great concern to the management of the 41% of universities that do not follow an accessibility policy as there are Australian Federal Government guidelines regarding accessibility access and these are not being followed. The reasons for this are again often resource and time related the managers who do not follow an accessibility policy typically do not have time to administer systems, develop policy and creatively add to the Web Sites they manage as well as ensure universal access. Those that did follow policy often stated that accessibility was an institutional goal and they used it as a guide in determining the design and technical nature of their Web Sites the policy was a reference tool they used as part of their work.
Eighty six (86%) of respondents use Web Site statistics in their Web Site work. Though the degree to which this was formalised and sophisticated varied considerably.
Many of the Web Site managers were aware of the potential pitfalls in interpreting Web Site statistics. A high traffic area of a Web Site may be because of a design fault that leads users to the wrong area. The traffic is high but not indicative of the design fault and the stats may be misinterpreted as being a positive result. Most of the respondents who use statistics often generate them for interested areas within their university, very often the marketing department. One respondent stated that they found that seventy percent (70%) of the traffic on their corporate Web Site was from staff accessing the Web Site internally. As such they altered a large part of the information structure and design on the corporate Web Site and migrated much of the staff accessed content to their staff Intranet. WebAlyzer and WebTrends were the two statistics packages most mentioned as being useful to Web Site managers though there were many instances of basic Analog system statistics being used. The high cost of WebTrends [HREF 1] and similar statistical products were seen to be barriers to the use of the more sophisticated and useful statistical products available.
When asked whether they believed their Web Site to be centralised or decentralised 65% of respondents stated that they felt their site was decentralised (see Figure 12).
Figure 12 Would you describe your Web Site as centralised or decentralised?
Managers of decentralised Web Sites reported higher levels of frustration with their ability to maintain site design and policy consistency across the university. Often internal institutional politics was seen to be the cause of the evolution of the decentralised systems. Many of the decentralised universities were actively pursuing development or tendering for CMS to assist in the greater centralisation of university sanctioned templates and as a means of clearly separating the responsibility for content from the Web Site management team. The interview data showed this seemed to be a definite trend.
It was very apparent that the Web Site managers did not want to be responsible for the management of content on the Web Site but that they did definitely want to enforce consistent design and policy on the areas outside their immediate control.
CMS were seen to be crucial in the solving of this situation. Content creators would be given high levels of autonomy for their sections and Web Site managers could ensure corporate consistency of content from the devolved organisational units.
The Web Site managers were asked whether they though that their team were able to keep up with emerging technologies and whether they felt their training was adequately resourced. Most felt that they were well resourced but that time and task management issues meant that they were often unable to be proactive in obtaining training. Most teams kept abreast of emerging trends when they could fit in the time and it was typically self-directed. This was seen to be a generally unsatisfactory situation by the respondents as they felt reactive and often had to “catch up” on emerging issues rather than be at the forefront.
Respondents were asked about their role in training other staff within their universities’. Many conduct workshops and training for content producers on how to use the approved HTML templates or how to use the university CMS. Most of those who were trainers saw the training sessions as part of their job but time away from their desks invariably meant catching up on work at a later date. One university brought in external trainers to take the workload off the Web Site management team and to make it obvious to the devolved content areas that they were to pay for the training.
It was very apparent that the Web Site managers did not want to be responsible for the management of content on their Web Site but that they did definitely want to enforce consistent design and policy on the areas outside their immediate control. Use of a University wide CMS was seen to be crucial in the solving of this situation. Content creators would be given high levels of autonomy for their sections and Web Site managers could ensure corporate consistency of content from the devolved organisational units.
Development of CMS was often seen to be an upcoming primary focus of the Web Site management team barring budgetary or political concerns. Some respondents looked forward to CMS technology but also thought that training will become a larger issue for Web Site management team as devolved content creators will need to be trained in the use of the CMS.
Further analysis of the data will hopefully show whether or not there is a relationship between University size and Web Site management issues that have been highlighted in this preliminary research.
The analysis will also hopefully show if there is a significant difference between the way that the so called “Sandstone” universities, and their technical and regionally based competitors.
This study identifies the perception that the eventual rollout of CMS on an organisational level to be an essential requirement for optimal delivery of Web Site data and that use of a CMS will assist in good human resource planning, marketing, and potential fund raising through more effective presentation to local and international markets.
Bates, T,. (1997) The impact of technological change on open and distance learning,. Journal of Distance Education, Volume: 18, Issue: 1, Pages: 93-109
Bates, T,. (2001) International distance education: Cultural and ethical issues,. Journal of Distance Education, Volume: 22, Issue: 1, Pages: 122-136
Ellis, A. & Patterson, K. (2002) Design and Navigation: A Survey of Australian University Homepages, Proceedings of AusWeb02, pp 483-485 Southern Cross University Press, Lismore, ISBN 1 863844538
Fiore, M. (1997). "Colleges ponder the pros and cons of having students
design Web sites." The Chronicle of Higher Education 43(47): A22.
Kirkwood, A. (2000). Learning at home with information and communication technologies,
Journal of Distance Education, Volume 21, Issue: 2, Pages: 248-259
Neilson, J., (2000), Designing Web Usability. New Riders Publishing, USA.
Nielson, J; Snyder, C; Molich, R; Farrell, S. (2000), E-Commerce User Experience
Methodology, Nielson Norman Group
O'Reilly, M., & Patterson, K., 1998 Assessing learners through the WWW,
Computer Networks and ISDN Systems, 30, pp 727-729, Washington.
Patterson, K., Ellis, A. & Brice, D. (2000) Client Versus Browser: A Case Study from Southern Cross University, in M. Wallace, A. Ellis & D.Newton (eds) Proceedings of the Moving Online Conference, pp190-202, Southern Cross University Press:Lismore ISBN 86384 466 X
Spool, J., Scanlon, T., Schroeder, W., Snyder, C. & De Angelo, T., 1999, Web Site Usability: A Designers Guide., Morgan Kaufman Publishers., San Francisco CA.
Youngblood, P; Trede, F; Di Corpo, S,. (2001) Facilitating online learning: A descriptive study, Journal of Distance Education, Volume: 22, Issue: 2, Pages: 264-284