Assessing University Homepages from Web standard and usability perspectives

Tran Quoc Bao [HREF1] Graduate College of Management, Southern Cross University [HREF2]. btranq10@scu.edu.au

Associate Professor Allan Ellis [HREF3] School of Commerce and Management, Southern Cross University [HREF4]. allan.ellis@scu.edu.au

Abstract

Academic institutions and universities are often Web-technology pioneers. Because they often have their own IT department to build and manipulate their Web sites, universities frequently touch up their Web sites, especially the homepage, to produce better marketing performance.

Over recent years published pilot studies have reported research involving Web standard and usability issues. This research uses a combination of measures to analyse relevant 2007 concerns about homepage redesign and performance and uses some case studies to compare and highlight current differences. A pilot investigation was undertaken of 31 institutions around the world: 10 from Australia, 3 from Asia, 4 from England, 1 from France, 13 from the US. In term of focus and disciplines, there are 21 universities (general curricula) and 10 business schools.

The results reveal that British and Australian institutions appear to be more compliant with Web standards and usability issues. Other institutions show wide variation in the way they organise information on their homepage. Most have scope for significant improvement.

Introduction

Alexander (2005) conducted a study of usability and search ability issues involving 13 Australian and 2 overseas university Web sites. She concluded with 5 major action orientated recommendations (pp317-318):

• Design an information architecture that meets prospective student needs
• Create content that meets the needs of prospective students
• Improve the performance of search
• Do not assume that prospective students have the relevant domain knowledge
• Do not use PDFs - the primary format for web content

Ruwoldt and Spencer (2005) used a printout of home page screen shots as the basis for a questionnaire involving 68 Australian and overseas universities that sort comments on specific aspects of content, labelling and navigation, design and branding. They concluded that best practice in terms of information architecture involved providing….” multiple navigation paths into the broader web site: (p431)

• Group static links into audience and topic categories; label the groups "For" and "About"
• Where appropriate, provide two or more links from the home page to a key content page; these links should have different titles
• Visually emphasised links to key content
• Allow users to choose between using a search engine or browsing a site map or index/directory.”

Nichani (2006) surveyed 25 universities, mostly from Australia, British and the US, and focused on Web standards and navigation structures. She concluded Web site re-design projects were breaking new ground and that considerable experimentation was going on. DeWeaver and Ellis (2006) surveyed a representative sample of 9 universities (NSW and Queensland) on 28 marketing parameters. They concluded that even though universities had over 15 year Web marketing experience there was surprising variation with some rating very low. They suggested that …”in order for Web-Marketing to be effective there needs to be greater integration of design with content linked to as better understanding of how visitors move around on a website as they seek information”(p.14).

This pilot study builds on these earlier works by examining and rating the full range of important factors including information architecture, usability concerns, technical issues and marketing effectiveness.

Methodology

This pilot study involved examining and rating a sample of university Web sites based upon survey methods and an amalgamation of criteria adopted from the published studies just cited. Universities and business schools are from the top-tier institutions, always quoted in the Higher Education Supplement 2006 [HREF5] or Financial Times MBA Ranking [HREF6] and Australia Group 8 [HREF7], all over the world and Asia Pacific region. They are selected because of the strong belief that they can outperform in their Websites. There are 10 institutions from Australia, 3 from Asia, 4 from England, 1 from France, 13 from the US. In term of focus and disciplines, there are 21 universities (general curricula) and 10 business schools.

Researchers first validated these homepages by Markup Validation Service [HREF8], Cascading Style Sheet (CSS) Validation Service [HREF9]. If any failed in either validation tests, the Hypertext Markup Language (HTML) code was examined. Usability metrics adapted from (Nielsen & Tahir, 2002) were used to assess the usability of those homepages. Results were presented in Excel Spreadsheet and grouped by disciplines. In some metrics, the program SPSS was used for more in-depth statistical analysis. It allowed us to both observe the general trends and evaluate each individual exception.    

Criteria

No standard, agreed set or criteria exists to examine and rate university Web sites. Drawing on previous studies this survey uses criteria selected from various areas: Web standards, usability concerns, and marketing performance. The assessment criteria include:

• Web standards ensure that everyone from various demographics can have equivalent access to the Web site. For example, people with poor eyesight can get pages rearranged and magnified for reading easily or people using handheld devices can browse the Website as easily as those using a workstation. Likewise, Web standards also produce better search results and maintain visual consistency throughout the site.
• Usability ensures the “ease to navigation” of a Web site. It  includes numerous issues such as file sizes, the layout of search utility and sitemap (Nielsen & Tahir, 2002). Smaller file sizes make the pages load faster. Most Web users navigate a Web site through search tools or a sitemap rather than scanning the content on the homepage. Therefore the search utility and sitemap are essential audience tools.
• Information architecture is the hierarchical structure of the Web site, i.e. topic-based or audience-based, primary navigation and utility navigation methods (Nichani, 2006). Consistency is the basis of a positive user experience. In the university context, it is difficult to get all its departments, schools and faculties to agree on a common layout. Hence information architecture helps provide a common guideline and allows university sub-sites to determine their own content without influencing the user experience.

Universities in the study

In the world of international business and Web sites, English is the dominant language. More universities in English-speaking countries tend to be sought after by international students. It is hard to find an international student who does not recognise the high-profiled names like Harvard, MIT, Stanford, Cambridge or Oxford. Moreover, Asia-Pacific, the fastest growing economic region with countries such as Japan, China and Singapore, has proved its effective educational strategy in economic development. So it should be included. Consequently selected institutions in this paper are the leading institutions from those countries because of the strong belief that these leading institutions should perform better in terms of their Web strategies.

In addition leading business schools were investigated because they generally outperformed other schools in the same university. Sometimes their reputation was even more popular than their parent university. Of the 31 institutions sampled, 21 are universities and 10 are business schools. The disciplines and location of those institutions are presented in table 1.

Table 1: Institutions in this study

There were some apparent overlaps in this selection such as the Australian Graduate School of Management (AGSM) is a school of the University of New South Wales. The Melbourne Business School is a joint venture and one of the stakeholders is the University of Melbourne. The Harvard Business School is a faculty of Harvard University and the Stanford Graduate School of Business is a faculty of Stanford University. However in practice, these business schools operate largely independently of their parent organisation.

Over the last decade universities have been progressively offering courses in an online learning format in the hope of attracting both “mature learners” and providing more flexible course offerings.  Hence, online distance learning has spawned the development of online only universities such as the US based Phoenix University.

Results

These are discussed under the subheadings of Web standards, design, usability, information architecture and metadata. They refer to what sites were displaying early in 2007. Some sites will no doubt change by the time this paper is published.

Web standards

Universities adopting Web standards and Cascading Style Sheets (CSS) are listed in table 2.

Table 2: List of universities compliant with both XHTML 1.0 and CSS mark-up

Surprisingly, there are no business schools that are compliant with both XHTML 1.0 and CSS mark-up. Common mistakes, for which institutions fail XHTML 1.0 validation, include: Use of inline JavaScript; JavaScript with some characters, such as &&, that are not validated by an XHTML parser; use of depreciated tags.  For example many homepages still use<font> or some other attribute which has become depreciated in HTML 4.0.

Design

There are 9 institutions applying table-less design in their homepages. Of the 9 institutions, there are 4 from the U.S.A., 2 from Australia, 1 from England and 1 from Asia. The list of institutions appears in table 3.

Table 3: List of institutions using table-less structure

Analysis shows that some 4/10 (40%) business schools and 5/21 (42%) universities apply a table-less structure. It appears that business schools tend to use table-less design rather than universities. Because their information architecture is much simpler and less hierarchical than those of universities, it is more feasible for business schools to redesign their website. Combining Web standards, CSS and table-less mark-up criteria: Stanford University is the leading institution that strictly follows these criteria.

Usability

Nielsen and Tahir (2002) suggest a usability list for the corporate homepages. A subset of these has been selected to evaluate the usability of university homepages. They are listed in table 4.

Table 4: Usability assessments for institutional homepages (Adapted from Nielsen & Tahir, 2002)

Maximum possible score is 30. Table 5 displays sub-scores and total scores on usability assessment rating for universities.

Table 5: Usability scores for university homepages

There are six universities achieving very high scores (28 to 30), 4 from the U.S. and 2 from Australia. Peking University gets the lowest score of 17. The average usability score of a university homepages is 25.8. Business schools’ usability is slightly lower than that of institutions. Table 6 shows the usability of business school homepages.

Table 6: Usability scores of business schools’ homepage

4 business schools score very high (28 to 30) on the usability rating scale: 2 from the U.S.A., one from England and one from Australia. The Stern business school in the U.S.A. gets the lowest score of 19.

In term of high usability, there are 6 from the U.S.A., 3 from Australia and 1 from England or 46%, 27% and 20%, correspondingly. These figures suggest that American institutions put more effort in usability when compared with institutions from other parts of the world.

What can be learnt from institutions in the bottom of usability assessments? There are a few usability concerns in the lowest-ranking institutions:
• No search utility (Colombia Business School and Stern) or the search box is too small (less than 10 characters).
• No or less visible links, “Contact” and “Privacy Policy”.
• Small text size, less than 12 pixels.
• No footer navigation (Columbia Business School and Peking University)
• Broken or invalid links.
They are the typical usability mistakes that institutions should be reviewed as redesigning and maintaining their Web sites.

Information architecture

If a homepage is well-structured, it is often easy to use. Information architecture is the blueprint of a Web site. Clear visual information architecture results in a Web page with user-friendly appearance. Hence the following sections will discuss the information architecture of institutions’ homepages in order to understand the “Do’s” and “Don’ts” in organising information in the homepage.
 

There are two main classifications for top-level Web site structures. They are the audience-based or topic-based structure:

• Information for (audience-based):  Prospective students, Current Students, Faculty, Staff and Alumni and Family etc.
• Information about (topic-based): the University, Courses, Library, Teaching and Learning, Research and Services

Back to the study, 19 of the 31 institutions incorporate elements of both a both topic-based and audience-based structure. It is the illustration of the so-called “current industry standard” in information architecture of the university homepages (Ruwoldt & Spencer, 2005). All Australian universities, excluding Australian business schools, follow this combination. In addition, most Australian-based institutions add ‘International Students’ as a category in the primary navigation or sub-level, reflecting their internationalised marketing purpose.

Left-hand rail, links across the top and categories cascading in the middle of the page (or Yahoo-style) are dominant in primary navigation structures. 16 of the 31 institutions employ those methods although they are space-consuming.  Visual-effect menus using JavaScript, such as drop-down and fly-out menus, are also popular. There are a total of 11 institutions using this approach despite debates about their usability shortcomings. The key benefit of those approaches is to save the screen real estate for other important information. The disadvantage is that visitors cannot see the sub-categories of a main category unless they “mouse” over the category. 

How many news or events appear in the homepage? Statistics in this study are shown in table 7 below.

Table 7 Mean value of news and events appeared in institutions’ homepage from this study

For business schools, there are usually up to 8 news and 2-3 events because they have relatively more space. For a university, there are typically two news items and one event. Those numbers are much less than those in a business school because the university needs to save the screen real estate for other items. Summing up, current practice is to have 3-4 news items and 1-2 events appearing on an institution’s homepage. In addition such dynamic media items as news, events or announcements should be syndicated using RSS. 30 out of 31 institutions surveyed used RSS for syndication.

At present the use of podcasts by institutions is mostly for teaching and learning activities. In the near future it is highly likely that podcasts will be widely used for media and marketing events and public relations (PR) activities. Indeed some universities have started to use podcasts to promote their branding. For example, Sydney University has started to use podcasts to publish their public lectures under the link ‘Listen to our public lecturers’. Likewise the Australian National University uses podcasts to publish its public talks.

Marketing effectiveness

Metadata

What is the difference between the University of Sydney and Sydney University? Most people will think there is little difference. But to the search bots (i.e. Google search engine HREF41), this variation is significant. The following figure (figure 1) presents the number of search results with the keyword: Sydney University. 

Figure 1: Google search results with the keyword: Sydney University

And figure 2 illustrates the number of results with the keyword: the University of Sydney.

Figure 2: Google search results with the keyword: the University of Sydney.

While the conventional name of the university is “the University of Sydney” the number of search results with this keyword is 11% less than a less formal name “Sydney University”. Examining the HTML code of the university’s homepage, there is inconsistency in the way it names itself (figure 3).

Figure 3: metadata of HTML code of Sydney University Homepage

Similarly, the University of Melbourne: keyword “the University of Melbourne”: 688,000 responds, keyword “Melbourne University”: 852,000 responds. It is a difference of 24%.

Institutions can pay millions of dollars to have their banners appearing in other Web pages to improve their Web accessibility. However just a small rephrasing in their metadata keywords can improve their accessibility, e.g. up to 11% for the University of Sydney and 24% for the University of Melbourne. This illustrates the significance of controlled vocabulary in improving accessibility and Search Engine Optimisation (SEO).

In fact, of the 31 institutions, there are 10 not using metadata in their homepages. They include 3 from Asia, 1 from France, 2 from Australia and the other 4 from the U.S.A. Most Australian institutions actually provide more that just minimal metadata. They use not only the frequent metadata tags “description” and “keywords” but also Dublin Core Metadata schemas to improve their accessibility. Monash University even goes beyond this and has its own metadata schema bases named “monash” [HREF42]. Figure 4 displays a sample of Monash metadata.

Figure 4: Monash metadata

In general, British and Australian institutions make considerable use of metadata compared to others. All British institutions and 82% Australian institutions use metadata. However, there are various concerns relating to thesaurus design and controlled vocabulary in their metadata, that the scope of this study cannot cover.  

PageRank

Google PageRank™ [HREF43, HREF44] is to measure how relatively important a page is. It is a whole number between 0 and 10. It does not rank the academic quality and teaching practice of an institution; but partially indicates the significance of the website in the WWW.

Although, PageRank is a reference Web metric rather than a precise Q&A indicator, the rank of those institutions’ page can reveal many interesting aspects of their online marketing effectiveness.  

• MIT is the only institution with a rank of 10. It should be acknowledged that only a few corporations or portals in such a high rank, i.e. Microsoft [HREF45], Yahoo [HREF46] or W3C
• Most US and British leading universities are ranked of 9 but the Imperial College London (8).
• Australasian universities, including Australia Group 8, NUS, Peking and Tokyo, are ranked of 8, except the University of Adelaide (7).
• Most business schools’ pages are ranked less than those of the universities. The exceptions are Harvard, Stanford and London Business School (8).

MIT stands out from other universities because it is always quoted as the leading technology and computing university. The US and British leading universities are still the dominant power in academic and teaching; therefore, their ranks are higher than those of Australasian colleagues. Business schools’ pages are often ranked less than those of universities since their academic and teaching scope is less intensive.

Google analytics (GA) [HREF47, HREF48]

GA generates Web statistics about the visitor to a website. It can be a useful indicator to improve content quality and navigation structure of a website. In the sampling, there are up to 16 institutions using GA to record the statistics of the traffic to their website. It is obvious that institutions recognise the usability and content of their website to improve online marketing effectiveness.

Issues for dual-language websites

NUS, Tokyo and Peking University share a similar solution to maintain a dual-language mode for their portal. They separate their local language pages and English pages into distinct information flows.  
     
 Figure 5: dual-language mode of Asian institution websites in the study.

The default homepage of NUS and Tokyo University is English (2) whereas in Peking University (PKU), it is Chinese (1). NUS and Tokyo use an identical layout for their (1) and (2) but PKY applies totally different layouts for its (1) [HREF49] and (2) [HREF20].  The NUS arranges its Chinese pages in a subfolder [HREF50]. For example, in NUS, (1) (1.1) (1.2) etc… are in a directory and (2), (2.1), (2.2) etc… in directories. Tokyo maintains the dual-language mode in the same directory. It differentiates pages by appending a postfix for the page name. For instance, (1) is named index_j.html (j for Japanese), and (2) is name index_e.html (e for English).

Maintaining a dual-language mode for institutions is a traumatic task in this way. As generating content in English, the Webmaster created an equivalent content (translation) in the local language with an identical layout (in case of NUS and Tokyo). In PKU, the task seems harder because of different layout.

Discussion

In Web standards, Australian institutions appeared to be more compliant. In usability, U.S. institutions are more advanced. There is no golden model for institutions from this study but the following institutions have actually done well in terms of the main assessment areas of this study:

• Stanford University is the leading institution because of XHTML Strict 1.0, CSS, table-less mark-up and very high usability score (28 out of 30).
• Oxford University and Monash University share the second position because of XTHML Strict, CSS mark-up and 27 out of 30 usability score.

Compared to institutions from other countries, Australian institutions are more unified in terms of their use of information architecture and compliance with Web standards and CSS mark-up. Moreover, in their homepages, they often include ‘International Students’ as an audience option, which demonstrates that Australian universities have a strong focus on international students.

Business schools tend to present the content like a news portal. It allows a much higher number of media items on their homepages than that in universities. It is noteworthy that business school are typically an independent unit or a partnership with a university and the size and its offerings are smaller and fewer than those of the entire university. It results in the need to display less information in a business school’s homepage relative to that of the whole university. Such specialisation seems to promote the use of additional utilities such as search, jobs or ‘enlarge the text size’ and especially news and events items.
 

Conclusion

This pilot study provides a brief assessment of university and business school homepages in the key areas of Web standards, usability and marketing effectiveness. While these are important clusters of factors they are not the only factors and a host of other minor factors play a role in making a Web site fully functional and attractive to users. A trend emerging from successful sites is to employ a combination of industry standard practices in relation to coding and layout combined with a mix of topic-based and audience-based structure.

References

Alexander, D. (2005) How usable are university websites? A Report on a Study of the Prospective Student Experience, AusWeb05: Proceedings of the 11th Australasian World Wide Web Conference, Southern Cross University, Lismore, pp 303-320.

DeWeaver, L. & Ellis, A. (2006) University Web-Marketing: A Report Card,  in AusWeb06: Proceedings of the 12th Australasian World Wide Web Conference, Southern Cross University, Lismore pp 9-14.

Digital Inspiratio (2005) Prevent Google Analytics from tracking your visit.   Retrieved 12 -Dec, 2006, from http://labnol.blogspot.com/2005/11/prevent-google-analytics-from-tracking.html

Monash University. (2004) Search and metadata.   Retrieved 13-Jan, 2007, from http://www.monash.edu.au/staff/web/search/

Nichani, M. (2006) The Changing Face of University Websites.   Retrieved 15-Dec, 2006, from http://www.pebbleroad.com/article/the_changing_face_of_university_websites/ 

Nielsen, J. & Tahir, M. (2002) Homepage usability: 50 websites deconstructed. Indianapolis, Ind: New Riders.

Phillipson, G. (2006) Getting Connected, Campus Review, April 11, pp12-13.

Ruwoldt, M. L. & Spencer, C. (2005) Navigation and content on university home pages. AusWeb05: Proceedings of the 11th Australasian World Wide Web Conference, Ausweb2005, Southern Cross University, Lismore, pp 431-434.

Hypertext References

HERF 1

http://keronii.wordpress.com/

HREF 2

http://www.scu.edu.au/schools/gcm/

HREF 3

http://www.scu.edu.au/staffdirectory/person_detail.php?person_id=88

HREF 4

http://www.scu.edu.au/schools/comm/

HREF 5

The Higher Education Supplements http://www.thes.co.uk/statistics/international_comparisons/2006/top_unis.aspx?window_type=popup

HREF 6

Financial Times MBA rankings http://rankings.ft.com/rankings/mba/rankings.html

HREF 7

Australia Group 8 http://www.go8.edu.au/

HREF 8

W3C Validation Service http://validator.w3.org/

HREF 9

Cascading Style Sheet Validation Service http://jigsaw.w3.org/css-validator/

HREF 10

Australian Graduate School of Management http://www.agsm.edu.au/

HREF 11

Columbia Business School http://www0.gsb.columbia.edu/

HREF 12

Harvard Business School http://www.hbs.edu/

HREF 13

Insead http://www.insead.edu/

HREF 14

London Business School http://www.london.edu/

HREF 15

Melbourne Business School http://www.mbs.edu/

HREF 16

New York University: Stern http://www.stern.nyu.edu/

HREF 17

Stanford Graduate School of Business http://www.gsb.stanford.edu/

HREF 18

The Wharton Business School http://www.wharton.upenn.edu/

HREF 19

University of Chicago Graduate School of Business http://www.chicagogsb.edu/

HREF 20

Peking University http://www.pku.edu.cn/eindex.html

HREF 21

Cambridge University http://www.cam.ac.uk/

HREF 22

Harvard University http://www.harvard.edu/

HREF 23

Imperial College London www3.imperial.ac.uk/

HREF 24

Massachusetts Institute of Technology http://web.mit.edu/

HREF 25

Melbourne University http://www.unimelb.edu.au/

HREF 26

Monash University http://www.monash.edu.au/

HREF 27

National University of Singapore http://www.nus.edu.sg/

HREF 28

Oxford University http://www.ox.ac.uk/

HREF 29

Phoenix University http://www.phoenix.edu/

HREF 30

Princeton University http://www.princeton.edu/

HREF 31

Stanford University http://www.stanford.edu/

HREF 32

The Australia National University http://www.anu.edu.au/

HREF 33

The University of Adelaide http://www.adelaide.edu.au/

HREF 34

The University of Queensland http://www.uq.edu.au/

HREF 35

The University of Western Australia http://www.uwa.edu.au/

HREF 36

Tokyo University http://www.u-tokyo.ac.jp/index_e.html

HREF 37

University of California, Berkeley http://www.berkeley.edu/

HREF 38

University of New South Wales http://www.unsw.edu.au/

HREF 39

University of Sydney http://www.usyd.edu.au/

HREF 40

California Institute of Technology http://www.caltech.edu/

HREF 41

Google http://www.google.com

HREF 42

"monash" namespace http://www.monash.edu.au/staff/web/search/

HREF 43

Google PageRank™ at Google Technology http://www.google.com/technology/

HREF 44

Google Page Rank at Wikipedia http://en.wikipedia.org/wiki/PageRank

HREF 45

Microsoft http://www.microsoft.com/en/us/default.aspx

HREF 46

Yahoo http://www.yahoo.com

HREF 47

Google analytics (GA) at Google http://www.google.com/analytics/

HREF 48

Google analytics (GA) at Wikipedia http://en.wikipedia.org/wiki/Google_analytics

HREF 49

Peking University (PKU) Chinese website http://www.pku.edu.cn/

HREF 50

National University of Singapore (NUS) Chinese website http://www.nus.edu.sg/chinese/

Copyright

Tram Quoc Bao & Allan Ellis © 2007. The authors assign to Southern Cross University and other educational and non-profit institutions a non-exclusive license to use this document for personal use and in courses of instruction provided that the article is used in full and this copyright statement is reproduced. The authors also grant a non-exclusive licence to Southern Cross University to publish this document in full on the World Wide Web and on CD-ROM and in printed form with the conference papers and for the document to be published on mirrors on the World Wide Web.