Jeni Li [HREF1], Arizona State University, PO Box 870101, Tempe, AZ 85287-0101. jeni.li@asu.edu
User-Centered Redesign, Accessibility, Usability
This paper details the "G5" redesign project culminating in the Fifth Generation of Arizona State University's Web site. The redesign's major goals include accessibility, usability, and ease of maintenance, as well as a nicer image for the university. The paper explains the project's genesis, goals, method, pitfalls, and saving graces. Results of user focus groups and online surveys conducted during G5 are included and/or linked to within the paper.
Arizona State University[HREF2] began its G5 Project (so called for the fifth generation of the ASU Web) in the Spring of 1998. This project is far more comprehensive and user-centered than previous Web redesign efforts at ASU, with strong emphasis on user feedback as the basis for the new design and on resolving Web developer needs to ensure a consistent and well-maintained site.
This paper details the project's genesis, goals, data collection methods and results, obstacles, and lessons learned along the way. Current information about the project is available on the G5 site [HREF3].
Arizona State University's Web site is well into its third year with essentially the same design and organizational structure. Frankly, our duct tape is showing! The site's top levels have undergone little substantive change over this time: Elements have moved around on the pages, new information and services have been "patched in" deeper into the site, and new features have been added to the top of the site with little regard for their impact on ease of use. The end result is that users take more time and experience greater difficulty finding what they're after on our site.
The current ASU Web shows little design consistency from page to page and section to section. Design standards exist, but they're loosely defined, poorly disseminated, and seldom enforced. A rough analysis of users' electronic mail questions indicates that the current site's inconsistent design contributes to user frustration and a general sense of uncertainty and confusion about the site.
We continue to receive positive feedback about the current ASU Web; however, we know it can be much better for all concerned. The G5 Project is a proactive effort to make the ASU Web work better for our users, our developers, and our image as a Research I university.
User survey results [HREF4] indicate that 40-60% of our audience accesses our site with a modem connection (split down the middle between 28.8 and 56k). This is fairly consistent with the results of GVU's 10th WWW User Survey [HREF5].
The current ASU Web's "entrance" page (along with many other pages on the ASU Web) carries a total document weight of nearly 100k -- far too hefty for the impatient and bandwidth-challenged user. Our target for the new design is a total document weight of no more than 10-20k per "core" page.
In order to navigate the current ASU Web easily, one must be familiar with ASU's organizational structure and terminology. This is a stretch for, say, the seventeen-year-old college-shopping secondary student or the overseas prospective graduate student.
Early data indicated that a user-categorized site ("I know who I am") would work well; however, organizational concepts based on this approach were less successful than we'd hoped. Focus group input and survey responses are now leading us toward more of a "front door/back door" approach, offering different entry points for individuals who are new to ASU and for individuals already affiliated with ASU.
In the United States, the Department of Justice recently ruled that Web sites of public institutions (e.g., state universities), like their physical facilities, must accommodate individuals with physical disabilities under the Americans with Disabilities Act. In our particular environment, the primary concern is supporting the visually impaired. A site that properly accommodates blind users will generally be accessible to users with other impairments, including the mobility-impaired. Since we have very little audio content, little accommodation is required for the hearing-impaired.
A text-based design offers many advantages over a graphical design for the visually impaired: Text can be enlarged on the client, foreground and background colors can be overridden for greater contrast, and of course a text-to-speech reader can deliver the page verbally.
The current ASU Web is designed to accommodate a "text-only" version of each page. This puts the burden on developers to maintain two versions of every page. Compliance is spotty at best, and where text-only versions are maintained, they're often out of sync.
A major goal of the new design is to render acceptably in text-only and text-to-speech browsers within a single set of pages, thereby eliminating the need for dual maintenance of content and the risk that text-only versions might be inaccurate or lacking.
As our measurement of the site, it should comply with the W3C's Page Authoring Guidelines [HREF6] and pass Bobby [HREF7]. Finally, and perhaps most importantly, the site should incorporate what the project team has learned in focus groups with visually impaired students at ASU (more on this appears later in the paper).
Our access logs [HREF8] indicate that a broad range of browsers are being used to hit our site. Even among the "big two", Netscape Navigator and Microsoft Internet Explorer, several versions are still in use. In fact, only about 58% of our accesses are made with version 4.x browsers at present -- a remarkable statistic when one considers that this includes our remote computing site machines with 4.x browsers installed and home pages pointing to the ASU Web on startup. Actual use of the ASU Web with 4.x browsers in the "real world" must then be considerably lower. (Note: While the access logs presented here are fairly old in Web years, the 4.x browsers have not gained greatly on their older counterparts in our more recent site statistics.)
Supporting multiple browsers and multiple browser versions meant foregoing some newer standards and enhancements in favor of older techniques with wider browser support. We love cascading style sheets, but the browser support for them isn't there yet. Dynamic HTML allows for spiffy effects, but renders like garbage in a text-to-speech browser like pwWebSpeak [HREF9] or a Windows-based screen reader like JAWS [HREF10]. The new design's approach is not a "lowest common denominator" approach, but one of intelligent enhancement choices and consideration for the site's rendering in multiple browsers.
We're exploring longer-term solutions to this issue, including a database-driven design that would allow content delivery "on the fly" using a number of browser-specific templates. A database-driven design would have advantages in consistency as well as in customization, but would require greater resources in server hardware to ensure adequate performance. Resource requirements can be overcome. Greater obstacles exist in developers' desire for creative licence and in the ASU Web community's difficulty in grasping the concept of database-driven design.
Search engine results are unpredictable on the current ASU Web, largely because developers' use of metadata is inconsistent. The new design is to include standards for document metadata, and will probably require developer tools to update new and existing pages with appropriate metadata.
Content is published on the ASU Web by departmental "information holders" -- individuals, often administrative assistants or student workers, with little or no training and experience in Web development. If the new design is to be adopted successfully, it must be mind-blowingly easy to comply with. Interestingly, many developers have had great difficulty using the server-side includes that are part of the current ASU Web style guide. The project team is in the process of authoring Web-based developer tools to create and edit Web pages as well as tools to assist in converting existing pages to the new style.
The project team solicited input by electronic mail for several months and has received approximately 200 responses. These responses have guided development and refinement of design prototypes. A sample of electronic mail input is published online [HREF11].
The project team hosted a number of facilitated discussions with ASU faculty, staff, and students to gauge user reactions to the current ASU Web and pinpoint areas for improvement. Feedback from some of those focus groups has been published online [HREF12]. Many of the new design's goals directly reflect feedback obtained in these discussions. This has been our single most valuable source of information so far.
Most of the focus groups were conducted using the ORID discussion method from the Institute for Cultural Affairs' [HREF13] "Technology of Participation" series. An ORID discussion is a guided discussion that leads participants through four stages:
A focus group with visually impaired students, part open discussion and part face-to-face usability test, proved most enlightening [HREF14]. We had understood that many Web sites are highly graphical and inaccessible to visual impairments. Hearing these students speak, however, brought home a harsher reality: The Web itself is all but unusable to many blind users. This group of students had no concept of what the Web is and how to get around it. They asked us, in as many words, what all the fuss was about and whether they were missing much.
Usability testing was conducted in ASU's DRS computer lab, in accordance with Jakob Nielsen's guidelines for basic usability testing [HREF15]. In a very short time, a number of very serious problems became apparent to us. When faced with such a simple navigation element as a link in the middle of a page with the label "Back to Top", blind users became uncertain as to whether they had inadvertently navigated back to the top of a page. The tools they used exacerbated this situation: Some screen readers would ignore images (and, distressingly, their alt text as well!) while some would stop on the first image encountered. Some screen readers would immediately begin reading the text of a page, while some would read the title tag and stop. Some readers would ignore the page's text and force the user to tab through the links, reading only the link labels without context. And this was all in ASU's computer lab for disabled students!
Observing the visually impaired students in action has led to some simple site modifications with profound effects. Rather than begin pages with the university logo as before, we elected to provide the site index and search tools at the very top. The phrase "Click here for ..." will be forever abolished from the ASU Web! Some decorative images are now included as backgrounds for table cells in some of the prototypes, with the effect that screen and text-to-speech readers completely ignore them. Most members of our VI focus group indicated a preference for very plain content, given that most of the usual dynamic features slowed their download time and/or interfered with their ability to access the content. For the most part, their view of a Web site was much more utilitarian than that of most audiences today.
A survey was conducted online to determine users' affiliation with the university, Internet connection speed, familiarity with the Web, willingness to use plug-ins and accept cookies, and other related information as well as to target user priorities among speed, simplicity, and visual appeal in a Web site. Key survey results are listed on the survey summary page [HREF4].
To increase randomness and, if anything, bias the survey toward outside visitors, the online survey was linked to from the very front page of the ASU Web, which is not the default home page on computing site machines. Since the sample is self-selected, we cannot guarantee it is representative of the entire ASU Web audience. However, as stakeholder input for a service improvement project, the sample is quite valuable and larger than we could have hoped for by telephone or face-to-face.
Several organizational concepts (site structures, or site maps if you will) were advanced as possible components of the new design. A usability test was conducted online to gauge each organizational concept's effectiveness and to identify troublesome areas in each. This test yielded a phenomenal amount of data, which is still being analyzed.
Separate test instruments were created for students, faculty and staff, and other visitors to the site. Each test instrument consisted of a set of 20 to 30 items, relevant to the user's affiliation with the university, to look for on a university site (e.g., "You're looking for admissions information"). These items became progressively more esoteric.
Test respondents were presented with text-only organizational concepts at random (to avoid the "learning factor") and asked to select the link where they believed they would be most likely to find the item. When a link was selected, a CGI script recorded the following information:
This allowed us to track the amount of time taken to answer a given question from a given organizational concept, along with the accuracy of the answer as compared with our vision for each site structure. It also helped us to pinpoint "trouble spots" within each concept, identified by a high frequency of "search" links chosen for a particular item.
One weakness of this test was that it tracked links to only one level. This contributed to a predictable result: More detailed organizational concepts yielded longer response times, but more accurate results. Future tests of this nature should track users through an entire site several levels deep, ending on one of several possible conditions:
We believe a usability test of this nature would give broad-based results -- not as rich as, but still not too dissimilar from, face-to-face testing.
When the organizational concepts are refined and a concept has been chosen, the concept will be tested with a group of users under observation. The method will be similar to that of the online usability test, but observation will allow for greater understanding of the user experience with the organizational concept. Information from this usability test will be used to identify any problems with the new structure and refine the structure as necessary before developing prototypes.
When design prototypes are completed and final prototypes selected by the project steering committee, an in-depth usability test will be conducted online to compare the prototypes objectively. The usability test will be similar to the test of organizational concepts, but will track user navigation in greater detail. General feedback will be solicited on the prototypes at the same time.
In an in-depth focus group discussion with departmental webmasters and content providers [HREF16], several areas of concern were identified. Budget, training, and communication among developers were chief among these, and it is clear that these issues must be resolved if the new design is to succeed.
The focus group included a Reverse Brainstorming exercise, Force Field Analysis, and Technology of Participation "Workshop" (another technique from the Institute for Cultural Affairs [HREF13]). In the Reverse Brainstorming exercise, participants emphasized a user's perspective. They were asked first to brainstorm characteristics of "the Web site from Hell". Then, with that list visible, they were asked to brainstorm characteristics of an ideal Web site.
Once the participants were focused on the user perspective, they were asked to identify driving and restraining forces -- conditions that propelled ASU Web developers toward or away from "User Heaven".
Finally, after considering their goal (User Heaven) and their current conditions (driving and restraining forces), participants identified ASU Web developers' needs (in terms of resources, support, and/or organizational conditions) to accomplish a better ASU Web. This was done using ICA's Workshop Method, which is similar to Affinity Grouping or Nominal Group Technique. Given a focus question, participants write their top answers to that question on index cards, one to a card. As a group, they then categorize the index cards in an orderly fashion. The categories this group developed (in priority order) were as follows:
Their complete responses are recorded in the minutes of that session [HREF16].
Higher administration sometimes calls the Web our "farthest-reaching document", and sections of the ASU Web are considered to be the source of record for legal document archival purposes. Yet there is no review process for Web materials at ASU.
A common argument for lax Web development standards is the desire for creative licence. This would be unheard of with regard to printed materials at the university. Printed materials undergo rigorous review and simply aren't printed if they fall short of the university's graphic standards. Web materials, on the other hand, can be published and modified at any time, unbeknownst to anyone other than the developer.
The definition of the Web as "akin to printed materials" or "a living document" seems to be largely a matter of convenience to the discussion at hand. This complicates communication among parties interested in the project, to say nothing of actual policy development.
The project's steering committee has recently become divided over the university Web site's main purpose. Some state that the Web site should be primarily a marketing and recruitment tool, while others maintain that delivering information and services to the ASU community is the first priority. Most committee members have agreed that the quality of information and services delivered to the ASU community will be a powerful form of marketing in itself.
This difference in purposes, coupled with the fact that there are very distinct "external" and "internal" audiences to the site, has increased the emphasis on having two sites -- or at least two entry points to the site -- for users external and internal to the university.
Despite higher administration's commitment that the new site structure and style will be a university-wide requirement, garnering developer interest in the details of that new structure and style has been difficult. In truth, there is no reporting structure between the university webmaster and departmental content providers, nor are there clear consequences for non-compliance with university Web standards. As previously stated, there is no review process for Web materials at this time. Addressing this issue is a key component of the project; views on the issue are mixed. The highly distributed nature of Web development at ASU complicates this matter greatly.
Early in the project, there was great difficulty convincing ASU Web users and departmental content providers of the team's sincerity in seeking user input to the process. Project team members were greeted with suspicion, almost hostility at times. Some specific departmental concerns included:
This is somewhat typical of a departmental love/hate relationship with a centralized department of Information Technology in a distributed computing environment. Debunking such suspicions and actively demonstrating sincerity and responsiveness became (and continues to be) critical to the success of the project. The project team has felt it necessary to address every concern of this nature openly and to seek input more actively and on a broader scale than originally planned. The project itself has undergone significant change based on departmental concerns.
Ultimately, user and developer buy-in is critical to the G5 Project's success. In order to obtain that buy-in, the project team has redefined the scope of the project and taken every measure possible to demonstrate responsiveness to the input received. The team has literally answered every response to its request for feedback (approximately 200 to date) and is constantly updating the project site to reflect changes made to the project as a result of feedback from the ASU community.
This has taken a tremendous amount of time and effort, but we believe that this more than anything else will ensure the project's long-term success. If developers are disenfranchised by the process, they'll be less inclined to comply with new style guidelines. We seek to create ownership of the site among the user community and the developer community.
The G5 Project has a steering committee with representation from all university campuses and departments. Certain members of the committee have been instrumental in providing ways to gather user input on the project.
The steering committee will make the final recommendations to higher administration regarding the new design, the degree of "image legislation" necessary on the Web, developer education requirements, and quality-control procedures.
Several members of the steering committee are likely to serve on a voluntary Web site review board that will review submitted sites for style compliance and make comments to the authors. This review board will probably also approve submitted URLs for inclusion on top-level navigation pages.
Designing Web pages is easy. Designing a university Web site for multiple audiences is a bit more difficult. Designing a process and an environment that encourages consistent, high-quality distributed Web design is the real challenge. For all our talk of user input and user-centered redesign, developer support is a key component of the G5 Project that must not be overlooked. Our focus to date has been on the product: A new Web site. Equally important are the training, tools, and style guidelines the project team is preparing to assist developers in creating and maintaining Web content in an orderly, intuitive site. After the new site is launched, the next focus for G5 is on back-end automation.
The G5 Project is a highly ambitious undertaking. Resource constraints and "committee compromise" may water down its results from what was originally intended. However, by reaching for the stars, we think there's a good chance we'll manage to land on the moon.
Jeni Li
Webmaster, Arizona State University
PO Box 870101
Tempe, AZ 85287-0101
USA
(602) 965-4915 voice
(602) 965-8698 fax
jeni.li@asu.edu
http://www.asu.edu/webmaster
As the Webmaster of Arizona State University, Jeni Li administers ASU's seven main Web servers and supports Web application development and content publication at ASU. Jeni's educational background is with ASU as well (M.Ed. Educational Administration & Supervision, 1998; B.Sc. Computer Science, 1991). Jeni served as Program Chair for WebdevShare98 and N.A.Web'98. She has made several conference, class, and workshop presentations on CGI, Web site redesign, Web security,