Conceptualisation of the Web and Disorientation

Abstract

The study explored the extent of computing students’ perceived disorientation on the Web and the relationship of their disorientation to their conceptualisations of a Web-based learning environment. Intensive data collection methods were used to gain insights into how students work within the Web environment and deal with disorientation and the problems it can give rise to. Understanding how experienced Web users have adapted to the Web environment has provided information about factors that should be considered when providing Web-based learning environments. Also found was that students who had constructed simple mental models of the website experienced less disorientation. A further finding of this research was that students used a range of techniques to navigate on the Web which seemed to be useful in helping their orientation within this environment.

Keywords

Introduction

The Web is now established as a core medium for course delivery in tertiary education. Most courses of tertiary study include online resources as part of the learning environment provided for students: often a variety of resources are incorporated into an integrated Web-based system that may comprise part of or the entire learning environment. With the extensive use of the Web it is important that this medium is used in a way that provides an effective and engaging learning environment for students. It is also imperative that educators and instructional designers make appropriate use of the technology within a sound pedagogical framework that considers the abilities and experience of their students.
Disorientation, or the perception of being lost, is acknowledged as one of the key problems associated with learners working in electronic environments. Norman (1998) discusses this in terms of the extra cognitive load placed on learners by the interface of the electronic environment. With the widespread use of the Web disorientation is an issue which is of interest to most educators. Since the introduction of hypertext applications into the educational arena, various research studies have investigated the factors that can lead to disorientation and the effects of this phenomenon on learning experiences and learning outcomes (Beasley & Waugh, 1996; McDonald & Stevenson, 1996). Many of these studies have explored characteristics of the learners and how they respond within the electronic environment. However, some of these studies are now over ten years old. Many students who commence tertiary study now are very familiar with the Web environment and have extensive experience using the Web for recreational and educational purposes. These students present a different profile to students from a decade ago when the Web was very new. The results of these previous studies must therefore be viewed in the context of a different demographic profile. A study by Altun (2000) found that experienced users tended to develop their own navigation strategies. This leads us to wonder to what extent are students now adapted to working in the Web environment and whether disorientation is still a problem. To explore these questions, the research reported in this paper has investigated how experienced users conceptualise the Web environment and how this relates to their orientation within the Web and the navigation techniques that they use.
This paper presents findings of a study of computing students using a Web-based learning environment that was built to support on-campus students enrolled in an industrial experience project unit. The students were experienced Web users and familiar with the Web environment used in the study. The study explored the extent of students’ perceived disorientation within the environment and the relationship of this to their conceptualisation of this environment. Examining the techniques the students used to navigate and orient themselves within the environment gave insights into how students work within these environments and deal with disorientation and the problems it can give rise to. Understanding how experienced Web users have adapted to the Web environment has provided information about factors that should be considered when providing Web-based learning environments.

Disorentation on the WEB

The concept of disorientation when applied to electronic environments can be broadly viewed as the user having uncertainty about where they are in the virtual space. However, when we consider the different factors that can cause a feeling of disorientation, this becomes a complex issue. The user may not know where they are in relation to the rest of the environment, how they reached their current location or how to proceed to an intended destination. A user experiencing disorientation within an electronic environment can face a variety of problems. According to Hammond and Allinson (1989) the user may: become lost, find it difficult to gain an overview of the material, not be able to find information they need or ramble through the information in an ineffective way.
In addition to these problems, Foss (1989) proposes that disorientation can cause the user to experience a sense of lack of ‘closure’. This is manifest by the user not knowing whether they have explored all relevant parts of an environment or have missed something important. Foss was commenting on stand-alone hypertext applications; with the boundless virtual environment of the Web these problems are exacerbated.
The issue of disorientation is a concern to educators who provide Web-based learning environments for their students. Various studies have shown that disorientation can have a negative impact on learning outcomes (Baylor, 2001; Chiu & Wang, 2000). In addition, it can result in poor learning experiences leading to negative reactions from the learners. As Simpson and McKnight (1989) suggest, if users “are frequently lost, then they will become frustrated and cease to use the system”  (p.82) . However, some suggest there is a positive aspect to disorientation. Baylor (2001) proposes that a certain level of disorientation may be advantageous to the learner and may contribute to incidental learning.
What do we know about the causes of disorientation? Elm and Woods (1985) propose that disorientation is a “result of a failure to consider man and computer together as a cognitive system” (p. 927). Experiencing disorientation on the Web suggests that the learner does not have a clear conception of relationships within the environment. This implies a low correspondence between the learner’s mental representation and the physical representation of the hypertext network.
Various research studies have shown that successful navigation and orientation within the Web environment are dependent on the user forming accurate cognitive maps of the system (Lai & Waugh, 1995; Stanton, Taylor, & Tweedie, 1992). The representation of this understanding in the mind is called a mental model. Norman (1996) discusses how a simple mental model can facilitate successful use of an artifact. In investigating how learners use the Web it seems important to understand how they conceptualise the system. A difficulty here is that a user’s mental model is not easy to determine. They are not observable and cannot easily be articulated (Carmel, Crawford, & Chen, 1992). Not surprisingly therefore, a search of the literature found few studies that had attempted to do this. A study by Gray (1990) requested users to draw their models of their system and used think-aloud protocol to help interpret these. This study found that novices hypertext users presented models which included linear sequences, hierarchies or trees, grids and webs. Their conceptualisation of the system appeared to be influenced by prior understandings and models. In another study by Stanton (1992)  students were asked to draw an outline of the hypertext system on paper, showing the organisation of the screens. This study found that the user’s ability to navigate effectively was related to the development of a good cognitive map.
Our study used similar techniques to Gray (1990) and  Stanton (1992) to elicit students’ conceptualisations of the website under study.

Context of the Research

The WIER website is an integrated environment providing facilities and support for students doing their Industrial Experience project work. The site provides various resources including general project information, a toolkit for project management including a task/time tracker, time graph generator, file management facility, risk management facility, event scheduler, and various forms of communication facilities via news groups and discussion forums. The website also provides access to a repository of resources including standards documents, document templates and samples of past projects.
From a structural perspective, the website consists of more than 130 pages with each page categorised as belonging to a resource. The students access the site from a login page that leads them to the home page. Each resource is accessible via menus from this home page. No mechanisms are provided to enable navigation from one resource to another. This was a deliberate decision of the designers to facilitate orientation within the website. Rather than a heterarchical structure with a network of links, this forms a hierarchical structure of pages about three or four levels deep. More details about the WIER site can be found in another paper (Ceddia, Tucker, Clemence, & Cambrell, 2001).

Study Methods

Students enrolled in a 3rd year undergraduate Industrial Experience Project unit were invited to participate in this study. The students were informed of the study at their weekly seminars early in second semester 2002. At the time of the study there were approximately 175 students in the industrial experience project unit. Eighteen students volunteered to participate.
Each student attended a session with the researcher. During this session they completed a questionnaire to determine demographic information, website access and usage, and their opinions of its usability and the usefulness of its contents. Each participant was then observed as they completed a series of tasks inside the Web environment. Tasks ranged from determining details of a forthcoming event scheduled on the calendar, to entering details of a new task in the Time Tracker. The evaluator then asked follow-up questions, following a semi-structured interview schedule. The focus of these questions was determining how the students conceptualised the WIER website and the Web in general, how they oriented themselves on the Web and techniques they used to navigate on the Web. Each session was from 30-45 minutes in duration.
The data gathering apparatus were prepared for the study by the researcher in consultation with the IE Project coordinators. This data collection was held from the 6th to 8th week of 2nd semester 2002.
In addition, log file data collected from student interactions with the WIER website was used to determine the frequency of access and time spent at the site.

Results

    Student Demographics

The eighteen students who participated in this study were aged from 19 to 24 years. Fifteen were male and three were female. This is a higher percentage of males than were enrolled in this unit. Eleven students claimed their average course performance was at the distinction level, six at the credit level and one at the high distinction level. The mean performance is higher than the average course performance for third year students in this course.
The students were all regular users of the Internet. Using a seven point Likert scale, where 1 indicated not used and 7 indicated heavily used, the students indicated heavy use of the Internet for coursework (M = 6.00, sd = 1.19) and for general use (M = 5.78, sd = 1.44). All except one student accessed the Internet daily.
All students claimed they accessed the WIER website regularly. The most frequent access mode was twice weekly (10 students), with four students accessing the site weekly and four accessing it daily. From the log file of student interactions with the website during the semester, the frequencies of access to the site and the mean times spent using the site were determined. This showed the mean number of accesses was 59 and the mean total time spent on the site was 7.3 hours.

    Perceptions of Navigation and Orientation

In the pre-interview survey the students rated various aspects of usability of the WIER website in the survey using 7 point Likert scales. The three aspects of interest to this study were: ease of navigation (N), ease of locating information (L) and ease of orientation (O). Pearson's correlation coefficients calculated for these usability aspects against the total number of sessions  showed no significant relationships. However, there was a significant relationship (R2 = 0.664, p < 0.01) between the ease of navigation and the ease of locating information. During the interviews the students were asked whether they ever experienced a feeling of disorientation while working on the Web. Their responses of frequently, sometimes or never are shown in Table 1. Pearson's correlation coefficients were calculated to determine any relationship between these responses and the number of sessions and their ratings of aspects of usability of the WIER site. There was a significant negative relationship between whether they experienced disorientation on the Web and their ratings of the ease of orientation on the WIER site.

   Conceptualistations of the WIER Website

During the interview the students were asked to represent on paper how they conceptualised the website. They were told this could be done as a description or a drawing. All students chose to depict the website as a diagram. The students’ drawings presented a variety of views of the WIER website in a range of detail. These were classified as follows:
    1. single screen: (11 students) A full or half page drawing of the home page screen, usually incorporating details of the screen layout and menu. This presented a two dimensional view of the website. Most drawings had details of menu structures and navigation buttons. A typical example of these drawings is shown in Figure 1.                                                                         
    2. multiple screens: (4 students) Two or more screens. These were drawn with minimal detail. Two were depicted with arrows to indicate navigation between the screens. These were shown as linear sequences. An example is shown in Figure 2.
    3. hierarchy diagram: (2 students) A hierarchical structure of Web pages. This presented a structural view of the website drawn in a conventional top down hierarchy. An example is shown in Figure 3.
    4. tree diagram: (1 student) An arrangement of web pages emanating from a single point. This was essentially a hierarchy diagram. This is shown in Figure 4.
Details of the students who produced these diagrams are shown in Table 1. It is interesting to note that the only students who claimed never to be disoriented on the WIER website drew a single screen view of the website. A cross tabulation showed that their experience of disorientation on the Web was significantly different across the three groups of website representations drawn (c2 (2, N=18) 10.22, p < 0.05).

Discussion

The students who participated in this study were all experienced users of the Web. They had all used courseware websites in previous units in their course. Furthermore they designed and developed websites as part of their computing course. Most had shown regular and consistent use of the WIER website. It was interesting to find therefore, that more than half claimed that they had experienced a feeling of disorientation while using the Web and with three claiming this happened frequently.
Analysis of the results in Table 1 showed that the only students who did not experience disorientation drew a screen view of the site. To gain insights into how their conceptualisation of the site related to disorientation the interview data was analysed. In this analysis we focused on the techniques the students used to orient themselves in the site and the navigation techniques they used.

    Single Screen View

The most common conceptual model of the WIER site presented by students in this study was a single screen. This represented a Portal view of the site. The students who drew these images claimed they did not think of the site beyond the screen view. A typical comment was, “the WIER site is just a page with changing information”. For a couple of students, this conceptualisation was a deliberate choice. As one commented:
I try to think of it as much as I can as a single level … I don’t think of it in levels. Like I know it is multilevel - but that’s not how I view it.
Although the students had all built websites and understood how they were constructed, as users they did not want to think of the site as having structure or a series of pages that were linked together in any way. As one student claimed, “the rest of the site is a list of links on the left hand side - menu and topics”. Another student just saw it as a series of frames that appeared on the screen. This conceptualisation appeared to aid their orientation by removing a level of complexity.
I prefer to look at it like that [a screen] … but that [the hierarchy] adds complexity - look at how complex it is!
To avoid having to remember sequences of links, a common navigation approach was to open another browser. Another technique was heavy reliance on the back button.
None of the students used a site map. One student saw advantages in opportunities for incidental learning:
… don’t like site maps … I like to explore … if you don’t use a site map you will discover new things along the way as you try to get where you are going
Almost half the students who drew this single screen view claimed they were never disoriented on the Web. As one stated, “I never [get lost] I know exactly where I am fitting in to the whole thing”. For the students who did experience disorientation it did not appear to be a huge concern. One student commented that if she became disoriented she would just wonder, “what is wrong with the site”. Inferring that there was something wrong with the site rather her own navigation skill could be seen as an indication of her confidence in working in this environment.
All students in this group claimed that if they became disoriented they would finish the task they were working on before trying to re-orient themselves. As one student remarked, “We hang in there - or, if it’s not important we just give up … so we just go to another site.”

The student responses shown here suggest that they used surface learning approach when using the website. Ramsden has an extensive discussion of student learning approaches (Ramsden 2003, chpt 4). The two main approaches discussed are surface learning and deep learning. Surface learning is characterised by rote learning and imitation whereas deep learning is characterised by the students trying to understand the big picture of the topic. Ramsden also reports that a deep learning approach consistenly leads to better performance in assessment than surface learning. If we interpret their simple view of the website as indicating a surface learning approach then the students particpating in this research seem to have gone against the trend explained by Ramsden. Why this is the case can only be speculated upon as the original survey questions used in this research did not try to capture reponses from students to confirm their usual learning style; typical reponses to questions designed to determine learning style are listed in Ramsden, page 52.

Given that 12 of the 18 participating students have better than average grades, there is a reasonable possibility that they are deep learners. It may be then that these students have used their past skills to learn or remember the least amount required to cope with the task at hand. For example they used two browser windows to help them navigate rather than one. It is only by having a deeper knowledge of the workings of the web that could present this as a viable option to overcome disorientation.

    Multiple Screens View

Four students conceptualised the site as a group or series of screens or pages. One student claimed that she thought of the site as different pages but did not see them connected. However, the other three students showed more concern for how the screens or pages linked together. As one student commented, “it’s all in my mind … a series of links”. This presented a linear view of the navigation.
In their comments the students emphasised the need to remember links to navigate and orient themselves in the site. As one student commented:
We follow the links and links and links … next time you are there you don’t have a blank … you just have to remember, ‘last time I was here I did this’ …
Another student spent time learning the site before the start of semester:
… if it is a new site then I have to read  through all the pages and really find out information … usually find where I am by reading all the pages …
One student recommended having a site map to aid navigation and orientation within the site, “I think the best way is to have a site map with the different categories”.
All students in this group experienced disorientation on the Web and one claimed it happened frequently.
The students in this group appear to be surface learners judging by their approach to mastering the web site; they try to remember all the links and navigation. This was obviously difficult at times as shown by their reported disorientation.

    Hierarchical or Tree View

The WIER website was constructed as a hierarchy of pages to a maximum of four levels deep. However, only three students conceptualised it this way. These students discussed how the site was structured in detail and all claimed that they thought about the structure when navigating through the site. As one commented:
I think of it as a structure because whenever you want to go somewhere in the site you would go to a site map or something, which has like a home page and there is a tree structure and a branch structure.
Furthermore,  the students orientation in the site appeared to be a continuous concern while they were working.
… like when I’m reading through the content, it’s at the back of my mind trying to pick up key words or hints that I’m in the right area.
The students all claimed that knowing where they were in the site at all times was important to them and if they perceived that they were lost they would immediately try to orient themselves. They would do this by going back to the home page or using the back button. One student felt a site map was important.
The students in this group claimed they experienced disorientation and two of them claimed this happened frequently.
The same comments about the students' learning approach may be made here as for the students who adopted a multiple screen view even though this group used a different mental model.

Navigation Techniques

The students in this study exhibited a range of navigation techniques that appeared to be useful in helping them overcome problems in disorientation. The students who claimed that they never or only sometimes experienced disorientation while working on the Web mentioned one or a couple of the following strategies that they used to orient themselves on the Web:
1.    Relying on a website following established conventions for website strucure.
2.    Developing an understanding of structure of the website by:
        -exploring the website to gain familiarity with it;
        -using a site map to gain an overall view of the site;
        -learning the sequence of links to a desired  destination.
3.    Using orientational aids:
        -referring to a site map;
        -working with multiple windows;
        -returning to the website home page.
4.    Using navigational aids:
        -back button;
        -linking directly using a URL

Conclusion and Further Work

The results of this study were a surprise to the designers of the WIER website. It was expected that the students, as experienced Web and WIER users, would reproduce hierarchical models of the WIER website as their conceptualistions, reflecting the organisation of the website. However, this model was adopted by only a couple of students. Most of the students constructed simple mental models of the website. Furthermore, students who had simple cognitive models experienced less disorientation than those who had more complex models. This confirms what other interface design researchers such as Norman (1996) and Nielson (1990) have found, namely that a simple mental model places less cognitive load on users and so the users are better able to function with the interface.
The students had developed a range of strategies to navigate on the Web and these appeared to be successful in helping them orient themselves within this environment. Future work will study novice users to compare their conceptualisations and navigation strategies with their experiences of disorientation. This should help inform the design of Web-based learning environments that are considerate to the experience and cognitive styles of the learners.

References.

Altun, A. (2000). Patterns in cognitive processes and strategies in hypertext reading: A case of two experienced computer users. Journal of Educational Multimedia and Hypermedia, 9(1), 35-55.
Baylor, A. L. (2001). Perceived disorientation and incidental learning in a Web-based environment: Internal and external factors. Journal of Educational Multimedia and Hypermedia, 10(3), 227-251.
Beasley, R. E., & Waugh, M. L. (1996). The effects of content-structure focusing on learner structural knowledge acquisition, retention, and disorientation in a hypermedia environment. Journal of Research on Computing in Education, 28(3), 271-281.
Carmel, E., Crawford, S., & Chen, H. (1992). Browsing in Hypertext: A cognitive study. IEEE Transactions on Systems, Man and Cybernetics, 22(5), 865-883.
Ceddia, J., Tucker, S., Clemence, C., & Cambrell, A. (2001). WIER - Implementing artifact reuse in an educational environment with real products. Paper presented at the Thirty-first Annual Frontiers in Education Conference, Reno, Nevada.
Chiu, C.-H., & Wang, F.-M. (2000). The influence of navigation map scope on disorientation of elementary students in learning a Web-based hypermedia course. Journal of Educational Computing Research, 22(2), 135-144.
Elm, W. C., & Woods, D. D. (1985). Getting lost: A case study in interface design. Paper presented at the Twenty ninth meeting of the Human Factors Society.
Foss, C. L. (1989). Tools for reading and browsing hypertext. Information Processing & Management, 25(4), 407-418.
Gray, S. H. (1990). Using protocol analyses and drawings to study mental model construction during hypertext navigation. International Journal of Human-Computer Interaction, 2(4), 359-377.
Hammond, N., & Allinson, L. (1989). A learning support environment: the hitchhikers guide. In R. McAleese (Ed.), Hypertext: theory into practice (pp.62-74): Oxford: Intellect.
Lai, Y.-R., & Waugh, M. L. (1995). Effects of three different hypertextual menu designs on various information searching activities. Journal of Educational Multimedia and Hypermedia, 4(1), 25-52.
McDonald, S., & Stevenson, R. J. (1996). Disorientation in hypertext: The effects of three text structures on navigation performance. Applied Ergonomics, 27(1), 61-68.
Nielsen, J. (1990). The art of navigation through hypertext. Communications of the ACM, 33(3), 297-310.
Norman, D. A. (1998). The Design of Everyday Things. New York: Basic Books.
Norman, D. A., & Spohrer, J. C. (1996). Learner - centered education. Communications of the ACM, 39(4), 24-27.
Ramsden, P. (2003). Learning to Teach in Higher Education. 2nd edn. RouteledgeFalmer, London. Chapter 4.
Simpson, A., & McKnight, C. (1989). Navigation in hypertext: structural cases and mental maps. Paper presented at the Hypertext II conference.
Stanton, N., Taylor, R. G., & Tweedie, L. A. (1992). Maps as navigational aids in hypertext environments: An empirical evaluation. Journal of Educational Multimedia and Hypermedia, 1(4), 431-444.

Copyright