Peter Brusilovsky, Human-Computer Interaction Institute, School of Computer Science, Carnegie Mellon University Pittsburgh, PA 15213, USA firstname.lastname@example.org
Elmar Schwarz, Department of Psychology, Carnegie Mellon University Pittsburgh, PA 15213, USA email@example.com
adaptive, authoring, hypermedia, navigation, textbook, user-model, WWW
This paper examines some recent research in the area of adaptive navigation support, a class of adaptation in user-model based interfaces, and specifically discusses the authoring and delivery tool for adaptive electronic textbooks (ETs) called Interbook [HREF2], implemented on the WWW. Interbook uses history-based, knowledge-based and prerequisite-based adaptive annotation of links to suggest to the individual user an appropriate path through a learning space. We describe the authoring environment and the principles upon which it is based, and discuss our efforts to experiment with, and to evaluate, this tool.
For a moment, consider a textbook such as one of the many "visual quickstart" guides that may help a novice or intermediate user of software such as Clarisworks, or Word for Windows. These textbooks are typically structured through the use of chapters, sections, headings and subheadings, and the material is sequenced from "begin here" to "more advanced". It has an implied set of prerequisites so that it is intended to be read, overall, from beginning to end. There are also navigational devices in this textbook: A Table of contents at the start, an index at the back, and throughout the book there are pointers to other places in the text that may be relevant to what is being discussed. This textbook may also have a "test yourself" section at each chapter, so that readers may find out how much of the material they have understood before proceeding to the next chapter. In all these features of the textbook, the expert has placed their understanding of the content, and their understanding of how novices might best learn from it, into the book, and thus onto the knowledge contained within it. The pages provide a sequence to the knowledge in the book, but there is also the opportunity to move around it. Similarly if this book were put in the form of a hypertext document, it could read in a linear manner, or traversed very easily using the hyperlinks.
Two problems with this common form of textbook present themselves: The textbook, whether in print or on the WWW is the same for all users - it has no understanding of who is using it, and cannot change it's behaviour in terms of what it presents on that basis. This is a problem because readers have different knowledge, goals, computer experience and will learn the material at different rates. Secondly, providing the readers (or users) with navigation aides assumes that they can make sensible decisions about when to use them, and about where to proceed in the body of knowledge. However, there is some evidence (eg Hammond, 1989) which tends to suggest that they do not always make well informed and careful decisions. Once again some readers will be able to handle certain navigation features - others not: So perhaps the book should be structured and indexed differently for different classes of readers/learners/users?
We contend that the often cited problem of "becoming lost in hyperspace" is really one of loosing sight of objectives, of being unable to make sense of the new material, and in terms of learning, there is no distinction between this problem in paper-based text or in hypertext, or from a book and a hypertext document.
Given that the hypertext version of the textbook will allow greater possibilities to implement individualised support, the question posed is: How can we design a Web-delivered learning environment in the form of a textbook which goes some way toward addressing these problems?
This paper is specifically concerned with adaptive navigation support on the Web, and we describe some ANS methods for the WWW (see also Eklund & Zeiliger, 1996). Using a Web-based adaptive educational system which uses ANS, namely InterBook ([HREF2], Brusilovsky, Schwarz, and Weber, 1996), we offer one possible solution to the problems of the passive nature of the media (its inability to alter itself to meet the needs of the individual user), without limiting the free-browsing, learner-controlled nature of hypermedia and more generally that of the WWW.
Adaptive navigation support techniques can be classified in several groups according to the way they adapt presentation of links (Brusilovsky, 1996): direct guidance, sorting, hiding, and annotation. These will be briefly described.
Direct guidance can be applied in any system which can decide what is the next "best" node for the user to visit according user's goal and other parameters represented in the user model. ELM-ART (Schwarz, Brusilovsky, and Weber, 1996) is an example of an adaptive system implemented on the WWW that uses this technique. ELM-ART [HREF6] generates an additional dynamic link (called "next") connected to the next most relevant node to visit. Direct guidance has been criticised for being "too directive" as it provides almost no support for users who would like make their own choice rather than follow the system's suggestion.
In adaptive ordering technology all the links of a particular page are sorted according to the user-model using some easily recognisable means of conveying this to the user, such as having the more relevant links closer to the top (Hohl, Bocker, and Gunzenhauser, 1996). This technology exists within ELM-ART and Interbook. Adaptive ordering has a limited applicability: it can be used with non-contextual links, but it cannot be used for indexes and content pages (which usually have a stable order of links), and can never be used with contextual links and maps.
Hiding is an annotation technology which restricts the navigation space by hiding links to irrelevant pages. A page can be considered as irrelevant for several reasons: for example, if it is not related to the user's current goal (Brusilovsky, and Pesin, 1994; Vassileva, 1996) or if it presents materials which the user is not yet prepared to understand (Brusilovsky and Pesin, 1994; Perez, Gutierrez, and Lopisteguy, 1995).
Adaptive annotation technology augments the links with a comment which informs the user about the current state of the nodes behind the annotated links (Brusilovsky, Pesin, and Zyryanov, 1993; de La Passardiere, and Dufresne, 1992; Hohl, Becker, and Gunzenhauser, 1996; Schwarz, Brusilovsky, and Weber, 1996). Link annotations can be provided in textual form or in the form of visual cues, for example, using different icons, or colours, font sizes, or font types. Typically the annotation in traditional hypermedia is static, that is independent of the individual user. Adaptive navigation support can be provided by dynamic user model-driven annotation. Adaptive annotation in its simplest history-based form (outlining the links to previously visited nodes) has been applied in some hypermedia systems (for example, TopClass [HREF8], which shows a folder as unread with a "U" until all of the items within that folder have been visited), including several World-Wide Web browsers. Even the form adaptive annotation which distinguishes two states of links is quite useful.
The knowledge about the domain is represented in the form of a concept-based domain model. The simplest form of domain model is just a set of domain concepts. These concepts can be named differently in different systems - topics, attributes, knowledge elements, objects, learning outcomes, but in all the cases they are just elementary pieces of knowledge for the given domain. Depending on the domain and the application area, the concepts can represent larger or smaller pieces of domain knowledge. A more advanced form of the domain model is a network with nodes corresponding to domain concepts (and with links reflecting several kinds of relationships between concepts). This network represents the structure of the domain in a hypermedia system.
The domain model provides a structure for the representation of the user's knowledge of the subject. For each domain model concept, an individual user knowledge model stores some value which is an estimation of the user knowledge level of this concept. This can be a binary value (known Ğ not known), a qualitative value (good-average-poor), or a quantitative value, such as the probability that the user knows the concept. The individual user-knowledge model, which is called an overlay model, is most often used in adaptive hypermedia systems. An overlay model of user knowledge can be represented as a set of pairs "concept - value", one pair for each domain concept. The overlay model is powerful and flexible, it can measure independently the user's knowledge of different topics.
The knowledge about the course is represented by indexing hypermedia nodes containing various units of learning material (presentations, tests, examples, problems) with domain model concepts which are related to the content of the unit. This is a relatively popular direction for the development of education-oriented hypermedia systems. There are two major types of indexing: content-based indexing and prerequisite-based indexing. With content-based indexing, a concept is included in a page index if some part of this page presents the piece of knowledge designated by the concept (Brusilovsky and Pesin, 1994; Schwarz, Brusilovsky, and Weber, 1996; Zeiliger, 1993). With prerequisite-based indexing, a concept is included in a page index if a student has to know this concept to understand the content of the page (Schwarz, Brusilovsky, and Weber, 1996).
We identify two further methods for ANS on WWW: knowledge-based annotation and prerequisite-based annotation. The idea of the "knowledge-based" method is to distinguish different levels of the user's knowledge of the node. We suggest the use of three graduations: not-known, in-work (partially known) and well-learned, and annotate differently the links to the nodes of these three classes as in (Brusilovsky, Pesin, and Zyryanov, 1993; de La Passardiere and Dufresne, 1992). Here by "not-known" we mean that the user has never heard about some of the concepts from the node's outcome. "In-work" means that the user has acquired some information about all the concepts presented in this node (it does not necessarily imply that the user has just visited this node!). "Well-known" means that the user confirmed his or her knowledge of all the concepts presented in this node by answering tests or solving problems. This method requires a user model which can distinguish three levels of user knowledge of the concept: the user has never heard about a concept, the users has read some information about a concept, and finally that the user has correctly answered a test or solved a problem which requires this concept. It also requires the embedding of tests into the courseware, which has not yet been implemented in the version of Interbook described in this paper.
In the "prerequisite-based" method we distinguish nodes which are ready and not-ready to be learned as in (Brusilovsky and Pesin, 1994). The node is considered as not-ready-to-be-learned in two cases: first if any of the concepts in the prerequisite section of node index is not-known and second if any concept from the outcome section of node index has a not-known prerequisite concept. This method could be implemented with the simplest form of overlay model which only distinguishes known from not-known. Pre-requisite based adaptive annotation is a feature of Interbook.
All sections of an electronic textbook are indexed with domain model concepts. For each section, a list of concepts related with this section is provided (we call this list the spectrum of the section). The spectrum of the section can represent also the role of a concept in the section (each concept can be either an outcome concept or a background concept).
The knowledge about the domain and about the textbook content is used by InterBook to serve a well-structured hyperspace. In particular, InterBook generates links between the glossary and the textbook. Links are provided from each textbook section to corresponding glossary entries for each involved background or outcome concept. Similarly from each glossary entry describing a concept InterBook provides links to all textbook units that can be used to learn this concept. It means that an InterBook glossary integrates features of an index and a glossary. These links are not stored in an external format but generated on the fly by a special module that takes into account the student's current state of knowledge represented by the user model.
InterBook uses coloured bullets and different fonts to provide adaptive navigation support (Figure 1). Wherever a link appears on InterBook pages: in the table of content, in the glossary or on a regular page, its font and colour of its bullet will inform the user about the status of the node behind the link. InterBook integrates all three methods of annotation: history-based, knowledge-based and prerequisite-based. Currently four colours and three fonts are used. Green bullet and bold font means 'ready and recommended', ie., the node is ready-to-be-learned but still not learned and contains some new material. A red bullet and an italic font warns about a not-ready-to-be-learned node, while white means 'clear, nothing new', ie., all concepts presented on a node are known to the user. Violet is used to mark nodes which have not been annotated by an author. A check mark is added for already visited nodes. Currently, InterBook does not support tests and can not provide "well-learned" annotation. This is currently under development.
The user model in Interbook is initialised from the registration page via a stereotype model, and is modified as the user moves through the information space. New work on Interbook includes the provision of an "interview" to further specify the user model, and embedded testing for knowledge-based navigation support. The user model for each user is stored in a file on the server in a Lisp format.
InterBook recognizes the structure of the document through the use of headers. It means that the titles of the highest level sections should have a pre-defined text style "Header 1", the titles of its subsections should have a pre-defined paragraph style "Header 2", and so forth. The title of the textbook should have paragraph style "Title". The result of this step will be a properly structured MS Word file.
The second step in the authoring process then involves concept-based annotation of the Electronic Textbook (ET) to let InterBook know which concepts stand behind each section. This knowledge allows InterBook to help the reader of the ET in several ways, and the result of this step is an annotated (and structured) MS Word file.
An annotation is a piece of text of special style and format inserted at the beginning of each section (between the section header and the first paragraph). Annotations have special character style (hidden + shadowed) which are not visible in the text window to the reader of the ET. For each unit the author provides a set of outcome and background concepts. In this way, each section is annotated with a set of prerequisite concepts (or terms which exist in other sections which should be read before the current section), and a set of outcome concepts (terms which will be assumed known once the reader has visited the section). The format for the outcome annotation is: (out: concept-name1, concept-name2, etc.) and the format for the background annotation is: (pre: concept-name1, concept-name2, etc.).
Once the annotations are complete the file is saved in RTF format. The RTFtoHTML program [HREF5] with some special settings is used to convert the ET into HTML format. Then the .html extension on the file is manually altered to .inter so that it can be recognised by the Interbook system.
Lastly, when the InterBook server starts, it parses all interbook files in its "Texts" folder (i.e. all files with extension .inter) and translates it into the list of section frames. Each unit frame contains the name and type of the unit, its spectrum, and its position in the original HTML file. The obtained LISP structure is used by InterBook to serve all the available textbooks on WWW providing the advanced navigation and adaptation features. The content which is presented to the user is generated on-the-fly using the knowledge about the textbook, the user model, and HTML fragments extracted from the original HTML file. These features of InterBook are based on the functionality of the Common Lisp Hypermedia Server.
While Interbook undergoes development at Carnegie Mellon University, we are currently experimenting with the Interbook tool at the University of Technology, Sydney [HREF1]. An Interbook server has been established and we have been granted permission by Peachpit Press to use sections of Charles Rubin's book "MacBible Guide to Clarisworks4" (Rubin, 1996) as content for an adaptive ET. The database and spreadsheet sections of the textbook are being set up as Interbooks on the server through the authoring process described above. Second year students in an educational computing elective, which has a focus on the use of Clarisworks Database and Spreadsheet modules for information handling, will be using the ETs: One group with and a group without the adaptive annotation. We are attempting to measure both comprehension (via a multi-choice test administered after the sessions) and their navigation traits (via the use of audit trails).
Our hypothesis is that students using the ET with adaptive link annotation (ie using the adaptive version of the ET) will show paths which are "more purposeful" (less nodes visited, greater average time spent on each node, less movements to unrelated nodes), with increased comprehension (better results on tests), than those using the same ET without the adaptive link annotation.
Hook [HREF3] draws attention to the fact that the interface design is often inextricably linked to the adaptive component in an adaptive system. Removing adaptivity may remove a natural part of the system and its intended use. Other studies (Boyle and Encarnacion, 1994; Kaplan et al., 1993) have also centred on "with and without adaptivity". These experiments have offered some straightforward, if oversimplified, results in favour of adaptivity, but have not convincingly separated a range of confounding variables, such as those for which Hook argues. These can be deceptively simple. For instance, if we ask a student why they followed the annotated links, we may find that it is because they thought it would be the easiest way to get through the session! There is a clear need to integrate the results of a questionnaire, along with the hard numerical data from the audit trails, to make sense of experimental outcomes. Just as the evaluation of the tool cannot be separated from its development, measuring the effectiveness of its components cannot be successfully achieved in isolation from broader human factors.
In terms of these considerations for Interbook, the interface is reasonably complex although it does provide an easy entry level, as learners may use the hyperlinks like a book index, of which they will be reasonably familiar, or just click the "continue link" to read in a linear way. Fortunately also, the visible adaptive component of Interbook, namely the simple coloured bullets next to the links, is a straightforward addition to the interface and should not confuse the fact that we are measuring the addition or exclusion of adaptive navigation support with a measurement of the effects of changes in the interface. We also intend a number of sessions before the trial where students may familiarise themselves with the interface to minimise this affect.
Some early results will hopefully be available at the time of the presentation of this conference paper.
Brusilovsky, P. (1996) Methods and techniques of adaptive hypermedia. User Modeling and User-Adapted Interaction 6 (2-3), 87-129.
Brusilovsky, P. and Pesin, L. (1994) ISIS-Tutor: An adaptive hypertext learning environment. Proceedings of JCKBSE'94, Japanese-CIS Symposium on knowledge-based software engineering. Edited by H. Ueno and V. Stefanuk. Pereslavl-Zalesski, Russia, May 10-13, 1994, pp. 83-87.
Brusilovsky, P. and Pesin, L. (1995) Visual annotation of links in adaptive hypermedia. Proceedings of CHI'95 (Conference Companion). Edited by I. Katz, R. Mack and L. Marks. Denver, May 7-11, 1995, pp. 222-223.
Brusilovsky, P., Pesin, L., and Zyryanov, M. (1993) Towards an adaptive hypermedia component for an intelligent learning environment. In Human-Computer Interaction, Lecture Notes in Computer Science, Vol. 753, L. J. Bass, J. Gornostaev and C. Unger (eds), Springer-Verlag, Berlin. pp. 348-358.
Brusilovsky, P., Schwarz, E., and Weber, G. (1996) A tool for developing adaptive electronic textbooks on WWW. Proceedings of WebNet'96, World Conference of the Web Society. San Francisco, CA, October 15-19, 1996, pp. 64-69, http://www.contrib.andrew.cmu.edu/~plb/WebNet96.html.
Kaplan, C., Fenwick, J., & Chen, J. (1993). Adaptive hypertext navigationbased on user goals and context. User Models and User AdaptedInteraction, 3(3), 193-220.
de La Passardiere, B. and Dufresne, A. (1992) Adaptive navigational tools for educational hypermedia. In Computer Assisted Learning, I. Tomek (ed) Springer-Verlag, Berlin. pp. 555-567.
Eklund, J. and Zeiliger, R. (1996) Navigating the Web: Possibilities and practicalities for adaptive navigational support. Proceedings of Ausweb96: The Second Australian World-Wide Web Conference. , pp. 73-80.
Hammond, N. (1989). Hypermedia and learning: Who guides whom? In H. Maurer(Ed.), 2-nd International Conference on Computer Assisted Learning,ICCAL'89 (Vol. 360, pp. 167-181). Berlin: Springer-Verlag.
Hohl, H., Bcker, H.-D., and Gunzenhuser, R. (1996) Hypadapter: An adaptive hypertext system for exploratory learning and programming. User Modeling and User-Adapted Interaction 6 (2-3), 131-156.
Ibrahim, B. and Franklin, S. D. (1995) Advanced educational uses of the World-Wide Web. Computer Networks and ISDN Systems 27 (6), 871-877.
Perez, T., Gutierrez, J., and Lopisteguy, P. (1995) An adaptive hypermedia system. Proceedings of AI-ED'95, 7th World Conference on Artificial Intelligence in Education. Edited by J. Greer. Washington, DC, 16-19 August 1995, pp. 351-358.
Rubin, C (1996) The Macbible Guide to Clarisworks 4. Peachpit Press. California.
Schwarz, E., Brusilovsky, P., and Weber, G. (1996) World-wide intelligent textbooks. Proceedings of ED-TELECOM'96 - World Conference on Educational Telecommunications. Boston, MA, June 1-22, 1996, pp. 302-307, http://www.contrib.andrew.cmu.edu/~plb/ED-MEDIA-96.html.
Vassileva, J. (1996) A task-centered approach for user modeling in a hypermedia office documentation system. User Modeling and User-Adapted Interaction 6 (2-3), 185-224.
Weber, G. & Specht, M. (1997, in press). User modeling and Adaptive Navigation Support in WWW-based Tutoring Systems. Paper to be presented at UM97, The Sixth International Conference on User Modeling, Chia Laguna, Sardinia , Italy, June 2-5 1997.
Zeiliger, R. (1993) Adaptive testing: contribution of the SHIVA model. In Item banking: Interactive testing and self-assessment, NATO ASI Serie F, Vol. 112, D. Leclercq and J. Bruno (eds), Springer-Verlag, Berlin. pp. 54-65.
John Eklund, Peter Brusilovsky and Elmar Schwarz ©, 1997. The authors assign to Southern Cross University and other educational and non-profit institutions a non-exclusive licence to use this document for personal use and in courses of instruction provided that the article is used in full and this copyright statement is reproduced. The authors also grant a non-exclusive licence to Southern Cross University to publish this document in full on the World Wide Web and on CD-ROM and in printed form with the conference papers, andfor the document to be published on mirrors on the World Wide Web. Any other usage is prohibited without the express permission of the authors.
[All Papers and Posters]