Paper-based and Web-based resources: what do students value?

Judy Sheard, School of Computer Science and Software Engineering, Monash University, 900 Dandenong Rd, Caulfield East, Victoria 3145, Australia. judy.sheard@csse.monash.edu.au

Margot Postema, School of Computer Science and Software Engineering, Monash University, 900 Dandenong Rd, Caulfield East, Victoria 3145, Australia. margot.postema@csse.monash.edu.au

Selby Markham, School of Computer Science and Software Engineering, Monash University, 900 Dandenong Rd, Caulfield East, Victoria 3145, Australia. selby.markham@csse.monash.edu.au

Abstract

This is a report of an innovative study into how students value printed and electronic subject resources in the context of the growing use of the Web. The apparent lack of research into student valuation and use of resources has prompted this investigation.

The study involved surveying students (n=329) and lecturers in seven computing subjects within the Faculty of Information Technology of Monash University. The subjects chosen for the study provided students with various combinations of subject resources in paper format and on the Web. Ten different types of resources were considered. Eight resources were available in both paper and Web format and two were available only on the Web. Each subject had a dedicated subject Web page.

The results from the surveys indicated that students were enthusiastic about the Web resources provided however the interesting outcome from the research is that students within the Faculty of Information Technology still appear to be paper dependant.

Introduction

The widespread acceptance of the Internet and the use of World Wide Web (WWW or Web) technology in tertiary institutions have seen the emergence of a wealth of new resources for students. This is seen by some people as an indication of a paradigm shift in teaching and learning [3]. The electronic format of the Web allows for the development of interesting and exciting interactive resources, incorporating multimedia, and these are features not possible with paper-based format [2, 3, 7]. In addition, resources traditionally produced in paper-based format e.g. subject information, lecture notes, tutorial exercises are now commonly placed on the Web, providing the paper resource in an alternate format. Subject lecturers are now usually faced with the decision of whether to provide students with resources in paper-based format, electronic format, or both [4].

Web resources provide advantages over paper resources. The Web makes possible interactive resources encouraging student involvement, a fundamental requirement of the currently popular constructivist theory of learning [11, 12]. The Web enables immediate changes to information; resources may be accessed by students on or off campus and at any time [1]. However, they can be time consuming for lecturers to produce and the Web is not always easily accessible by students [9, 13]. Is all the extra effort benefiting students and having an impact on their learning? A search of the literature has failed to produce systematic investigation of these questions.

The study being reported here aimed to establish the relative valuations students placed on Web or paper resources and what factors influence these. Students in seven undergraduate subjects within the Faculty of Information Technology of Monash University were surveyed. Information about resources provided was obtained from subject staff for the purposes of cross-referencing. This study is part of a larger project that will attempt to gain an understanding of what resources students use and find of value to their learning.

Project Description

Research Design

Selection of students for study

Students from seven undergraduate computing subjects within the Faculty of Information Technology were invited to participate in this study. These subjects were chosen from three different computing degree programs on two separate campuses. Each undergraduate year level was represented.

Research method

This study investigates whether students have a preference for resources in electronic or paper format, what influences any preferences and what impact they feel Web resources have on their learning. It was decided to use a survey to gauge the studentsā opinions of the value of resources. The availability of each resource available to students was determined by a separate questionnaire which was given to the lecturer responsible for each subject.

The survey questionnaires were trialed beforehand with students and lecturers who were not involved in the survey.

Monash University ethics committee approval was given for this study. In accordance with the ethics committeeās requirements, the surveys were anonymous with no identifying information asked of the students. Students were given a cover sheet with their questionnaire explaining the purpose of the survey.

Resources

A total of ten different types of resources were considered in the study. These are listed in Table 4. Eight of these were available in both paper and Web format and two were available only on the Web (subject updates and Anonymous Feedback). The availability of each type of resource within each subject was obtained from the subject lecturers' surveys. The subjects in the study provided various combinations of both paper and Web resources. All subjects had a dedicated Web page that contained, at least, basic subject information and any other resources. All subjects also provided subject information in paper format.

Student survey

The student questionnaire asked students to rate the value of each type of resource on a seven-point Likert scale, with a separate option to select if a resource was not provided. The questionnaire also contained questions to establish a profile of the students, to determine what access students had to the Internet and what use they made of the Internet. At the end of the questionnaire provision was made for students to add additional comments.

The students were surveyed during the second last week of the semester. By this time they had had the opportunity to use every resource provided. All the students who attended classes during this week were given a questionnaire to complete. Participation in the survey was voluntary, however most students were happy to complete the questionnaire. It was decided to use paper questionnaires, rather than a Web survey, as some of the surveys were conducted in classrooms without computers. It was also felt that a Web survey might bias the sample away from people who did not have easy access to the Web, or were not familiar or comfortable users of the Web.

Lecturer survey

The lecturers of each subject included in the study were surveyed at the end of semester. The questionnaire asked lecturers to indicate what types of resources they made available for their subject. The details of this survey will be presented in another paper.

Results

Student Profile

A total of 329 students participated in the study. Most of these students were full-time students (94%) and the remainder were part-time. There were no students studying by distance mode. The majority of students were male (75%).

All students have access to the Internet on campus and a high percentage of students surveyed (82%) had access to the Internet off campus.

The Internet usage is shown in Table 1. The students indicated showed a moderately high usage of the Internet for accessing course administration and teaching materials, for communications such as email, chatting and newsgroups, and for general use such as surfing and hobbies. However they showed only a moderately low use of the Internet for recreational activities such as games.


 
Use of the Internet for:
Mean
Standard deviation
Accessing teaching materials
5.4
1.5
Accessing course administration
5.2
1.6
General use (hobbies)
5.4
1.7
Communication (email, chatting, newsgroups)
5.4
1.6
Games
2.7
2.2

Table 1 Internet usage (A rating of 1 indicates no Internet usage and a rating of 7 indicates high Internet usage)

Web or Paper?

Do students value more highly a resource in Web or paper format?

The students were asked in the surveys to rate the value of each resource provided. The means and standard deviations of the ratings for each resource available in paper and Web format are shown in Table 2. For each resource a t-test (a method of testing for a significant difference between two means) was used to determine whether there was a difference between studentsā value ratings of the paper and electronic format of the resource. The results of these tests show students rated significantly more highly lecture notes, assessable exercises, assignments, reading material, and reading lists, in paper format rather than on the Web. However, for subject information, references to other resources, non-assessable exercises, and self-evaluation exercises, there were no significant differences.


 
Resource
Paper format
Web format
 
Mean
Standard deviation
Mean
Standard deviation
 
Subject information
5.0
1.5
5.0
1.5
Lecture notes
5.5
1.6
5.1
1.8
*
References to other resources
4.4
1.5
4.5
1.6
Tutorial/non-assessable exercises
4.8
1.6
4.8
1.6
Assessable exercises and assignments
5.3
1.5
5.1
1.6
*
Reading material
4.5
1.5
4.2
1.6
*
Reading lists
4.2
1.5
4.0
1.6
*
Self evaluation exercises
4.4
1.8
4.4
1.7
* indicates t test results significant at p <= 0.05

Table 2 Results of comparisons of paper and Web resource value ratings. (A rating of 1 indicates that the resource was not useful and a rating of 7 indicates that the resource was very useful.)

Is there a relationship between students' valuations of a resource in paper and Web format?

To determine if there was a relationship between studentsā valuations of each resource in paper and Web format, Pearson's product moment correlation coefficients were calculated. A correlation coefficient gives a measure of the degree of a relationship between two variables. Correlation coefficients were calculated for each pair of paper and Web resource ratings. The results can be seen in Table 3. There was a significant relationship between the studentsā ratings for each pair of resources, indicating that for each resource students who find the resource valuable in paper are also likely to find an electronic version of the same resource valuable.
 
Resource
Pearson Correlation
Subject information
0.60
*
Lecture notes
0.38
*
References to other resources
0.54
*
Tutorial/non-assessable exercises
0.59
*
Assessable exercises and assignments
0.52
*
Reading material
0.59
*
Reading lists
0.66
*
Self evaluation exercises
0.71
*
* correlation significant at the 0.01 level

Table 3 Results of correlations between paper and Web resource value ratings

Do students who are high Internet users place more value on Web resources?

The students were categorised into low or high Internet users for various types of Internet access. For each resource available in Web and paper format, a t-test was then used to determine if there were any differences in the value ratings of resources between low and high Internet users. The results recorded in Table 4 show significant differences between some ratings, depending on the type and the extent of Internet usage.

These can be summarised as follows:

·Students who are high users of the Internet for accessing course administration rated every type of Web resource as more valuable than low Internet users.

·Students who are high Internet users for accessing teaching materials rate every type of Web resource as more valuable than low Internet users except for self evaluation exercises and Anonymous Feedback.

·There is no significant difference between resource ratings low and high users of the Internet for hobbies.

·Students who are high users of the Internet for communication rate Anonymous Feedback as more valuable than low users.

·Students who are high Internet users for games rate lecture notes on the Web as more valuable than low users.


 
Resource
Accessing teaching materials
Accessing course administration
General use (hobbies)
Communication (email, chatting, newsgroups)
Games
 
Mean
Mean
Mean
Mean
Mean
 
Low
High
Low
High
Low
High
Low
High
Low
High
Subject information
4.5
5.4
*
4.6
5.5
*
5.0
5.0
4.8
5.1
4.9
5.1
Subject updates
4.7
5.4
*
4.7
5.6
*
5.0
5.2
5.1
5.1
5.0
5.2
Lecture notes
4.6
5.5
*
4.8
5.4
*
5.0
5.2
5.0
5.1
4.8
5.2
*
References to other resources
4.1
4.8
*
4.2
4.9
*
4.5
4.5
4.4
4.6
4.3
4.7
Tutorial/non-assessable exercises
4.5
5.0
*
4.5
5.2
*
4.9
4.7
4.7
4.9
4.8
4.8
Assessable exercises and assignments
4.9
5.3
*
4.8
5.4
*
5.2
5.1
5.1
5.1
5.2
5.1
Reading material
4.0
4.4
*
4.0
4.5
*
4.2
4.2
4.2
4.3
4.2
4.3
Reading lists
3.8
4.2
*
3.8
4.3
*
4.1
4.0
3.9
4.1
3.9
4.1
Self evaluation exercises
4.3
4.6
4.2
4.7
*
4.4
4.4
4.3
4.5
4.4
4.5
Anonymous feedback
4.0
4.4
3.9
4.7
*
4.3
4.2
3.9
4.5
*
4.1
4.4
* indicates t test results significant at p <= 0.05

Table 4 Results of comparisons of low and high Internet user ratings of Web resources. (A rating of 1 indicates that the resource was not useful and a rating of 7 indicates that the resource was very useful.)

Does the quality of a Web resource influence students' opinions of the resource?

To determine if there was a relationship between the students' opinions of the quality of the Web resources provided and usefulness of each Web resource a Pearson's product moment correlation coefficient was calculated separately for each student rating of navigability, currency of information and relevance of Web resources against the rating for each resource. The results can be seen in Table 5. There is a significant correlation between each quality rating and each resource value rating indicating that the quality of resources influences student opinions of their usefulness.
 
Resource
Navigability
Currency
Relevance
Pearson Correlation
Pearson Correlation
Pearson Correlation
Subject information
0.44
*
0.36
*
0.45
*
Subject updates
0.46
*
0.47
*
0.52
*
Lecture notes
0.40
*
0.35
*
0.41
*
References to other resources
0.42
*
0.39
*
0.42
*
Tutorial/non-assessable exercises
0.38
*
0.36
*
0.44
*
Assessable exercises and assignments
0.35
*
0.43
*
0.43
*
Reading material
0.33
*
0.40
*
0.40
*
Reading lists
0.30
*
0.29
*
0.35
*
Self evaluation exercises
0.31
*
0.33
*
0.31
*
Anonymous feedback
0.45
*
0.26
*
0.33
*
* Correlation significant at the 0.01 level

Table 5 Results of correlations of resource quality against usefulness

What is the impact of resources on students' learning outcomes?

A multiple regression was performed in order to look at what impact the values students placed on each Web resource had on the importance of Web resources to their learning. Multiple regression was used because it allows the evaluation of the effect of a set of possible predictor variables (resources) on an outcome variable (importance to learning). The results are shown in Table 6. The two valuations, subject information and subject updates have significant standardised coefficients, indicating that both of these contribute to the perceptions students have of importance of Web resources to learning. The remaining six resources make no significant contribution.

None of the paper-based resources produced a significant interaction with the rating of importance of paper resources to learning outcomes.


 
Resource
Standardised coefficient (Beta)
t
Subject information
0.28
3.2
*
Subject updates
0.25
2.4
*
* Regression significant at the 0.01 level

Table 6 Significant results of regression of Web resources against importance of Web resources to learning

Qualitative Responses

Students were given the opportunity at the end of the survey forms to make extra comments about Web resources provided in their subject. Students indicated a strong interest in the Web resources, many commenting positively about resources available in their subject

·"excellent site - used constantly"

·"this is the future, thank you"

·"it is very convenient for both the student and the lecturer"

Some students had suggestions about how the resources could be improved:

·"more examples of code and sample programs"

·"could have included some practice exercises and self review questions"

There were also, however, a few negative comments from students complaining about the quality of Web resources in their subject:

·"Web site needs to be updated more regularly"

·"Lecture notes need to be put up before the lecture"

·"A bit difficult to navigateā

·"Very unorganised. All downloads should be in one place, not scatteredā

One student commented:

·"I did not know a Web site existed for this subject"

Many students expressed a preference for resources in printed form especially lecture notes:

·"please print lecture notes - even if they have to be purchased"

·"I would prefer to have printed lecture notes handed out by the lecturer rather than downloading them"

·"If all the notes are on the Internet, it costs too much to print out"

Discussion

The results of this study are complex. In the extra comments on the survey forms students indicated a strong appreciation of the Web resources. Many remarked positively about availability of Web resources available in their subject. However, when asked to rate the usefulness of individual resources, students indicated that the resources in paper format were more valuable or that there was no difference in value. In the comments some students stated that if a resource was only available on the Web, then they would print a copy. This confirmed observed behaviour of students during semester. Many students are excited by and interested in the Web, but use it purely as a storage medium and a distribution mechanism from which to obtain printed resources [7].

Students rated printed lecture notes as more valuable than lecture notes on the Web. In all subjects in the study, students are encouraged to take a copy of the lecture notes to lectures and annotate these with any necessary extra explanations. If lecture notes are only available on the Web then students have the task of printing a copy. From their comments this is a cost and inconvenience that some students obviously resent

Students who were high Internet users tended to place more value on individual Web resources than the low Internet users. The students who were high Internet users for course administration found all Web resources more valuable, indicating that they had a high engagement in the subject. Students who were high Internet users for course materials indicated all the Web resources were useful except for Anonymous Feedback and self-evaluation exercises indicating a lower level of interactive engagement. However there were no relationship with users of the Internet for games and recreation and the value of resources. Students who used the Internet for communication found the Anonymous Feedback valuable indicating that this resource is used by student who are comfortable with this means of communication but not necessarily those who need help.

Yeatman [12] and Hara [7] suggest that students who are not comfortable using the Internet or who have poor access to the Internet will be reluctant to use it and may be disadvantaged. However results from this study showed that students who found paper resources useful also found the corresponding Web resources useful, indicating that students who are high resource users will tend to use a resource regardless of what format it is in.

Studentsā opinions of the quality of resources influenced how valuable they felt each resource was. They commented on the currency and navigability of resources available for their subject. This indicates that it is important to the students for Web resources to be kept up-to-date, easy to locate and download.

There was a significant relationship between the value of the Web resources of subject updates and subject information and how important students felt Web based resources were for their learning. Although it is reasonable to expect a level of causality, where the values placed on resources should influence the importance to their learning, these particular resources are able to provide regular information and communication with students and can be important for their successful progress through the semester. This relationship implies that students may perceive learning as success in passing a subject however it seems that the provision of functional Web based material will impact on students learning.

Conclusion and Further Work

The widespread acceptance of Web technology in our tertiary institutions has brought about profound changes in the style, quantity and variety of resources that are made available to students. This has, in many subjects, fundamentally changed the learning environment provided for students. Some people believe that there are indications that students are shifting to a new learning paradigm, from one which is paper-based to one which is electronic-based. However the results of this study have indicated that so far an undifferentiated shift or no shift has taken place. The study found that students are staying with what they feel most comfortable. They are enthusiastic about Web resources but they find resources in paper form more useful than in electronic form and they will obtain resources in paper form if they are not provided in this format. The conclusion that can be drawn from this study is that students are still dependent on paper for their learning.

References

1.Arnold, M., (1997) Using the Web to augment teaching and learning, Proceedings of the ASCILITE conference, Perth, WA, Australia.

2.Arnow, D. M. and Clark, D., (1996) Extending the conversation: integrating email and Web technology in CS programming classes, ACM SIGCSE Bulletin, Vol 28, Special Issue, pp 93-95.

3.Boroni, C. M., Goosey, F. W., Grinder, M. T., Ross, R. J., (1998) A paradigm shift! The Internet, the Web, browsers, Java, and the future of computer science education, Proceedings of the 29th SIGCSE Technical Symposium on Computer Science Education, ACM, Atlanta, Georgia, USA. pp 145-152.

4.Debreceny, R. and Ellis, A., (1997) The management of World Wide Web resources in Australian universities, Proceedings of the ASCILITE conference, Perth, WA, Australia.

5.Ellis, A. et al (1998) A collaboration strategy for developing shared Java teaching resources to support first year programming, Proceedings of the 3rd Annual Conference on Integrating Technology into Computer Science Education, Dublin, Ireland, 1998

6.Goldberg, M. W., (1997) WebCT and first year: student reaction to and use of a web-based resource in first year Computer Science, ACM SIGCSE Bulletin, Vol 29, No 3, September 1997, pp 127-129.

7.Hara, N. and Kling, R. (1999) Students' frustrations with a Web-based distance education course, First Monday, Vol 4, No 12, December 1999.

8.Lefoe, G., (1998) Creating contructivist learning environments on the Web: the challenge in higher education, Proceedings of the ASCILITE conference, Wollongong, Australia

9.Lowder, J. and Hagan D., (1999) Web-based Student Feedback to Improve Learning, Proceedings of the 4th Annual SIGCSE/SIGCUE Conference on Innovation and Technology in Computer Science Education, Krakow, Poland, June 1999, pp 151-154.

10.Miranda, J. E. P. and Pinto, J. S., (1996) Using Internet technology for course support, ACM SIGCSE Bulletin, Vol 28, Special Issue, pp 96-100.

11.Pilgrim, C. J. and Leung, Y. K., (1996) Appropriate use of the Internet in computer science education, ACM SIGCSE Bulletin, Vol 28, Special Issue, pp 81-86.

12.Yeatman, H. and Stace, R., (1997) Travelling in the slow lane of the information highway, Proceedings of the ASCILITE conference, Perth, WA, Australia.

Copyright

Judy Sheard, Margot Postema, Selby Markham, © 2000. The authors assign to Southern Cross University and other educational and non-profit institutions a non-exclusive licence to use this document for personal use and in courses of instruction provided that the article is used in full and this copyright statement is reproduced. The authors also grant a non-exclusive licence to Southern Cross University to publish this document in full on the World Wide Web and on CD-ROM and in printed form with the conference papers and for the document to be published on mirrors on the World Wide Web.

[ Proceedings ]

AusWeb2K, the Sixth Australian World Wide Web Conference, Rihga Colonial Club Resort, Cairns, 12-17 June 2000 Contact: Norsearch Conference Services +61 2 66 20 3932 (from outside Australia) (02) 6620 3932 (from inside Australia) Fax (02) 6622 1954