Web-based delivery of Electrical Engineering : Challenges faced and Outcomes

Melissa Norfolk [HREF1], Department of Electrical & Electronic Engineering, The University of Melbourne, Parkville, 3052, Australia. m.norfolk@ee.mu.oz.au

Som Naidu, Multimedia Education Unit, The University of Melbourne, Parkville, 3052, Australia. s.naidu@meu.unimelb.edu.au

Iven Mareels [HREF2], Department of Electrical & Electronic Engineering, The University of Melbourne, Parkville, 3052, Australia. i.mareels@ee.mu.oz.au

Abstract

The "EE1 Online" project [HREF3] is a large collaborative team project which aims to develop a prototype for an online learning environment for undergraduate engineering students. Our chosen subject is an introduction to the fundamentals of electrical engineering. It is difficult to teach, due to the theoretical and sometimes dry nature of the subject matter, but primarily due to the large variation in students' backgrounds. We aimed to make the content more interesting to students through visual animations and circuit simulations, and by presenting the material at various levels of complexity and abstraction, thus making it a valuable tool over the world wide web whilst catering for the diversity of student backgrounds. The current work evaluates the project, the methods used, the resulting online learning environment and how this project may impact on future projects in the Faculty of Engineering. The evaluation of the team dynamics is also important due to the cross collaborative nature of the project.

Introduction

The project brought together nine team members from The University of Melbourne, Box Hill Institute of Teaching and Further Education (TAFE) and Kangan Batman Institute of TAFE, including content experts and web/multimedia developers. The project began in the second half of 1997 and in semester 2, 1998, the first group of students used the site.

Prior to implementation we sent a questionnaire to the student group who had completed the subject in 1997 and also those who were to complete the subject in 1998. This aided some of our decisions and made us aware of student needs and expectations. A third, larger end of semester questionnaire collected further opinions and feedback from the student group who used the web site in 1998. Other evaluation carried out included a questionnaire to the academic and teaching staff involved either in the project team or in teaching the subject, and also small informal focus groups with students to stimulate discussion and gather feedback and opinions that could not be collected from a short answer questionnaire.

The web site was launched to the public on Friday 11th December 1998.

Analysis

The initial planning stages of the project began in late 1997. A considerable amount of effort and time was invested into clarifying the projects goals and it's scope, including the different needs of the University and TAFE sectors.

An evaluation of the team dynamics was completed July 1998, where two people external to the project conducted individual interviews with each team member. The interviews were based on eight questions probing the role played by each team member in the project, identifying subteams, the actual working relationships and generally identifying the strengths and weaknesses of the project.

Early analysis also targetted students who completed the subject in 1997 as well as those enrolled to complete the subject in 1998. Questions gathered student feedback on the way the course was delivered, the material available for downloading on the web, what they liked and disliked and suggestions for improving.

Results showed

• All students had used the web before and felt comfortable being able to access information on the web,
• students strongly felt that an on-line subject should not replace any contact hours,
• students felt that the online subject would be of benefit if used in conjunction with traditional lectures and tutorials.

Methodology and Site Structure

We trialled several commercial tools for building interactive online course environments [HREF4, HREF5] , but we could not find one that met all our requirements. The subject we chose as our prototype, is an introduction to the fundamentals of electrical engineering. It is difficult to teach, due to the abstract nature of the subject content and also the diversity of students entering first year and therefore required a novel approach to make the content more interesting to students and valuable over the WWW. The site is particulary useful in that students can pick their own pace and level of approaching the material.

The development stage of the project began in January 1998. Our goals were to develop an integrated learning environment incorporating everything needed to complete the course. This was to tightly integrate with existing teaching and the delivery of lectures. Lectures were delivered by four different lecturers via a shared set of Powerpoint slides. We integrated these slides with the online material so that students could use them to recall the main topics introduced in the lecture and easily navigate to areas where they had trouble. Hypertext links then took them to the intended pedagogical value of the site. The material is presented at different levels of simplification and detail. The lecture slides providing an overview, the essence of the ideas, which in related and hyperlinked material gets explored in depth through more theory, tutorial exercises and circuit simulations. The aim was not to replace any aspect of face-to-face teaching but enhance students' learning options. Students are now able to explore their study material online, in an interactive way, to achieve better understanding.

Other site features include a weekly online quiz for students to test their understanding on a regular basis, a search facility, glossary and mailing list for each tutorial group. During 1999, we hope to improve the interactive help available from the site to allow students 24 hour access to a virtual tutor.

Evaluation

Our approach to the monitoring and evaluation of the outcomes of this project has been utilisation focussed (Patton, 1986). As such our focus was the use and utility of the instructional innovation for students, lecturers and tutors. We are also interested in the effectiveness of the courseware for achieving the learning and instructional outcomes that have been identified in the above. Use refers to the actual usage of the materials by students in their study of the subject matter content and their levels of satisfaction with it. Utility refers to the actual nature of students' and lecturers' use of these materials such as in and out of class and for particular purposes such as in private study. The goals of the evaluation were as follows; a) to ascertain the use of the EE1 on-line course materials by students in their study; b) to ascertain the use of the EE1 on-line course materials by lecturers in their teaching; and c) to ascertain students' and lecturers' satisfaction with the EE1 on-line course materials.

Instrumentation

The following approaches were used to gather this information:

1. End of semester feedback from all students completing the subject in 1998 (Appendix 1);
2. End of semester feedback from the teaching staff involved with the subject (Appendix 2);
3. Focus-group interviews with volunteer students completing the subject in 1998 (Appendix 3).

End of Year Student Questionnaire (See Appendix 1)

This questionnaire was administered on-line through the subject web page and replaced the traditional end-of-semester questionnaire generally handed out in the last lecture for the year. It was similar to the previous questionnaire, but longer, and asked more specific questions on several aspects that make up the web site (i.e., the slides, the theory and animations, the online quizzes and the interactive circuit simulations) as well as a few questions on tutorials and the text book used. There was also much more room for extended answers throughout the questionnaire, and over half the respondents made use of these sections. In total, 123 students responded with 62 making extended comments, this was a little disappointing considering there were 515 students enrolled in the subject.

Comments about the online slides indicated that :

• On average, students accessed the site fortnightly.
• The lecture slides were felt to be valuable and served as a good summary of material when studying for exams. Some students felt that they could rely on the WWW slides instead of taking notes and as such pay more attention to what was said in class.
• Some students revisited lecture slides to follow up lecture slides they had not understood.
• Some students felt the quality of lectures was diminished by the delivery of the www slides in lectures, because the lecturers did not present any extra material and there were not enough examples shown in lectures.
• Very few students tried to print out or download the contents of the site.
• Some students requested the www slides be different from the slides delivered in lectures to broaden the web material's coverage of the course.

Comments about theory, animation and circuit simulations :

• The circuit simulations were rarely used. But were used as intended by those who did use them. The most common reported reason was the difficulty of installing and other problems including platform dependence, and lack of advertising.
• Animations and examples were perhaps too simple and too few, but these did contribute to learning

"The additional links filled in the gaps of information and showed some practical examples which helped me understand the work."

• Some students greatly praised the animations but they were used less often and felt to be of less value than the textual lecture notes and definitions.
• Technology problems also deterred some students from making use of the animations. (ie. animations not working on UNIX, high bandwidth, wouldn't print, couldn't put on floppy).

Comments about on-line quizzes indicated that :

• Regular quizzes encouraged students to keep up to date , but 1% marks per quiz was too low for some.
• Quizzes were seen as very difficult, but challenging and good preparation for the examination.
• Students observed lots of cheating, which gave a free 10% to some, but some saw the greater returns of not cheating.
• Many enjoyed the feedback, some wanted more feedback.
• More than 80% attempted these quizzes, and about the same number said that the quizzes were useful in their understanding of the week's material.

"The quizzes were difficult, but working on it gave me experiences and a fine build up of confidence and realize the manipulation of electrical engineering knowledge in electrical engineering problem solving."

Academic/Teaching Staff Questionnaire (See Appendix 2)

The seven academic and teaching staff involved in the course delivery and/or development of course content were asked for their feedback at the end of semester. This included the four lecturers who delivered the course in 1998, first year coordinator and two TAFE teachers who helped in the development of course content for the EE Online site.

A set of eleven questions where asked which included what their role was in the online delivery of EE1, how much time they contributed to the project, overall perceptions of the EE1 web site, comparisons between EE1 Online and conventional delivery formats, suitability with their approach to teaching, suggestions for improvements and problems experienced with the site.

Summary of Results

• All responses indicated that they were either using the site in 1998 or would make use of the site for TAFE course delivery from 1999.
• Overall it was indicated that the course materials were good and provided a framework but there was room for expansion and improvement of the materials.
• Responses varied from those that felt it was superior, those that felt it was worthwhile if used in addition to lectures and tutorials, and those that felt it was essentially not very different in it's present form and needed to make use of more multimedia features.
• All responses, except for one, indicated that the web site suited their approach to teaching and was even more useful for large class sizes.
• There was an overwhelming request for more interactive, motivational material, feedback for quizzes and expansion of the current course material.
• Only one respondent indicated they weren't comfortable using online teaching methods and this was due to technology issues.
• All except one indicated that they felt this approach should be used more widely in the Department and the TAFE sector, and this comment was due to limited resources being available.
• Some problems experienced while using the online material in lecturers were technology problems and problems keeping the students' attention as they had already seen the material on the web, or felt they could look at it in their own time.
• Several felt that the site had improved the standard of the course content but others felt it had restricted lecturing styles and had not improved standards.

Student Discussion Forums (See Appendix 3)

Three small group discussions where arranged between students and a non-teaching member of the development team. This was to stimulate discussion and perhaps get more useful feedback than could be provided through the online questionnaire. Twelve students volunteered their time for these forums and all of them gave mostly positive feedback.

Of those interviewed, half resided in university colleges and used the online material extensively as they had unrestricted access to the Internet. The other half accessed the site from home and their main complaint was the cost of being on the Internet, which limited the amount of time, they could make use of the online material.

Questions asked included what they liked and disliked about the site, if it suited their approach to study, what the general feeling amongst the rest of the students was and what they felt could be done to improve the site as a whole.

Summary of Responses

• The majority felt the site benefited them and used it in conjunction with the textbook.
• All of them attempted the quizzes each week and felt this helped them keep up with weekly material covered in lectures, as well as in preparing for the examination.
• All but one preferred the online delivery with other conventional delivery methods.
• Most used the web extensively and felt it suited their approach to study, two commented that they found it hard to study from the screen for long periods and preferred to print it out or make their own notes.
• All agreed that the hyperlinks from the theory slides where valuable, but they would have liked to see more, plus the material could be written in simpler English in some instances. This may be partly due to the fact that 30% of the students are from a non-english speaking backgrounds. A number of international students also commented that the animations aided their understanding better than textual explanations.
• All agreed that they would like to have a printed version of the lecture slides available to them.
• The majority of the students did not make use of the circuit simulation package but would like to in the future. Also they would like to see the online model used in other subjects in the Engineering faculty.

Conclusions

Our experience shows that developing www based materials to assist the learning outcomes for large and diverse classes is a challenging and expensive task, but an extremely valuable one. The technology does offer advantages not available in lectures. As always change implies uncomfortableness, and this was unearthed in the feedback obtained. Also www technology at present does not achieve individualised teaching and learning, but we believe that we have taken the first steps in that direction.

References

Patton, M. Q. (1978). Utilisation-Focussed Evaluation. Beverly Hills, CA: Sage.

Patton, M. Q. (1980). Qualitative Evaluation Methods. Beverly Hills, CA: Sage.

Gellie, K., Hull, J., & Tumbri, B. (1998). The TAFE Contribution to the UMTC EE1 Online Project. EE1 Online Launch: The University of Melbourne [HREF6]

.

Hypertext References

HREF1

http://www.ee.mu.oz.au/staff/melissa/

HREF2

http://www.ee.mu.oz.au/staff/iven/

HREF3

http://www.ee.mu.oz.au/project/

HREF4

http://www.wbtsystems.com/

HREF5

http://www.creator.com.au/

HREF6

http://www.ee.mu.oz.au/project/umtcpaper/

Appendix 1: On-Line Student Questionnaire

Instructions: We are interested in your perceptions of the study materials in this subject. The information you provide here will have no bearing on you personally or your assessment in this subject. Please respond to all the questions in the spaces provided. Your feedback will help us to improve the delivery of this unit in the future.

Lecture Slides: The lecture slides for this subject are available to you online.

1. Please indicate the nature of your use of these slides in the study of this subject.

	All the time	Usually	Rarely
I download the online lecture slides on my PC
I print all the online lecture slides for study later
I rely on the online lecture slides instead of taking notes in the lecture
I use the online lecture slides in studying for my exams
I just browse through the online lecture slides
I browse through the online lecture slides and make my own notes
I print them out and add my own annotations to it
I take my own notes instead of using the online lecture slides

2. How often did you use these online lecture slides?

Daily

Weekly

Fortnightly

Monthly

Only during the last week before exam

3. Did the online lecture slides help you better understand the material presented in the lectures?

• In what way&&&&&&&&&&&&&&&&&&&&&&&&&&..

4. Rate the value of these online lecture slides in your study of this subject.

Extremely valuable

Valuable

Not valuable

Notes and animations linked to lecture slides: These are the lower level notes and animations that are linked to the top level lecture slides on the web.

5. Please indicate the nature of your use of these in the study of this subject.

I download the notes/animations on my PC	All the time	Usually	Rarely
I print all the notes/animations for study later
I rely on the notes/animations instead of taking notes in the lecture
I use the notes/animations in studying for my exams
I just browse through the notes and animations
I browse through the notes/animations and make my own notes
I print out the notes/animations and add my own annotations to them
I take my own notes instead of using the online notes/animations.

6. How often did you use these online notes/animations?

Daily

Weekly

Fortnightly

Monthly

Only during the last week before exam

7. Did the notes/animations help you better understand the material presented in the lectures?

• In what way&&&&&&&&&&&&&&&&&&&&&&&&&&..

• Why not?&&&&&&&&&&&&&&&&&&&&&&&&&&&..

8. Rate the value of the notes/animations in your study of this subject.

Extremely valuable

Valuable

Not valuable

Weekly Quizzes: The weekly online quizzes have been presented to encourage you to test your understanding each week of the new material covered in lectures.

9. How many of the online quizzes did you attempt?

None

One or Two

Most of them

All of them

If you didn't do any, or only one or two say why&&&&&&&&&&&&&&&&&&&.

10. Please indicate the nature of your use of these online quizzes.

	All the time	Usually	Rarely
I used these quizzes for study in the course
I used these for quizzes in preparation for the exam
I relied on these quizzes instead of the textbook
I relied on these quizzes instead of the lecture material

11. Were the online quizzes useful in your understanding of the material covered in the lecture of that week?

• If yes, then how?&&

• If no, then why not?&&.

12. Rate the difficulty level of these online quizzes.

Very hard

Hard

Moderate

Easy

Very easy

13. Rate the value of these online quizzes in your study of this subject.

Extremely valuable

Valuable

Not valuable

Interactive Circuits: The interactive circuit simulation enables you to measure the voltage and current in a circuit, or view the response on an oscilloscope in order to see the circuit working and get a better understanding for how it works.

14. Did you attempt any of the circuit maker exercises?

• Any comments?&&&&&&&&&&&&&&&&&&&&&&&&&&.

• If not, why not?&&.

15. Please indicate the nature of your use of this interactive circuit exercise.

	All the time	Usually	Rarely
I used the interactive circuits for the study of this topic
I used the interactive circuit facility in preparation for the exam
I prefer this material above that provided in the textbook

16. Did the circuit maker exercises inspire you to change circuit values and components and test the outcomes?

• Why not?&&.

17. Rate the value of this material on interactive circuits in your study of this subject.

Extremely valuable

Valuable

Not valuable

18. Rate the level of difficulty of the interactive circuit exercise?

Very hard

Hard

Moderate

Easy

Very easy

The Textbook: The lecture material follows the prescribed textbook quite closely, but does not cover all the material in it. The textbook is the most complete source of material for this course, and allows you to extend your knowledge beyond the minimal requirements.

19. Did you purchase the prescribed textbook?

• Why not?&&.

20. Did you borrow the textbook from the library?

Never

Rarely

Regularly

21. How often did you consult the textbook?

Never

Rarely

Regularly

22. If you used the textbook, how do you rate it?

Excellent

OK

Poor

23. In your opinion does this textbook provide a good explanation of the subject?

• If not, what do you think is its weakness?&&&&&.

 24. How often did you do the exercises presented in the textbook?

Never

Rarely

Regularly

The Tutorials: The tutorials complement the content of the lectures and the textbook. They provide an opportunity to apply the lecture and textbook material in a simple problem-based format.

25. Did you attend T1 and T2 tutorials on a regular basis?

• If no, then why not? &&&&&&&&&&&&&&&&&&&&&&

26. Did you prepare for T1 tutorials?

Never

Rarely

Regularly

27. If you prepared regularly for T1 tutorials. What type of preparation did you do?

	Regularly	Rarely
Read the textbook
Made T1 set problems
Studied the Web material
Tried exercises from the textbook

28. How often did you download the T1 tutorials from the subject Website?

Never

Rarely

Regularly

29. How helpful did you find your tutor in class?

Very

Somewhat

Not at all

30. Do you feel the tutorials assisted you in improving your understanding of the lecture and textbook material?

Very

Somewhat

Not at all

Appendix 2: Teaching Staff Questionnaire

Interviewee: ________________

Interviewer: _______________ Date: __________

Probes:

1. Describe your specific role the online delivery of EE1?
2. Estimate how much time you would have spent on this project?
3. Please describe your use of the EE1 materials in your teaching.
4. On the whole, what did you think of the EE1 Online course?
5. How do you rate the EE1 Online course against its conventional delivery format?
6. Does this approach match with your personal approach to teaching? If so how, and if not then why not?
7. What do you like about the online delivery of EE1?
8. What DON'T like about the online delivery of EE1?
9. What is the word among the academic staff about the online delivery of EE1?
10. What can be done to improve the online delivery of EE1?
11. Do you feel confident with on-line teaching?
12. Would you recommend its use widely in the Department? If yes, please explain and if not then why not?
13. What kinds of problems have you experienced with the use of EE1 online materials in your teaching?
14. What aspects of these materials do you think need improvement?

Appendix 3: Student Focus Group Discussions

Instructions: We are interested in your perceptions of the on-line delivery of EE1. The information you provide here will have no bearing on you personally or your assessment in this subject.

Focus Group: _______ # of Participants: _________

Date: __________ Interviewer: _____________

Probes:

1. What did you think of the EE1 Online course?
2. How would you compare the on-line format with the unit 431-102 which is delivered in the conventional manner?
3. Does the online delivery of EE1 suit your approach to study, if so how, and if not, why not?
4. What do you like about the online delivery of EE1?
5. What DON'T like about the online delivery of EE1?
6. What is the word among the students about the online delivery of EE1?
7. What can be done to improve the online delivery of EE1?

Copyright

Melissa Norfolk, Som Naidu and Iven Mareels, © 1999. The author assigns to Southern Cross University and other educational and non-profit institutions a non-exclusive licence to use this document for personal use and in courses of instruction provided that the article is used in full and this copyright statement is reproduced. The author also grants a non-exclusive licence to Southern Cross University to publish this document in full on the World Wide Web and on CD-ROM and in printed form with the conference papers and for the document to be published on mirrors on the World Wide Web.