Using innovative online quizzes to
assist learning
Abstract
Online quiz tools are commonly used in online education. These tools can be
used to generate multiple choice questions for a range of purposes, including
summative and formative assessment. This paper provides examples of 4 different
types of quizzes that assist learning and the develop particular graduate
attributes, and discusses how they can be further adapted to improve their
effectiveness. The power of quiz tools to assist teaching and learning through
innovative designs that support personal construction of understandings,
simulate group feedback for individual learners, provide self-assessment with
tailored feedback and build confidence in learning online, are discussed. The
quizzes are analysed using Diana Laurillard's conversational framework, graduate
quality development potential and other attributes, to gauge their effectiveness
as learning tools. The analysis was useful to steer future development of the
teaching resources.
Introduction
The Web and education
The Web holds potential for facilitating self-directed learning and
streamlining assessment processes. Computer-assisted learning resources can help
to explain difficult concepts through combining the use of text, graphic, video,
sound and links (Velan et al., 2002). The benefits of the online medium
for delivery of this teaching and learning experience are its flexibility and
Web accessibility. It has the potential to give students the freedom to access
learning environments when it best suits them and in the power to control their
own learning (Glasser 1999; Schweizer, 1999). A drawback of the Web as an
educational environment is its accessibility, which includes the technological
requirements to access Web-based learning experiences and the sometimes
intermittent nature of its availability. At the University of South
Australia an in-house online teaching and learning platform is used called
UniSAnet.
Graduate qualities at the University of
South Australia
The University of South Australia (UniSA) has embedded a key skills
framework, called Graduate Qualities, throughout the curriculum and assessment
processes. Seven attributes are developed. These include the body of knowledge
(graduate quality 1), information literacy and life long learning through self
evaluation (graduate quality 2), problem solving skills (graduate quality 3),
the ability to work autonomously and collaboratively (graduate quality 4),
consideration of social and ethical issues (graduate quality 5), effective
communication (graduate quality 6) and awareness of international and cultural
aspects of their professional discipline (graduate quality 7).
[HREF6].
These qualities are embedded into the curriculum such that the unit value of
each course is examined to reveal the proportion of teaching and assessment that
contributes to the development of each of the 7 skills (graduate quality profile
of the course). Course profiles can be combined to reveal the qualities
developed in a program. Not all graduate qualities are developed in each course,
but when a student graduates from a program, all seven graduate qualities are
said to be developed. Strengthening links between graduate quality development
within a course, and its assessment, has been identified as a strategic teaching
and learning priority for UniSA in 2003
[HREF7].
Types of assessment
Assessment can be categorised by the way it can be used to measure student
performance (Airasian and Madaus, 1972). Formative assessment does not
contribute to the students’ final grade. It is usually done during a
course and feedback is provided to the student to help them make changes to
improve their performance in subsequent assessment activities. Summative
assessment usually occurs at the end of a learning experience and is used to
determine the grade for the student. It provides information on the success of
the whole of the learning activity. This paper focuses on the use of online
quizzes to provide formative assessment to support learning, and extends its use
to that of a teaching and learning tool. Assessment contributes powerfully to
teaching and learning, in fact,
... for most students, assessment requirements literally define the
curriculum. Assessment is a potent and strategic tool for educators with which
to spell out the learning that will be rewarded and guide students into
effective approaches to study. Equally, however, poorly designed assessment has
the potential to hinder learning or stifle curriculum innovation. (James et al.,
2002: p7).
Current view of multiple choice
questions
One type of formative assessment is an online quiz containing multiple
choice questions. Briefly, multiple choice questions, sometimes called
“items”, are composed of a question stem and several answer options.
True/false questions are a simplified form of a multiple choice question.
Students need to select one option and they are then assessed for accuracy
(summative quiz) or provided feedback on their responses (formative quiz). As an
assessment tool, the current view of multiple choice questions has been jaded
(Linn and Gronlund, 1995), and there are concerns amongst educators that
multiple choice quizzes promote surface learning and transmission teaching
styles.
There is the predominant use of multiple-choice questioning, which is
typical of the transmission approach as it serves to check that the message has
been received. Such assessment is based on how much, and how accurately,
information is known rather than what is understood, whereas a more student
centred approach focuses on what is understood (Brooksbank et al., 1998:
p51).
This paper examines a new genre of multiple choice questions made possible
by an online quiz tool, and attempts to analyse their potential as learning
tools using Diana Laurillard’s conversational framework and their ability
to support development of different graduate qualities.
Analysing learning technologies using
a conversational framework
According to Diana Laurillard, there are several basic types of learning
technology, narrative (where the learner experiences attending and
apprehending), interactive (where the learner investigates and explores),
adaptive (where the learner experiments and practises), communicative (where the
learner discusses and debates) and productive (where the learner articulates and
expresses) (Laurillard, 2002). Online quizzes fall into the category of
interactive or adaptive media. Quizzes containing intrinsic feedback are
adaptive, and can assist learning, in fact,
... intrinsic feedback is extremely valuable to the learner. It enables
them to know how close they are to a good performance, and what more they need
to do. It is individualised, private, formative feedback, which helps to build
their understanding of the internal relations between theory and practice.
(Laurillard, 2002: 126-127).
In addition, Diana Laurillard has analysed how students learn and from
this, developed a 'conversational framework' by which to evaluate educational
media in their ability to support the learning process.
Figure 1. The Conversational
framework
(Laurillard, 2002a; p 87, and reproduced from Laurillard 2002b; used with
permission
[HREF8].This framework was designed to be applied to any learning situation, bar
‘everyday’ learning (Laurillard, 2002: p 87), to assess if the
environment can foster all aspects of the learning process. Educators have used
this framework as a basis for planning the mix of online technologies required
to effectively teach using technology (Pascoe and Abernathy, 1996; Hannon and
Umble, 2002).
UniSAnet online
quizzes
In UniSAnet
[HREF9] quizzes
can take one of the following forms: multiple choice, multiple response, fill in
the blanks, true/false, match/label items, order text and enter text. For each
type of question the author can customise questions, answers, and feedback
for each response and for the question as a whole. Responses are stored in a
database so that scoring and management of questions is possible. Quizzes are
‘learning objects’ that can be used in a range of
contexts.
At present, quizzes in UniSAnet are only available as formative assessment
tools and the quiz author has no direct access to statistics that reveal the
activity that has occurred between learners and the quiz. As such, UniSAnet
quizzes are a private, independent learning tool for learners at UniSA. By their
nature they develop information literacy and lifelong learning (graduate quality
2) as they promote independent study techniques and self critique. They are
often used to develop a body of knowledge (graduate quality 1). However, when
online quizzes are adapted to one of the 4 different types described below,
their potential for the simultaneous development of other graduate qualities
increases.
The aim of this paper is to
- demonstrate different ways in which online quizzes can be used to assist
learning
- analyse the different quiz types to determine how they can support learning
- describe the development of graduate qualities through the use of online
quizzes
Quiz type 1: Online quizzes to assist
learning by supporting personal construction of
understanding
Introduction
Feedback to the learner, whether it be intrinsic (embedded and automatic)
or extrinsic (a comment on the action) is essential to the learner and to the
personal construction of understanding (Laurillard, 2002).
The teacher has to devise situated actions that elicit meaningful intrinsic
feedback to the student, or redescribe the student's description in a way that
gives meaningful extrinsic feedback to the student (Laurillard, 2002:
p58).
In any discipline there are crucial understanding, pivotal points of
learning that students must surmount to move forward in the topic. Academics
will know these points well - they are the ones that require much time and
effort to redirect misconceptions. These are the points where students approach
teachers directly to attempt to expand their understanding. The learners may
present their current understanding of the issue in different ways, depending on
their personal experiences, however a similar learning pathway will need to be
travelled, but with slightly different support.
The empirical result that possible misconceptions are few in number
reflects the experience of most teachers that the student errors they encounter
in tutorials, assignments and examinations are the same every year. It is rare
for a student to come up with a wholly new way of getting it wrong. So if the
forms of error are relatively few in number, why are they not documented so that
we can address them in future teaching? (Laurillard, 2002: p184).
It is these learning misconceptions that are worth investing time in as a
teacher, to elucidate the most effective teaching and learning strategies. The
use of online quizzes is one way that this can be achieved, and online quiz type
1, which will be discussed below, is an example of how this can be
achieved.
Constructing
understanding
Quizzes can be developed in ways that they play less of an assessment role,
and more of a teaching role. They assist learning by utilising teaching
techniques that determine where students have their current understanding, and
then taking them forward from that point. The main difference in type 1 quizzes
compared to standard multiple choice questions is that there are no
“right” or “wrong” answers identified. Multiple
viewpoints, as realised through teacher’s previous teaching experience,
are presented as answer options. Each conception (answer option) is valid in
some way and needs to be built upon to create deeper understanding, which is an
element of Constructivist pedagogy;
This shift in perspective, from viewing knowledge as something external to
be "mastered" to an internal construction or an attempt to impose meaning and
significance on events and ideas lies at the heart of what has been called the
constructivist paradigm. In this view, learners are not passive beings who
respond to "stimuli," and learning is not merely the appropriation of previously
devised labels and categories. Instead learning is an active process of
constructing meaning and transforming understandings. One corollary of this is
that because no two people have had identical experiences, each person
constructs a more-or-less idiosyncratic explanatory system: a unique map of the
topography that we call reality (Candy,1991: p251-252).
and
What people construct from a learning encounter depends on their motives
and intentions, what they know already and on how they use their prior
knowledge. Meaning is therefore personal.... (Biggs, 1999: p13).
By responding to type 1 quizzes (selecting an answer option) students can
align themselves with the view most complimentary to their own personal
understanding in a non-threatening, private arena and receive feedback. However
care needs to be taken in the language used.
Writing constructive feedback about
personal understanding
Learning is an emotive exercise (Brookfield, 1990) and criticising people's
understanding requires consideration of the language used.
In critical thinking episodes or educational experiences described as
"transformative", there are times when what was thought to be fixed, true and
permanent is found to be relative, shifting, and culturally specific. .... What
were previously accepted as common sense, taken-for-granted, conventional
wisdoms are now seen as distorted and inadequate to account for reality. ...
This process is not entirely joyful, in fact it is often distressing and
disturbing (Brookfield, 1990: p46-47).
Care needs to be taken in writing feedback about personal understanding.
The feedback that is available within type 1 quizzes needs to acknowledge the
students’ contribution to the common knowledge as well as pointing out
where they need to continue to develop. The text can be written with a voice and
tone that can help build a sense of belonging and fun (Glasser, 1998a), in
addition to the content and direction. This supportive, encouraging voice, that
can sometimes bring a smile to your face, can be referred to as the language of
learning, and it is a feature of those learning environments that support high
achievement through perseverance (Glasser, 1998b).
Example A of Quiz type 1: Ethical issues in Medical Science
[HREF10]Example B of Quiz type 1: Misconceptions in responses to acute blood loss
[HREF11]
Analysis of online quiz type
1
Type 1 quizzes are a type of ‘adaptive media’, that is they are
capable of changing their state in response to the user’s actions
(Laurillard, 2002: p126). The teacher is able to describe their conception
through the question stem and answer options (step 1 of the conversational
framework). The student is able to communicate their conception by selecting an
answer option (step 2). Within the formative and supportive intrinsic feedback,
the teacher can re-describe the conception in light of the identified
misconception (step 3).
By using the back button on the browser it is possible for students to
explore other answer options, however, but as this is not a feature of the
design, type 1 quizzes do not necessarily allow students to redescribe their
conceptions in light of the teachers redescription (step 4). It is possible, by
using embedded hyperlinks in the feedback, to direct learners to a new task goal
in light of the way the student has responded (step 5). For example, in Example
A, students who were unfamiliar with the practice of medical ethics were
directed to the ethics page within the International Federation of Biomedical
Laboratory Science Web site. By experiencing this feedback, and visiting the Web
site, students would be in a position to adapt their actions (step
10).
This is where the presented type 1 quiz examples begin to fail in as far as
Laurillard’s framework is concerned. In both examples the teacher does not
go on to set a task goal (step 6) that the student can work through to achieve
(step 7). The teacher does not set up a subsequent world to give intrinsic
feedback based on the students’ action completing the task goal (step 8),
and the student is unable to modify their action in response to the intrinsic
feedback (step 9) nor reflect on the experience (step 11). Besides a secondary
evaluation of the quiz, the teacher is not able to reflect on the
learners’ action to modify their description (step 12) (see table 1 for
summary).
Type 1 quizzes are well suited to the examination of ethical and social
issues related to professional practice (graduate quality 5), improving
relational understanding through problem solving (graduate quality 3), as well
as developing the body of knowledge (graduate quality 1) (see table 2 for
summary).
The use of type one quizzes could be one way that we can address the
diversity of our student population and be more inclusive in the curriculum.
Tailoring feedback for the unique situation of the learner, acknowledging what
they bring, and expanding their view appropriate to their understanding is what
makes type 1 quizzes a valuable learning tool (see table 3 for
summary).
By using Laurillard’s framework as a guide for future development, it
would be possible to expand the feedback provided in type 1 quizzes to identify
further tasks for the student, depending on the current understanding of the
student. For example in the Medical ethics question, the student could be asked
to use the Web site to answer a subsequent quiz, or to submit a written task and
a reflection on learning. This would help stretch the learning potential of type
1 quizzes to include steps 6, 7, 8, 9 and 11. As mentioned above, the inclusion
of an evaluation exercise would need to be included to satisfy step 12 of the
conversational framework.
Quiz type 2: Simulating group feedback
for individual learners
Introduction
Online quizzes strongly support individual learners by their nature.
However, much learning is actually social and can be assisted by the learner
being part of a group, and receiving different types of feedback (physical,
emotional) from them. Teaching techniques such as role-play are powerful
learning tools as they involve students using the 'whole person -intellect,
feeling and bodily senses' to create real experiences (Brookfield, 1990: p 115).
Actions by one member impact on other members of the role-play. In addition,
learners are in a position to access the perspectives and experiences of other
members of the role-play. Unfortunately face-to-face (and online!) role-plays
are difficult to establish and run, and can approach impossible as student
numbers increase.
Online quiz type 2 is an online quiz that has been constructed to allow
individuals to participate in mock role-play activities. By including scripting
in the feedback of the UniSAnet quiz, responses to answer options can be in
context with the students’ previous responses. The decisions a learner
makes in the role play are also influenced by the feedback of the rest of the
group. In quiz type 2, responses can be presented using images, producing a
visual form of group feedback.
Analysis of quiz type
2
Using Laurillard’s framework to analyse online quiz type 2, it
becomes apparent that it offers additional learning potential when compared to
quiz type 1.
Type 2 quizzes are more adaptive than type 1 as the scripting in the
feedback allows contextualised feedback to be offered. The teacher is able to
describe their conceptions through the question stems and introductory
statements (step 1 of the conversational framework). The student is able to
communicate their conception by selecting an answer option to each of the 10
questions as well as journaling their thoughts (step 2). Within the graphical
feedback, the teacher can re-describe the conception of power in light of the
group and the student’s answer (step 3).
The quiz includes a reset functionality which allows the students to
reattempt the role-play exercise if they wish, thus allowing them to redescribe
their conceptions in light of the teachers redescription (step 4). At the end of
the quiz learners are directed to a task goal (step 6) which is unique depending
on their experiences (journaling their experiences and reflections to set
questions) (step 5). During this reflection students are able to act (step 7)
and receive feedback through non-online modes (step 8). They are also in a
position to adapt their actions in light of their often transformative learning
(step 10)(Table 1).
In the initial iteration of quiz type 2 the reflection was performed by
students in workbooks. Subsequent iterations will be further challenged by
posting their reflections to a facilitated discussion group. Steps 6, 7, 8, 9
and 11 of the framework will be enhanced, making this an comprehensive learning
experience as measured by Laurillard’s conversational framework.
It can also be noted that Laurillard’s model does not directly
examine the learning potential of group feedback.
Type 2 quizzes are well suited to exploring social and ethical issues
(graduate quality 5) and could be adapted to revealing international and
cultural issues relevant to professional practice (graduate quality 7). They
offer an independent online learning option that has an apparent sensation
(visually and emotionally) of being part of a group (Tables 2 and 3).
Quiz type 3: Self assessment for
tailored feedback
Higher education institutions is Australia are striving to improve
inclusivity of education, where inclusivity is
The essence of inclusivity is understanding and catering for the different
potentials and needs and resources of our students (DECS, 2002: p3)
[HREF13]However, in practice, inclusivity often means verbosity. Often, when
written information attempts to be highly inclusive, the delivery system becomes
swamped and there is a potential for individual learners to become lost. One way
to address this is to include self assessment of learning needs followed by the
provision of tailored feedback. Self assessment has been defined as;
... a process in which learners assess their existing competencies at the
start of a period of study ...to enable learners ... to identify learning needs,
so that rational plans can be made to meet those needs (Hammond and Collins,
1991: p116-117).
Self assessment puts the learner in control of the instruction provided and
information that is irrelevant to that individual can be eliminated from their
learning experience. This self-editing assists the construction of meaning by
screening out that which of no concern of the learner.
Online quiz type 3 allows this type of self assessment and tailored
feedback. Issues are assembled in order as multiple choice questions to which
the learners are able to select the answer option most suited to their
situation. No answer option is identified as correct or incorrect. The feedback
provides the information most relevant to their needs.
Analysis of online quiz type
3
Type 3 quizzes are well suited to situations where diverse learners need to
access information specific to their situation. They are a type of adaptive
media (Laurillard, 2002; p126). The teacher is able to describe their conception
through the question stem and answer options (step 1). The learner is able to
communicate their understanding of their position by selecting an answer option
(step 2). Within the contextualised feedback, the teacher can re-describe the
issue in light of the learners situation (step 3).
By using back on the browser it is possible for learners to explore other
answer options, and in this example this was a reality as staff operated in
different courses with different conditions (step 4). As in quiz type 1,
embedded hyperlinks in the feedback direct learners to new task goals in light
of the way the student responded (step 5) and can adapt their actions
accordingly (step 10).
The teacher sets the task goal (step 6) but it is the learner who, using
the feedback must select the appropriate action. External to quiz type 3, the
learner can work through the task (step 7), but the teacher does not set up a
subsequent world to give intrinsic feedback based on the students’ action
completing the task goal (step 8), and the student is unable to modify their
action in response to the intrinsic feedback (step 9) nor reflect on the
experience (step 11). Similarly, teachers are unable to access information to
reflect on how learners have progressed through the quiz (step 12) (Table
1).
Although the example presented has been used in professional development,
it is clear that type 3 quizzes would be useful in undergraduate teaching and
could foster the development of problem solving skills (graduate quality 3).
Careful design of type 3 quizzes in teaching can result in the development of
inclusive curricula that can be sculpted to suit the individual needs (Table 2
and 3).
Quiz type 4: Online quizzes for
information and confidence building
When students are new to online learning environments they can be somewhat
overwhelmed with how to navigate through the Web site, what the requirements
are, what they need to purchase to participate in the course and when the
assignments are due etc. The processes associated with online learning can be
distinctly different to learning techniques that students have previously found
to be successful and therefore the experience can be sometimes
debilitating.
Fear is an important element in every novice computer user’s first
attempts to use a new machine or new software: fear of destroying data, fear of
hurting the machine, fear of seeming stupid in comparison with other users, or
even to the machine itself (Rheingold, 1995: p10).
The doubt that exists in students’ minds that they do not fully
understand the educational processes to commence learning, can make them lose
their confidence to participate in an online environment.
This problem can be addressed to a degree by the construction of type 4
quizzes. These take the form of standard multiple choice questions, with correct
answer options available, however the topic that is assessed is not the
course content, but the processes by which it will be taught. Type 4 quizzes
allow students to check their understanding of the educational processes prior
to participating and can impact favourably on the level of confidence that they
have in themselves as participants.
The following example describes a quiz that is a precursor to an online
role-playing exercise that assesses, explains and redirects as necessary to
ensure students have essential understanding to participate.
Analysis of online quiz type
4
Type 4 quizzes are adaptive like the other 3 quiz types described above
(Laurillard, 2002). The teacher is able to describe their conception through the
question stem and answer options (step 1). The learner is able to communicate
their understanding of their position by selecting an answer option (step 2).
Within the contextualised feedback, the teacher can re-describe the issue in
light of the learners situation (step 3).
By using back on the browser it is possible for learners to explore other
answer options (step 4) but this was not a design feature of this quiz type. As
in quiz type 1 and type 3 quizzes, they can contain embedded hyperlinks in the
feedback to direct learners to new task goals in light of the way the student
responded (step 5) and can adapt their actions accordingly (step 10).
The teacher sets the task goal which will be the subsequent role play (step
6) and the learner can work through the task (step 7) albeit external to the
quiz. The teacher can set up a subsequent world to give intrinsic feedback based
on the students’ action completing the task goal (step 8), and the student
is able to modify their action in response to the intrinsic feedback (step 9)
and reflect on the experience (step 11), but again this is indirectly. The
teacher can access information about the role-play and reflect on how they might
describe their conceptions in the future (step 12). However if we consider quiz
type 4 as a singular element, distinct from the role-play, then it is not able
to deliver these latter aspects (Table 1).
Type 4 quizzes help develop problem solving skills (graduate quality 3) and
are well suited to situations where learners need to grapple with new learning
environments and develop confidence in participating in the learning environment
(Tables 2 and 3).
Conclusion and further
work
In this paper we have described innovative ways that online quizzes that
have been used at the UniSA using the online quiz tool within UniSAnet, UniSA's
online teaching and learning platform. Four types have been described, but they
are probably part of a larger typology of online quizzes that can be used to
assist learning. Although these quizzes have been used with students and staff
of the University, their evaluation as learning tools has been sporadic and
anecdotal. Further work is required to determine the effectiveness of each of
the quiz types as learning tools. It is hoped however, that by describing these
4 quiz types, the quiz design process, for those teachers interested in using
online quizzes as teaching tools, will be simplified.
Quiz types 1 to 3 are distinct from normal multiple choice questions as no
right or wrong answers are specified. The purpose is to give appropriate
feedback, whether it is to help construct understanding, improve understanding
of social and ethical issues through group feedback, or to self-select
information most relevant to the learner’s personal needs. The structure
of quiz type 4 is standard, in that correct answer options are identified,
however its purpose, to improve confidence by assisting learning of educational
processes, is distinctive. In addition, these types of online quizzes are not
only able to enhance the body of knowledge (graduate quality 1) and critical
self-reflection skills (graduate quality 2), their use can assist learners in
their development of other qualities including awareness of social, ethical,
international and cultural issues (graduate qualities 5 and 7) (see Table
1).
Each of the 4 innovative quiz types were analysed using Laurillard’s
framework for learning technologies. The framework was useful to determine the
limitations of the technology and to suggest further developments that could
improve the effectiveness of the learning experience. Laurillard’s
framework evaluates whether various conversations between teacher have been
allowed, but not about the nature of those conversations, and the potential
impact on the learner of the language used in learning technologies. For
example, the framework did not capture the language of learning as being
essential to provide constructive feedback on personal understanding, nor the
impact of group feedback on learning. Language, and the formation of supportive
communities, has been found to be important in learner success in online
teaching and learning environments (O’Regan et al.,
2002).
Type 1-4 quizzes can take a considerable amount of time to develop. When
deciding where to start, we suggest targeting areas of the curriculum that are
known to confuse students or are notoriously difficult to teach, for quiz
development. In this way the quality, efficiency and flexibility of the learning
outcomes gained by the student, will justify the investment of time and
intellectual effort by the teacher/developer.
Tools and frameworks that strive to analyse the learning attainable within
online environments are an important aspect of the broader need in higher
education to develop scholarly practice in teaching and learning (Glassick et
al., 1997). Through assessment of the value of new approaches to teaching
and learning, significant improvements and efficiencies can be gained.
Table 1: Summary of the evaluation of
the 4 quiz types using Laurillard's conversational framework (Laurillard,
2002)
|
Step
|
|
Type
1
|
Type
2
|
Type
3
|
Type
4
|
|
1
|
Teacher can describe conception
|
ü
|
ü
|
ü
|
ü
|
|
2
|
Student can describe conception
|
ü
|
ü
|
ü
|
ü
|
|
3
|
Teacher can redescribe conception or action
|
ü
|
ü
|
ü
|
ü
|
|
4
|
Student can redescribe in light of teachers redescription or student's
action
|
¢ (back)
|
ü
(reset)
|
ü
|
¢ (back)
|
|
5
|
Teacher can adapt task goal in light of student's description or
action
|
ü
|
ü
|
ü
|
ü
|
|
6
|
Teacher can set task goal
|
¢
|
ü
|
ü
|
ü
(RP)
|
|
7
|
Student can act to achieve task goal
|
¢
|
ü
|
ü
(external)
|
ü
(RP)
|
|
8
|
Teacher can set up a world to give intrinsic feedback on actions
|
¢
|
ü
(external)
|
¢
|
ü
(RP)
|
|
9
|
Student can modify action in light of feedback on action
|
¢
|
¢
|
¢
|
ü
(RP)
|
|
10
|
Student can adapt actions in light of teachers description or student's
redescription
|
ü
|
¢
|
ü
|
ü
(RP)
|
|
11
|
Student can reflect on interaction to modify redescription
|
¢
|
ü
|
¢
|
ü
(RP)
|
|
12
|
Teacher can reflect on student's action to modify redescription
|
¢
|
¢
|
¢
|
ü
(RP)
|
|
13
|
Students can receive peer feedback on their actions
|
¢
|
ü
|
¢
|
ü
(RP)
|
|
14
|
Design supports constructive feedback
|
ü
|
¢
|
ü
|
ü
|
|
15
|
Design is inclusive
|
ü
|
¢
|
ü
|
¢
|
|
16
|
Design supports development of graduate quality 1 (body of
knowledge)
|
ü
|
ü
|
ü
|
¢
|
|
17
|
Design supports development of graduate quality 2 (life-long
learning)
|
ü
|
ü
|
ü
|
ü
|
|
18
|
Design supports development of graduate quality 3 (problem
solving)
|
ü
|
¢
|
ü
|
¢
|
|
19
|
Design supports development of graduate quality 4 (autonomous and
collaborative)
|
ü
|
ü
|
ü
|
ü
|
|
20
|
Design supports development of graduate quality 5 (social and ethical
issues)
|
ü
|
ü
|
¢
|
¢
|
|
21
|
Design supports development of graduate quality 6 (communication)
|
¢
|
¢
|
¢
|
¢
|
|
22
|
Design supports development of graduate quality 7 (international and
cultural perspectives)
|
ü
|
ü
|
¢
|
¢
|
ü indicates that the quiz
complies
¢ indicates non-compliance
(RP) indicates the quiz complies only when examined as part of the
associated role play exercise, but independently the quiz does not
comply.
(back) indicates quiz complies when the back button is used on the browser
to reattempt the quiz.
(external) indicates that an external activity linked to the quiz assists
with compliance, but independently the quiz does not comply.
Table 2: Summary of the evaluation of
4 quiz types for the UniSA graduate qualities
|
Design supports development of graduate quality 1 (body of
knowledge)
|
ü
|
ü
|
ü
|
¢
|
|
Design supports development of graduate quality 2 (life-long
learning)
|
ü
|
ü
|
ü
|
ü
|
|
Design supports development of graduate quality 3 (problem
solving)
|
ü
|
¢
|
ü
|
¢
|
|
Design supports development of graduate quality 4 (autonomous and
collaborative)
|
ü
|
ü
|
ü
|
ü
|
|
Design supports development of graduate quality 5 (social and ethical
issues)
|
ü
|
ü
|
¢
|
¢
|
|
Design supports development of graduate quality 6 (communication)
|
¢
|
¢
|
¢
|
¢
|
|
Design supports development of graduate quality 7 (international and
cultural perspectives)
|
ü
|
ü
|
¢
|
¢
|
ü indicates that the quiz
complies
¢ indicates non-compliance
Table 3: Summary of the evaluation of
4 quiz types for other factors known to promote learning
|
Students can receive peer feedback on their actions
|
¢
|
ü
|
¢
|
ü
(RP)
|
|
Design supports constructive feedback using the language of
learning
|
ü
|
¢
|
ü
|
ü
|
|
Design is inclusive
|
ü
|
¢
|
ü
|
¢
|
ü indicates that the quiz
complies
¢ indicates non-compliance
(RP) indicates the quiz complies only when examined as part of the
associated role play exercise, but independently the quiz does not
comply.
References
Airasin, P. W., and Madaus, G. J. (1972). "Functional types of
student evaluation" in Measurement and Evaluation in Guidance, 1972 v.4,
p. 221-233.
Brooksbank, D. J., Clark A., Hamilton, R., and Pickernell D. G.
(1998). "A critical appraisal of the WinEcon and its use in a first year
undergraduate economics program." in Association for Learning Technology
Journal, 1998 v.6 n.3 p.47-53.
Candy, P. C. (1991). Self-direction for lifelong learning: a
comprehensive guide to theory and practice. Jossey-Bass, San
Francisco.
Department of Education and Community Services (DECS)(2002) "The
inclusivity challenge: within reach of us all."
[HREF13]
(accessed 20/7/03).
Glasser, W. (1998a). Choice theory: a new psychology of personal
freedom. HarperCollins, New York.
Glasser, W. (1998b). Choice theory in the classroom. HarperPerennial, New
York.
Glassick, C. E. (1997). Scholarship assessed: Evaluation of the
Professoriate. Josey Bass, San Francisco.
Hammond, M., and Collins, R. (1991). Self-directed learning: critical
practice. Kogan Page, London.
Hannon, P. A., and Umble, K. E. (2002). "Gagne and Laurillard's Models of
Instruction Applied to Distance Education: A Theoretically Driven Evaluation of
an Online Curriculum in Public Health" in
International Review of Research in
Open and Distance Learning 2002 October, (
[HREF16], accessed
21/3/03).
James, R., McInnis, C., and Devlin M. (2002). Assessing Learning in
Australian Universities: ideas, strategies and resources for quality in student
assessment. Centre for the Study of Higher Education, Melbourne, and The
Australian Universities Teaching Committee, Canberra.
Laurillard, D. (2002a). Rethinking University Teaching: a conversational
framework for the effective use of learning technologies (2nd ed.)
RoutledgeFalmer, London.
Laurillard, D. (2002b). Rethinking University teaching in the digital age
(
[HREF17],
accessed 21/3/03)
Linn, R. L., and Gronlund, N. E. (1995). Measurement and
Assessment in Teaching (8th ed.) Prentice-Hall, New Jersey.
O’Regan, K., Quinn, D., and Wache, D. (2002). "Can a virtual
community be a reality?" Proceedings of the 2002 Effective Teaching and Learning
Conference, Brisbane, October 2002.
Pascoe, R., and Abernathy D. (1996) "Teaching a Diploma in Medical
Informatics using the World Wide Web". N.A.WEB 96 - The Second International
North America World Wide Web Conference
[HREF18] University of New Brunswick.
(
[HREF19],
accessed 21/3/03).
Rheingold, H. (1995). The Virtual Community, Minerva,
London.
Schweizer, H. (1999). Designing and teaching an on-line course: spinning
your web classroom. Allyn & Bacon, Boston.
Velan, G. M., Killen, M. T., Dziegielewski, M., and Kumar, R. K. (2002).
"Development and evaluation of a computer-assisted learning module on
glomerulonephritis for medical students." in Medical Teacher 2002, v.24
n.4 p.412-416.
Hypertext References
Copyright
Diana Quinn and Ian Reid © 2003. The authors assign to Southern Cross
University and other educational and non-profit institutions a non-exclusive
licence to use this document for personal use and in courses of instruction
provided that the article is used in full and this copyright statement is
reproduced. The authors also grant a non-exclusive licence to Southern Cross
University to publish this document in full on the World Wide Web and on CD-ROM
and in printed form with the conference papers and for the document to be
published on mirrors on the World Wide Web.
