In this paper we present computer-based instructional
materials developed as part of the Open
Learning Initiative (OLI) at Carnegie Mellon University that, upon completion,
would constitute an entire online course in Statics. These materials reflect recent progress in re-thinking Statics
instruction, including a recently proposed object-centered approach to teaching
Statics that deliberately separates out individual concepts and treats them
sequentially. These materials also
benefit from studies of conceptual knowledge in Statics, and the development
and psychometric analysis of a Statics Concept Inventory. The computer-based implementation of
instruction incorporates many general lessons from the learning sciences that
are broadly relevant. The structure of
the course materials is presented, including how it reflects a sequence of
learning objectives, which are addressed through means that fully capitalize on
the capabilities of the computer.
Assessment at multiple levels is embedded into the materials, with the
aims of both facilitating learning and monitoring progress. The effect of these materials on learning is
quantified for its first use in a traditional statics course.
_____________________________________________________________________________________________________________________
INTRODUCTION
Statics continues to be a mainstay of engineering
education in many disciplines, forming an important prerequisite for many
subsequent courses. It remains a course
in which student achievement is rarely satisfactory to instructors,
particularly in follow-on courses such as design [1]. A detailed critique of traditional Statics instruction was
recently offered [2]. In most institutions,
Statics is taught with an emphasis on the mathematical operations that are
useful in its implementation, but without enough emphasis on modeling the
interactions between real mechanical artifacts. Often, students who learn Statics in this traditional way fail to
learn to utilize Statics adequately in the analysis and design of mechanical systems and structures which they
confront subsequently. Moreover,
most widely-used Statics textbooks follow essentially the same sequence of
topics as put forth in the first modern textbook in the subject dating from the
1950’s. Changes reflecting a rethinking
of the core concepts in this subject, or observations of the pitfalls to which
many students are prone, appear to be minimal.
In seeking to address these
deficiencies, the authors combined a variety of instructional techniques known
to increase learning, such as active learning, collaboration, integration of
assessment and feedback, and the use of concrete physical manipulatives, to
devise a sequence of learning modules [3].
These learning modules provided stimulating activities for the classroom
that make visible the relation between forces and the object-interactions they
represent. They also reflected a more
deliberate, sequential approach to addressing concepts in Statics. To strengthen the basis for instruction that
addresses concepts, the authors along with others undertook research to
identify key concepts in Statics [4] and to develop and refine a testing
instrument, the Statics Concept Inventory, to measure a student’s ability to
use those concepts in isolation [5-8].
It has been a goal of the authors to expand upon the object-centered,
concept-driven approach to include the full range of ideas and skills that one
needs to learn in Statics and to make the approach more widely available to
students and instructors.
It is natural to explore the
potential of the web-enabled computer for providing broad and potentially
effective access to this new approach.
To harness this potential, the drawbacks of conventional instruction
need to be acknowledged. For example, lecture is sometimes
inconvenient: the words and drawings of the instructor come only once, not
necessarily when the student is most prepared to assimilate them. Traditionally, the opportunities for
displaying phenomena dynamically were minimal.
While computers and projection systems in the classroom do allow
instructors to show such phenomena (should they take advantage of them), they
are run according to the instructor’s whim, not the learner’s. The principal activity outside of class is
solving homework problems. Here, feedback during practice would be most
beneficial, but the feedback loop is particularly weak: students typically get
“graded” homework back, say, one week later, possibly even after they have
completed the subsequent assignment and too late to be useful. Rather than
waiting until exams to recognize their deficiencies, students would benefit
from early, if not instantaneous, assessment.
There have been efforts to take advantage of the computer to enhance
instruction in Statics. “Multimedia
Engineering Statics” [9] spans an entire topic list in a standard
textbook. For each topic it includes a
case study, the relevant theory for addressing the case study, a solution of
the case study, and a simulation.
“Shaping Structures - Active Statics” [10 ] seeks to develop intuition
for the forces in structures. In a
series of simulations encompassing various structural configurations, one can
change the parameters such as the load direction, and see the resulting change
in the forces within the different structural members. In addition there is a series of specific
exercises using the simulations that are recommended to reveal key relationships. “Interactive Learning – Practice to
Perfection” [11] seeks to teach students a consistent series of solution steps,
such as recording data, recording assumption, drawing the free body diagram,
and so forth. It also allows user
input, such as drawing vectors and writing text on the free body diagrams;
although they are not processed by the system, numbers entered for the final
solution are processed. “Self Assessment: Structural Analysis I” [12] offers a
series of exercises on drawing free body diagrams (by dragging loads to points
on a blank diagram) and writing down equilibrium equations; some limited
feedback is offered. “Engineering Mechanics – Statics” [13] offers one-hour
lectures on CD-ROM and on the web covering the full range of topics in a
Statics course. The voice accompanies a
periodically changing graphical display.
“Working Model 3-D Simulations” [14] has a set of Working Model
simulations that accompany selected problems from the textbook. The “Free Body Diagram Assistant” [15]
allows the user to interactively draw elements of a free body diagram, which
the system interprets and then gives the user feedback on the correctness of
various choices. As can be inferred
from this partial review of computer based materials, educators and developers
have certainly sought to take advantage of the simulative capabilities of the
medium, and to some extent the possibility of offering feedback.
While the previous efforts surveyed above offer
examples of learning materials, here we discuss a project to design an on-line Statics
course that enacts instruction, allowing users to learn Statics even if
they do not have the benefit of an instructor or a class. We view such materials as ultimately being
of benefit in the full range of potential use, for the fully independent learner,
as well as for instructors and their students to use to supplement and
complement class instruction.
To enact dynamic, flexible, and responsive
instruction that fosters learning we draw heavily upon the current
understanding of cognition and learning, as described in the next section. We then provide a description of the scope
and structure of the course. Next, we
show interactive examples of several of the activities in the course, pointing
out the basis in lessons from the learning sciences. Potential ways that this course can be used are then identified,
along with how feedback on student activities in the course can benefit various
constituencies. Finally, some initial experience using the materials, including
the responses of the students and preliminary quantification of learning gains,
is briefly presented.
RELEVANT
LESSONS FROM LEARNING SCIENCES THAT GUIDED THE COURSE DEVELOPMENT
Many lessons from
the learning and educational sciences regarding good instructional practice
have guided the design of the OLI Engineering Statics course. It recognized that instruction in general,
and educational courseware in particular, should have clearly articulated Learning Objectives [16]. The OLI Engineering Statics course has
reformulated Statics as a progressive set of learning objectives that are
appropriate for students entering virtually all Statics courses, and that
gradually build student knowledge in this subject.
Students who are actively
engaged learn more [17]. The OLI Statics course currently (as of spring
2008) offers about 200 interactive computer-based tutors of various types,
which are thoroughly embedded in the learning materials, with suitable amounts
of text in between activities. Detailed
description of the types of the tutors with interactive examples will be
offered below.
It is recognized that assessment should be thoroughly integrated into the learning
process [18], with students given ample opportunity to test their knowledge and
receive feedback on their progress.
Many learning studies have shown that learning improves and
understanding deepens when students are given timely and targeted feedback on their work [19, 20]. Furthermore, the best learning outcomes
occur when feedback comes immediately after the students’ response, although
not before the student is ready to revise his or her understanding [21]. Assessment is thoroughly embedded into the
OLI Statics course, and occurs in the context of computer tutors. Students
receive feedback immediately based on their response, and can alter their
responses accordingly.
Providing hints
and scaffolding on demand is a general instructional technique
[22] which allows students to progress in a task as long as they are able, and
provides only what students need should they get stuck. All tutors in the OLI Statics course have
hints available for students; some also have embedded scaffolding. In a typical scaffolded tutor, students can try to
complete an entire task, requiring potentially several steps, on their
own. If they are unsuccessful, the
tutor asks for the result of the first step and so forth for subsequent
steps. A detailed description
of computer tutors with hints, feedback, and scaffolding with interactive
examples, will be presented below.
In conceptually complex domains, self-explanation is found to improve learning [23]. While
the underlying mechanism is not fully understood, learners who self-explain
tend to construct better problem solving procedures and to understand
underlying principles more completely.
One style of tutor used frequently in the OLI Engineering Statics course
is a “Submit and Compare” tutor.
Students are shown some situation or phenomenon, and are asked to answer
a question and provide an explanation for their answer. After submitting their answers, students are
able to compare their answer and explanation with an expert’s answer and
explanation.
A number of principles have been established for
multimedia learning, for example, principles related to contiguity (graphics
and explanation nearby) and effects that constitute distractions to
learning. Among these principles is the
modality principle, which states that receiving complementary information in two modalities, for example
viewing diagrams and listening to an explanation, are often better than seeing
the diagrams and reading the same explanation [24]. The OLI course uses videos that combine evolving graphics and a
spoken explanation, for example to illustrate a procedure that involves
drawings and sometimes equations (we designate such a video a
“Walkthrough”). One might compare a
Walkthrough with a small portion of lecture.
Simulations
can be used to explain certain concepts far more succinctly, and less
ambiguously, than words can. In
particular, they can help learners to connect calculations and numbers with
physical representations [16]. The OLI
Statics uses guided interactive simulation selectively where it
would appear to offer particular benefits, for example, to explain phenomena
that involve motion, including the effect of changing parameters.
Finally, for many subjects in the sciences or
technologies, physical referents or manipulatives can serve to enhance
learning. The use of manipulatives
accommodates students with a greater range of learning styles. As an example relevant to our
implementation, students that learned about pulleys on real pulley systems were
better able to solve real world problems compared to students who learned from
line diagrams [25]. Earlier work by the authors to revise Statics instruction led to a more
object-centered approach, in particular the balancing of simple objects by
hand. This theme also runs through the
OLI Statics course materials.
DESCRIPTION OF THE SCOPE AND STRUCTURE OF THE COURSE
Scope
The course will be divided
into five units, comprising approximately twenty modules. The first two units
(9 modules) are completed (as of spring 2008), and the completion of the
remainder is planned for the fall of 2008. To
access the course go to: http://www.cmu.edu/oli/courses/enter_statics.html
and click on “LOOK inside the free & open OLI Engineering Statics course”. Each module is broken into a set of pages,
and each page is devoted to a carefully
articulated learning objective that is independently assessable. A typical page of the course is displayed in
Figure 1.
The first unit encompasses
the treatment of bodies in planar equilibrium with simple interactions, such as
normal contact, weight, attached cords, and springs. In the second unit, complex interactions between bodies,
beginning with the couple, are introduced, followed by static equivalence and
its applications, and finally with the representation of planar interactions of
common engineering connections. The
third unit addresses the modeling (including reduction via symmetry to 2-D) and
analysis of single and multiple body systems, with simple interactions as well
as engineering connections, with a single solvable subsystem. The fourth unit
deals with configurations commonly referred to as frames, machines, and
trusses. The modular nature allows an
instructor many options, for example to cover trusses before or after frames
and machines. To promote the integration of knowledge addressed in this course
and to help students retain “the big picture”, the major steps in a Statics analysis are
articulated in the course introduction and revisited at the start of each unit
and module (Figure 2).
Figure 1. Typical page from
OLI statics course.
Figure 2. Diagram
illustrating major steps in a Statics analysis.
Learning Objectives
From any page of the course,
students have access to the learning objectives for the current module by
clicking on the objectives button in the top or bottom of the navigation bar.
(see the symbol 
in Figure 1). Most
of the learning objectives are addressed through three highly interactive
elements: exposition (content), formative assessment on conceptual
understanding and problem solving, and summative assessment.
Exposition
In the exposition, the relevant concepts, skills and
methods are explained. Besides words and static images that are the mainstay of
textbooks, basic content is presented through other means. Self-discovery learning is promoted by Non-Interactive Simulations that are
initiated by the student, and might be viewed as analogous to in-class
demonstrations. After each such
simulation, there is always a short “Observation”:
one or two sentences to ensure that the student takes away the intended lesson
of the simulation. In Interactive Guided Simulations, students adjust parameters and see
their effects (what-if analysis). These
are often initiated by a question which the student is supposed to answer, or
suggestions of various outcomes to achieve by adjusting parameters. The extensive use of motion to convey basic
concepts in Statics is part of the authors’ pedagogical philosophy of making
forces and their effects visible [1,3].
The course seeks to take advantage of digital images of relevant artifacts and video clips of mechanisms, to the
extent that they solidify material presented.
Also, consistent with the authors’ pedagogical philosophy of focusing
initially on forces associated with manipulating simple objects, students are
at times guided to manipulate simple objects to uncover relevant lessons. To help students review the key points, each
page, which is devoted to a specific learning objective, ends with a brief
summary called “To Sum Up”.
Problem Solving and
Formative Assessment
After presenting a concept the course offers
opportunities for students to test their understanding of the concept. These frequently involve questions with
yes/no or multiple choice answers; these tutors offer hints and feedback. Since Statics is a subject that requires
doing as well as understanding, some learning objectives are to master
important tasks. Larger procedures have
been carefully dissected and are taught as a series of steps. Several approaches are used to help students
learn such procedures. First, such a
procedure would be explained in straight text.
Second, we often demonstrate the application of the procedure with a
worked-out example or more likely with a “Walkthrough”:
an animation combining voice and graphics that walks the student through an
example of the procedure.
Students themselves first engage in problem solving
procedures typically in “Learn By Doing”
exercises (referred to as LBD’s). These
are computer-tutors in which students can practice the new skill, within a
structure that offers hints and feedback that is similar to tutors that assess
conceptual understanding. When an important, complex procedure is to be
learned, early LBD’s might lead students explicitly through the steps. In some instances, later LBD’s might be
scaffolded, with the student able to work out the solution independently, but
able to request intervention as needed.
In some instances, multiple versions of a problem can be generated with
altered parameters; these enable students to practice a procedure as many times
as needed to master it.
Summative Assessment
At the conclusion of each
learning objective, students have an opportunity to assess their learning
through “Did I Get This?” exercises (referred to as DIGT?).
Such assessments capture the concepts covered in the learning objective,
as well as any procedure which the student was intended to master. The student can then determine whether
further study of previous material is warranted. As with LBD exercises, some tutors are scaffolded, and some can
generate additional versions of the problem, offering the student further
opportunities to practice and test their skill.
DESCRIPTION OF TYPES OF
TUTORS WITH INTERACTIVE EXAMPLES
Tutors with Hints and
Feedback and Scaffolding
These types of tutors were designed to provide both
formative (LBD) and summative (DIGT) assessment
with opportunities for the user to receive hints
and scaffolding, and to get timely
and targeted feedback on their answers.
Hints, often with increasing degrees of
specificity, are available to the student. In the case of multiple hints, the
first hint reminds the student of the relevant underlying idea or principle,
and the second hint may link the general idea to the details of the problem at
hand. Where the answer involves input
of a number rather than selecting from a finite set of options (multiple
choice), bottom-out hints virtually give the correct answer.
In addition, each answer input by the student provokes
feedback. When possible, feedback is intended to
provide information that encourages the revision or refinement of thinking. Thus, in some cases the feedback is tailored
to each incorrect answer, particularly when a likely diagnosis of the error can
be made. In other cases feedback may be
generic "That's not right".
There are clearly benefits of such immediate feedback as compared with
traditional paper and pencil homework that is graded and returned far too late
to be of value.
To illustrate how the hints and feedback are used we
show in Figure 3 a tutor in which students practice calculating the moment of a
force applied at point P about point O using its components parallel to and
perpendicular to OP. This tutor appears
as a “Learn By Doing” exercise in
Module 4 entitled Effects of Multiple Forces (module 4 / Calculating Moments
Using Components (2 of 2); EXAMPLE: Calculating Moments Using Components 4).
This appears in a portion of the module where students learn to find the moment
of a force by resolving it into components, an effective approach when the
moments due to individual components are simple. While hints are generally in the form of words, this tutor
illustrates how hints may be provided in graphical form.
Click the image on the left to open and
interact with the tutor.
Figure
3. Tutor on calculating moments using perpendicular and parallel components,
illustrating hints in verbal and graphical form
Some tutors offer more elaborate scaffolding. First, the student is given the opportunity to
solve the problem entirely independently.
If unable to do so, the student can request help and be reminded of the
first step, with hints and feedback available to complete that step. If this step is the road block that prevents
solution, then helping the student get the correct result for this step will
ultimately lead to the correct answer.
If this step is NOT the road block, little effort was wasted in asking
for the result of that step since it had to be done anyway. At any
point, the student can choose to complete the problem independently, or if
necessary request scaffolding for another step.
We first illustrate scaffolding with a tutor shown in
Figure 4, also from Module 4 on Effects of
Multiple Forces, which asks students to
resolve forces described in distinct ways and sum them. This tutor appears as a “Did I Get This?” exercise and is an opportunity for
students to do a "self-check" to make sure they understand the
concepts (module 4/ Summing Forces; second DIGT). The student is presented
with a graphical representation of the problem and asked for the answer. If the
student is unsure of the procedure for solving the problem, the first hint
provides a link which, when clicked, expands the tutor into the various steps
needed to solve the problem. The tutor provides scaffolding to support the
student to learn the steps of the procedure when needed. The hints and feedback
given by the tutor change depending on which part of the exercise the student
is attempting. Since the problem statement, hints, feedback and answers are
dynamically-generated, the student can work through the tutor multiple times,
receiving a different problem each time, until the student is confident that he
or she understands the concept and has developed fluency with the procedure.
This provides the student with virtually unlimited opportunities for supported
practice.
Instructions
for Tutor 4
Click the image on the left to see a video of a user
interacting with this tutor (allow up to 2 minutes to download), and then click on the image on the
right to open and interact with the tutor yourself.
If you are not sure how to proceed,
click the hint button.
You may need to click the link in
the first hint that expands the tutor into the multiple steps that are required
to solve this problem.
Type your answers into each box and
do not hesitate to ask for hints for each step as you work through the problem.
There are multiple levels of hints
for each step; you may continue to ask for hints by clicking the 'get next
hint' link at the bottom of the hint window until you reach the final hint that
gives you the answer for that step and allows you to continue working on the
problem.
Allow up to 2
minutes to download video
Figure 4. Tutor on resolving
and summing forces, illustrating scaffolding.
As second illustration of scaffolding, we show in
Figure 5 a tutor which features two trucks, each with a crane, which tip over
because the loads are too large. From
the given free body diagram, users are to find the reactions on the tires or
supports, and interpret the results of the solution. This tutor appears as a “Learn
By Doing” exercise in Module 5 entitled Equilibrium under 2D Forces (module
5 /Forces In
One Direction (2 of 3) EXAMPLE: Equilibrium under forces acting in the same
direction 4). As can be seen
at the right, scaffolding has been provided at the student’s request (the steps
refer to four overall steps in solving problems using equilibrium). Notice that students can be prompted to
think about strategy, and to write down a particular equation of equilibrium,
with the algebraic equation to be constructed from multiple pull-down menus.
Click the image
on the left to open and interact with the tutor[AD1] .
Figure 5. Tutor on applying
the conditions of equilibrium, illustrating scaffolding.
Submit and Compare Tutor
A number of tutors in the OLI Statics course require students
to answer and to explain their answers.
Such tutors were designed to take advantage of the potential benefit of self-explanation.
In some cases, students select from several possible
explanations. Other tutors request free
form input which is expected to be a one or two-sentence response to the
question. After the student submits her
answer, an expert’s answer appears and the student may compare them. Such “Submit
and Compare” exercises seek to foster critical thinking on the part of the
student.
By way of example, we show on Figure 6 the first two
from a series of four “Submit and
Compare” tutors, which addresses the representation of forces between
various bodies. These tutors feature
the scenario of a hand gripping a cord which is attached to a cart and appear
as “Learn By Doing” exercises in
Module 1 entitled Representing Interactions between Bodies (module 1/ EXAMPLE: Cable and
Attached Body). The student is asked to consider
successively the various forces between the cord, the cart, and the hand,
whether the sense of the force can be determined, and why. Because there is a series of four questions,
students who submit their answers and study the expert answer have a chance of
improving their argument.
Click the image
on the left to open and interact with this series of tutors[AD2] .
Figure 6. Tutors on identifying
senses of forces exerted by and on cords, illustrating Submit and Compare.
Walkthrough
These types of tutors were designed to demonstrate
procedures or explain complex ideas that would be difficult to follow with
conventional written text and diagrams. Here the system provides complementary information in distinct
modalities; this capitalizes on the advantages of using multiple
pathways (aural and visual) to convey information. Further, the diagrams evolve in synchrony with the voice so the user’s
attention is appropriately focused (consistent with the contiguity
principle). Compare this with the
burdens of going back and forth between text in a textbook and the figures on
the side or on the next page. When such
a presentation is provided with standard video controls, the user has full
ability to pause, stop, rewind, and repeat.
As pointed out above, such a presentation is analogous to a small
portion of lecture. While an instructor
can provide as good an explanation involving voice and graphics, students
cannot readily ask the live instructor to repeat
selected portions of lecture multiple
times the way they can replay a video file. (Of course, a video file cannot respond with an altered
explanation based on a student query.)
Such capabilities also allow for more convenient review of material.
We illustrate this technique with a “Walkthrough”, taken from Module 8
entitled Application of Static Equivalency (Figure 7). This “Walkthrough”,
(module 8/ Center
of Gravity (3 of 4); EXAMPLE: Center of Gravity 2), explains the method of determining the center of
gravity by decomposing a body into simple shapes, each of which has an obvious
or tabulated center of gravity.
Click
the image below to activate this tutor and use the play, pause, stop, rewind,
and repeat buttons
Figure 7. Walkthrough describing procedure of determining center of
gravity for composite body.
Interactive Guided Simulation
Students learn in part through a process of constantly
comparing their understanding and predictions with observations. In many subjects, dynamic simulations can provide
observations to be compared with predictions.
Simulations can also help significantly in conveying complex ideas that
are difficult to convey with static images.
With regard to Statics, the digital environment allows
us to make forces and their effects visible to students in ways that are not
possible in the traditional classroom. In teaching Statics, simulations of
motion are critical to conveying the various effects of forces, and the
conditions for equilibrium. In a traditional classroom, neither a traditional
textbook, nor an instructor, can offer dynamic simulations with parameters which
are controlled by the learner seeking to explore relevant phenomena. In the OLI
statics course, learners can experiment with the parameters and see the effects
of their experimentation in Interactive Guided Simulations such as the
ones shown below. We often introduce the guided simulations with a question for
the student to answer and follow it with a succinct description of an
observation the student should have made.
To illustrate simulation, we show in Figure 8 a tutor
from Module 5 entitled Equilibrium under 2D Concentrated Forces. This simulation appears in the context of a
problem where the balancing of a uniform rectangular bar by a pair of fingers
is considered (module 5/Forces In One Direction (1 of 3); EXAMPLE: Equilibrium under
forces acting in the same direction 1; click on “Show FBD”, “Next”, and then the simulation appears). The user can alter the magnitudes of the two
forces representing supporting fingers, and view the resulting motion. We use this as a discovery learning
exercise; it strengthens the idea that equilibrium (keeping the bar motionless)
involves consideration of both translation and rotation, requiring the
independent balance of forces and of moments.
The course utilizes motion extensively to show the effects of forces; in
all cases, the forces are turned on for a brief period of time, so any
acceleration they produce results in a constant velocity. In this tutor users also see the immediately
updated equations that capture force and moment summation; this serves to strengthen
the relation between the algebraic result and the physical result
(motion). Moreover, this exercise is
guided, in that users are prompted to produce several different outcomes; of
course, users can freely explore as well.
Click the image on the left to open and
interact with the tutor; clicks on “Show FBD”, then, “Next”, and the simulation
appears
Figure 8. Guided Simulation motivating the discovery that forces on a body
independently control the translational and rotational tendencies, and that
both must be zero for equilibrium.
INSTRUCTIONAL
ROLES INTENDED FOR OLI STATICS
The OLI Statics course is capable of being used in
several distinct modes described below.
Blended into
traditional course with instructor
It can be used in a blended mode, serving
as supplemental material, or electronic textbook and tutor, for students in a
traditional instructor-led course. The modular format permits instructors to
include all or only selected elements of the courseware. Since the materials are designed to be used
independently by students without supervision outside of class, they also
enable asynchronous/distance learning for students who might be off-campus
during some period to stay abreast of the course. Since the materials give students constant feedback as to whether
they are on track, components of the course may be assigned as “required
learning” as opposed to “required reading” outside of class, with instructors
receiving reports on student usage.
Some of class time may be freed to focus more productively on, for
example, design projects, more advanced critical thinking, and problem solving.
Major instructional source
for class with course coordinator
When institutions are limited by the
availability of instructors for a particular course, an OLI course can function
as a fully stand-alone course. Credit
for such courses is offered through academic institutions that connect to OLI,
and there are currently no charges for institutions. Resources may allow for an individual to serve as course
coordinator, with the bulk of instructional responsibility falling on OLI. In this way, OLI courses increase the
options available to a broad range of institutions, including small engineering
programs and community colleges, which may wish to offer Statics courses, but
find themselves on occasion without a suitable instructor.
Fully independent learners
Finally, OLI courses can serve individual,
independent learners who wish to learn subjects without receiving credit. Individuals can register so that their
progress is tracked from one session to the next, or even work
anonymously. For such students, the
course materials constitute an electronic textbook with a private tutor. This may also serve the needs of learners in
non-traditional programs where background in certain subjects, but not credit,
is necessary. Furthermore, the OLI
course materials could form a resource for students who have completed Statics
and are reviewing either for a follow-on course or for professional licensure
examinations.
Feedback to constituents
Virtually all of a student’s interactions with the system can be
logged. Obviously, this facility, if
suitably exploited, can enable students to track their progress and complete
“required learning” assignments. It
likewise enables an instructor or course coordinator, if present, to monitor
whether students are keeping up with assignments. Logged interactions, together
with data-mining technologies, also offer the potential for constructing
patterns of success and failure that signal to the instructor areas where the
class as a whole, or sizable groups of individuals, need additional
instruction. In fact, data-mining can
provide evidence to course developers on which to base further improvements in
the course and data to cognitive scientists who study, for example, learning in
an on-line environment.
Testing
Extensive user testing of OLI courses prior to the development
of the OLI Statics course established the usability of interface elements that
are common to many OLI courses. Some interface elements that were developed
specifically for the OLI Statics course were user-tested at CMU in Spring 2006
by experts in human-computer interaction.
Hired students spent one hour on various portions of modules and then
took a test related to their learning; these students had taken physics, but
had not completed, nor were enrolled in, a Statics class.
The first five modules were used in a blended mode
during the first six weeks of two sections of a Statics class at Miami
University in Spring 2007. Students
worked through portions of modules in class, so the instructor could observe
and offer help if needed. The completion of modules was assigned to be done
outside of class. In the first six
weeks of the semester there was no lecture, and no textbook homework; only the
OLI course was used with the exception of two lectures devoted to couples and
static equivalency (topics beyond the first five modules).
Preliminary Assessment of Student Learning
Pre- and post-tests (paper and pencil assessment
problems) corresponding to learning objectives in each of the modules were
administered to all of the students taking the course at Miami University,
immediately prior to (pre), and immediately after (post) using each respective
module. In addition, we monitored, for
comparison purposes, the performance of students on the class exams, as well as
on the nationally-used Statics Concept Inventory [5-8].
Results of the analysis of gains as measured by the
paper-and-pencil assessment tests are shown in Table 1. The pre- and post test for each module were
administered on different days; hence the sample size N varies across the modules. The analysis only included results for which
both pre- and post-test scores were available.
For each student, the normalized gain is the increase in score
(post-pre) normalized by the maximum possible gain (100% - pre). (Note that the mean of the normalized gain
can and does differ slightly from the gain calculated directly from the pre-
and post-test means.) To determine whether the gain was significantly different
statistically from the null hypothesis of no gain, the t-statistic, which is the mean gain normalized by the standard
error (standard deviation normalized by
) was computed. The
associated probability p that the
sample could have been randomly selected from a population with zero mean gain
is seen to be extremely low in all cases, except for module 4, which is still
below a 0.05 threshold. Regarding the
magnitudes of the mean normalized gains, we offer the comparison with Hake [26]
who used normalized gain to compare scores on the Force Concept Inventory (FCI)
data at different schools. Hake found
the mean normalized gain for the FCI in traditional classes to be 0.23 ± 0.04
and 0.48 ± 0.14 in classes with interactive engagement. Again, it must be
emphasized that only OLI courseware was used for these topics (no lectures).
Table 1. Results of analysis of gains on
paper-and–pencil assessment tests
|
Module |
N |
Pre-test (%) (Mean) |
Post-test (%) (Mean) |
Gain (Mean) |
Normalized Gain (Mean) |
t-statistic |
p |
|
1 |
32 |
38 |
81 |
43 |
0.67 |
11.96 |
< 0.0001 |
|
2 |
33 |
51 |
94 |
43 |
0.81 |
10.90 |
< 0.0001 |
|
3 |
30 |
38 |
70 |
32 |
0.50 |
5.061 |
< 0.0001 |
|
4 |
14 |
45 |
66 |
21 |
0.32 |
2.266 |
< 0.02 |
|
5 |
27 |
21 |
60 |
39 |
0.51 |
10.85 |
< 0.0001 |
We sought to understand whether the learning tested by
the paper-and-pencil assessments was relevant to the Statics course overall,
such as measured by the final exam.
Since each of the assessments focused on a small set of concepts in the
course, one should not expect a significant correlation with such a broad
measure as a final exam. However, in
the case of module 2, which addresses free body diagrams (the forces that ought
to be represented on separated bodies), such a correlation was found. The Pearson correlation between the gain on
this assessment test and the final exam was 0.502 (p = 0.003). As a comparison, the correlation between the
other three class exams and the final were 0.611, 0.663, and 0.758. Thus, the material tested by the module 2 assessment
is closely related to other learning in the course.
We also sought to establish the significance of
learning during the one third of the semester that used the OLI course by
utilizing the nationally-used Statics Concept Inventory (SCI). This inventory addresses the core concepts
in Statics and reports out sub-scores on nine individual concepts. While the first five modules of the OLI
course are designed to lay a solid foundation for many Statics concepts, they
relate directly to only one concept tested by the SCI: selecting the correct
forces to be represented in the free body diagram of a subset of bodies
extracted from a larger system. (This
is largely the subject of module 2.)
Analysis of results over the past years has shown that this concept
sub-score of the SCI correlates strongly with final exams at many institutions.
The performance on this concept sub-score of the cohort of Miami students using
the OLI course was compared with Miami students who took Statics three years
prior with the same instructor (co-author A.D.), who used the Learning Modules
developed by the authors [2, 3] but not OLI, with Miami students who took
Statics in 2005 with a different instructor (also without OLI or the Learning
Modules), and with other universities in Fall 2005. (Miami students did not take the SCI in 2006.) The mean of this sub-score is shown in
Table 2; there is no essential difference between the scores when A.D. was
instructor with and without OLI. The
mean for instructor 2 (2005) was compared to the students who used OLI (2007)
via a t-test; the differences were
found to be very significant (t =
2.72, p < 0.001). Furthermore, it must be remembered that the
OLI students did not have any instruction in this part of the course outside of
the OLI materials.
Table 2 Scores on Statics Concept Inventory sub-score
related to Free Body Diagrams (subject of modules 1 and 2)
|
Class |
2004 (Miami) A. D. using L.M. |
2005 (Miami) other instructor no L.M. |
2005 at different universities |
2007 (Miami) A. D. OLI |
|
FBD Sub-score |
0.623 |
0.327 |
0.32 to 0.79 |
0.645 |
User Feedback and Perceptions
Miami students were surveyed at the end of the Spring
2007 semester (33 out of 38 responded).
Students were asked to rate a series of statements on a Likert scale from 0
(strongly disagree) to 4 (strongly agree). With regard to aspects of the course materials that
most enhanced their learning experience, students were most strongly positive
about the materials’ “allowing me to repeat selected portions
of the course” (mean rating
3.73), “allowing me to work at my own pace” (3.64), “providing opportunities to
repeat (selected) exercises to get more practice” (3.58), “allowing me to
conveniently review the material before exam” (3.18), and “allowing me to control and
observe simulations, and draw conclusions”(3.12). Students seem
to value most the following features of the course: “hints in
"Learn by Doing" and "Did I Get This" tutors” (3.15);
“interactive simulations” (2.97); “wrong answer feedback provided by
"Learn by Doing" and "Did I Get This" tutors” (2.94); and
the “capability of some "Learn By Doing" and "Did I Get
This" tutors to automatically generate additional problems for me to work
through” (2.94). With regard to “The opportunity
to practice the concepts I learned in the course (i.e. the amount of available
exercises or problems)”, the mean student response was (2.03), with the scale defined by the range from 0 -
too little (I could have used more practice), to 2 - just right, to 4 - too
many (I didn't work through them all).
Learners are encouraged to submit comments about the
courseware through “My Response” links at the end of each module. Comments from students are taken seriously
and routinely incorporated into improvements in the course.
SUMMARY
This paper describes a
web-based course that seeks to fully enact instruction in Statics. The course draws heavily upon previous work
to enhance Statics instruction in the classroom, and to identify key conceptual
difficulties that students have. In
fully enacting instruction, the course provides interactive content, as well as
opportunities to practice and receive feedback on both conceptual and
procedural elements of the subject.
Materials are suitable as a standalone course for an independent learner
or for blending into an instructor-led course.
The course is structured into units, each consisting of modules, which
in turn are broken into pages, each with an independently assessable learning
objective.
Design
of the course materials draws upon many lessons from the learning
sciences. Students need to remain
active during the learning process, and to be given frequent opportunities to
assess their progress. This assessment
should offer timely feedback, targeted to the students’ specific
trajectory. Hints should be available
to provide scaffolding to students at early stages, but there should be
opportunities to practice independently, with additional scaffolding available
when needed. Students should be encouraged
to develop deep rather than shallow understanding, through opportunities to
explain their thinking and receive feedback on it. The multimedia capabilities of the computer should be
appropriately exploited. For example,
many students better process and integrate information when they receive it via
multiple modalities, such aural (voice) and visual. Also, simulations can be beneficial if students are guided to
derive the intended lessons from them.
By comparison with traditional classrooms, the OLI Engineering Statics
course offers students far more fine grained feedback on their progress, as
well as convenience of study and review.
The monitoring of student activities allows detailed data to be
accumulated by the system; when fully harnessed, instructors will have
actionable feedback to inform classroom instruction.
Initial
experience gained in blending a limited set of modules (five) into an
instructor-led Statics course was described.
Based on specially designed pre- and post-tests that focus on the
concepts covered in each module, evidence of learning gains attributable
exclusively to the OLI modules was found.
In addition, we monitored class performance on the one concept covered
by the Statics Concept Inventory (drawing of forces appropriately on a
collection of parts) that is fully addressed in the limited OLI modules
used. Even though the bulk of students’
exposure to this topic was through OLI, students performed at least as well as
a previous class having classroom instruction from the same instructor.
ACKNOWLEDGEMENT
The authors are
indebted to the principal developer/programmer on this project, Ross Strader,
who has contributed greatly to the realization of this project. We also appreciate the programming
assistance of Renee Fisher, and the thoughtful input of Candace Thille, Marsha
Lovett, Bill Jerome, and Aaron Bauer.
Support by the William and Flora Hewlett Foundation, by Miami University
Department of Mechanical and Manufacturing Engineering, and by the Department
of Mechanical Engineering at Carnegie Mellon University, is gratefully
acknowledged.
REFERENCES
1. Harris, T.A. and Jacobs, H.R., 1995, J. Eng. Educ., Vol. 84, p.343.
2. Steif, P.S. and Dollár, A., Reinventing The Teaching Of Statics, International Journal of Engineering Education, Vol. 21, No. 4, pp 723-729, 2005
3. Dollár, A. and Steif, P.S., Learning Modules for Statics, International Journal of Engineering Education Vol. 22, No. 2, pp 381-392, 2006 (interactive paper at: http://www.ijee.dit.ie/OnlinePapers/Interactive/Statics/Learning-Modules-for-Statics-Dollar-Steif.html )
4. Steif, P.S., An Articulation of the Concepts and Skills which Underlie Engineering Statics, 34th ASEE/IEEE Frontiers in Education Conference, Savannah, GA, October 20-23, 2004.
5. Steif, P.S. and J.A. Dantzler, A Statics Concept Inventory: Development and Psychometric Analysis, Journal of Engineering Education, Vol. 33, pp. 363-371, 2005.
6. Steif, P.S., Dollár, A., John A. Dantzler, Results from a Statics Concept Inventory and their Relationship to other Measures of Performance in Statics, 2005 Frontiers in Education, Indianapolis, Indiana, October 2005
7. Steif, P. S. and M. A. Hansen, Comparisons Between Performances In A Statics Concept Inventory and Course Examinations, Int. Journal of Engineering Education, Vol. 22, 2006, pp. 1070-1076
8. Steif, P.S. and M. A. Hansen, “New Practices for Administering and Analyzing the Results of Concept Inventories", Journal of Engineering Education, Vol. 96, pp. 205-212 (2007).
9. Multimedia Engineering Statics, Kurt Gramoll,
10. Shaping Structures- Active Statics, Simon Greenwold and Edward Allen, 2003
http://acg.media.mit.edu/people/simong/statics/Start.html
11. Interactive Learning – Practice to Perfection, P. Venkatazaman, 2004 http://www.rit.edu/~pnveme/plig_2004/Statics/?downloadURL=true&loId=82251379-A59F-4512-AE82-AFF798E92D04
12. Self-Assessment: Structural Analysis I, Educative Technologies LLC, 2004
13. Engineering Mechanics-Statics CD ROM associated with textbook by William F. Riley and Leroy D. Sturges on "Engineering Mechanics",
http://www.gatewaycoalition.org/files/Engineering_Mechanics/page2.html
14. Working Model 3-D Simulations; supplement for Beer & Johnston, Statics and Dynamics,
15. Free Body Diagram Assistant, Robert J. Roselli, 2004,
16. Teng1, X., Tront, J.G., Muramatsu, B., and Agogino, A. Best Practices in the Design, Development and Use of Courseware in Engineering Education, 35th ASEE/IEEE Frontiers in Education Conference, Indianapolis, IN, October 19-22, 2005.
17. Dufresne, R.J., Gerace, W.J., Leonard, W.J., Mestre, J.P., and Wenk, L. (1996), Classtalk: A classroom communication system for active learning, Journal of Computing in Higher Education, 7, pp. 3-47.
18. Black, P. and William, D., “Assessment and Classroom Learning, “Assessment in Education, Vol. 5(1), 1998, pp. 7-73.
19. Butler, D. L., & Winne, P. H. (1995). Feedback and self-regulated learning: A theoretical synthesis. Review of Educational Research, 65, 245-281
20. National Research Council. (2004). How people learn: Brain, mind, experience, and school. Expanded Edition. Washington, DC: National Academy Press
21. Corbett & Anderson, 2001, Proceedings of ACM CHI-2001: Locus of feedback control in computer-based tutoring: Impact on learning rate, achievement and attitudes
22. Wood, D., Bruner, J.S., & Ross, G. (1976). The role of tutoring in problem solving. Journal of Psychology and Psychiatry. 17, pp. 89-100.
23. Chi, M. T. H., Bassok, M., Lewis, M. W., Reimann, P. & Glaser, R. (1989). Self-Explanations: How Students Study and Use Examples in Learning to Solve Problems. Cognitive Science, 13, 145-182.
24.
R., Moreno,
R.E, Mayer, Cognitive principles of multimedia learning: the role of modality
and contiguity, Journal of Educational Psychology, 91, 358-368, 1999.
25. Ferguson, E. L. and Hegarty, M., 1995, Cognition & Instruction, pp. 129-160.
26. Hake, R, (1998), Interactive Engagement Versus Traditional Methods: a Six-Thousand Student Survey of Mechanics Test Data for Introductory Physics Courses, American Journal of Physics, 66, 1, pp. 64-74.
|
This
work is licensed under |
Carnegie Mellon University 5000 Forbes Avenue Pittsburgh, Pennsylvania 15213 |
Funding provided by |
