| 
January/February
2004 - Discussion Board-Based Group Work in a Large
Enrollment First Semester Organic Chemistry Lecture Course
By: Tyson A. Miller, Visiting
Teaching Associate, Department of Chemistry, University of Illinois at
Urbana-Champaign, tmiller2@uiuc.edu
Keywords: chemistry;
large enrollment; online course; online education; discussion
board; group work; organic chemistry
Note: This version contains a detailed account of the work discussed.
There is a synopsis for
those who would like only the highlights.
Abstract:
This work examines
the practicality of a single facilitator implementing online discussion
board-based group work in classes with large enrollments (> 200 students).
Simultaneously, student perception of online group work value and utility
via this methodology was also measured and provided insight on the benefits
and costs with implementing discussion board training on this scale.
Comparisons between opinions from students in face-to-face and distance
education students
are also made. 669 students in a first semester non-majors organic chemistry
lecture course were teamed in small subgroups of 5-6 students to solve
a potential exam question via discussion board on the WebCT course management
program. Over the course of two weeks, students oriented themselves with
one another, developed a timeline for solving the problem, discussed
their
answers, and posted their collective answer on a group discussion board
forum. Students were surveyed before and after the exercise about computer
background, experience with online tools and courses, perceptions about
group work, and value of the assignment. Their opinions merited the following
conclusions: a) opinions about group work did not change overall from
the experience, 2) an overall positive experience from the assignment was
gained,
3) students benefited most by experiencing group work online, learning
how to use discussion boards, and learning how to better use one of the
functions
of WebCT. A stark increase in student confidence using discussion boards
was reported. A higher percentage of distance education students (vide
infra) benefited from the assignment as compared to the rest of the class.
Facilitator
effort in design, construction, and execution, and grading of the assignment
was rather intensive. A reduction of student support for the exercise
in later midterm and end of semester course surveys was determined. This
result
predictably reflects the need for the integration of online work in a
sustained manner within the design of the course to be practical for the
facilitator
and popular with the students.
Background:
Students who enroll in the first
semester non-majors organic chemistry course at the University of Illinois
Urbana-Champaign are generally science or pre-professional majors who need
the course to fulfill requirements. Enrollment size is typically 600-800
students in the fall semester and 300-400 students in the spring semester.
Fall Semester 2003 was slightly
unusual from previous semesters. Due to schedule changes and overlap of
times in the required courses of the students enrolled, students were split
into three sections over two different lecture times to accommodate the
overload and the conflicts. Also, two instructors were used for the course:
one instructor at 8:00 AM on Monday, Wednesday, and Friday (Section A) and
one instructor at 9:00 AM on the same days with the students seeing the
lecture face-to-face (Section B) and via simultaneous closed-circuit live
projection in a separate classroom (Section B1).
This was the first time the chemistry
department at UIUC had offered a course via closed-circuit projection.
It was discovered early that many students in the video section had little
to no experience with this type of educational setting, and so students
were treated in this study as distance education students.[1] Predicting
that some students who were forced to register for the video Section
B1
might be prime candidates for dropping the course in frustration,[2]
the need existed to redesign the course in two principal ways: 1) the distance
students should have functional connections with the students from face-to-face
sections to avoid feelings of isolation; 2) the distance students needed
a medium by which they could have an elevated amount of access to the
instructor
and to other students. Face-to-face students could interact with the
instructor during lecture in real time. Therefore, it was felt that increased
instructor
access for the distance education students around the examination times
might restore educational equity in the students’ minds. Discussion
boards offered a technological solution for offering help to students
in a controlled and universally accessible fashion.
In addition to overcoming these
obstacles, the class enrollment at the time of designing the experiment
was 675 students. As of this writing, there is no reported precedent for
online discussion board small group work (4-6 students) as part of a large
enrollment course of this magnitude in any discipline.[3] Effective student
group work means designing the assignment for maximum interaction of the
students, which means instructor facilitation of discussion amongst the
groups.[4] This is multiplied by a factor of 100 in this case. To maximize
the chances for an overall increase in student confidence using the discussion
boards, clear evidence of instructor involvement at each stage of the assignment
is crucial to success.[4] Therefore, the design of the assignment needed
to include automated functions that demanded a minimal consumption and a
maximum efficiency of the use of facilitator time.
Objectives
of the Work:
The proposed solution to both
of these issues was to incorporate online group work into the course that
a) permitted students across the three sections to interact with others,
b) gave them hands on training in the use of technology, and c) increase
their proficiency and confidence in using that technology. If students could
gain confidence in using online discussion boards, then increased distance
student access to the instructor via discussion boards might be a viable
response to educational environment equality concerns. In addition, discussion
board-based group work may increase student efforts to continue building
online study groups with one another after the conclusion of the assignment.4
Therefore, the “WebCT Assignment” (name
given for this project) was designed to answer the following questions:
1) Can large classes be trained to use WebCT in online discussion board
work efficiently and successfully? 2) Can student confidence in discussion
board use increase as a function of online group work? 3) Can a single
primary instructor effectively facilitate online discussion board group work
for
a large class? 4) Will distance education students believe they have
benefited from online group work experience as much or more than their face-to-face
student peers? 5) What will be the overall level of student satisfaction
with the exercise? 6) Is discussion board-based facilitated small group
work practical for instructors of courses with large enrollments?
The WebCT
Assignment:
Students were given two weeks
to introduce themselves and work together with 4-5 other students from other
lecture sections to solve one potential test question via online discussion
board. Students were also required to fill out online surveys before and
after the assignment to provide insight on the following issues: student
experience with classroom technology, student confidence using classroom
technology, student attitudes regarding group work, and student perceptions
about the assignment and its benefits.
Students were given a detailed
handout that provided the following information: 1) an introduction to
the purpose and details of the assignment, 2) a step-by-step list of how to
complete the assignment for full points, 3) a suggested timeline for
the
completion of the assignment, 4) a detailed set of instructions for how
to post messages on a discussion board, 5) a detailed list rules for
proper online Netiquette behavior, 6) a list of deadlines for each phase of
the
assignment, 7) a rubric that outlined objective criteria for earning
all of the points for the assignment. The suggested timeline included recommendations
for meeting other subgroup members and agreeing to post and discuss answers
within the first week. This would provide ample time for students to
get
familiar with the technology and for subgroup members to get to know
one another. It was recommended that the final week be used solely for posting
answers, discussing answers, deriving a consensus answer to the question,
and agreeing who should post the subgroup’s answer in the Group forum.
Three superceding criteria were
established in the rubric that nullified credit for the assignment in
the event of failure to do the orientation activity, of failure to participate
in the subgroup’s effort to solve the problem assigned, or for Netiquette
violations (unprofessional behavior; i.e., profanity, offensive language,
etc.). The assignment itself was worth 1% of the overall course grade.
However, 2 of the 14 questions from the assignment were offered verbatim
on the first
exam, increasing the overall value to about 2.5% of the overall course
grade. Students appeared motivated by the prospect of providing the answer
key
to a few questions on the exam.
Experimental
Design:
At the time the assignment began,
669 students were enrolled in the 3 sections of the course: Section A – 264;
Section B – 319; Section B1 – 86. Subgroups of students were
established based on two goals: 1) having an appropriate number of students
to work on a problem of moderate difficulty that could end up as an exam
question; 2) having every subgroup in an approximate 2:3:1 ratio of students
by section (A:B:B1, respectively). 112 subgroups of 5-6 students (109
with 6 students, 3 with 5 students) were then established at the start of
the
assignment based on the criteria.
Figure 1 shows the organization
of the subgroups and the assignment. 112 subgroups can be easily combined
into 8 groups of 14 subgroups. The students communicated with their own
subgroup team to solve the potential exam question and would have a different
exam question from each of the other subgroups in that group. However, with
a group-level forum, they could begin to communicate on a social or professional
level with other students in the class. This design limits their maximum
student contact to others in their group and subgroup only (84 maximum)
for the duration of the assignment and keeps the discussion boards from
cluttering into a disarray of postings from the onset. Monitoring and facilitating
discussion board activity at the subgroup (5-6) and group (83-84) level
seemed much more reasonable for a single facilitator to accomplish.
Experience with the Making the
Virtual Classroom a Reality coursework from the Illinois Online Network
led to the appreciation of an orientation activity and having a social
discussion board forum where students can go to talk about anything other
than organic
chemistry.[4,5] The orientation activity chosen was the “8 Nouns” activity,
where each subgroup member lists eight nouns plus a brief description
of each noun that best describes that member to the other members of the
subgroup.[6]
The social forum was called the Expresso Bar, and was established at
the group level to allow students to converse with classmates beyond their
subgroup
at a manageable level.[5]
The discussion board forums that
each student had access to at the onset of the assignment included the
following: a) Main; b) Notes; c) Instructor’s Office – Group (Name); d)
Expresso Bar – Group (Name); e) Orientation – Subgroup (Name);
f) Subgroup (Name); g) Group (Name). Group names were simply the first eight
letters of the Greek alphabet: Alpha, Beta, Gamma, Delta, Epsilon, Zeta,
Eta, Theta. Subgroup names were surnames of famous chemists, past and present,
and were generated to strike up interest in chemistry beyond what a generic
subgroup number would. The actual spreadsheet used to organize the groups
and subgroups (students’ names omitted) is provided.
The Main forum was locked to student
postings and hosted announcements from the facilitator for students to
see. The Notes forum was locked to student postings and contained reminders
about
the appropriate timeline for subgroup progress called “Milestones.” The
Instructor’s Office forum was accessible at the group level and was
the forum where students can ask questions about the assignment or about
anything pertaining to the course. The Expresso Bar forum was accessible
at the group level and was a social forum where students were welcomed to
talk about anything other than organic chemistry. The Orientation forum
was accessible at the level of the individual subgroup. This is the forum
that the 5-6 members would post their “8 Nouns” and introduce
themselves to one another. The Subgroup forum is where the members of
the subgroup would find their assigned question to answer, discuss potential
answers to that question, and come to a consensus answer for the subgroup.
Finally, the Group forum is where each subgroup posted its final answer.
The Group forums remained locked until 48 hours before the end of the
assignment
to maintain an orderly forum.
Each of the 14 subgroups within
a particular group had a different question to answer, so it was not
possible for a subgroup’s answer for a question to be useful to another late
subgroup within the same group. Yet, the students within a group can see
all 14 questions (and answers for them) for the first time at the end of
the assignment and benefit from the work and knowledge gained from up to
84 students. After the assignment concluded, a WebCT Assignment Answers
forum was created to display all of the subgroups’ answers to a particular
question in an organized fashion to all students. Comments from the facilitator
at the end of this forum guided the students to which answers were the
most correct amongst the 8 subgroups who answered a particular question.
The 14 questions used in the WebCT
Assignment were created with several criteria in mind. The first criterion
was that the question had to contain sufficient difficulty such that
it required the work of 5-6 students to achieve a correct or adequate answer.
Topics that had not yet been covered in lecture but were available for
investigation
in the textbook were excellent for this. The second criterion was that
the question needed to be phrased such that an objective answer could be provided
for easy grading for the WebCT Assignment and on the subsequent exam.
The
third criterion was that the questions needed to broach relevant topics
that would be discussed throughout the semester. The final criterion
was that students should be able to answer the questions in a text-only format.
No attachments, links, graphics, or files were allowed as part of the
answers,
because it was felt the answer should be accessible to all students in
the class without the need for extra programs, plug-ins, or other external
software.
Sample questions: “Predict the position of the substituents (axial/equatorial)
in the lowest energy conformation for (1R, 3S, 4R, 6S)-4,6-Dimethylcyclohexan-1,3-diol.
Rationalize your answer.” and “List and describe the similarities
and differences between a carbocation, a carbanion, and a radical.” The
first question required the students to build the model of the molecule
with the correct stereochemistry correctly and then challenged the students
to express what they see in a textual form on the discussion board. The
second question required students to look up definitions and trends for
the three species and then discuss these trends with their colleagues
until they come to a consensus agreement.
The discussion board forums mentioned
previously were created on WebCT and totaled 251 for the assignment (Main,
Notes, 8 Instructor’s Office, 8 Expresso Bar, 112 Orientation, 112
Subgroup, 8 Group, and a compiled answers forum). About 3 hours time were
needed to create all of these forums, but roughly 40 hours of dedicated
work were required to establish the roster of members of each of the private
forums. Posting “Purpose of the Forum,” “Tyson’s
8 Nouns,” and “How to Post Your Answer” messages in each
discussion board forum to orient the students to using each forum properly
required about 5 hours work. Finally, another 4 hours were needed to
write and construct the online Pre-Assignment and Post-Assignment surveys.
The principal consumption of time
in preparing the assignment is maintaining and monitoring the most updated
roster of students throughout the assignment. Assigning and establishing
access for new students while closely monitoring those students who drop
the course before the assignment starts is the most difficult aspect of
executing the assignment properly. Failing to do so affords the risk of
creating subgroups that begin with less than the optimum number of expected
members and complicates the situation for students who are just beginning
to get oriented with their team members. Dynamic rosters are common with
large classes at the beginning of the semester. However, it was a variable
that had to be negotiated if the assignment was to serve its stated purpose
of training students with the technology and increasing student confidence
of discussion board use before the first course examination
Facilitation
of the Assignment:
Unless students possess significant
experience in online group work, instructor facilitation[4,7] can be crucial
to the success for students who are inexperienced with group work performed
through an online medium. Facilitating small classes with relatively few
discussion boards to monitor can be time-consuming.[4] One of the challenges
of this work was to see if one person could serve as an effective facilitator
for a larger class.
With 200+ discussion boards to
monitor simultaneously, it was felt that certain concessions had to be made
about facilitation levels that one person could offer over the course of
two weeks. Additional help in maintaining order during the course of the
assignment was provided by the graduate and undergraduate teaching assistants
of the course to ensure that the main facilitator was always informed of
any problems in communication between students that could lead to flashpoint.
Their primary mission was to monitor subgroup discussions on a daily basis
and look for Netiquette violations or any other signs of trouble in communication
amongst students at either the subgroup or group level. Facilitation of
discussion was welcome, but not necessary. Each of the two graduate teaching
assistants was given four groups (56 subgroups) to monitor, while each of
the four undergraduate assistants was given two groups (28 subgroups) to
monitor. While a couple of assistants posted an occasional comment, most
served as lurkers. With help in place to minimize the chance for missing
out of control situations, the facilitator concentrated on promoting discussion
amongst students.
Before the assignment began, initial “Purpose
of the Forum” messages from the facilitator in each forum were necessary
to orient the students to what each forum was for and how it was used for
the assignment. In addition, the Subgroup forum’s “Purpose of
the Forum” message contained the question that the subgroup needed
to answer, as well as links to 3 online references on how to be successful
in completing online group work.[8] In the Orientation forum, an additional
post of the facilitator’s own “8 Nouns” offered the students
an example of how it was to be accomplished. Finally, an additional “How
to Post Your Answer” post in the Group forum detailed the exact format
that each subgroup’s final answer was to follow.
After the assignment began, the
facilitator’s role varied from reviewing and responding to students’ 8
Nouns to answering student questions about how to begin the assignment.
The most time-consuming role was helping students figure out how to post
on a WebCT discussion board. Between answering questions in the Instructor’s
Office and pointing them to the hadout that explained how to post, most
students figured out how to use the boards by the end of the first week.
Students were also encouraged to contact the facilitator by private e-mail,
if there were any problems or if the student was embarrassed in any way.
It became apparent quickly that the time to read and respond to each student’s
8 Nouns (~5 mins. each) simply did not exist. The facilitator was reduced
to checking that the students had done the 8 Nouns correctly, injecting
encouraging comments when difficulties arose. In addition, students who
asked questions in the Instructor’s Forum about how to solve the problem
were offered directions to investigate the answer, but were not given
any explicit or implicit hints about the answer to that problem.
At the half way point into the
assignment (one week), some subgroups had made progress toward completion
of the assignment, while others had simply not started. A posting was
made to all 112 subgroups about their level of activity with one of two titles
in the subject heading: Progress or Lack of Activity. Progress was broken
into two categories with corresponding messages: a) every subgroup member
had posted their 8 Nouns activity and a timeline had been established
for
completing the assignment by those members, or b) some subgroups members
had not posted their 8 Nouns but the participating members established
a timeline for completing the assignment. (Dropped students were not factored
in at this point in the assignment because it was too time consuming
to
track them all.) Lack of Activity posts were given to all subgroups who
had not established a timeline for completion, regardless of 8 Nouns
postings. Roughly 10 hours of facilitator time was needed to read each subgroup’s
postings and provide the appropriate progress report message.
Subgroups that had only one person
checking in at the halfway point received significant attention from
the facilitator. This came in the form of private encouragement e-mails and
postings in the subgroup’s forum that only the students who have participated
with 72 hours to go may post on behalf of the subgroup. Those who had not
offered any substantive help by that time would be considered delinquent
and may lose points for the whole assignment. The warnings motivated most
of those who had lingered up to that point to begin. However, in two subgroups, “subgroup
leader” status was awarded to the person or 2 persons who had participated
by that deadline and were given the power to post on behalf of the participating
members of the subgroup. They proceeded to do so accordingly, which was
empowering for them despite the loss of the online group work experience
they sought.
With 48 hours remaining, the Group
forums were unlocked to allow subgroups to post their final answer. Credit
for the assignment hinged on co-authorship on the subgroup’s final
posting, which came only if substantive posts were offered before the final
posting. Therefore, a subgroup’s Orientation and Subgroup forums were
locked immediately after the final answer was posted in the Group forum
to prevent lingering students from posting answers after the contributing
members had completed discussion. At that point, the class was reminded
to complete the Post-Assignment survey after posting the final answer.
Subgroups were given an additional 24 hours beyond the full credit deadline
to post
their answers; only 2 subgroups were late with their posts. All forums
were locked down after the 15th day expired.
For 14 straight days, an average
of 8 hours a day was spent monitoring, reading, and responding to students.
The hours increased around the one week mark when progress posts were made
and at the end of the assignment when forums needed to be locked down after
final subgroup answers were posted.
Grading
the Assignment:
It was foreseen that keeping
track of which students were contributing to their subgroup’s efforts
and which were getting by on the work of others would become fairly subjective
and difficult after carefully reading the discussions of 20 subgroups,
much less 112. So, the creation of an objective rubric was
critical to ensure fair grading for all 600+ students. For parity, all grading
was done by only the assignment facilitator.
The Pre-Assignment and Post-Assignment
surveys were critical to learning about the value of the assignment from
the students’ perspective; therefore, 10% of the value of the assignment
was awarded for the completion of each survey by the deadline. The Pre-Assignment
survey was accessible during the first week of the assignment, while the
Post-Assignment survey was accessible during the last 3 days of the assignment
and for 3 days after the end of the assignment. Surveys on WebCT do not
track student responses with the student who made them; instructors only
receive a summary of answers and are therefore anonymous surveys. This aspect
was important to receiving frank assessments about the assignment. Responses
from the three sections (Section A, B, and B1-Distance) were lumped together
for the Pre-Assignment survey, but kept separate in the Post-Assignment
survey to gain insight about student perceptions from the different sections.
The Post-Assignment surveys were labeled with their section (i.e., WebCT
Post-Assignment Survey – Section B1-Dist.) to the students and only
accessible to students registered in that particular section. However,
the surveys were identical in every way for all three sections.
The 8 Nouns activity was itself
worth no points, but was required to gain any points from the assignment.
It was felt that blurting out answers to a group of people one doesn’t
know without introducing oneself was rude, and carried the potential to
harm or disrupt group synergy among students who are inexperienced with
online group work. In addition, unprofessional behavior or language (Netiquette
Violations) or failure to participate toward the subgroup’s efforts
were considered “superceding criteria” and constituted a loss
of all credit for the assignment.
Individual points were awarded
(up to 30% of the value of the assignment) based on postings that “significantly
contributed” to the subgroup’s final posting. Three “significant” posts
were required for full individual points. It was announced in lecture multiple
times that a) agreeing to a timeline, b) proposing an answer to the question,
c) discussing others’ answers to the same question, and d) deciding
on a consensus answer and who should post it constitutes four significant
contributions to the subgroup and only three were needed for the points.
When beginning to grade the assignment, first contact postings in the Subgroup
forums varied from a strong effort answer to a basic greeting. Some students
who checked in late simply stated that what someone else answered was good
enough, without offering anything new. To standardize the grading while
rewarding those who completed the assignment in good faith, the following
standards were used for credit. The first 10% was given for any posting
of any kind before the subgroup’s final answer and would constitute
co-authorship in the final answer. The second 10% must contain 2 full sentences
of material relevant to answering the subgroup’s question. This could
take the form of offering one’s own individual answer, or a good faith
promotion of discussion of another’s answer. The third 10% could be
any comment that logically follows the progression of the subgroup’s
final answer. The post constituting the second 10% was required in order
to be considered for the third 10%. If the student’s very first posting
in the Subgroup forum met the criteria for the second 10%, then the next
two postings that met the criteria for the third 10% were considered
for credit.
A distinction was made between
points for an individual’s effort and points for the subgroup’s
collective efforts. In haste, a subgroup’s team points were called “Group” points,
despite the use of “Group” earlier with a definition of 14 subgroups.
Fortunately, no students indicated confusion with the syntax and the change
of name to “Subgroup’s Points” will be made to the title
of team points when future iterations of this project develop. Presentation
(30%) and promptness of answer submission (20%) were considered for the
team points.
It became apparent that some subgroup
answers were thorough to respectability at the graduate level, while other
answers were consistent with what would be expected from a team of 5-6 undergraduates
engaging organic chemistry for the first time. Correctness was asked for
in the rubric, but not enforced
as long as the answer reflected a good faith team effort toward answering
the question and followed the format asked for in the “How to Post
Your Answer” message in the Group forum. Given the open-endedness
of some of the questions,
a detailed critique of each answer would have required a concentrated
effort on the instructor’s part to enforce – this was judged
as too time consuming.
The main requirement that was
emphasized multiple times is that the final answer of the subgroup must
be justified by the evolution of subgroup discussion posts online. This
was to prevent arranged face-to-face meetings as the primary source of
collaboration for the subgroup’s final answer. The rubric had
to reflect this notion, but in the end, every subgroup obeyed the spirit
of the assignment. Promptness of posting the final answer was enforced;
only two subgroups submitted their answer after the full credit deadline
and received half credit (10% out of 20% possible). In essence, participating
members of all subgroups but two received full credit for the “group” points.
This kept scores fairly high for those who participated and rewarded
the students for their efforts.
Assigning points for completing
the surveys, posting the 8 Nouns, and posting the final answer was relatively
automated and simple. In short, the students either completed that part
of the assignment or they didn’t, with the rubric establishing
credit in each case. Grading all of these aspects was completed within
5 hours. However, grading the individual points was painstakingly slow because
of the need to carefully read and follow the flow of every discussion,
judging
each member’s efforts and intentions along the way. The average time
it took to carefully read each subgroup’s discussion forum was 20
minutes. Some took longer than the average, only a few took less. A reasonable
estimate of time taken to complete this portion of the grading was about
45 hours (between 2-3 subgroups per hour). This intensive phase of the
grading had to be spread over the course of two weeks to accomplish, given
other
professional duties.
Results:
Statistics, Scores, Participation, and Discussion Postings
An immense amount
of data was collected throughout the course of the assignment, as well as
from the surveys. While future publications regarding this work will eventually
disclose a more thorough presentation of the data collected, a discussion
of the key highlights is afforded here.
Table 1 outlines the number of
students that participated in the WebCT Assignment and its surveys at various
times in the semester. 669 students were involved in the assignment at one
point; 3 of which joined the class after it began but before the first week
ended. A high number of WebCT-delivered Pre-Assignment surveys (585, 87.4%)
and Post-Assignment surveys (609, 91.0%) were collected from the students
which reflected the fact that credit was assigned to the completion of the
surveys. However, optional paper surveys given at the midterm 3 weeks after
the end of the WebCT Assignment (302, 45.1%) and at the end of the semester
(265, 39.6%) reflected normal lecture attendance, but still provided enough
data to analyze student satisfaction long after the assignment had concluded.
Table 2 shows the scores that
students earned for the assignment, which was worth 10 points in the
course (out of 1000). Student scores of 6-10 out of 10 are earned by participating
in their subgroup’s efforts toward answering the question. Students
who earned 0/10 did so by meeting one of the three superceding criteria
in the rubric (vide supra). Students found guilty of superceding criteria
were given an option to consult the facilitator after the assignment about
their violation and earned individual points were returned, though group
points were not restored. Those under this category had scores ranging from
1-5 out of 10 points. Counting all students who started the assignment,
the average was 8.5/10. However, when discounting the scores of those who
dropped out the course before the end of the assignment, the average jumps
to almost 8.9/10. Furthermore, when considering only those students who
completed the assignment and participated (discounting drops and those who
didn’t help their respective subgroups at all), the average jumps
to 9.3/10. When taking into account those who finished the assignment but
dropped the course later in the semester, the averages didn’t change
much whether they participated or not, demonstrating that WebCT Assignment
score had absolutely no correlation with dropping the course at a later
date.
Though the score distribution
data is not shown here in its entirety, 62.0% of the students starting the
assignment earned perfect 10/10 scores and 88.9% of the students earned
scores of 6 or higher out of 10. When disregarding those who dropped before
finishing or those who did not help their subgroup, the percentage of students
earning 10/10 jumps to 67.9% and the percentage of those earning 6 or higher
raises to 97.4%.
Out of 669 students who began
the assignment but earned no credit for it, 26 students (3.9%) dropped
the course before the end of the assignment. An additional 14 students who
finished
the assignment dropped before the end of the semester. 32 students (4.5%)
did not contribute toward their subgroup’s efforts. 34 students (5.1%)
did not post their 8 Nouns activity. 3 students (0.4%) committed a Netiquette
Violation, while another 6 students (0.9%) received private e-mail warnings
that their language approached the standard for a Netiquette Violation.
Netiquette Violations also led to revoking the student’s discussion
board privileges in the course for the remainder of the semester.
Progress was also tracked at the
Subgroup level. One week into the two-week assignment, each subgroup
was evaluated for progress toward completing the assignment and given either
a “Progress” or “Lack of Activity” post with encouragement
to press ahead to the finish. The basis for which message was given was
discussed earlier (see Facilitation of the Assignment). 22 subgroups
(19.6%) had made progress with all members having introduced themselves
via 8 Nouns.
37 subgroups (33.0%) had made progress in spite of not having all subgroup
members introduced or participating at the halfway point. In total, 52.7%
of the subgroups made progress toward completing the assignment according
to the timeline suggested in the WebCT Assignment handout. 53 subgroups
(47.3%) had not established an agreed upon timeline for completion of
the assignment. However, 110 out of the 112 subgroups (98.2%) had submitted
their final answers by the deadline, with 2 posting their answers during
the 24 hour grace period after the deadline.
Though the assignment was designed
to make the subgroups as even as possible, students who dropped the class
or chose not to participate in their subgroup’s efforts inevitably
reduced the effective number of members of their respective subgroups.
Table 3 displays the number of subgroups that had diminishing numbers of
members
after the start of the assignment. Overall, the results do show a promising
number of subgroups that had optimum participation.
At the onset, 109 subgroups had
6 members while 3 subgroups started with 5 members. Fortunately, the
5-membered subgroups did not suffer any reductions in members due to drops
or non-participation
throughout the course of the assignment. However, the 6-membered subgroups
did not fare nearly as well. 24 of the 6-membered subgroups started with
at least 1 student who dropped the course during the course of the assignment.
22 of the 6-membered subgroups began with at least 1 student who stayed
in the course but did not participate in their subgroup’s efforts
toward answering the assigned question. Indeed, when both sets of absent
members are factored in, some subgroups suffered significant losses to their
team rosters. One subgroup lost 4 members while two subgroups lost 3 members.
The effective 2-membered subgroup and one of the 3-membered subgroups had “subgroup
leaders” appointed by the facilitator (see Facilitation) as a way
to let the participating members know they had the attention of the instructor
and could proceed on their own. Overall, 72 subgroups (64.3%) operated
with full participation of all its original members. This is significant
as it
shows that a minority of subgroups will likely lose members from a large
class design and that only a handful of subgroups are likely to lose
significant numbers of members to the point of requiring special attention
from the
facilitator.
Finally, the number of discussion
board postings by everyone involved in the assignment was simply astonishing,
given that all of these posts occurred over a span of just two weeks.
Table 4 displays the total number, the average, high, low, and standard deviation
of posts in each type of forum. These include postings from students,
the
facilitator, and occasional comments from teaching assistants. 5672 posts
total were made as part of the assignment. Only 499 of those (8.8%) were
initial messages from the facilitator on usage of the forums and completing
the assignment, including: Purpose of the Forum, Tyson’s 8 Nouns,
mid-assignment subgroup progress post, How to Post Your Answer, announcements,
and the compiled answers in the WebCT Assignment Answers forum. As a comparison,
in the 13 weeks that followed, only 1937 posts were generated on post-WebCT
Assignment Instructor’s Office and Expresso Bar forums during the
remainder of the semester.
The Main and Notes forums were
used only for minor announcements over the course of the assignment.
The Instructor’s Office forum averaged 51 posts per group and contained
as roughly as many questions about the course in general as about the WebCT
Assignment. The Expresso Bar ranged from extremely active (202 posts) to
essentially non-active (39 posts). A fair amount of these posts were simple
greetings (i.e., Hello.) from students who had little experience. Most of
the activity was led by a relative few students who appeared to be discussion
board savvy. The 8 Nouns activity in the Orientation forum led to friendly
conversation in some subgroups, while a fair majority of others had little
to no response to their colleagues’ 8 Nouns. Reasons for this are
speculated on based on survey data presented later. The Group forums possessed
only the “Purpose of the Forum” post, “How to Post Your
Answer” post, and the answers from the individual subgroups, except
for two subgroups who posted additional modifications to their original
answers.
As expected, the Subgroup forums,
where the subgroups posted their discussions with one another, are where
the bulk of discussion board activity lied in the assignment. 3125 posts
(55.1% of all posts, 3349 minus 224 “Purpose” and “progress” posts)
were generated toward the effort of advancing subgroup answers over the
course of 2 weeks. The subgroups averaged 30 posts per subgroup. Most
of the subgroups had one or two members who provided a thorough answer with
subsequent critique and modification by the other members. Roughly a
quarter
of the subgroups had each member provide his/her own answer and then
discussed the similarities and differences. Though most subgroups used an
agreed timeline
for asynchronous postings, two subgroups brilliantly devised a plan to
begin posting to their subgroup board asynchronously at a specific time,
essentially
transforming an asynchronous discussion board into a modified synchronous
chat room. Against expectations, no subgroups were found to have any
adversarial discussions (nor were any reported) or problems with conflict
resolution.
Despite the large and diverse population of students in this course possessing
a variable range of computer skills and group work experience, perhaps
the lack of any serious (or even minor) subgroup conflicts was one of the
most
pleasant outcomes from this project.
Results
Student Surveys
Can large classes be trained
to use WebCT in online discussion board work efficiently and successfully?
Can student confidence in discussion board use increase as a function
of online group work?
The answer to the first question is an unequivocal yes. The evidence shows
that the overall student participation rate in the assignment was very high
(88.9% of all students), despite the fact that the assignment was worth only
1% of the overall course grade. 98.2% of the subgroups submitted their answers
by the deadline and the remaining subgroups posted within 24 hours of the deadline.
Efficiency and success was reflected in student opinions as well. 58.6% of
all students felt that expectations for the assignment were clear; another
35.8% felt expectations were clear on some points but not so clear on others.
The total amount of time students took to complete the assignment (0 to 7 hours,
94.1%) matched what they expected to spend on the assignment (90.1%). Of those
students, a majority of them spent between 2 to 4 hours on the assignment (42.1%
expected, 46.0% actual). The students felt overall that the question difficulty
was either very appropriate, somewhat simple, or too easy for a team of 5-6
students to solve (78.0%, 47.3% - very appropriate). Only 11.7% felt the questions
were somewhat difficult or too difficult for a team of 5-6 students.
Survey results also showed that
student confidence in discussion board use did increase as a function of
the WebCT Assignment. Before the assignment, only 45.1% of students stated
they had some experience with discussion boards prior to taking the course.
However, after the assignment concluded, 50.9% of students stated their
confidence in communicating with discussion boards increased as a function
of the WebCT Assignment. 13.8% stated that their confidence has increased
slightly as a function of the assignment, but still have some problems.
17.1% claimed their confidence did not increase from the assignment because
they already knew how to use discussion boards. Only 7.2% stated their confidence
did not go up at all and 11.0% had no opinion one way or the other. Going
from 45.1% of students who had any experience whatsoever to 81.8% that could
use discussion boards with at least some degree of confidence is a very
significant result.
Furthermore, 61.3% of students
felt the assignment was a somewhat positive, positive, or very positive
experience. 11.8% had no feelings one way or the other. On a continuous
scale from 0 to 10 (0 = not beneficial at all; 10 = very beneficial),
the overall class average, mean, median, and mode were all 5.0/10 with S.D.
= 2.7. The median (6.0/10) and mode (8.0/10) were slightly higher for
Section
B1-Distance, but the section average was approximately the same as the
class average for all sections. Out of a random-ordered list of 10 ways this
assignment
could be beneficial or not (check as many as applied), a majority of
students expressed they benefited most by learning how to use discussion boards
(50.2%),
learning how to use one of the features of WebCT (53.2%), and it gave
them the opportunity to experience online group work (54.4%). Table 5 presents
this data with respect to the class as well as by section. Coincidently,
the fourth-largest voted benefit was that they found the 8 Nouns activity
to be a nice icebreaker in meeting new people (24.1%), which is consistent
with the approximate number of subgroups witnessed to have significant
social
discussions in the Orientation forum about their 8 Nouns. Students also
found the Instructor’s Office forum useful and helpful (51.6%); most
of the remaining students either were indifferent about the Instructor’s
Office forum’s utility and helpfulness (30.0%) or did not know whether
it was helpful or not (14.6%).
Finally, the WebCT Assignment
didn’t appear to change students’ feelings about group work
overall. Before the assignment, 71.6% of the students either liked group
work (25.1%) or believed it was OK and saw the need for it in the real world
(46.5%), while 14.2% of the students either didn’t like group work
because they felt that they carried the load for others (13.5%) or saw
no value in group work whatsoever (0.7%). 9.6% were completely indifferent.
When asked if their feelings about group work/collaborative work changed
after the WebCT Assignment, 41.0% had no opinion about it one way or
the
other. 15.4% felt the assignment positively impacted their feelings about
group work, while 17.1% always liked group work irrespective of the WebCT
Assignment. Meanwhile, 11.5% felt the assignment negatively impacted
their feelings about group work in general and 14.8% always disliked group
work
and the assignment did nothing to change that opinion. While a slightly
higher percentage of students either gained or maintained perceptions
about group work than those who possessed negative feelings, the overall
change
in perception in the course is minimal.
Can a single primary instructor
effectively facilitate online discussion board group work for a large
class?
The answer here lies in how much time one can dedicate to designing, constructing,
monitoring, facilitating, and grading discussion boards around other duties.
In the case of this course, only one instructor participated in the WebCT Assignment.
The total number of hours of instructor labor associated with this one assignment
(just the hours mentioned in this article) is 214 hours, not counting the time
taken to accumulate and analyze score and survey data. However, this was an
assignment that had 669 students at the beginning. This same course offered
in the alternate semester usually has enrollments in the 300-350 range. This
project repeated again under this new scenario could prove to be much less
demanding. The overall scores and percent completion rates at the individual
and subgroup level are promising indicators of facilitator influence.
Indeed, the students noticed the
facilitator’s efforts. When asked the question “In your opinion,
how would you describe the facilitator’s (Mr. Miller) online presence
during this assignment? (Please check all that apply)”, students felt
the facilitator was friendly (99.2%), helpful (97.2%), and available for
help (99.1%), as opposed to unfriendly, not helpful, and unavailable for
help, respectively. Students also gauged the facilitator’s overall
online presence during the assignment. 70.8% of students described the facilitator
as “everpresent (I could sense Mr. Miller was always there watching)”,
while 24.9% described the facilitator as “sometimes there (I could
sense that Mr. Miller was there for us sometimes, but not others)”.
Despite the facilitator postings, constant monitoring, and announcements
that only one instructor was facilitating the assignment, 4.3% described
the facilitator as “never there (I didn’t sense any evidence
that Mr. Miller was involved in this assignment)”.
Similarly, the teaching assistants’ (graduate
and undergraduate) optional role in facilitating the assignment was also
highlighted in student responses. When asked the question “How would
you describe the TA [graduate teaching assistant] or SI [Supplemental Instruction – undergraduate
teaching assistants] assistant facilitators presence during this assignment
(Please check all that apply)” and having the same options listed
for the facilitator question, they received quality marks for being friendly
(100%), helpful (75.4%), and available for help (83.2%). However, 73.2%
of the students described the assistants as “never there”, 22.7%
described them as “sometimes there”, with only 4.1% describing
them as “everpresent”. These numbers reflect the fact that the
assistant’s primary role was to watch for student arguments, unprofessional
behavior, and violations and that it was not mandatory to facilitate
by posting.
To summarize, a single person
can successfully facilitate large classes of discussion board-based group
work. However, the time costs associated with proper monitoring and facilitation
can be very large. When this project is repeated in a semester with a lesser
enrollment, more can be said about the student/time ratio and whether this
type of work on this scale is practical.
Will distance education students
believe they have benefited from online group work experience as much
or more than their face-to-face student peers?
The surveys indicate that a higher percentage of distance students may have
benefited from the assignment as compared to the class overall.
It was reported previously that
the Section B1-Dist. students had a slightly higher median (6.0/10) and
mode (8.0/10) with similar class average (5.0/10) on the 0 to 10 scale describing
how beneficial the assignment was. So, the activity overall was not found
to be any more beneficial to the video section students than to the class
overall. However, Table 5 shows a significantly higher percentage of Section
B1 students reported benefit in all of the major categories mentioned previously
by at least 8% or more over the class or the face-to-face sections: learning
to use discussion boards, using a feature of WebCT, and opportunity to experience
online group work. In addition, fewer distance students saw no benefit to
the assignment at all immediately after the assignment ended. Student support
for the assignment would wane over time though, as will be discussed later.
Furthermore, a higher percentage
of distance students reported an increased confidence in using the discussion
boards as a result of the WebCT Assignment as opposed to the class overall
or the sections of face-to-face students. Table 6 shows that 61.4% of the
distance students felt the assignment increased their confidence in using
discussion boards, as opposed to 50.9% of the whole class, 52.1% of Section
A, and 46.7% of Section B. Another interesting statistic is that only 10.2%
of the distance students had confidence in using the discussion boards such
that the assignment did not affect it, as opposed to 17.1% for the class,
17.5% for Section A, and 18.8% for Section B. While there are more students
enrolled in Section A and B which can affect the class average, the numbers
show that far fewer students with significant discussion board experience
were in the distance section and needed the experience gained from the assignment.
It is possible that the distance
students may have had a perceived need to gain more out of the assignment
(given that the assignment was designed to benefit the distance students
the most) and therefore reflected this sentiment in the surveys. Actual
benefit was not measured in other ways (more detailed pre-assignment surveys
about the above topics might gain more insight on the matter).
What will be the overall level
of student satisfaction with the exercise?
It has been shown previously that an intermittent use of technology
in the classroom without requiring its use in further work is perceived
negatively
by students over the course of time.9 Since there were no other group work
assignments in the course and since the use of discussion boards were not required
after the WebCT Assignment, it was expected that student support and perceived
usefulness for the assignment would diminish over time. The question was whether
the assignment’s impact on student confidence in using discussion boards
would translate to moderate support for its implementation as a standard tool
for large classes. Another question was whether the distance students would
support the assignment more overall than the rest of the class.
Table 7 displays the survey data
associated with student support for the WebCT Assignment just as the
assignment ended, at the midterm of the course 3 weeks later, and at the end
of the
semester. The numbers clearly show a sharp decline in perceived value
of the assignment over time. Just after the assignment ended, the “beneficial” value
of the assignment was 5.0 out of a 0 to 10 scale, including a slight increase
in median and mode for the distance Section B1 students. Three weeks later,
student support dropped as over half the class perceived the assignment
as either “mildly useful” or a “waste of time.”
However, examining the midterm
survey numbers carefully affords an interesting trend. Both sections
of students that did not have the WebCT Assignment facilitator as their face-to-face
classroom instructor gave a higher percentage of high-ratings for the
assignment
than those students who had the facilitator as their primary lecture
instructor. Section B lecture was taught face-to-face by the WebCT Assignment
facilitator
and had the highest enrollment of all sections. Only 27.0% of the students
in Section B gave the WebCT Assignment a “Useful” or higher
rating on the midterm survey. 37.8% of the distance students in Section
B1 (which had the WebCT Assignment facilitator as their video instructor)
and 40% of the students in Section A (who had an instructor different from
the WebCT Assignment facilitator) gave a “Useful” or higher
rating on the midterm survey. The numbers may reflect that students who
have an instructor different from the person running the online assignment
gain at least an equivalent perception of assignment usefulness as distance
education students. Furthermore, the numbers may reflect that students
who have increased face-to-face contact with the person running an online
group
assignment tend to perceive discussion board-based interactions as less
necessary to classroom success.
The end of the semester survey
results reflect an overall further erosion of student support with time
and similar trends in support within sections as seen in the midterm
survey. Fewer students reflected on the WebCT Assignment as a “Waste of Time” in
both Sections A and B1, with B1 as the lowest. Interestingly, the percentage
of students who rated the assignment “Useful” and “Mildly
Useful” bears further analysis. In the case of Section A students
who had a different instructor from the facilitator, more students rated
the assignment as “Useful” than “Mildly Useful”,
unlike for Sections B and B1. The distance students of B1 had the highest
percentage of “Mildly Useful” ratings than any section in the
class, but it was higher than the percentage of students that rated “Waste
of Time” which did not happen for any other section. After the 2nd
midterm exam roughly 70% of the way into the course, the distance students
voted to have face-to-face instruction from the Section A instructor
(with occasional TA guest lecturers) over a consistent single video lecturer.
It is felt that the Section B1 student support for the assignment at
the
end of the semester may have waned further than what was observed for
Section A as a function of having face-to-face instruction for the first
time during
the semester.
In conclusion, had discussion
board-based group work been designed into the course beyond a one-time assignment,
student support for it might have been sustained throughout the course.
As is, it was shown that discussion board training for the purposes of increased
instructor access for distance students in large classes alone was not sufficient
to maintain student support for the assignment.
Conclusion:
Is discussion
board-based facilitated small group work practical for instructors of
courses with large enrollments?
The answer to this
question is perhaps the most interesting part of the story and is the impetus
for this research.
Unfortunately, the answer to the
underlined question above can’t yet be given, simply because there
is but one data point: 214 facilitator hours yields one 2-week discussion
board assignment for 669 students (not counting midterm and end of semester
survey data workup). Fortunately, this experiment is being repeated this
Spring Semester 2004 with a student enrollment of approximately 330+
students and may be repeated in Summer Semester with fewer students still.
A facilitator
time/student enrollment ratio can then be established that would give
significant insight as to how practical single facilitator small group discussion
board-based
work is in large enrollment classes. That ratio would give course designers
an idea of what it would take to implement, sustain, and support this
type of student work throughout a course curriculum for classes of all sizes.
The good news is that it can be
done by a single facilitator and it can be done better. As mentioned
earlier, not all of the data collected in this experiment was reported here.
One,
there were many other questions in the surveys that gave useful information
about various details and elements of the assignment that could have
been asked more precisely. Two, deadlines will be changed to other days of
the
week besides Fridays – this will hopefully increase student participation
in surveys and in the assignment in a more timely fashion. Three, the amount
of undergraduate teaching assistant support overall in the course changes
from semester to semester. There is the potential for increased student
perception in value for the assignment in semesters where teaching assistant
availability decreases while instructor availability remains the same. Four,
increased integration of discussion board-based group work (or discussion
board work in general) over the course of the semester (not just one time)
may improve student perceptions of the assignment’s value. Five, allowing
students to utilize graphics or attachments as part of their answers
may enhance the learning experience for those students who are visual learners.
Finally, perhaps Table 8 shows
one of the best reasons of all for doing this type of work. Of those
who used the discussion boards through the semester, 82.4% of the students
found
them at least “Mildly Useful” with 50.2% of those rating them “Useful” or
better. Furthermore, of the distance students, 97.2% found the discussion
boards at least “Mildly Useful” with 61.1% rating them “Useful” or
better. When one begins with the fact that only 45.1% of the entering
class had any discussion board experience at all and ends
with 81.8% of the students having at least some confidence in using discussion
boards, one underlying fact remains: if a student doesn’t leave an
organic chemistry course with any confidence in organic chemistry, at
least he or she can obtain some confidence in learning and using a technological
and educational tool that will likely be a vital part of his/her future.
Student learning of that kind is always worth the effort and time.
Acknowledgements:
The author would like to give
thanks to Professor Stanley Smith for his mentorship and advice. Sincere
thanks to Professor Jeff Moore for his efforts and collaboration in teaching
the course. Grateful thanks to Dr. Patricia Phillips-Batoma for her expertise
and help in the use of WebCT and in the maintenance of the course website
as a whole. Also, thanks to Dr. Iris Stovall and the Illinois Online Network
for the opportunity to share this research with the online educational world.
References:
1. a)
Schamber, L. Delivery Systems for Distance Education. ERIC Digests http://www.ericfacility.net/databases/ERIC_Digests/ed304111.html;
b) Oblinger, D.G. The Nature and Purpose of Distance Education. http://ts.mivu.org/default.asp?show=article&id=647
2. Wang, A.Y.; Newlin, M.H. Predictors of
Performance in the Virtual Classroom. T.H.E. Journal http://www.thejournal.com/magazine/vault/A4023.cfm
3. Renzi, S.; Klobas, J. Steps Toward Computer-Supported
Collaborative Learning for Large Classes. Educational Technology & Society
3 (3) 2000. http://ifets.ieee.org/periodical/vol_3_2000/d07.html
4. a) Funaro, G.M.; Montrell, F. Pedagogical Roles
and Implementation Guidelines for Online Communication Tools. ALN Magazine.
3 (2) 1999. http://www.aln.org/publications/magazine/v3n2/funaro.asp; b) Edelstein,
S.; Edwards, J. If You Build It, They Will Come: Building Learning Communities
Through Threaded Discussions. Online Journal of Distance Learning Administration
5 (1) 2002. http://www.westga.edu/~distance/ojdla/spring51/edelstein51.html;
c) Berge, Z.L. The Role of the Online Instructor/Facilitator. eModerators 2001
http://www.emoderators.com/moderators/teach_online.html; d) Palloff, R.M.;
Pratt, K. Effective Strategies for the Online Classroom. http://itlearningspace-scot.ac.uk/courses/keynotes/module1/main.cfm
5. Britt, K. Tool Talk. http://webct.com/service/viewcontentframe?contentID=2515824
6. Bonk, C.J.; et al. Advances in Pedagogy: Finding
the Instructor in Post-Secondary Online Learning. AERA 2000 http://www.indiana.edu/~bobweb/temp/online_ped.pdf
7. West, M.L.; Luetkehans, L. Ten Great Tips for
Facilitating Virtual Learning Teams. PSD Corps. http://www.psdcorp.com/dislearn.htm
8. a) http://www.nwlink.com/~donclark/leader/leadtem2.html;
b) http://www-honors.ucdavis.edu/vohs/sec04.html; c) http://www-honors.ucdavis.edu/vohs/sec04-1.html
9. a) Teaching Through Technology Guide: Getting
Started with Learning Technology. Teaching Support Services. http://www.tss.uoguelph.ca/TGuides/ltintrotips.html;
b) Smith, B.D.; Jacobs, D.C. TextRev: A Window into How General and Organic
Chemistry Students Use Textbook Resources. J. Chem. Ed. 2003 80 1 99-102.
|
