| 
Grading
the Assignment:
It was foreseen that keeping
track of which students were contributing to their subgroup’s efforts
and which were getting by on the work of others would become fairly subjective
and difficult after carefully reading the discussions of 20 subgroups,
much less 112. So, the creation of an objective rubric was
critical to ensure fair grading for all 600+ students. For parity, all
grading was done by only the assignment facilitator.
The Pre-Assignment and Post-Assignment
surveys were critical to learning about the value of the assignment from
the students’ perspective; therefore, 10% of the value of the assignment
was awarded for the completion of each survey by the deadline. The Pre-Assignment
survey was accessible during the first week of the assignment, while the
Post-Assignment survey was accessible during the last 3 days of the assignment
and for 3 days after the end of the assignment. Surveys on WebCT do not
track student responses with the student who made them; instructors only
receive a summary of answers and are therefore anonymous surveys. This
aspect was important to receiving frank assessments about the assignment.
Responses from the three sections (Section A, B, and B1-Distance) were
lumped together for the Pre-Assignment survey, but kept separate in the
Post-Assignment survey to gain insight about student perceptions from the
different sections. The Post-Assignment surveys were labeled with their
section (i.e., WebCT Post-Assignment Survey – Section B1-Dist.) to
the students and only accessible to students registered in that particular
section. However, the surveys were identical in every way for all three
sections.
The 8 Nouns activity was itself
worth no points, but was required to gain any points from the assignment.
It was felt that blurting out answers to a group of people one doesn’t
know without introducing oneself was rude, and carried the potential to
harm or disrupt group synergy among students who are inexperienced with
online group work. In addition, unprofessional behavior or language (Netiquette
Violations) or failure to participate toward the subgroup’s efforts
were considered “superceding criteria” and constituted a loss
of all credit for the assignment.
Individual points were awarded
(up to 30% of the value of the assignment) based on postings that “significantly
contributed” to the subgroup’s final posting. Three “significant” posts
were required for full individual points. It was announced in lecture multiple
times that a) agreeing to a timeline, b) proposing an answer to the question,
c) discussing others’ answers to the same question, and d) deciding
on a consensus answer and who should post it constitutes four significant
contributions to the subgroup and only three were needed for the points.
When beginning to grade the assignment, first contact postings in the Subgroup
forums varied from a strong effort answer to a basic greeting. Some students
who checked in late simply stated that what someone else answered was good
enough, without offering anything new. To standardize the grading while
rewarding those who completed the assignment in good faith, the following
standards were used for credit. The first 10% was given for any posting
of any kind before the subgroup’s final answer and would constitute
co-authorship in the final answer. The second 10% must contain 2 full sentences
of material relevant to answering the subgroup’s question. This could
take the form of offering one’s own individual answer, or a good
faith promotion of discussion of another’s answer. The third 10%
could be any comment that logically follows the progression of the subgroup’s
final answer. The post constituting the second 10% was required in order
to be considered for the third 10%. If the student’s very first posting
in the Subgroup forum met the criteria for the second 10%, then the next
two postings that met the criteria for the third 10% were considered for
credit.
A distinction was made between
points for an individual’s effort and points for the subgroup’s
collective efforts. In haste, a subgroup’s team points were called “Group” points,
despite the use of “Group” earlier with a definition of 14
subgroups. Fortunately, no students indicated confusion with the syntax
and the change of name to “Subgroup’s Points” will be
made to the title of team points when future iterations of this project
develop. Presentation (30%) and promptness of answer submission (20%) were
considered for the team points.
It became apparent that some
subgroup answers were thorough to respectability at the graduate level,
while other answers were consistent with what would be expected from a
team of 5-6 undergraduates engaging organic chemistry for the first time.
Correctness was asked for in the rubric,
but not enforced as long as the answer reflected a good faith team effort
toward answering the question and followed the format asked for in the “How
to Post Your Answer” message in the Group forum. Given the open-endedness
of some of the questions,
a detailed critique of each answer would have required a concentrated effort
on the instructor’s part to enforce – this was judged as too
time consuming.
The main requirement that was
emphasized multiple times is that the final answer of the subgroup must
be justified by the evolution of subgroup discussion posts online. This
was to prevent arranged face-to-face meetings as the primary source of
collaboration for the subgroup’s final answer. The rubric had
to reflect this notion, but in the end, every subgroup obeyed the spirit
of the assignment. Promptness of posting the final answer was enforced;
only two subgroups submitted their answer after the full credit deadline
and received half credit (10% out of 20% possible). In essence, participating
members of all subgroups but two received full credit for the “group” points.
This kept scores fairly high for those who participated and rewarded the
students for their efforts.
Assigning points for completing
the surveys, posting the 8 Nouns, and posting the final answer was relatively
automated and simple. In short, the students either completed that part
of the assignment or they didn’t, with the rubric establishing
credit in each case. Grading all of these aspects was completed within
5 hours. However, grading the individual points was painstakingly slow
because of the need to carefully read and follow the flow of every discussion,
judging each member’s efforts and intentions along the way. The average
time it took to carefully read each subgroup’s discussion forum was
20 minutes. Some took longer than the average, only a few took less. A
reasonable estimate of time taken to complete this portion of the grading
was about 45 hours (between 2-3 subgroups per hour). This intensive phase
of the grading had to be spread over the course of two weeks to accomplish,
given other professional duties
Background
Objectives of the Work
The WebCT Assignment
Experimental Design
Facilitation of the Assignment
Grading the Assignment
Results - Student Surveys
Acknowledgements and References
 Printer
Friendly Version
|
