SUBJECT: Teaching students to write scientific reports; teaching TAs to
grade reports
DATE: 5/94; 1/95
Hi all,
I'll start off by naming one (out of many) problems I'd like to see
discussed. In a large, multiple lab section course with many TAs, how do
you all get your TAs to grade reports the same way? I have tried using all
sorts of checklists that they can go through, but there is always something
left to subjective interpretation. One thing I tried this semester is to
have every major report graded by 2 TAs. I then can look at each TA's
comments, and usually end up assigning a grade that is a compromise of the
2 assistants. This seemed to work, but the turnaround time on reports was
very slow. (My situation may be a little different from others -- we are
an undergraduate college, so I have undergraduate TAs to help me with each
lab section.) I'd be interested in hearing people's experience in TA
training.
Mike O'Donnell
--------------------------------------------------------
Mike O'Donnell
michael.odonnell@mail.trincoll.edu
Here at Carolina we have a set of "Standards" of things that the
TAs are
allowed to award points for. The TAs spend a half day at the beginning
of the semester training on the standards. Then they swap reports for
grading and the students' 3 reports are all graded by a different TA.
This helps even out the effect of varied grading difficulty. It is an
aggravation, but by the end of the semester, the TAs agree that it is the
most fair way to operate. The undergraduates do not like having their
reports graded by different people (understandably), but we are persuaded
that it is the most fair procedure. I would like to see what others use
to grade by. Jean DeSaix, Univ of NC at Chapel Hill
We use a grading checklist for reports. Students have a copy of it in their
Writing Guide so they know how they'll be evaluated, and instructors are
given a version that assigns points to each item. Before the first report
is graded, the instructors grade a sample paper. Then they discuss their
grading in small groups at a prep session. When we reassemble as a whole
(20-some instructors, including faculty and TAs), each group reports
their consensus score and explains what they agreed and disagreed on. I
try to summarize and provide answers for common concerns. (Then
everyone goes and does what they darn well please. One faculty member
gave perfect or almost-perfect scores for a report.)
Novices find that it takes a long time to grade the first report, since
they
have to keep referring to the gradesheet, but it gets easier.
Reports are the major part of our students' grades in one course, and I'm
not satisfied with the results of our brief prep session exercise on this.
I'd
like to hear more about Jean DeSaix's training on standards.
And does anybody have any suggestions for handling the instructors whose
grades are out of line?
Jean Dickey
Clemson University
dickeyj@clemson.edu
It would be interesting to see the grading checklists that you provide to
your teaching staff. I've tried such a thing in the past. We have only
two lab reports each semester but I frequently change the lab topics on
which they write them. We have had a lot of trouble with copied papers
from fraternity or sorority files so I like to modify labs often. This has
made it very difficult to produce thorough grading checklists of what I
expect to find in each lab report.
As to handling out of line grades, this is a tremendous problem for all
of
us. It has to be handled during the semester - end of the semester is too
late. I ask my TAs to look through at least half of their papers before
they start grading, to get a feel for what a good paper is and what a bad
paper looks like. Then I ask them to try to adjust their grading to an
average (75-80%). After the first lab reports are graded, I look over the
averages, talk to the TAs with outlying averages, and ask them to adjust
their grading on the next reports to bring them more in line.
Obviously, this does not always work. Additionally, if a TA has
particularly good students, or better prepares his or her students to
produce better lab reports, these reports ultimately may receive the same
grade as a poorer paper from a student of another TA.
Kent
*******************************
Kent A. Vliet, Ph.D.
Coordinator of Laboratories
University of Florida
Division of Biological Sciences
Gainesville, Florida 32611
Office: (904) 392-1565
Fax: (904) 392-3704
In response to more info about training on standards: During our two or
three day TA training before the semester starts, we have a half day when
TAs grade sample reports. We go over standards (like your check list, I
expect) and then everyone grades a paper we have xeroxed. We then go
around the room and ask for grades assigned. Everyone is suprised at the
diversity of grades since we all "agreed" on the standards. We
continue
to grade more papers, discussing why people did or did not award points,
and we sometimes modify the standards. When we reach convergence (every
one within two points of one another on a 10 point scale) we quit. But
TAs then continue to monitor one another and discuss the standards
because other TAs grade the "home" TAs reports. The home TA is
in a
position to advocate for his or her own students and can go back to the
grading TA to negotiate points. The TAs are further motivated to reach
convergence by their knowledge that the same scale will the used for all
the sections and they don't want all the As going to someone else. This
seems to work really well to keep the really hard TAs from being really
hard and the really easy ones from being really easy. When I started it,
however, there was a storm that erupted. I spent many hours talking to
TAs and to students who believed that I was being wrong and cruel. It
helped some that the TAs knew that I almost never dictate and refuse to
modify a policy in the face of such criticism. The fact that I was so
determined and so unwilling to negotiate persuaded them that I felt very
strongly about the issue. After one semester, the TAs were converted
into believers. Jean DeSaix Univ of NC
Discussion 1/95
Greetings to everyone!
I have just gotten started working with our English dept. on ways to more
effectively introduce scientific writing to my freshmen students. The
issues we are trying to address are:
1. How to effectively communicate what scientific writing is to beginning
students (both content and style).
2. As a staff, how to prioritize what we are looking for in the writing
we
get back from students. For example, which is more important, the
student's ability to logically think through a scientific process or their
ability to put it on paper with the correct grammar and spelling?
3. Methodology to enable my multiple TA's to grade writing assignments
with a reasonable amount variability between sections.
My questions to all of you (especially those with freshmen) are:
1. What sorts of writing assignments do you give your students? Do you
expect them to write in journalistic style (intro, material/methods/
results/discussion)? Do students practice the "parts" before they
are
expected to write a whole scientific paper? Do some assignments offer more
informal writing?
2. What criteria do you use for grading writing assignments? How much
emphasis on style, grammar, and spelling?
3. If grading is split between multiple TA's, do you have any effective
methods for getting the grading to be consistent?
Thanks for your help.
Mary H. Gray D.V.M.
Department of Biological Sciences
Lilly Hall of Life Science
Purdue University
West Lafayette, IN 47907
(317)-494-8185
FAX (317)-494-0876
We have used one of our introductory lab courses as the site of a writing
intensive experience for a number of years now. Analogous to you working
with your English Dept., we have been working with people in our Dept. of
Communications. We worry about the students writing science and they worry
about the grammar and flow of the papers.
Specifically, we have our students write their lab reports in the format
of
journal articles. We use Jan Pechenik, A Short Guiede to Writing about
Biology, 2nd ed, Harper Collins College Publishers, 1993 (ISBN
0-673-52128-1) as a reference for the students. Actually, there is enough
other material in this little book that it is useful in our upper level
courses as well.
The biggest problem we have found is the lack of ability (or experience
perhaps) of our beginning students to critically write about the experiments
they have done. Between the use of Pechenik's book and research journal
articles, we show our students how an experiment should be described and
analyzed in a paper. It takes at least a semester, but progress is made.
Each paper written by students consists of a first draft and an optional
second draft. The grade weighting on the first draft on papers early in
the
course carries less weight than the second draft, the ratio changes as their
experience increases. We encourage our students to stop by so we can go
over
informal outlines of what they propose to put in the first draft. This
approach works very well and allows the student to really get an
understanding of the process of scientific writing.
The issue of possible grading inconsistencies between TAs has long concerned
me. I usually sit down with them and show them the worst and best papers
in
my lab sections. These have been critiqued with extensive comments by me,
and that seems to get everyone on the same wavelength with respect to what
is important in the grading of a paper.
George Edick
RPI - Dept. Biology
Troy, NY 12180
edickg@rpi.edu
Mary,
We ask our beginning students to produce 4 scientific papers during their
2
semester freshman biology course. We provide a very detailed rubric of
what is expected in the paper and the points assigned for each portion.
This works because the students are very aware of what is expected as are
the TAs. All of the multiple sections follow the same rubric (this can
have its own problems) so all are graded relatively similar. I would be
happy to send you a sample rubric if you wish.
V. Christine Mahaffey Minor (vcmahaff@iastate.edu)
Laboratory Coordinator
154 Bessey Hall
Iowa State University
Ames, IA 50011
Since there have been some questions, I thought I'd try to answer all at
once. Purdue's English department sponsors a writing lab that is manned
by
their graduate students and provided help for all sorts of writing to
anyone on campus. I started this collaboration by calling the writing lab
director to see if the lab could handle several hundred biology students
trooping over there with their assignments. They were willing to give it
a
try. In our first go-around, this is what we tried: Our students take a
field trip early in the semester to compare/contrast the plant and insect
diversity of a grazed versus an ungrazed field. I first have them do a
homework assignment requiring the necessary calculations for results and
also ask them to interpret these results. This gives them the first
feed-back from the TA's. After this comes a lab dealing with library usage
and construction of scientific papers. Our lab manual has a chapter on
scientific writing and we also utilize a book by Alan Gubanich called A
Student's Guide to Writing a Scientific Paper. How to Survive the
Laboratory Research Report, Kendall/Hunt Publishing. Then we ask the
students to go back to this same information and present it in the
scientific writing format. These assignments are then graded. Students
are given the option of rewriting the papers for a higher grade PROVIDED
they show evidence that they have been to the writing lab for assistance.
Slightly less than half of my students took advantage of the rewrite
opportunity. Later on in the semester the students do a 3-week project on
the Life History of Daphnia magna and are required to write a paper in
scientific format that is worth 100 points.
Concerning models for the students to look at: We go over the scientific
method during the first week of class. I then give them a homework
assignment requiring them to read one of several journal articles on
reserve in the library. They are asked to state the hypothesis being
tested, describe how the hypothesis was tested, what the conclusions were,
and if the original hypothesis was supported or refuted. It was
challenging to find enough articles that I thought freshmen could actually
understand. Much of the literature is written assuming a good grasp of the
terminology and techniques of the field by the reader. For our big Daphnia
project, we have put models of previous student papers on reserve. This
is
useful for modeling but then we run into plagiarism problems. Hmmm...solve
one problem, create another.
Hope to hear from others of you out there.
From: <mgray@bilbo.bio.purdue.edu> (Dr. Mary Gray)
Christine,
We ask our first year students to do one report each term and also
supply a detailed outline for the students as well as a the TAs. I
would be very interested in seeing your rubric since we still have
problems with getting consistency in marking between sections despite
the outline and a pre-marking meeting with the TAs!
When it comes to questions of grammar, sentence structure, etc., we
allot 1 mark out of 15. However, if the writing is particularly bad,
it is reflected in the other marks as well since you often can't
figure out what the student is trying to say so they lose marks on
content.
Lynn (Hauta) Ruxton
Laboratory Coordinator
Dept. of Biology, Lakehead University
Thunder Bay, Ontario, Canada. P7B 5E1
Regarding student lab submissions, the one thing i think we would all
agree on is that initially the reports are pretty awful. Initially the
students in the first year submitted short assignments (essentially report
sections) and one formal report. This did not work well; those teaching
second year complained about the students abilities. We then switched to
all submissions (experimental reports) being done in journal format. This
may seem a sudden shock to students coming from high school, but there
seems little point in teaching them different formats when journal format
is what will be required in all(?) of their science courses.
The first couple of reports are, usually, pitiful. But then they get
better once they have had a little practice. In order to alleviate the
pain students are to hand in the first lab as a practice lab. The lab
manual contains detailed instructions on how to prepare each section,
along with suggestions as to what should be included. The problems are
that: 1) they usually do not read the instructions (or do not understand
them), 2) their command of the English language is poor in many cases, and
3) they feel that they can submit the same material that they did in high
school and pass. Students are allowed to submit a draft report to the TAs
for correction (no grade assigned, only major errors noted) although few
avail themselves of this opportunity.
In terms of grading three reports are submitted each term and the best two
are counted toward their term mark. TAs grade the report from a detailed
marking key which breaks the reprot down in to the various sections
(intro., m&m, etc.) and gives the mark for each item which should be
present. This reduces, but does not eliminate variance between TAs. The
TAs are also given a handout on what they should and should not do when
marking papers. The main point that should be stressed to the markers is
that they should write LOTS of commments on the papers. Students want,
and need, the feedback. If the paper contains many major errors these
should be listed at the end where the mark is placed.
By the second term the majority of the students are able to write a pretty
good lab report, in journal format. Once learned it is not something they
forget.
From: mweber@ccs.carleton.ca (Mike Weber)
The recent discussion of teaching scientific writing in introductory labs
has been very interesting.
We teach journal format scientific writing in group-based laboratories in
our four core biology courses. These are investigative labs, most of
which have a "design-your-own" component. Students work in groups
of
three and take turns being the "PI" for each experiment.
In the first course (Bio 101) we devote a lab period to (1) discussing the
format, style, and content of journal format articles, (2) teaching them
how to critique scientific writing (we use two student papers which have
various problems - one reasonably well done, one poorly done) which helps
to prepare them to do peer reviews, and (3) teaching them library research
skills, especially using on-line search facilities. These exercises are
supported by appropriate documentation ( a style manual prepared in-house,
examples of well written scientific papers, etc.) This same presentation
is repeated in subsequent courses as needed for students just starting.
The students do the experiment, analyse the data, and prepare the data for
presentation (tables and Figures) as a group. We encourage them to
outline their paper as a group as well. [ We require the group to
bring a draft of their Introduction to lab the day of the experiment
for critique]. The PI then drafts the paper,
revises, and give copies to the other group members for peer review. They
mark up the draft and fill out a peer review form which asks
section-specific questions regarding style, content, clarity etc. The PI
then revises the draft using the peer reviews and the group submits a
final draft for evaluation by their instructor. Each group member must
sign the final draft to indicate they have approved it. If they don't, we
allow them to hand in an alternative "view", but only after consultation
their instructor (this is rarely used by the students). NOTE: The final
draft
is completed within a week of the experiment.
The first paper in Bio 101 is critqued by the instructor and returned for
revision, and then graded. This gives everyone opportunity to learn
(painlessly?) what is expected and to have higher level feedback right
away. papers are evaluated using a 100pt scale with points assigned to
the content of each section and to global aspects such as clarity,
adherence to scientific format, format and clarity of illustrative
materials, and to format and use of references. In Bio 101, we weight
format related points more heavily, and shift more points towards content
as they proceed through the curriculum. If a group score less than
80/100, they may revise the draft to obtain a maximum of 80pt (~B-).
We offer our students lots of writing support. Our writing center
provides a technical writing specialist who tutors students as well as
trains our Technical Writing Assistants - undergraduate TAs who work in
the lab and hold evening office hours to help with student writing. We
(the instructors) also try to meet with each group to give a verbal
critique of their work and to clarify any comments we made on their paper.
Our experience has been that we get very reasonable (and many excellent)
papers from first semester students. By the time they complete the core
they been through the routine 12-13 times and have a very good handle on
scientific writing and critiquing, and on the process of science from
start to finish.
[NOTE:Our core courses range from 50 - 180 in size.]
I look to forward to reading more.
Greg Anderson
Department of Biology
44 Campus Ave.
Bates College
Lewiston, ME 04240
ganderso@abacus.bates.edu
This year we initiated a linkage between the second term of our freshman
general biology sequence and the second term of the freshman English
compositon sequence. There are five sections of lab in bio and each is
paired with a section of HU (English comp). Thus, these students are
together for 3 hours of lecture, 3 hours of lab, and 3 hours of HU. They
work in small groups of 3-4 in both lab and HU (the same groups). The
course starts with winter field trip and a writing assignment about the
field
trip from the point of view of one of the organisms. Then they talk about
posing questions, pose a biological question related to the lab they are
doing on circulation, and write a brief proposal stating their question
and methods.
Then, they must choose a project topic on behavior, and this
year we asked them to relate the project to some consequence of global
warming. They wrote a preliminary proposal with questions to be answered,
methods, and a list of materials. We used these to prepare needed
materials (we earlier gave them a list of possible organisms). They
visited the library in their HU class and learned how to use the various
search services. After doing library work, they wrote a more formal
proposal with a referenced introduction as part of their HU course. They
had two weeks of lab time in which to carry out their projects, and they
were asked to submit a second proposal to their BL GTA for revisions of
their methods and/or objectives during the second week of experimentation.
Some additional time was provided for those who needed additional
observations.
Our objectives were relatively simple. We followed the philosophy of
letting them learn by their mistakes. Few of them have had any statistics
and have no clue about how much replication is needed. However, when they
repeated their experiments and got different results from the first try,
they understood the variability of biological organisms, even the same
organism. We talked about several examples in lecture. A particularly
appropriate one was a comparison of an HU reading by Gordon & Suzuki
on
global warming that discussed boiling frogs and the frogs just sat there
as the temperature got hotter, compared with a student experiment where
the students moved a frog from room temperature to 30 deg C and the frog
became very active whereas the room temperature and 10 deg frogs were
quiet. Students hypothesized reasons for the differences in response.
The next week we tested some of their hypotheses in lab. The frogs
behaved differently then.
Students will soon turn in to both the HU and BL instructor a
journal-style article on their group experiment. In most cases, they are
writing this as a group. They have discussed excerpts of early drafts in
HU, using peer review. I will grade these papers for scientific style,
scientific argument, appropriate referencing, and validity of
interpretation and conclusions. At this stage I am less concerned about
the design of the project itself, but rather with their recognition of its
problems now that they have completed it.
Their final project-related assignment will be to rewrite their research
paper for a lay audience.
The HU readings all related to biology and science in some way. They
used biological examples to examine paradigms, asking questions,
differences in opinions and interpretations among scientists, ethical
questions on using primates for research, discussions of why biologists
need to write and how they learn by writing (using an article by Randy
Moore), articles that dealt with temperature effects on animals (in
scientific style and others in layman's style), and much more. Students
had a basis to discuss style and audience, techniques of persuasion, and
differences in styles between journals in the same field of science.
In addition to all this, students are required to write a weekly email
journal entry to which others in their section may respond. Their HU
faculty and I respond to most of them.
This course has more than met my expectations. I am ecstatic. Students
have been enthusiastic (except for the upper classmen who changed majors
into biology and thought they were through with freshman comp). They have
integrated material more than I had ever dared to hope. It is like giving
them two biology courses. They are getting lots of types of exposure to
many of the concepts I talk about, sometimes by coincidence, sometimes by
design. They have brought in prior experience and other courses in their
journals. They listen to TV and email me with questions. And they love
designing their own experiments. Most of them chose simple questions, but
simple usually does not turn out to be so simple.
They also do not complain (yet at least) when I grade them down for
grammar and spelling errors in the email essay questions and answers they
submit to me. These are submitted weekly for a class file to be used for
preparing for their tests in biology. Since this is the first time the
course has been taught, and the first time most of them have had to take
essay exams, it is a way to help them learn to write adequate rather than
superficial answers.
Coordination between the two courses is important. We spent more than
100 hours developing the two tracks. We had an added incentive of
submitting a proposal to the NSF/NEH/FIPSE interdisciplinary call for
papers. Although the reviewers didn't feel we were interdisciplinary
enough, I have been quite surprised at how interdisciplinary we are and
how well the students are contributing to that integration. I wouldn't
change the structure of this course if I were offered a grant for doing
it! I am convinced that more faculty involved in these two courses would
be overkill and would only serve to create disconnectedness, not
connectedness as we now have. Team teaching can be very confusing for
students, especially freshmen who are already struggling with all the new
expectations of college courses. By maintaining distinct grades and
syllabi, students know who is "boss" in each part of the course.
I have
team-taught before, and this way of doing it seems to me to be far
superior to the more traditional ways I have joined before.
Janice M. Glime
Dept. Biol. Sci.
Mich. Tech. Univ.
Houghton, MI 49931-1295
jmglime@mtu.edu