In part two we explore a method for fast, effective formative assessment by leveraging EssayTagger's strengths and incredible built-in data reporting.
Part two: Fast, effective formative assessment with EssayTagger
"If students receive feedback often and regularly, it enables better monitoring and self‐regulation of progress by students."
- Nicol and Macfarlane-Dick
At face value EssayTagger's core function--grading essays more efficiently--seems more well-suited to end-of-unit essay evaluation (summative assessments). But as you'll see we can easily leverage EssayTagger's strengths to hit all three formative assessment keys discussed in part one: speed, detailed diagnostics, and quality feedback.
Basic approach
Develop open-ended, journal-style written response questions aligned with unit goals and then evaluate students' work in EssayTagger, focusing on short, targeted feedback. Then review the evaluation results data to refine class-wide instruction and target individual reinforcement or remediation. Ideally you would repeat 2-3 times throughout the unit before the end-of-unit summative assessment.
A concrete example: The Tempest, Sophomore English
When studying Shakespeare with sophomores we need to work on the mechanical skill of processing the complex text and would like to see the students develop an engagement with the text at an emotional, human level. A final summative assessment might come in the form of an essay prompt like, "Do Prospero's ends justify his means?"which would require a detailed understanding of the text and characters along with an expectation of referencing appropriate textual evidence.
To build students up to that goal, I created a short journal assignment for Act I of The Tempest. The conclusion of Act I is a good place to pause and take stock of where the students are at. The journal assignment consists of three parts: summary, opinion with evidence, and text decoding.
The summary is a straightforward way to see how well they're following the text at a high level. It also becomes a useful refresher for the students at the end of the unit when they want to look back at the previous acts.
The opinion with evidence section is my favorite. It asks an engaging opinion question: "What's the deal with Prospero? Is he just a big jerk?" It's open-ended and there's no right or wrong answer. It gives them a chance to reflect on and process what they've seen of Prospero thus far (which is admittedly quite confounding!). Just about any take on him is fair game--provided they offer at least one piece of evidence from the text.
The text decoding section offers a complex excerpt from the text and asks the student to do her best in interpreting its meaning and explain how she came to that conclusion.
In each section we're looking for no more than a paragraph of writing and the assignment is something that sophomores should be able to complete in one night or perhaps even in one class period if they have access to computers.
Rubric
I created this EssayTagger rubric to closely match the assignment:
- Summary (paragraph-level)
- Character Development & Impact (paragraph-level)
- Textual Evidence (sentence-level)
- Text Decoding (paragraph-level)
Evaluate fast, fast, fast
At this point you're ready to review the students' work. You only have four things to evaluate: each short paragraph and one piece of evidence.
The "Summarize Text" rubric element is configured as a paragraph-level item. All you have to do is drag the "Summarize Text" rubric button over the text it will bracket whichever paragraph it's over.
Just release the button when it's over the summary and the evaluation options and feedback comments pop up.
Select an appropriate feedback comment from the desired quality level column or add a new comment as needed.
Then move on to quickly evaluate the remaining two paragraphs ("Character Development & Impact" and "Text Decoding"). The "Textual Evidence" rubric element is a sentence-level item that we expect to find in the second paragraph where the student discusses her take on Prospero.
And, of course, each rubric element has its own dedicated collection of feedback comments that you create.
Less is more
In a survey of research on feedback, Nicol and Macfarlane-Dick note one study that found "three well‐thought‐out feedback comments per essay was the optimum if the expectation was that students would act on these comments." This makes intuitive sense but is rarely the case when evaluating student writing. We tend to try to solve all problems at once.
But the beauty of fast, repeated formative assessment is that we can make a series of small nudges and have students work on addressing deficiencies in manageable chunks. The journal assignment offered here is perfectly suited to "limiting the amount of feedback so that it is actually used."
So keep your feedback comments focused and make sure that they "provid[e] corrective advice, not just information on strengths/weaknesses" so that your students can take action and follow your coaching.
Reusable feedback comments
An important part of keeping this evaluation process fast is creating reusable feedback comments.
Consider the following comments for Summary that essentially say the same thing:
- "Wait, who's Caliban? You mention his name but you don't tell us anything about him."
- "Be sure to give at least a brief introduction for each character that you reference."
It will take a bit of a mental adjustment and some practice to get used to this targeted-but-reusable approach. What might help is to think of it this way: Plan to use these comments on the next text. If you're reading The Tempest, try to make sure they'll work for Death of a Salesman.
Return work and review results data
The site can email students their graded work back to them when you're done. One click and delivery is done!
At this point you'll also have a wealth of diagnostic data to pore over. EssayTagger collects and analyzes your evaluation data for you, displaying overall classroom performance data for each rubric element as well as individual student drill-down charts with automatic outlier analysis to see exactly who is struggling and where.
You'll be in a position to view overall trends where a class-wide intervention might be in order or group up specific students who all share a specific area of weakness. You could even use the data to pair a student who excelled at Summary with one who really struggled.
Provide student samples
You can even retrieve text samples pulled directly from your students' work. Want to show them some examples of a strong summary? No problem. The site can pull up every summary by its evaluation level. Want to highlight some really nice reflections on Prospero's character? It's all trivially easy to access.
Mission Accomplished
Granted, this sort of formative assessment is not lightning fast. But I do think it strikes the right balance between fast-enough speed, phenomenal detailed diagnostic data, and targeted, specific feedback. You'll quickly find that you can evaluate each journal entry nearly as fast as you can skim them. As you make your way through your first class set, you'll be adding fewer and fewer new feedback comments because you'll be reusing more and more as issues you've already seen will reappear in subsequent students.
Call me crazy, but I'd bet that you could get this down to about 30 seconds per student. And if you're grading your second class section or reusing your comments on a subsequent journal assignment, you'll be flying right from the first student.
There's no other way to move that fast, that efficiently, and produce valuable diagnostic data, AND provide targeted feedback. This is why I find EssayTagger so exciting. It's a win-win-win.
Aligned with research
The open-ended nature of the journal questions is important because it emphasizes the student's opinion and interpretation rather than a single correct answer determined by the teacher. As Black, Paul, and Wiliam note, "Discussions, in which pupils are led to talk about their understanding in their own ways, are important aids to increasing knowledge and improving understanding" (7). But too often in-class discussions will only feature the most outspoken or more confident students. A written journal response brings opinions out of all students. Indeed, "What is essential is that any dialogue should evoke thoughtful reflection in which all pupils can be encouraged to take part, for only then can the formative process start to work (8)." All students can participate on the written journal response and then even use those thoughts as a starting point to bolster confidence for in-class discussions.
Sources
Black, Paul, and Dylan Wiliam. "Inside the Black Box."
Nicol, David J., and Debra Macfarlane‐Dick. "Formative assessment and self‐regulated learning: A model and seven principles of good feedback practice." Studies in higher education 31.2 (2006): 199-218.
