Accountability and Incentives: A Cautionary Tale, pt1

Introduction

I understand the rationale behind the push for ever-increasing teacher accountability -- it's natural to want to insure that your kids are getting the quality education they deserve. Teachers can't just sit back and say, "Trust me, I know what I'm doing." Those days are over. Fair enough.

But we teachers need the rest of the country to understand that when we push back against how teacher accountability is being implemented, we are not just scurrying around trying to hide our incompetence or protect our "cushy" jobs.

We can handle the scrutiny. Most of us would be eager to let our students' successes serve as evidence of our effectiveness as educators. I don't think we fear accountability -- if that accountability is implemented properly, if success is defined properly.

It's not about rigging the game in our favor, lowering the bar, or any of that nonsense; it's about making sure that we're judged for the things that really matter, the things that we teachers do that actually improve students' lives.

The American public needs to realize that not all forms of teacher accountability are created equal. Worse, the most popular methods can be misleading or downright detrimental. Even "no-brainer" approaches like rewarding "effective" teachers with bonuses have already ended in failure. Just because greed-maximizing incentives (sort of) work for capitalism doesn't mean they work for education.

However, hearing these arguments and really feeling them are two different things. And somehow teachers have been turned into the enemy -- or at least that's how it feels to us -- to the point where no one seems much interested in listening to the people who have the most expertise on the subject.

But I think the power of satire and absurdity has a chance to get the message across. So, in that vein, I offer you:

Latest Updates: Going global + enhanced security for student info

We are now global! 
Everyone on the planet can now grade essays faster and more efficiently!

Users from anywhere in the world can register on the site and use the free trial to grade a class set of essays (full details on the registration page).


Enhanced security for student information
Part of our to-do list for going global was to make sure that we were doing everything in our power to secure and protect students' personal information. Here are the steps we've taken:

Students' personal information is encrypted in our database using industry-standard best practices to guide our selection of encryption algorithms and encryption strength. If a hacker were to gain access to our records, s/he would only see the encrypted values:

firstName	lastName	email
BsBzpBQkQIN9s6RwV7fr58G2eY8qn2a6	yNd9sZ056n5xS14Qe5uZd3hefgY3s59i	vA8ezh8Vlm3W9WscfkFGQw4C

Each piece of information is individually encrypted in such a way that "cracking the code" for one value does not make it any easier to decrypt the other values.

In fact, EssayTagger's own internal view of the database makes students' information appear just as garbled to us as it would be to an outside hacker.

We have taken these precautions even though it is highly unlikely that our database would be compromised. We are running on Google's computing infrastructure and therefore Google's efforts to safeguard their own computers are our first line of defense. Their computing infrastructure is among the most robust and powerful in the world and any improvement they make to the security of their system automatically protects EssayTagger.com under the same umbrella.

We will never contact a student unless it is an assignment-related notification (e.g. to tell them that their graded essay is available for their review) or if there is a technical issue that requires action on the student's part in order to be resolved.

Note: Instructors can decide whether or not they want the system to collect and store student email addresses. This will allow teachers to remain in compliance if there are any local or national restrictions regarding student contact info.

We will not sell or share students' personal information with any third parties or use students' information for marketing or any purposes outside of what is required to facilitate student-teacher interaction in our system.

Students are not added to any mailing lists.

Open Beta extended and subscription bonus announced!

I have two important EssayTagger.com updates to share:

Open Beta extended
Our free Open Beta feedback period has been extended to February 10th! That gives you and your colleagues a little more time to try us out as much as you like without committing a single dime. After Feb 10th our $10/month pricing will go into effect.

I'm super-grateful for the valuable feedback we've received during this Open Beta period. So many of your questions, comments, and criticisms have really helped to guide my thinking and have already helped EssayTagger evolve.


Subscription bonus
I think you early adopters deserve to be rewarded. It takes courage and patience to dive into new technology that does not yet have a proven track record.

So I'm pleased to announce our Open Beta subscription bonus: Anyone who subscribes to EssayTagger before the end of Open Beta will receive their first month of service for $1.

Specifics:

• Commit to a subscription anytime before Feb 10th, 11:59pm (CST).
• Your first month will be $1.
• You won't be billed until Feb 11th.
• Subsequent months will be billed at our standard $10/month rate.
• Anyone is eligible--including new users--as long they commit before the deadline.
• See the FAQ for info on canceling, summer break, and group licenses.

Help us spread the word!
Now is obviously the time to try us out. Please help us build a spike of interest in these last two weeks of Open Beta!

Tweet a link to this post, blog about it, tell your colleagues and friends.

And keep that feedback coming!

Latest Update: Rubric drill-down analytics

This is one of the features I was most excited about when I first conceived of EssayTagger!

Read on or watch the tutorial video!

When you grade your essays in our system you're tagging and evaluating the students' text and storing it all in our database. That means that we can then retrieve all that information in powerful ways that aren't possible with traditional grading methods.

Grade a class set of papers and then go to the assignment's rubric performance analysis screen.

"analytics" tab > view aggregate rubric results

From here you can now drill down into each individual rubric core element. We're going to jump into the "Topic sentence" core element:

Click any of the "show text excerpts" checkboxes and the site will hit the database and retrieve your data:

The bold text is the feedback comment you selected for the student and the quoted text is the corresponding excerpt from each student's essay! The list is filtered to display, in this case, only the excerpts that were evaluated as being of "Level 4" quality. You can display the associated text excerpts for each quality level.

To make all of this more concrete, let's take a look back at the grading app:

Notice the highlighted essay text just above the comment selection box: "Students spend most of their days outside of home." Since that's a bad topic sentence, I'm going to select a Level 4 feedback comment. I chose the "This is a trivial fact" comment (of course you can always add a new comment if the existing ones aren't sufficient).


And if you look back at the drill-down analytics, you'll see that tagged text excerpt near the bottom of the list:

You can also select the "show students" checkbox to retrieve the name of the student who wrote each excerpt in the list:

Thank you to my friends who allowed me to use their names as sample students!

You can also uncheck "show text excerpts" and just display the student names:

This is a very simple--but very powerful--example of what EssayTagger.com can do now that all of your essay evaluations are in our database. Super-cool stuff!

Next I'll give you some ideas for how to use this data in your classroom.

Latest Update: Class rosters!

Major new update: we now fully support class rosters in the system!

I've designed it to be as little work as possible for teachers to build and manage their class rosters. Under the ideal scenario (outlined below), you can let the students build the rosters for you automatically when they upload their first assignment.

Here's the tutorial video for creating and managing your class rosters in EssayTagger.com:

The student upload process has also been slightly altered to accomodate the changes. Here's the new version of the tutorial for student uploads:

And you can still upload essays yourself. It isn't ideal, but we know that not everyone is ready to have the students do the uploading. The biggest downside of uploading essays yourself is that it takes a bit more work to then associate each essay with a student. Here's the instructor upload tutorial:

Latest update: Rubric quality level direction

This was another feature that was requested at NCTE.

You can now specify if your rubric quality levels progress from best-to-worst or from worst-to-best. You'll find the setting on the Create Rubric screen under the first quality level input field

When you click on it you'll see the two simple options:

The direction determines which end of the rubric gets tagged with green (best) markers or red (worst) markers in the grading app and in the aggregate results grid.

The worst-to-best direction opens up the possibility of using massive year-long rubrics that have way more than five quality columns. We spoke to one publisher at NCTE whose system uses 14 quality levels. Sounds intense, but each individual assignment only targeted a sub-range within those 14 levels. An early assignment's rubric might only go from level 1 to 5 whereas an end-of-year assignment's rubric might range from columns 10 through 14.

Latest update: Student uploads!

Update 12/16/2011: This process has changed slightly and a new video has been recorded to cover the latest version of the student upload process.


At NCTE a lot of teachers asked us if we could have students upload their own essays. At the time - last weekend - the only option was to have teachers collect and upload the documents themselves. Not ideal.

I began telling people that my goal was to support student uploads by this Friday (today!). I said it to enough people that I realized it was now a promise!

So I made good on that promise.

In the Detailed Assignment View you'll now see a student upload URL:

The upload URL contains a five-character code that is unique for each assignment. Simply provide this link to your students and they'll be able to hand in their assignments online themselves. You can also still upload documents for the assignment yourself.

Here's what they'll see:

No student login is required; the link goes straight to the upload page.

We ask for their first and last name so that you can identify whose work belongs to whom once it's uploaded. They also must specify the class section they belong to so you don't have to sort through them later.

I also added a checkbox that basically says "I swear this is my work." It won't stop plagiarists, but it's one more piece of leverage you'll have over them.

All fields are required and they cannot submit until they have been completed.

I also built in a lot of safeguards against bad document files and incorrect file formats. I had a lot of problems with this on moodle when I'd go to open a kid's document and it was all garbled or in a format I couldn't open (e.g. ".pages"). I didn't want to relive that here.

For starters the "select document" button will only allow students to select a DOCX, DOC, RTF, or TXT file. These are the only formats we are currently able to support.

When they hit "Submit" their document is uploaded and they are forwarded to a status page:

Once the processing is complete they'll see their document embedded in a confirmation page:

At this point they can see their document and confirm that there weren't any errors along the way. Each essay is also timestamped with a submission date/time so you can see who was on time (due dates are configured in the Assignment setup). Submit time is set by our server - not the student's computer - to guarantee consistency and integrity.

Students can upload their document multiple times if they have revisions - you'll just see multiple copies listed under their name with different submit times.

I recommend that you try out the process yourself and demo it for your students in class. You can easily delete the test documents you upload.

As with everything on the site, I tried to make this as foolproof as possible - which is an added challenge when you're talking about asking students to follow instructions!

**Note: We attach the students' first and last names to the document filename but we don't yet support full student integration into the site. That'll take a bit more work but is a high priority on my to-do list.

Key takeaways:

• Each assignment has a different, unique upload link.
• Tell your students to wait for the confirmation screen.
• You can still upload documents yourself if you need to.
• The site supports DOCX, DOC, RTF, and TXT files.