Grading with EssayTagger on your iPad via Photon browser

Today we discovered that the EssayTagger grading app can run on an iPad with a little help! Here are your step-by-step instructions for accessing EssayTagger through the Photon browser.

Install Photon on your iPad
From your iPad, go to the app store and search for "Photon browser". Make sure you select the iPad version and not the cheaper iPhone version. When I installed it today it was a $4.99 purchase.

Launch Photon

Photon is a web browser and really isn't all that different from any other web browser. There is one big special function you'll need to know about to enable Flash support, but Photon already does a good job of explaining it. You'll see.

Go to EssayTagger.com and either launch the interactive demo from the "try the demo" tab or log in to your account and click "start grading" to launch the grading app for one of your assignments.

When the grading app window opens you'll see a Flash error message instead of the grading app. But Photon explains what to do:

Once you click the lightning bolt icon at the top right, the grading app will be able to load:

Yay! The grading app is running on the iPad!

Adjust Photon settings

Now click the gear icon at the top right to enter the settings options. Change the following two settings:

Bandwidth: 6 - to maximize responsiveness and text quality. Photon runs the grading app remotely on its own server and streams it to your iPad. That's how they're able to support Flash on the iPad--because it's actually running Flash somewhere else! If the bandwidth setting is too low, the essay text will look chunky due to the transmission compression.

Mode: Web - I found this produced the best looking text and further minimized compression chunkiness.

That's all we need to do here. Photon will remember these settings so you won't have to worry about these again.


Using the grading app
A couple tips to make your life easier. Photon's default interactive mode is not a great match for the grading app's drag-and-drop interface. Once the grading app loads, switch to the "grab" mode at the top right:

With this mode enabled, dragging and dropping works extremely well. You can also easily select text passages to mark for errors or enter a free comment.

I would also take advantage of the grading app's built in font size adjuster buttons on the left. Increasing the essay font size will make it easier to read and easier to select text. Here's a set of before and after screenshots:

Use a bluetooth keyboard
The iPad on-screen keyboard does not work well when using the grading app in Photon. It often ends up covering up the text box so you can't see what you're typing:

Add a new comment...

D'oh! Covered up the text box!

My bluetooth keyboard worked just fine. The only slight oddity was that I had to press the keyboard button at the top right of Photon to get it to start receiving my typing.

Other minor tips

Horizontal alignment: The grading app makes the best use of space if you launch it while the iPad is oriented horizontally.

Essay scrolling: Scrolling through an essay is a little difficult because the vertical scrollbar in the grading app is so narrow. It will click-and-drag just fine but you'll have to be a bit precise to "grab" it.

It might be easier to just tap the gray area above or below the scroll bar to move the essay.


Quick tips: It's also a little hard to access the "Quick tips" rollout help topics at the top of the grading app. Switch to the mouse pointer icon mode to make it easy to "hover" over the Quick tips help items:

That's it for now! Go grade and feel all super-high tech!

Yes!! EssayTagger on your iPad!

EssayTagger's patent pending interactive grading app is built in Flash and works great in any web browser. However, Steve Jobs decided years ago that Apple would not to support Flash on iOS devices (iPad, iPhone). I've experimented with a special version of our grading app that can work as an installed iPad app, but it's a long way from being ready to put in the iTunes app store. I figured iPad support would just have to wait.

But then Alaina Langdahl of Parkrose High School in Portland, OR suggested we take a look at a few iPad web browser apps that serve as an alternative browser to the built-in Safari browser and, most importantly, they support Flash!

I'm ecstatic to announce that the Photon browser does a surprisingly good job of bringing the EssayTagger grading app to life on an iPad!

Holy awesomeness!!!

Drag-and-drop interactive grading on a tablet! This is exactly how I imagined using EssayTagger when I first started this company!

All grading app features are fully supported when used through the Photon browser. There are some important settings that will vastly improve the experience. I'll update this post soon with step-by-step instructions.


The downside
Photon browser is a $4.99 app purchase. I know, that stinks.

We have no relationship with Photon and will be evaluating other Flash-enabled iPad browsers. Hopefully a free option will emerge that offers support for all of the features required to run the grading app. Ideally Apple would finally come around and support Flash, but that's not very likely (in fact, even Android is moving away from Flash with its latest Android Jelly Bean 4.2 OS).

Until we identify a viable free solution, it'll be up to you to decide if grading on your iPad is worth $4.99.


Updates:
- Step-by-step instructions for using EssayTagger with the Photon browser are now posted!

- Alaina is reporting that she's having success with the free Puffin iPad browser. My testing with Puffin was less successful but I'll take a closer look at it as soon as I can.

System alert: Partial system outage - resolved

FINAL UPDATE 5:31pm
The last lingering effects of the Google App Engine server problems seem to have been cleared up and our own testing now shows full functionality restored.

This sort of downtime is frustrating, but it's still only the second Google App Engine outage since we launched EssayTagger 14 months ago. All sites suffer some downtime but we still believe in Google's reliability and ability to react and recover faster and more robustly than we could if we were managing our own server hardware.


UPDATE 1:23pm
Message from Google:
"We are still working to resolve the issue related to Google App Engine serving. At this point error error rates for affected applications should be declining. We will provide another status update by 11:30 AM PST."

EssayTagger is responding again, albeit slowly. Grading app is still severely impacted.


UPDATE 12:07pm
Google App Engine servers continue to see problems and it seems to have spread beyond the backend task queue. They've rescinded their earlier resolution note and are now only saying "We will update this message shortly when the incident has been resolved."

These server issues are now affecting the main EssayTagger site.


UPDATE 10:35am
Message from Google:
"This morning some Google App Engine applications reported elevated error rates and increased latency. This issue should be resolved as of 8:10 AM US/Pacific time. We apologize for the inconvenience and thank you for your patience and continued support. Please rest assured that system reliability is a top priority at Google, and we are making continuous improvements to make our systems better."

There still seems to be some intermittent slowdowns for backend processes (uploading an essay, marking an essay as graded). Though Google says the issue is resolved, I still recommend caution.


ORIGINAL POST 1/15 10:28am
At approximately 9am CST Google's App Engine servers suffered a problem with their "task queue" service which EssayTagger uses for behind-the-scenes processing. The majority of the site has not been affected.

Specific interruptions occurred on the student upload page which relies on the task queue to process incoming essay files. The other major impact was that the grading app was unable to process essays when they were marked as graded. Normal grading activity (evaluating rubric elements, adding comments, etc) should not have been affected.

The task queue service is currently intermittent. Take care if using the grading app and keep an eye on the "synced" or "syncing..." message at the top right. If it remains on "syncing..." for more than a few seconds, pause before doing any more work. Your work is only guaranteed to be saved when you see the "synced" message.

Updates will be posted as new information arises.

And... we're back!

Friday, Oct 26th, 2pm:
EssayTagger.com is back up and seems to be responding normally. See my earlier post and its live updates during the morning's downtime.


What happened?
Google App Engine suffered a worldwide outage around 9:30am (CST) Friday after which they slowly restored services. The outage knocked out major sites like Dropbox, Instagram, Khan Academy, and anyone else running on Google's infrastructure. During most of this outage EssayTagger.com was either inaccessible or experienced excruciatingly slow load times. The site reached stability around 2pm.

Is this normal?
Nope. An outage of this scale is unprecedented. Most tech folks view Google's infrastructure as being as robust and as close to invulnerable as you can get and their track history had borne that out, until this morning.

Are you going to drop Google App Engine now?
For the moment, no. This was an aberration. The realities of website hosting are that downtime happens, no matter which infrastructure you're running on. And, to be honest, I have much more faith in Google's engineers than I do in anyone else -- including myself. Yes, their system failed this morning, but in the brief 11 months of EssayTagger's life, Google App Engine has been remarkably stable and much more reliable than anything else out there.


Read Google's mea culpa, their analysis of what happened, and the new preventative measures they've put in place:
http://googleappengine.blogspot.com/2012/10/about-todays-app-engine-outage.html

"We know you rely on App Engine to create applications that are easy to develop and manage without having to worry about downtime. App Engine is not supposed to go down, and our engineers work diligently to ensure that it doesn’t. [...] We know that hundreds of thousands of developers rely on App Engine to provide a stable, scalable infrastructure for their applications, and we will continue to improve our systems and processes to live up to this expectation."

Site down: Google's servers experiencing problems

Friday, October 26th, 9:53am: 
The site is currently down. We run on Google's "App Engine" (GAE) infrastructure and they are currently experiencing problems that render our site -- and other prominent App Engine sites like KhanAcademy.org -- inaccessible.

Google App Engine status can normally be viewed here, but even the status page is failing to respond.

Needless to say, this is inconvenient but also rare; Google's infrastructure is among the best in the world and they rarely ever see interruptions of their App Engine service.

Interestingly, google.com search service is still functioning (as is blogger.com -- as evidenced by this post being publishable!). Not surprising that they'd have a separate set of servers for their core business.

Follow the latest updates on #GAE via Twitter:
https://twitter.com/search?q=%23GAE&src=hash

And my own Twitter account:
https://twitter.com/KeithMukai


UPDATE 11:10am
EssayTagger.com has begun to respond again, but service is intermittent. Google App Engine is not yet stable.


UPDATE 11:35am
From Google's Max Ross: "At approximately 7:30am Pacific time this morning, Google began experiencing slow performance and dropped connections from one of the components of App Engine.  The symptoms that service users would experience include slow response and an inability to connect to services.  We currently show that a majority of App Engine users and services are affected.  Google engineering teams are investigating a number of options for restoring service as quickly as possible, and we will provide another update as information changes, or within 60 minutes."


UPDATE 12:51pm
From Google's Christina Ilvento: "We are continuing work to correct the ongoing issues with App Engine.  Operation has been restored for some services, while others continue to see slow response times and elevated error rates.  The malfunction appears to be limited to a single component which routes requests from users to the application instance they are using, and does not affect the application instances themselves.

We’ll post another status update as more information becomes available, and/or no later than one hour from now."

EssayTagger.com is now responding more consistently. Cautiously optimistic that we're through the worst of it.


UPDATE 1:45pm
The App Engine status board is looking better. The error spike is returning to more sane levels but the system is still in an "elevated" problem state.

EssayTagger.com performance is still a little unpredictable with intermittent reports of documents that couldn't be uploaded to the system. We rely on Google Docs under the hood to process incoming documents so even if our site is working, this integration point with Google might still see issues.


Update 2:07pm

From Google's Christina Ilvento: "At this point, we have stabilized service to App Engine applications. App Engine is now successfully serving at our normal daily traffic level, and we are closely monitoring the situation and working to prevent recurrence of this incident.

This morning around 7:30AM US/Pacific time, a large percentage of App Engine’s load balancing infrastructure began failing. As the system recovered, individual jobs became overloaded with backed-up traffic, resulting in cascading failures. Affected applications experienced increased latencies and error rates. Once we confirmed this cycle, we temporarily shut down all traffic and then slowly ramped it back up to avoid overloading the load balancing infrastructure as it recovered. This restored normal serving behavior for all applications. 

We’ll be posting a more detailed analysis of this incident once we have fully investigated and analyzed the root cause."

So in theory EssayTagger.com and all other affected websites should be back to full power.

Latest Updates: Going global + enhanced security for student info

We are now global! 
Everyone on the planet can now grade essays faster and more efficiently!

Users from anywhere in the world can register on the site and use the free trial to grade a class set of essays (full details on the registration page).


Enhanced security for student information
Part of our to-do list for going global was to make sure that we were doing everything in our power to secure and protect students' personal information. Here are the steps we've taken:

Students' personal information is encrypted in our database using industry-standard best practices to guide our selection of encryption algorithms and encryption strength. If a hacker were to gain access to our records, s/he would only see the encrypted values:

firstName	lastName	email
BsBzpBQkQIN9s6RwV7fr58G2eY8qn2a6	yNd9sZ056n5xS14Qe5uZd3hefgY3s59i	vA8ezh8Vlm3W9WscfkFGQw4C

Each piece of information is individually encrypted in such a way that "cracking the code" for one value does not make it any easier to decrypt the other values.

In fact, EssayTagger's own internal view of the database makes students' information appear just as garbled to us as it would be to an outside hacker.

We have taken these precautions even though it is highly unlikely that our database would be compromised. We are running on Google's computing infrastructure and therefore Google's efforts to safeguard their own computers are our first line of defense. Their computing infrastructure is among the most robust and powerful in the world and any improvement they make to the security of their system automatically protects EssayTagger.com under the same umbrella.

We will never contact a student unless it is an assignment-related notification (e.g. to tell them that their graded essay is available for their review) or if there is a technical issue that requires action on the student's part in order to be resolved.

Note: Instructors can decide whether or not they want the system to collect and store student email addresses. This will allow teachers to remain in compliance if there are any local or national restrictions regarding student contact info.

We will not sell or share students' personal information with any third parties or use students' information for marketing or any purposes outside of what is required to facilitate student-teacher interaction in our system.

Students are not added to any mailing lists.

Avoiding Session conflicts on save() due to Hibernate lazy loading

With Java Reflection we can force Hibernate to instantiate lazily-loaded (aka proxy) member objects before we attempt to save the containing entity.

Yesterday I posted my Spring + Hibernate + Cloud SQL sample project code to help other developers get up and running with Hibernate and App Engine.

Today I had to make some DB changes and slightly rearchitected my code. That uncovered a problem. I'm using Spring's OpenSessionInViewFilter to support lazy loading in the view layer. It's dang convenient. And it's currently configured for singleSession=false. That means that it'll use a read-only Session for gets but a separate save/update Session for writes.

Calling getHibernateTemplate().save() had been working fine until my code reshuffling exposed a problem. If an entity has any member objects that have not yet been initialized (due to lazy loading), the save() call will throw a session conflict exception. Specifically:

org.springframework.orm.hibernate3.HibernateSystemException: illegally attempted to associate a proxy with two open Sessions

Here's a simple example:

The problem is that the read-only Session created the lazy-loading proxy Owner within oldCat but hasn't been asked to initialize the proxy yet. Then the save/update Session gets the newCat and tries to access its Owner and now both Sessions are trying to handle the non-initialized proxy, thus the "two open Sessions" complaint.

So folks will generally do something lame like this to get around it:

I'm not judging. I've done it in the past too. But it's clearly a crappy solution. If that chunk of code doesn't need to access any fields in Owner, it shouldn't have to just to make the DAO/ORM layers happy.

So here's my new solution: programmatically search for non-initialized proxies and call getHibernateTemplate().initialize() on them before we try to save.

This requires Reflection to inspect the Cat and look for fields that might need initialization.

Unfortunately my DAO doesn't have permission to access Cat's private member variables. So instead we scan for the public getters and look at their return types. If the return type is a domain object that is mapped by Hibernate, then we "get" it and initialize as needed.

How do we know it's mapped by Hibernate? I'm just filtering on the return type's package name; I only care about DTOs from com.essaytagger.model. The proxies that need initialization are easy to spot--they return true to (obj instanceof HibernateProxy) tests.

Here's the new version of my _GenericDaoHibernateImpl's save() method (my GenericDAO approach is kind of complicated. Track back through the Spring+Hibernate+CloudSQL post first and then see the more in-depth discussion of the original Objectify-based version that was adapted from David Chandler):

This GenericDAO work is getting me more and more into Java Reflection. It's pretty dang cool! But I'm not completely confident in this approach just yet. We'll see if any other odd use cases popup before we call this a win.

What do you think? See any problems down the road?

Spring + Hibernate on Google App Engine's new Cloud SQL!

Full project code for a Spring + Hibernate version of Google's "guestbook" sample webapp that accesses Cloud SQL!

Download the project code!

UPDATE: See the new version of my GenericDAO's save() method to avoid Session conflicts due to lazily-loaded member objects.

I was super-thrilled that Google approved my application to be part of the Cloud SQL limited preview. In gratitude I wanted to share the sample code that I used to do my proof-of-concept tests for accessing a SQL DB in App Engine.

I'm accustomed to using Hibernate with annotations for my ORM layer and was excited to be able to go back to that world in App Engine. But annotations won't work with App Engine as long as they limit the JPA spec to v1.x (I tried and failed to get around the limitation). So that means we're back to using Hibernate Tools and reveng.xml to generate the Hibernate *.hbm.xml mapping files. It's a pain and it feels like it's 2003 again, but whatever. Once you dust off that part of your memory it's not so bad.

BitmapButton: A custom-skinnable ActionScript button

Source code for a simple, skinnable ActionScript button class that includes a really handy arbitrary data field.

Now that I'm a whole 20 days into learning how to code in Flash's ActionScript language, I'm getting pretty dang good.

Yesterday I needed to create a button for the Flash grading app, but I didn't want to use the default fl.controls.Button class that resembles standard HTML form buttons.

I wanted to be able to skin the button with my own Bitmaps that I would generate in Flash. I figured that the easiest way to do this would be to build my own custom button class. So I did:

Progress update: 3,605 lines of code so far

Lines of code don't really tell you much of anything; it's all about quality over quantity and good programmers tend to code more concisely and efficiently than novice coders. But it is fun to check the numbers from time to time. Here's where I'm at thus far:

3,605 lines of actual code. That's pretty solid, especially considering that I've put a lot in place to avoid repetitive "boilerplate" code. Each Java file averages only about 50 lines of code; that's pretty light and efficient--the way I like it!

Code "comment" lines are super-important; that's where the developer explains to other developers (or him/herself) what the code is doing. Comments should count toward the total.

And even "blank" lines are important, otherwise all of your code is jammed together and is a lot less readable. I program like I write essays--with distinct paragraphs where each paragraph contains its own complete thought. Whitespace matters.

So if we add back in the comments and the blank lines, we get 5,630 lines!

Whew! Toldya I was busy!

How to programatically upload a file to Google Docs when you can't create a java.io.File in App Engine

Google's Data API (gdata) makes accessing their suite of tools and sites a lot easier than making direct HTTP calls to the Web-based service interfaces. But there don't seem to be too many code snippets out there--or at least not enough that do what I want to do.

I have a document that was uploaded through a Servlet and is now sitting in a byte[] array. Now I want to upload that file to my app's Google Docs account. Here's what Google's documentation offers:


Fairly straightforward, but there's one catch: The setFile() method is expecting a java.io.File. Normally that isn't a problem, but if you're developing in Google App Engine (GAE), then you know that you can't create filesystem files.

Digging through the DocumentListEntry code (thank you, Google, for making gdata open source!!) I was able to figure out how to avoid this File creation entirely:

Elegant coding with Objectify, Java generics, and interfaces

David Chandler posted code for an ObjectifyGenericDao on his blog. It takes care of all of the typical Objectify datastore operations you might need to perform on any kind of entity type.

It works great as-is. Here's a code snippet of the generic DAO in action:

Pretty simple. You define the generic type as you directly instantiate the DAO and you get instant access to ObjectifyGenericDao's suite of basic datastore operations.

But directly instantiating any service layer should set off an alarm in your head: "Wait! I'm supposed to code to interfaces!"

Google App Engine update: OAuth2 access into Google Storage!

This isn't the biggest accomplishment ever, but it is a necessary step for what we want to do.

I still don't like that I have to use three-legged OAuth2 authentication from my company's Google App Engine site into my company's Google Storage cloud storage account. I should be able to trivially directly connect them (tighter Google Storage integration is high on the GAE team's to-do list).

But instead I'm forced to use three-legged authentication which, by design, protects my Google Storage username and password from my GAE app. Call me crazy, but I'd much rather just store my Google Storage account info in a protected properties file and let my GAE app log in directly. I don't need the level of indirection/protection that three-legged authentication provides.

In fact, it makes me worry more. Instead of a permanent link between my app and my cloud storage, I have a refresh token that I use to ask Google Storage for permission to do anything. If anything goes awry with that refresh token, my app can no longer access its own cloud storage provider. That's a problem. And resetting that connection requires manual intervention on my part--I have to re-initiate the authorization process by hand and approve the connection, then feed the resulting access code to my GAE app so it can then get a new refresh token from Google Storage.

Sound complicated? It is. And for direct internal access, it's way too cumbersome. I'm going to have to build an interface to streamline the process of re-authorizing my site with my storage just in case something goes wrong with the current refresh token.

But the good news is that the code is working and I'm able to make authorized requests into Google Storage.

Up next: Upload a file from a Web browser and push it into the cloud.

After lunch. Maybe.

Diving into Google App Engine!

As discussed in a previous post, it's become obvious to me that Google App Engine (GAE) is really the only viable choice, given how I feel about server uptime, config, backups, disaster recovery, etc.

GAE's cutesy logo

So this is Day One of figuring out this weird, sheltered, somewhat limited environment that is GAE. I won't be posting daily updates, but I will post major progress points. This blog is as much a public information portal as it is a diary for me to remember how this company got itself off the ground (well, let's hope it does!).

And this will definitely prove to be a major piece of the puzzle; the odds of this company's success ride squarely on my ability to hitch ourselves to that silver and blue engine.

Here we go!

Source code control on the cheap: Assembla

A solitary developer at a tiny startup can use Drobox as a poor man's source code control system. In fact, that's what I started with to use as a just-in-case code backup. It's also super-convenient for syncing work between multiple computers (see my extended post on Dropbox). Not so great for team-based coding though.

Pretty soon though you'll want to incorporate a real source code control system--CVS, SVN, take your pick.

A Big Decision: Use a traditional Web host or commit to Google App Engine (GAE)?

Oh tradeoffs, why must you torture us so?

 
The dilemma:
One of the hundreds of things on my to-do list is to figure out Web hosting. For what we want to do a Virtual Private Server (VPS) or cloud VPS seems like the best option. A standard Tomcat host environment won't work because our current designs rely on having OpenOffice installed as a service that we can make calls into. A VPS solution essentially gives us a fresh server that may or may not even have an OS installed on it yet and we can install and configure it however we like.

A VPS setup gives us that freedom and a cloud-based VPS gives us scalable horsepower. Awesome.

But what happens when the server instance crashes? What happens when the database crashes? Presumably a VPS (and certainly the cloud variety) would have redundant-enough distributed/RAID data storage, but what happens if we do suffer data loss?

Any half-decent dot-com needs to have recovery strategies in place to handle any possible calamity, regardless of how unlikely they are.

And here's our first problem: I'm not much of a hardware guy. I really do NOT want to muck through all of this.

But even if we had the world's greatest server admin, we would merely be in good position to recover from disasters when they happen; no one--no matter how good--can make server disasters disappear altogether.

And a VPS--or any Web host or colo option--requires too much machine/environment maintenance and disaster recovery.

The solution(?): Google App Engine
The best way to offload these concerns seems to be to run your code off of Google App Engine (GAE) instead of a traditional Web host/Web server.

Two-legged authentication via OAuth2 to Google Storage: Easier than it seems

First of all, if you're reading this post on purpose and have any clue what the title actually means, then you're in the right place.

If you aren't a tech geek, give up now and make better use of the next five minutes of your life.

Corporate email accounts via Google Apps

I set up the EssayTagger.com email accounts today through Google Apps. I already use Gmail for my personal mail and my school district transitioned to Google Apps-hosted services last semester.

As far as I'm concerned, Google Apps for your organization is a no-brainer. My overall philosophy for EssayTagger.com is to leverage as much outside expertise as possible--which is another way of saying: offload as much as possible, wherever practical. Yes, I know how to do a whole lot of the tech stuff on my own, but there's a lot of it that I shouldn't be doing on my own because other people can do it better and more cost-effectively.

Domain name purchased!

It's hard coming up with a good domain name these days. We decided that our domain name should:

• Describe the service.
• Be professional and appropriate for both teachers and students.
• (no goofy, random names--like google, moodle, GoDaddy, etc.)
• (and nothing that suggests teacher laziness: FasterGrading, EasyGrading, etc)
• And yet still be at least a little memorable.
• Be reasonably easy to spell since students may be typing in the URL.
• Last, but not least: be available for purchase!

In the early phases when I was first playing with this idea I grabbed WritingBrilliance.com. But I think you'll agree that it's too long and too prone to misspellings.

After a lot of brainstorming and a lot of registrar lookups we ended up with: EssayTagger.com.

It fits all of the criteria reasonably well and, as you'll see in the coming weeks, it really does describe our service.

I went with Register.com as my registrar. I've used them for most of my other sites and I've been happy with them. I'm put off by the flashier registrars and their marketing campaigns.