The Logic of the Contextual Guidance API

In my last post, I provided background on this project, and why it's useful to be able to provide users with help that fits their experience with the application. I'd like to shift now to looking at the business logic that would make this happen.

I've already covered the method signature for this very simple API, but how does the API know to return a value of "beginner," "intermediate," or "advanced,"? To make this determination, we have to think beyond a simple, reductive value. I could certainly write something simple to say that "If someone has logged in to the app more than 20 times, they've got to be an intermediate user." That's easy, but misleading.

People tend to forget things. I work in academia, and work here is highly cyclical. There's a flurry of activity at the start of each academic term, and the place is a ghost town in the months of May and June (well, faculty and students can't be found. Staff is working as hard as ever). Faculty and TAs working on courses tend to do a lot of work in a concentrated period of time, then walk away and don't look back until the next time they work on a course. I think that many users are the same way: they get what they need to get done ASAP, then walk away without a look back until the next time they need to use the app. There are exceptions, of course (email clients, social networking and financial sites come to mind). Most of the time, though, a user will need to use your application for a little bit and then come back to it only when needed again. In this away time, users forget a whole lot of stuff. Humans are creatures of habit and need reinforcement to retain processes. If someone uses your application for two weeks and then doesn't use it again for eight months, they're going to have forgotten how to do things. They won't be starting entirely from scratch, as, ideally, bits and pieces of how the application works will come back to them, but they'll definitely need a refresher.

The contextual guidance API needs to take simple forgetfulness into account. We can't just say "If someone has logged in to the app more than 20 times, they've got to be an intermediate user." We have to take into account time — specifically, the time between their last visit and their current one. But we can't stop there. What if they visited our app a lot nine months ago, but then came back to it for the first time yesterday. Are they still an intermediate user? Have they remembered all the basic stuff that they needed to remember? Probably not. So we need to look a bit at their history and factor that into our decision.

One possible version of the business logic would work as follows:

1. Store a record of a request for a page in the app
   • Insert if it's a new user
   • Update if it's an existing user
2. Look at two stored values: lastVisit and totalVisits
3. Run a simple decision tree based on these values (see below for discussion)
4. Return "beginner," "intermediate," or "advanced," as appropriate.

So what would the decision tree look like? There are a number of ways of approaching this, but I plan to say something like this:

• If lastVisit < 14 days and totalVisits < 10 ===> new
• If lastVisit < 14 days and totalVisits between 10 and 25 ===> intermediate
• If lastVisit < 14 days and totalVisits > 50 ===> new
• If lastVisit > 14 days and totalVisits < 50 ===> intermediate
• If lastVisit > 30 days and totalVisits < 75 ===> intermediate
• If lastVisit > 180 days ===> new

There are lots of choices you can make here in terms of the "rules" for determining someone's experience with an application. But we're only making decisions on two pieces of data. It's useful, as Edward Tufte might say, to be multivariate. We need to factor in previous visits to the application and not just look at the lastVisit value. What if someone's lastVisit was yesterday, their totalVisits was 64, but the previous visit before yesterday was 9 months ago? Should they really still be labeled an "intermediate" user even though they haven't been using the app for more than a single day in the past nine months?

Beyond examining simple date values, we could also look at the amount of time the user spent working on the app in the past few days. "The amount of time" is always a highly flawed piece of data on the Web, as a user can sit on a Web page for hours but not do anything during that time. Instead of tracking hours, minutes, and seconds, we can look at the number (or even kind) of requests a user is making in our application and factor that in. You could even go so far as to weight certain, difficult tasks more than others. I'm not going to go quite that far, but it's another option. Looking at all the requests a user made and not just their number of logins can help determine if they're really using the app, or just browsing.

What about in-house staff or "experts"? Should they be a part of this decision tree, or should they always receive the "advanced" view? That's probably outside of the scope of this simple API, as the application that's calling this API can decide if the user is an expert and always return "advanced," if appropriate.

So that's a look at the business logic of the contextual guidance API. Next up will be a look at the (simple) database table structure, and some sample code in ColdFusion, my development language of choice.