Doing Web Evaluations the “Trinity” Way
This post is part of our “tools and tips” series that focuses on the many online options available to bolster your communications activities and ensure they support your overall strategies. In his second column on web evaluations, Louie Herr, a consultant based in Portland, Ore., discusses the “trinity” approach.
Guest Post: Louie Herr
In my first post on this topic, I wrote about how following a methodology called “Trinity,” developed by Avinash Kaushik, author of the leading web analytics and research blog Occam’s Razor, can help you track and monitor your website usage, and also analyze behavior of visitors to make adjustments that will help support your organization’s strategic communications goals. This post provides some tips on how to put Kaushik’s approach to work for you.
The first step in website evaluation is understanding the different kinds of reports you need to properly analyze what visitors are doing, including the pages they are viewing, where they spend most of their time, what they like and don’t and where and how soon they exit from your site, among other things.
These are the most useful kind of reports:
- Behavior reports answer the question, “What do we know about what people are doing on our site?”
- Outcomes reports answer the question, “Is my website doing what we need it to do?”
- Experience reports answer the question, “Why are visitors behaving this way?”
What’s the best way to collect that kind of data? Thankfully, there are a few affordable tools that let you do this:
For collecting data on website behavior, Google Analytics is the gold standard. Even better, Google Analytics is free and easy to install, and that makes it too good to pass up.
As outlined in my previous column, the Trinity approach calls for clickstream analysis more sophisticated than the traditional visits/pageviews/time-on-site reporting more commonly used. Kaushik’s approach drills deeper into data to help you learn how different classes of visitors behave on your site.
4Q Survey is my favorite web evaluation tool. The beauty of 4Q is that it’s a simple pop-up web survey that asks your visitors what Kaushik calls “The Three Greatest Survey Questions Ever:”
1. Based on today’s visit, how would you rate your site experience overall?
2. Which of the following best describes the primary purpose of your visit?
3. Were you able to complete the purpose of your visit today?
4a. (If 3 is yes) What do you value most about the [sitename] website?
4b. (If 3 is no) Please tell us why you were not able to fully complete the purpose of your visit today?
User responses to these questions can be phenomenally useful inputs for redesign discussions.
4Q Survey may be simple, but that’s also its strength. It provides feedback that’s easy to interpret, and unlike some other survey tools, it isn’t disruptive to your users. Because it works as a popup, visitors can opt-out if they don’t want to respond.
The basic 4Q tool is free, and more sophisticated versions are available for a fee. I recommend installing the free version on your website–it’s a simple process–and setting it to popup at a rate that won’t visitors won’t find disruptive. Then, leave it on for as long as it’s useful, and review the data collected as part of a monthly and quarterly web assessment.
There is a range of tools available for creating reports that help you answer the question, “Why are visitors behaving this way?” The tools below are designed to run tests with users to find out what works and doesn’t about your website. I don’t have a favorite, but here are two options:
- UserTesting.com is a fee-based service that’s offers very fast reporting and is considered the easiest, fastest, most-efficient way to how people are likely to spend time on your site.
- Silverback is a souped-up screen-recording application that captures high-quality video of both the participant’s screen and face as well as audio of anything the participant or moderator says or does. Additionally, user inputs like mouse clicks and keystrokes are shown on the screen as well. Silverback (and a methodology) is really all you need to conduct your own usability study.
Doing usability tests requires some planning, and if you haven’t undertaken this kind of work before, I recommend the Handbook of Usability Testing: How to Plan, Design, and Conduct Effective Tests to help you learn how to design user-based studies. I’ve found it to be an excellent top-to-bottom guide to usability testing.
You can always opt to hire a consulting or testing firm to take on this work for you, but it will cost considerably more. If you opt to go this route, I also recommend you prepare ahead of time, and as mentioned, read The Handbook of Usability Testing, or a book like it, and do as much preparation as you are able.
So what do you do with all the data you collect?
What follows is a sample of an evaluation using the Trinity approach:
Let’s say the 4Q Survey results show that many users are dissatisfied with a feature on your website. However, because 4Q only surveys a small sample of users, you don’t know how many actual users feel the same way.
Reviewing clickstream data from Google Analytics you also discover that this feature is the second most-trafficked part of your site. Additionally, people seem to be exiting your site from these pages at a higher rate than elsewhere. That should alert you to a problem.
However, because you don’t know what is causing visitors difficulty and why they’re exiting from that page you need to take a closer look and come up with some possible theories about what’s causing the problem as well as some likely fixes. After your web team implements these fixes you wait for the next round of reports to see if those changes solved the problem. How will you know for sure?
During your next quarterly review, you review Google Analytics data to see if the exit rate for those pages has decreased and look at 4Q Survey to see, among other things, if users’ satisfaction rate has increased. If the results show the fixes have worked, you can be more confident your site is supporting your organization’s communication goals.
A Trinity evaluation is cyclical, reflexive and greater than the sum of its parts. Increased familiarity with these tools (and tools yet to be developed) will enable new, more powerful analysis techniques.
One personal request: Does your organization have a dynamite web evaluation strategy? If so, I’d love to interview you. Email me at firstname.lastname@example.org. Or share in the comment section below. Hope to hear from you!