Evaluation Anxiety? You’re Not Alone
Three lessons for taking ownership of your evaluation efforts.
by RJ Bee and Kate Pazoles
The word “evaluation” can strike fear into the hearts of communicators. On one hand, we all want to measure what works and what doesn’t. On the other hand, we communicators are often held responsible for unrealistic outcomes. And if we don’t ask the right questions or have access to the right information, we may be set up to fail.
So how can we, as communicators, gain ownership of evaluation so that we can hold our teams and ourselves to ambitious but realistic goals? At our ComNet16 preconference session, “Road Map to Impact,” we discussed several lessons about communications evaluation drawn from our firm’s approach and client work.
- Vanity metrics do not an evaluation strategy make. This should come as no surprise, but many organizations still ask communicators to look at superficial statistics—social media engagement, web traffic and the like—to evaluate communications efforts. While these numbers may show an uptick in Facebook likes or Twitter followers, most of the time we don’t know if they reflect progress toward your organization’s actual goals.
To track impact, we need to take several steps back and ask ourselves basic questions:
- What is our goal?
- Who are our audiences?
- What is our objective?
- With each audience, do we want to raise their awareness, change their attitudes, or motivate them to take a specific action on a given issue?
Once we have our goal, audiences and communications objectives, we can begin to think about metrics for evaluation. Here are a few slides from our workshop deck to get you started.
- You can only measure what you can do. Huge outcomes and impact—changing the way people think about health, for example—are aspirational and inspiring. But as communicators, we can only measure what we, along with our grantees or allies, are actually doing.
In our preconference session, we introduced three elements of effective communication: 1) message consistency and effectiveness, 2) strategic storytelling and content creation, and 3) outreach and audience engagement. Thinking about these elements alongside your communications objectives (awareness, attitudes, action) will help you decide what to measure.
If you are hoping to start changing the conversation, for example, a first step is getting your team on the same page about how you talk about the issue. These questions can help with evaluating that task: How many people have you briefed on your message? How many staff/grantees have you trained on the message? How many people know how to tell stories about your issue?
- Evaluation can be overwhelming. Start small, learn—and evolve. A common reaction to our workshop at ComNet was, “Wow, that’s a lot!” It is. But you can take it one step at a time, learn and evolve. Start by training your staff and some select grantees on how to create stories that reinforce your narrative, with a specific call to action. You can use Storytelling for Good, a free storytelling platform for social-impact organizations that Hattaway developed in collaboration with the Rockefeller Foundation.
You could also conduct a storytelling workshop with a few program staff and grantees, and measure how many stories made it into public communication, such as speeches, presentations, web copy and social media outreach after two months. If none have, regroup with the team and ask what they need: more help on story development, more training on the tools, etc. Through testing, you can evolve and find the right metrics for your organization.
As communicators, we are all asked to measure our effectiveness—a reasonable and strategic undertaking. To do so, we have to ask the right questions, choose realistic metrics, and be willing to learn and evolve. We have to consider communications inputs and actions, as well as big picture outcomes.