Use these three ingredients in your elevator pitch to build understanding of evaluation
Evaluators frequently say that no one understands what we do.
In the words of Jean King: “It is awkward to be a program evaluator. On a personal level, people in social settings rarely seem to grasp the one-sentence summary of what you do.”
A research study
A few years ago my colleague Ashley Hunt and I ran a research study in which we asked 40+ evaluators how they describe evaluation—to friends, clients—even complete strangers. We wanted to find out what evaluators were saying that seemed to make our work so difficult to grasp.
Our core finding?
Evaluators spend a lot of time explaining why someone might do evaluation—before they explain what it actually is. We also gloss over what we evaluate, assuming that people know what “social programs” are (and what we evaluate).
The evaluators we spoke to made a number of suggestions about strategies that might help those who are new to evaluation grasp what it is that we do.
Another research study
We’ve put those theories to the test—running an online experiment with a representative sample of the US population to test (1) what the general public knows about evaluation, and (2) how we can talk about evaluation in a way that boosts understanding.
Four hundred and seventy-two Americans graciously took part in our experiment, answering questions about what evaluation is, what evaluators do, and when someone might do evaluation.
These participants were asked to imagine they were at a dinner party; and that their table neighbor (Sam), introduced themselves as a program evaluator. Participants were given different explanations of evaluation—and we coded their explanations about what evaluation might be.
Here’s what we found:
#1 If you think people don’t understand what evaluation is: you’re probably right!
When participants weren’t given any explanation at all (i.e. when Sam just said “I’m a program evaluator”), 74% had no idea what that meant.
#2 People think that means we’re computer programmers.
The biggest misconception was that a “program evaluator” is a computer programmer. Thirty percent of people in the “no explanation” group described evaluation as something to do with computer programming.
#3 Use an example.
Descriptions of evaluation with examples that explain what social programs are were better understood. This is the single most useful thing you can do to build understanding: give an example!
#4 If you’re in a pickle, use this.
“You know how government agencies and not-for profit organizations spend money on social programs like school lunch programs or substance abuse programs?
Well, I gather and analyze data about those programs to find out if they do any good or not.
The hope is that we’ll be able to provide data and feedback that will help decision makers improve the programs’ performance and their value to society.”
The three key ingredients
At a structural level this includes:
- An example of something that can be evaluated (relevant to the person you’re speaking with)
- A high-level explanation of what, tangibly, we might do
- A high-level description of what the world gets out of us.
Using this structure even helped me explain evaluation to my eight-year-old niece! So much so that she asked what we do if we find out that programs aren’t helping people or doing good.
Give these three ingredients a go, and make sure to share your experiences with! Take a moment to share your thoughts via email at masonsk@olemiss.edu!
