Batman and Evaluation

By Joey Rutherford

I recently went to the theatre to watch the most recent iteration of Batman. In this version, Robert Pattinson stars as the newest Batman. As a child of the 1980s, Michael Keaton’s performance as Batman is my gold standard. As a result, I have always evaluated George Clooney’s, Val Kilmer’s, Ben Affleck’s and Christian Bale’s performances against Michael Keaton’s rendition. I have nothing against these actors, but they never met my standard of performance for Batman.

Before settling myself down to watch the newest movie, I knew I came with a bias against Pattinson’s version of the Caped Crusader. However, as the movie started, I realized that I would not enjoy this version of Batman if I did not tune out my overwhelmingly favorable bias towards Keaton’s version.

I can now confidently say Robert Pattinson’s portrayal as the hero of Gotham City was phenomenal, and I may not have been able to appreciate that if I had not addressed my biases and tried to view Pattinson’s work as an independent piece.

Batman and bias—and evaluation

Now, what does this have to do with evaluation? Well, quite a lot. Consider my bias towards Michael Keaton as “the best” Batman. My bias was so strong that I never gave the other actors a chance. Evaluators face this dilemma frequently.

For example, we have entered a time in society that there are more virtual training options than ever before. Whether it’s professional development services for teachers or telehealth services for college students, I have a bias for in-person services. I prefer in-person teaching, professional development, counseling services and school interventions. This is a bias that I bring to the table when CERE evaluates virtual programs. If I do not acknowledge the bias, there is risk of evaluation design, data collection and findings being influenced by my views. Yet, the whole purpose of external evaluation is to evaluate programs openly and honestly. Virtual programs can be a force for good just as well as in-person programs. To do my job well, I have to check my biases, acknowledge why I carry such biases and actively work to mitigate them.

Ensuring that your biases do not enter into your responsibilities as an evaluator is not a simple task. However, it also doesn’t have to be complicated. Here are a few way that you can check your biases to ensure that the do not negative affect the evaluation:

• At the beginning of the evaluation journal your thoughts about the program or intervention that you are responsible for evaluating
• Explicitly discuss these perspectives at the beginning of an evaluation
• Engage in introspection: Take a self-reflective test to determine you biases
• Seek out a wider perspective: Work to gain an understanding of the people, program, or intervention being evaluated. How does the program align with (or diverge from) your prior experiences and perspectives?
• Ask your colleagues to let you know if your biases begin to creep into your evaluation.

At the end of day, more than one person can play the Caped Crusader, just as virtual and in-person services can both make a difference in the world.