Judgment during the time of COVID-19

By Sarah Mason

In the months (and years) to come, social and news media will be awash with critiques of local, national and global responses to the COVID-19 crisis.

‘Too slow,’ ‘too reckless,’ ‘just right’—whatever the perspective, we’ll be inundated by judgments about the nature and quality of these responses.

In other words, we’ll be inundated by evaluations of the global response to the COVID-19 pandemic. Whether or not the world knows it, people across the globe are about to engage in a very public evaluation process.

What role will evaluators play in these discussions?

A few weeks ago, Australia’s Tony Blakely—epidemiologist and professor at the University of Melbourne—argued for public deliberations about ‘what to do next’ in formulating an Australian response.

In his Sydney Morning Herald article, Blakely described the professions that would need to be present in those deliberations.

“We need to [also] hear from a wide range of experts to help us deliberate. Philosophers, ethicists, economists, public health experts and epidemiologists—to name just a few.”

Unsurprisingly, evaluators were absent from that list.

This should not come as a surprise. As our reserach has shown, we are a little-known profession that has difficulty articulating what we do, and even more difficulty articulating—let alone persuading the public—of our value.

But we have a chance to remedy this.

Our role

Evaluators have a key role to play in helping the global community formulate judgments about responses to the current crisis. I would like to argue that evaluators should be present in public—and private—discussions about ‘what counts’ as a good response to the pandemic.

While many of us may not have extensive technical (public health) expertise, we know how to work with communities to formulate criteria to assess policies and processes. We know how to facilitate conversations about ‘what counts as good’ and how to collect data to measure performance on those criteria.

I am, of course, talking about the logic of evaluation: a logic that warns against relying on one-dimensional criteria for success, but instead takes on the full range of community/societal needs (Economics! Mental health services! Physical security!) and values when determining merit, worth or significance.

Without a comprehensive set of defensible criteria, we run the risk of forming judgments in a way that is haphazard and driven only by emotion or fear. In times of heightened fear and emotion, there is more—not less—need for a systematic approach to evaluative judgment.

How can we do this?

Over the coming weeks, we will be launching a study to help collate public perspectives on criteria for assessing responses to the current crisis. This approach draws on previous work by Elena Harman and Tarek Azzam, which focuses on techniques for incorporating public values into evaluative criteria.

To that end, we’ll be asking members of the public to help us brainstorm a set of criteria and standards for evaluating COVID-19 responses. Our hope is that this will help us put together a usable tool—a rubric, of sorts—that could be used to spark discussions about the full range of criteria that might be needed to assess responses to a crisis like this.

Get in touch

We’re also looking for collaborators across the globe, individuals who want to work with us on the next steps in this journey. Reach out (masonsk@olemiss.edu) if you’d like to take part.