Is Systems Change the Answer? Depends on the Question

By Sarah Mason, izzy Thornton, Hannah Loomis

The non-profit, foundation, and evaluation worlds are abuzz with talk of systems change. 

Lasting social change, research tells us, requires change at the systems level. Changing a system, in turn, involves understanding the “root causes” of a problem and pursuing transformation in the structures, customs, mindsets, policies, etc. that hold these root causes in place. 

Yet when there are finite dollars to invest in social programs, how does a funder choose between helping people survive in a less-than-ideal system, and investing its funds in efforts to create a new (and perhaps better-in-the-long-run) system?  

At the Center for Research Evaluation (CERE), we recently explored this tension in an evaluation of systems change efforts in Mississippi’s early learning landscape. 

Mississippi’s early childhood education (ECE) system is complex

Across the US, many states use Quality Rating Improvement Systems (QRIS) to assess and report on the quality of childcare centers. These ratings help parents make decisions about local childcare services—and offer agencies a tool to guide accountability efforts and continuous improvements. 

In 2016, Mississippi disbanded its own early childhood QRIS. This decision was guided in part by widespread concerns about racial injustice and prejudice in its assessments of ECE providers. The elimination of Mississippi’s QRIS was an attempt to bring about systems change by altering the underlying policies and structures that held the “old” system in place.

But systems change is multifaceted.

Disarray

The resulting void destabilized the ECE field. It caused confusion among providers and led to increased competition among agencies and advocates vying to design a replacement system—which later efforts were unsuccessful.

Introducing: The Graduate Center for the Study of Early Learning 

Founded in 2015, the Graduate Center for the Study of Early Learning (GCSEL) at the University of Mississippi aims to shape and improve the ECE system across the state of Mississippi. 

GCSEL, under the leadership of Dr. Cathy Grace, provides a wide range of supports to Mississippi-based ECE providers, including professional development, public outreach about evidence-based practices in ECE, and direct support for ECE providers. 

The evaluation 

In 2022, CERE led an evaluation of GCSEL’s Kellogg-funded work, exploring the ways in which GCSEL’s work might contribute to change in Mississippi’s ECE system. 

Using Coffman’s Framework for Evaluating Systems Initiatives we undertook a series of wide-ranging key informant interviews, and a review of publicly available documents and participant surveys. We wondered: Could any shifts in the ECE system’s components, infrastructure, connections, context, or scale be attributed to GCSEL?

And we learned…much like systems change itself, evaluating systems change is also complex. 

Filling gaps, finding tensions 

We’d hoped for a simple, clear-cut answer. Instead, our work revealed a tension between the need to offer stability to those trying to survive within an imperfect system and an overall desire to pursue change towards an improved system. 

While GCSEL’s work was seen as essential to helping ECE providers survive within the current system, the time and effort required to do so meant there was less capacity for changing the system.

More specifically, GCSEL is one of the few organizations that offers direct service provision, in contrast to the many Mississippi ECE advocacy- and community-focused organizations. Because of its different nature, GCSEL fills critical gaps in the ECE system and adds to its stability as the system evolves. But GCSEL is not necessarily changing the ECE system. 

As a direct service provider, GCSEL is uniquely placed to foster trust across multiple levels of the ECE system. This trust building is critical as the overall field reorganizes in response to concerns about the QRIS, its removal and the absence of a new state system. 

We recommended that GCSEL continue its valuable provider-level work, while also exploring its role in the ECE systems-level work.

Ongoing debates

All of this led to a now-enduring debate among evaluators in our office: what are different ways to manage the tension between helping people survive in a less-than-ideal system, and investing funds in efforts to create a new (and perhaps better-in-the-long-run) system?  

What do you think?

Read our evaluation report here.