Personal Interwoven Narratives

By Shannon Sharp

Have you ever felt like conventional evaluation methods aren’t enough to capture participants’ personal lived experiences? Evaluators sometimes seek to understand perceptions about sensitive topics. They sometimes struggle to engage marginalized participants whose voices are often not sought out or heard. For these needs, traditional surveys, interviews, focus groups, etc. fall short.

Enter… Personal Interwoven Narratives (or PIN). PIN was born out of necessity. It started with a project designed to help traditionally underrepresented undergraduate students persist in STEM careers. Our evaluation’s conceptual model included program goals not only focusing on academic and career outcomes, but also on outcomes related to sense of belonging, various types of support (or lack thereof), identity, and quality of mentoring relationships, just to name a few. Understanding the need for a data-collection method that could encourage and embrace the recounting of personal stories around these sensitive topics, CERE’s Center Director (shout out to Sarah Mason!) thought back on something she’d had yet to pull out of her evaluation toolbox. She recalled learning about oral histories and thought this might be the perfect project to implement them.

In a nutshell oral histories sit on the open-ended side of the interview continuum. They elicit stories rather than statements and are often employed to gain a deep understanding of a shared historical experience (such as 9/11 or the Civil Rights Movement). You may be wondering at this point why I’m talking about oral histories when the focus of this blog is Personal Interwoven Narratives. Let me explain.

While oral history’s person-centered focus, storytelling format, and attentive interviewer-‘narrator’ (what oral history calls the interviewee) relationship seemed perfect for our project, as we developed, conducted and analyze our oral histories, our team realized that we had made certain adaptations along the way to meet our evaluation needs. I’d given several presentations and even published on our use of oral history for this project but always felt a little uncomfortable calling it that. Would oral historians feel that we stayed true to their method? I considered all the adaptations we made (see below) and decided that while our method definitely encouraged oral storytelling, it was different enough from oral history to need its own name.

Enter (again)… Personal Interwoven Narratives (or PIN), named as such because the rich narratives they elicit are both personal (individual) and interwoven (connected). So how do PINs differ from oral histories to help meet evaluation-specific needs?

• Shared experience: The shared experience for an oral history is an historical event while PINs typically focus on shared experience(s) related to the program or a program component.
• Prompts: Oral historians typically utilize a guide or outline of themes with open-ended narrative and reflexive prompts. Evaluators using PIN develop protocols of open-ended narrative and reflexive prompts but also include follow-up questions that ensure evaluation questions are adequately answered even if they don’t come up organically during initial storytelling.
• Length/frequency: Oral histories typically take as long as the narrator needs/wants to fully share their experiences and often include multiple sittings. Evaluators, however, must consider budget and time (theirs and the participants’) as well as what is needed to fully answer the evaluation questions. As such, PINs would typically involve less time over fewer (often only one per participant) sittings.
• Analysis: Oral history analysis considers the interview content in relation to the historical event and tend to be analyzed as the only or primary data collected. PIN analysis considers the interview content in relation to evaluation questions (or other evaluative goals), and we recommend that these data are used to complement quantitative data (such as from surveys).

Has this piqued your interest? Are you starting to think about projects that might benefit from Personal Interwoven Narratives? Here are some best practices for evaluators interested in incorporating PIN into their evaluations:

• Consider the program and evaluation’s focus: PINs are not appropriate for all types of evaluations, but are especially valuable for ones that seek to capture experiences at the intersection of identities, such as race or ethnicity, gender, religion, sexual preference, etc. Evaluation approaches that might benefit from PIN include (but are not limited to) culturally responsive, diversity focus, inclusive, personalizing, and social justice focus.
• Train and consult with an experienced evaluator: As with any data-collection method, it’s important to understand when and how to use the method. PIN is unique enough that it is important to learn about its nuances and how to incorporate it to best meet your needs while also making participants feel comfortable and heard. PINs, like oral histories, are different from conventional interviews. We trained with an oral historian to understand the method and its value, then consulted with her throughout the process to ensure our protocols, processes and analyses were as true to the oral history method as possible. Now that we’ve come to recognize that a new method emerged (PIN), we recommend similar training and consultation with an evaluator experienced with using PIN.
• Reflect on your own assumptions and biases: A best practice for evaluators in general, reflection is especially important when eliciting personal stories from individuals with backgrounds different from your own—to understand and map your individual positionality. This will allow you to reflect on your personal assumptions and biases and how they could impact how you approach the interview and analysis.
• Allow narrators to talk but also follow up: It’s important to allow narrators to tell their stories—that’s the whole point of using a storytelling method like PIN; however, it may be necessary to incorporate follow-up questions that specifically address evaluation areas the narrator’s stories do not adequately cover.
• Use PIN as part of a mixed-methods approach: While evaluators seek validity in data-collection measures, PIN data should be considered valid to the extent that they are how the narrator interprets their own experiences. Using PIN alongside quantitative data allows you to triangulate findings and verify trends observed in narrators’ stories while also painting a fuller picture of their experiences than would be available from qualitative or quantitative data alone.
• Consider time and cost: Because of the richness of the stories elicited through PINs, they take (much) longer to conduct and analyze. Since time is money, evaluators should consider their timeline and budget before incorporating PIN into their evaluation plans.