Introduction
Over 400,000 children and youth in the United States are in the foster care system at any given time (USDHHS, 2020). Consistent across their experiences is significant abuse and/or neglect, and extreme disruption in their lives associated with foster care placement. As a result of these experiences, current and former foster youth experience higher rates of mental illness than the general population necessitating access to high quality mental health assessment and, if appropriate, intervention (Hass & Graydon, 2009; Kerman et al. 2002; Scozzaro & Janikowski, 2015). Despite the need, successive and cumulative barriers create challenges in accessing and continuing care, which result in the majority of current and former foster youth not receiving any mental health services (dosRios et al., 2001; Kerker & Dore, 2006; Unrau & Wells, 2005).
One barrier to services involves the public perception and framing of foster youth’s mental health needs. Research shows that current and former foster youth experience a stigma associated with their placement in care, and that this is exacerbated when one also has mental health needs (Rauktis et al., 2011, Jee et al., 2014; Villagrana et al., 2018). Current and former foster youth share an increased difficulty disclosing and discussing their needs and a decreased likelihood of accessing mental health services (Jee et al., 2014; Villagrana et al., 2018; Ruff & Harrison, 2022). Psychotherapy outcomes identified in research, practice, and policy hold the potential to frame public perceptions and conversations about mental health need among current and former foster youth. As part, outcomes can influence what services are made available, to whom, and for how long. Still, outcomes will and should differ across individuals, circumstance, and time, creating an opportunity and inviting curiosity about what outcomes, or domains of “success,” matter to key stakeholders in child welfare.
For this study, we relied on community-based participatory action research methodologies to learn from key stakeholders in child welfare about outcomes of successful mental health treatment with and for current and former foster youth. Stakeholders were defined as those with personal and/or professional experience in child welfare, and included foster youth, foster parents, biological parents, child welfare workers, child welfare legal and policy professionals, as well as psychotherapists working with child welfare-involved youth and families. The aim was to learn from stakeholders about what success in therapy looks like to them with the longer-range goal of supporting client-centered services. While there are differing opinions about how to refer to those who have experienced foster care - for the purposes of this paper, we opted to use the language preferred by our community advisory board members: current and former foster youth.
Literature on Therapeutic Outcomes
Across the literature, there is heterogeneity in the outcomes identified for current and former foster youth in mental health treatment (Bergström et al., 2020; Van Andel et al., 2014). This variation in outcomes is necessary to capture the diversity among foster youth in and across age, identity and experiences leading to care and while in care. Still, research, practice and policy often require that needs be summarized and categorized to represent common needs of the majority. It is not clear as to whether or not these representations align with stakeholders’ goals and hopes for mental health intervention.
One common outcome in mental health intervention research with foster youth focuses on reductions in psychopathology and diagnostic symptoms (e.g., Burns et al., 1999; Taussig et al., 2019; Weiner et al., 2009; Kessler et al., 2008). Diagnoses vary, but there is consistent attention to post-traumatic stress disorder (e.g., Ahuna et al., 2021), depression (e.g., Stoner et al., 2015), anxiety (e.g., Chu & Kendall, 2004), substance use (e.g., Braciszewski et al., 2015), and/or diagnosis-related outcomes such as impairment in social-emotional functioning (e.g., Bellamy et al., 2010; Jacobsen et al., 2013) or internalizing and externalizing symptoms (e.g., Bergström et al., 2020; Mersky et al., 2016). This research represents diagnostic need and supports the development of aligned services, but it may not fully represent the larger conceptualization of stakeholders’ concerns. It is possible that current and former foster youth desire non-diagnostic outcomes in therapy (e.g., improved relationships, increased independence) and/or that they do not meet clinical thresholds at the start of therapy, but still require services. Furthermore, in some instances, it is important to consider that a fear of diagnosis itself serves as a barrier to accessing psychotherapy (Ruff & Harrison, 2022).
Alternatively, or in conjunction, an additional outcome commonly identified in intervention research with foster youth is that of decreased or eliminated system involvement. For example, success is sometimes operationalized by decreases in juvenile justice involvement (e.g., Pullmann et al., 2006), incarceration (e.g., Yamat, 2020), and /or recidivism, truancy and substance misuse (e.g., Robst et al., 2011). This outcome domain extends to child welfare involvement and includes increased placement stability (e.g., Frederico et al., 2017; Koob & Love, 2010), and/or decreased number of foster home placements (Schoenwald, 2000) and reentries into foster care (Fisher et al., 2005). For transition-aged foster youth, outcomes often include adult functioning (e.g., Anctil et al., 2007; Courtney & Dworsky, 2006; Miller et al., 2017) focused on educational attainment (e.g., Batsche et al., 2014; Barth et al., 2007; Greeson et al., 2015), employment (e.g., Hook & Courtney, 2011; Zinn & Courtney, 2017; Kim et al., 2019), and/or housing (Berzin et al., 2011; Greeno et al., 2018; Kim et al., 2019). Involvement in these systems supports a broader, long-term goal orientation, but does so at the risk of assuming that individuals in treatment are the appropriate focus of interventions for these societal outcomes. Here, attention could broaden from solely upon the individual to larger systems of inequality and bias that affect access to housing, education, and employment (Miller et al., 2013).
Also at the system-level, intervention research often gauges success through reduction in the use of subsequent mental health services, operationalized by decreased inpatient hospitalization (e.g., Hansen et al., 2011), emergency room visits, and/or psychotropic medication use (e.g., Hilt et al., 2015, Taussig et al., 2019). However, using markers associated with help seeking requires additional understanding. While it is understandable to assume a decreased reliance on services may indicate a reduction in need, it is also plausible that individuals still have needs, but do not use services because of stigma or barriers that prevent access to lower levels of care. It is also possible that seeking additional and/or appropriate services is a successful outcome. Consequently, a decrease in inpatient hospitalization or medication use may risk an incomplete assessment of individual mental health needs and/or progress.
In summary, despite the lack of a full consensus as to what indicates success in treatment, there are common success indicators identified by researchers and practitioners. In order to deepen our understanding and expand on existing research, we interviewed key stakeholders in child welfare about what they think are the most important indicators of successful psychotherapy with foster youth. We utilized community-based participatory action research to learn more about key stakeholder perceptions of psychotherapy outcomes and goals. The goal was to ensure the outcomes we identified and measured in our future research studies maximally aligned with and represented foster youth clients’ wants and needs in psychotherapy. Therefore, we aimed to capture what occurs during and after psychotherapy that is perceived by current and former foster youth to be successful.
Methods
Study Context
This study is phase one of a larger evaluation of A Home Within, a national nonprofit organization focused on the emotional wellbeing of current and former foster youth. Specifically, A Home Within offers pro bono mental health services to current and former foster youth (Heineman & Ehrensaft, 2006). Prior to evaluating the potential benefits of services through a randomized-controlled trial (phase two), we used community-based participatory action research methodology to conduct a mental health needs and service assessment with key stakeholders in the child welfare community. We interviewed key stakeholders – both associated with A Home Within, and not – to learn about mental health needs and resources for current and former foster youth as well as key indicators of mental health and well-being.
Researcher Positionality
The research team consists of two principal investigators who served as the first and second authors of this study. They are both licensed couple and family therapists with postsecondary degrees as well as a shared passion for mitigating barriers to accessing mental healthcare, especially for youth and young adults who have had a history of child welfare involvement. The first author has sixteen years of clinical experience and twelve years of child welfare research experience. The second author has over twenty years of experience conducting qualitative and mixed methods studies as well as providing direct clinical service and teaching experience centered on improving access to high quality mental healthcare for underserved populations. They have both been volunteers with A Home Within at various points in their careers and took several steps to mitigate any potential research bias, such as ensuring that all research design, data collection and analysis was conducted independently from A Home Within. By conducting the research through their affiliated educational institutions and organizations, they were able to maintain a boundary between A Home Within programming and the research.
The third author has six years of mixed-methods research experience within the field of education focused on underserved populations and has no affiliation with A Home Within. The fourth author grew up in foster care and has over ten years of experience advocating for foster youth in academic, social, and cultural settings. The fourth author has served on the Community Advisory Board for the current research program. The fifth author has five years of research experience in foster care, no affiliation with A Home Within, and extensive experience working with immigration.
Procedures
After receiving approval from the Institutional Review Board for the Protection of Human Subjects, the principal investigators used community mapping to identify key stakeholder groups for participation in the study. This community mapping process ensured that the perspectives and unique vantage points of each stakeholder group were represented and was an iterative process occurring simultaneously with initial data collection. This key stakeholder identification process provided context for how community members viewed both the accessibility and use of resources within the child welfare community (Shallwani & Mohammed, 2007).
Community members assisted with recruitment and established connections with individuals who were interested in and felt that they could make a sound contribution to the needs assessment. While it was important for the research team to collaborate with A Home Within as the key community partner around recruitment and data collection methods (i.e. the generation of qualitative interview questions), A Home Within staff and volunteers were not provided any access to the data or any other information that would enable them to identify participants.
Participants
Inclusion criteria required that all participants be over the age of 18 and have either lived and/or professional experience in the child welfare system. Twenty-two stakeholders in the child welfare community participated in qualitative semi-structured interviews for this study (see Table 1). Participants lived across the United States with the majority (68%) on the West Coast. Over half (54.5%) of the participants identified as White; three (13.6%) identified as Hispanic or Latino, two (9.1% identified as Black or African American, one (4.5%) identified as Native American, two (9.1%) identified as mixed or some other race, and two (9.1%) did not respond to the question about racial/ethnic identity. The majority of participants identified as female (63.6%) and as heterosexual (72.7%).
Table 1
Participant demographics
Age in years (Mean, SD) | 48.4 | 15.5 |
Current state you live in (n, %) | ||
California | 9 | 40.9 |
Massachusetts | 6 | 27.3 |
Oregon | 5 | 22.7 |
Washington | 1 | 4.5 |
New York | 1 | 4.5 |
Race or ethnic identity (n, %) | ||
White or Caucasian | 12 | 54.5 |
Hispanic or Latino | 3 | 13.6 |
Black or African American | 2 | 9.1 |
Native American | 1 | 4.5 |
Mixed or some other race | 2 | 9.1 |
Did not respond | 2 | 9.1 |
Gender identity (n, %) | ||
Female | 14 | 63.6 |
Male | 5 | 22.7 |
Fluid | 1 | 4.5 |
Other | 1 | 4.5 |
Did not respond | 1 | 4.5 |
Sexual Orientation (n, %) | ||
Heterosexual | 16 | 72.7 |
Queer | 2 | 9.1 |
Bisexual | 1 | 4.5 |
Gay | 1 | 4.5 |
Did not respond | 2 | 9.1 |
Stakeholder Status | ||
Psychotherapists | 9 | 41.0 |
Former foster youth | 8 | 36.4 |
Case manager / child welfare worker | 6 | 27.3 |
Foster parents / bio parents | 4 | 18.2 |
Child welfare legal / policy professionals | 4 | 18.2 |
Of the eight former foster youth (36%) who participated in the study, four (18%) were current A Home Within clients and one (5%) was the parent of an A Home Within client. Three participants (14%) were foster parents, one (5%) a biological parent of a foster child; two participants (9%) worked in juvenile justice. Thirteen participants were affiliated with A Home Within as a volunteer therapist, consultation group leader, clinical director, paid leadership role, or client. Several participants held many affiliations with A Home Within. Of those (n = 9) who were not affiliated with A Home Within, five were psychotherapists treating foster youth and four were social workers and case managers working in child welfare and/or with former or current foster youth involved with juvenile justice.
Data Collection
Methodology project consultants, A Home Within leaders and staff, two former foster youth (members of Community Advisory Board), and the A Home Within community volunteers collaborated to develop a semi-structured interview guide, consisting of a blend of open- and closed-ended questions. This collaborative process led to multiple iterations of the interview guide before finalization. The principal investigators often followed up interview questions with prompts for thicker description (Williams, 2015). The principal investigators used this guide to jointly conduct 90–180 min interviews with each participant, using Zoom. As part of the larger interview process, they collected data to answer this specific interview guide question: “What are the most important outcome indicators of successful mental health treatment for current and former foster youth?” In addition, they asked follow-up questions about how the indicators of success participants identified changed over time in treatment and whether they thought therapists and clients agreed about these indicators. Variations of follow up questions included: How would you know that therapy was working for you? What would you notice? What would others notice?
The principal investigators kept field notes throughout the data collection process and had a debriefing discussion after each interview. Consistent with grounded theory methodology, they concurrently collected and analyzed data (Foley & Timomen, 2015). Thus, there was some recursive modification of interview questions based on the field notes taken in the initial interviews (Strauss & Corbin, 1998). Participants consented and provided demographic data via a Qualtrics survey prior to being interviewed. All interviews were recorded and transcribed using Otter.ai and then cleaned for accuracy by trained research assistants.
Data Analysis
Given the focus on understanding shared views across key stakeholders of the child welfare system, the research team applied theoretical sufficiency, defined as a sufficient depth of understanding to allow for theory development (Vasileiou, et al., 2018), instead of theoretical saturation since the team could not provide evidence that they have reached true theoretical saturation across all stakeholder groups (Charmaz, 2014). At around nineteen interviews, during a regular debriefing meeting, the principal investigators discussed that there was some redundancy in the latter interviews from the earlier ones and actively explored what and whose viewpoints might still be missing. From that discussion, they identified three more stakeholders to interview – two therapists and one child welfare worker – to capture their unique vantage points. The goal of theoretical sufficiency was met upon the completion of these last three interviews as no new suggested themes or core ideas were generated.
Transcript data were analyzed using Grounded Theory qualitative methods allowing the research team to explicate implicit meanings, crystallize points of significance, compare data, and identify gaps in the data (Charmaz, 2014). By conducting the interviews themselves, the co-principal investigators were able to stay “close” to the data and use field notes to deepen their analysis process. Using constant comparative methods, they made comparisons at each analysis stage (Strauss & Corbin, 1998). They read the cleaned transcripts one time and then performed “open coding” separately, by carefully reviewing each line of the interview transcripts to identify common categories and themes. In addition, they used axial coding to identify relationships between open codes and adapted a coding scheme accordingly (Charmaz, 2014). The fifth coding scheme was the final one that was used to code all the data again. Finally, the co-principal investigators met to discuss and compare open and axial codes and then organize them into a set of overarching categories and subcategories.
Trustworthiness Strategies/Measures
The co-principal investigators took several steps to ensure the trustworthiness and dependability of the findings. First, when they had differing interpretations of the data, they discussed these further and asked follow-up questions to the key stakeholders that agreed to participate in a member checking process. These steps allowed for a reach consensus on the categories, subcategories, and their meanings, described in the findings section. The co-principal investigators engaged in member checking with interview participants by sending a synthesized and analyzed report of our findings to a representative subset of the study participants (Birt et al., 2016). They asked them to review the findings and respond to the following two questions:
1.
Do you see your experience captured in the findings?
2.
Is there anything you would add or change?
The co-principal investigators took a constructivist approach to member checking in that they allowed participants to comment and add to the data interpretation (Birt et al., 2016; Harvey, 2015). Five of the interviewees we asked participated in member checking. All indicated that the interpretation of the data and the findings captured their perspectives. Two participants suggested a few minor edits or additions, which were all integrated within the final reporting of the findings below. One participant seemed to use the member checking process as an opportunity to think about the implications of the findings, which the team tried to capture in the discussion section. The research team maintained an audit trail tracking the step-by-step process of data collection and analysis procedures and ensuring trustworthiness of the findings (Creswell & Creswell, 2018).
Results
The central theme that emerged across participants’ accounts was that successful psychotherapy is a developmental process and that key indicators of success need to be tracked across time and tailored to the individual and their unique contexts. Implied within this central theme is that all participants contended that it is possible to conceptualize what successful psychotherapy looks like. However, more than half of participants expressed their belief that there is a lack of clarity between researchers and clinicians for what constitutes successful psychotherapy outcomes, and a few went as far as to say that measuring outcomes should not be the focal point. Still, key stakeholder participants, even those seemingly more reticent to discuss the topic, offered mixed and oftentimes strong opinions about the types of outcomes that matter and whose perception of success should be prioritized.
Key indicators of successful therapy clustered into five interrelated categorical themes emerged from discussions on how successful therapy should be conceptualized and measured: (a) psychotherapy is a developmental process, (b) an indicator of improved mental health is a client’s increased understanding of themselves within their context, (c) therapy effectiveness ought to be measured based on individual goals, (d) client and other perceptions of helpfulness is a key driver of psychotherapy benefits, and (e) there are likely individual and relational outcome indicators of effective therapy. Within each of the five major themes, there were several subthemes that will be discussed and exemplified.
Overall, key stakeholders seemed to acknowledge the benefit of tracking indicators of successful mental health treatment and typically thought that there ought to be effective measures in place. However, participants described misguided efforts to measure successful treatment. One researcher and psychologist with a long professional history of working in child welfare and criminal justice systems said:
Why don’t we set very clear expectations for your outcomes, and then successively move towards those measuring progress, not perfection. And it’s hard for funders to hear that because they want a silver bullet, a magic bullet that does exactly what it’s supposed to do. Otherwise, they won’t fund it. But they [funders] have to understand that they need to fund the progression of ideas and increasing effectiveness over time.
This quote provides a sensical introduction into the first major theme detailed below.
A Developmental Process
At least half of the participants described success being more about a process that occurs across development and time than a specific and static outcome. The idea that the change process can wax and wane over time was illustrated by this former foster youth and A Home Within client: It’s about the journey, not the destination. Yes, you want to get to these outcomes and operate differently and feel differently and think differently, but a lot of that change is going to be incremental.
Similarly, a parent participant unaffiliated with A Home Within expressed opposition to using time-limited measures of progress by saying “Don’t put a time frame on it. Don’t try to measure it too soon. Just give it some time.” Another mental health professional and A Home Within volunteer emphasized the developmental progression of the therapeutic relationship and how this can tie to treatment goals: “…in the beginning, give me safe goals as a way to test our relationship.” This conversation included discussing how goals require continued review and possible revision.
Additionally, differential experiences across age and foster care experience often do not align with single outcomes applied universally to measure outcomes across all current and former foster youth. Participants emphasized the importance of measuring all progress instead of trying to find perfection in our measurements because any progress ought to be valued and celebrated. The following quote from one former foster youth illustrate these subthemes:
Therapy helps foster youth toward their goals. And helps them navigate stuff. But see when you talk about success, it can be a little weird because that suggests that a foster youth does not have other barriers, like all the barriers or obstacles are gone. I used to think that aging out of foster care was a good thing. I thought ‘Yes, I’m aging out - no more problems. I’ll have to deal with addressing my trauma and the problems that I had then, but once I do that, I’ll be soaring. I’m never going to face any more difficulties ever again. And I’m just going to be this successful person that never has anything else to worry about.’ But I found out there actually is more to it than I realized, the more I learned about society and my personal family history.
One A Home Within volunteer similarly discussed the importance for both the therapist and client to feel that there is positive momentum in their work together:
Yes, for me it is about setting realistic and reasonable goals/expectations that reflect barriers. It is very important for the therapist and client to feel that there is movement and progress, but if the end post is placed wrong, it can feel too daunting and be counterproductive. And the barriers need to be understood and discussed, as well.
Likewise, a mental health professional and foster parent expressed a desire for researchers to be holistic when trying to measure treatment outcomes and described factors such as the number of foster care placements, school setting that warrant consideration:
Understanding Self Within Their Context
The second major theme that indicates a client is benefiting from therapy is an increased understanding of themselves within their context. While it is likely that many current and former foster youth have experiences and exposures to trauma that informs their context, we also heard from former foster youth that they did not find it helpful when healthcare professionals made assumptions about trauma they may have experienced and centered their treatment around those assumptions. For example, one participant shared how these assumptions led a new provider to ignore her previous diagnoses and take her off medications that had helped her find stability for the previous two years. She said:
And I got signed off [because her previous provider left] to somebody else. So my case was passed to this gentleman that looked at my case and was like, ‘I want to do a whole new reevaluation.’ Even though my medications had been working, I had been stable and my anxiety levels had been going down, he wanted to do a re-evaluation and told me that I didn’t have anxiety, I didn’t have ADHD… he re-diagnosed me with PTSD. He only sat on the phone with me for about 15 min [for the re-evaluation] and only asked me questions about my childhood and my mom and dad. He didn’t ask me anything about what was going on now. And why I was struggling then.
Four subthemes emerged that provided greater understanding of the multiple dimensions of this major theme. The first subtheme related to current and former foster youth not immediately recognizing their positive changes and growth. The following quote from a former foster youth illustrates this subtheme:
I often don’t realize that change has happened or that I feel differently, until I have a very personal experience like, ‘Wow that was different.’ It might not come out at that moment until you have an ‘aha’ moment or epiphany further on where you’re like, ‘Oh shit, that was very different from how I’ve always reacted to this.’ Where you have a situation that previously would have triggered you and really upset you, and now you’re generally okay. So that part of things, I think, is hard to realize that change has happened or that you have grown, without having some distance between how you used to be and how you are now.
The second subtheme is that success can happen when a current or former foster youth gains a meta-perspective on who they are and where they come from, including their biological family and childhood experiences. For example, a participant who is both a former foster youth and mental health professional shared:
When they understand themselves better, they understand trauma, and they understand their caregivers better. Meaning they understand they were not at fault. It was due to their parents’ unmet mental health needs that led to their mistreatment or abuse or neglect. And that their parents possibly had unmet financial support, parental support, community support and that they may not have known how to access resources. When they want to go to therapy, they want to continue growing, and they have a growth mindset. They understand that they are not wrong and they’re good people.
A third subtheme was that a sign of progress is when clients begin making connections on their own either in or outside of therapy. A former foster youth who now works at a community organization with other foster youth said:
I just enjoy having a therapist who can meet me where I’m at. When I’m able to be honest and have the breakthrough. When I can make that connection like ‘oh my god, this is why I have anger problems.’ So, when a youth can make the connection on their own. That is a success right there, you [the therapist] helped find out the answer without telling them [the client].
The fourth subtheme identified as an indicator of success is when clients experience an increase in their understanding and sense of mastery over their stress responses can be an additional identifier of progress. This connects to the subtheme above related to the development of a meta-perspective. For example, one mental health provider and A Home Within volunteer described the willingness to approach distress:
When something distressing comes, how do you react? Are you even approaching the distress? Or are you disconnecting and hiding from it and doing the typical? So if we talk about unpacking your bag, do you just shut down? Or do you actually talk about what’s scary about packing your bag?
Measures of Individually Tailored Goals
Perhaps the most robust theme was that therapy effectiveness should be measured by individually tailored goals. As one mental health therapist we interviewed said, “What are the hopes, what are the aspirations at the beginning of treatment? What do you hope for yourself and what are they at the end? It is about reality-based hope or reasonable aspirations?” Goal attainment scaling was frequently mentioned even if participants did not use that specific language. The following quote from a licensed psychotherapist we interviewed is a good example:
Show me the client. Who’s the client? I can’t tell you how to measure outcomes if I don’t know the client. Is this a kid that never unpacks their bags, when they get to a foster placement? Then unpacking their bags probably is a significant deal. Right? They never go to school. And they’re now attending two days a week. That’s an improvement. Right? That’s the outcome. So, I think all outcomes probably boil down to a moment of strain. How do you approach that moment of strain? And I think that’s probably the simple, ‘if I could boil it down’ we can figure out how they are progressing and if therapy being effective?
A licensed mental health professional who works with foster youth and is a former foster youth herself said, “Let the clients tell you what their treatment goals are, and whether they’re making progress on those. That is the only thing you can do that is developmentally appropriate.” Another mental health professional said, “To me, it is the conviction that young people should be setting their own goals.” Likewise, a former foster youth participant said:
I think measuring success is very individual. As more problems and traumas occur in adult life, you will have setbacks. I think the only success one can have is whether or not the person feels loved and valued at any point they are at.
Another licensed mental health professional mentioned that letting clients decide upon their own goals is an especially useful strategy for empowering former and current foster youth who have had little control over their lives. Most participants expressed confidence that within the context of individually tailored goals, progress can be tracked, recognized, and celebrated, which then builds momentum for even more growth and success over time.
Client and Others’ Perception of Helpfulness
Another powerful theme that emerged from the data is that effectiveness is driven by client and others’ perception of the helpfulness of the therapy. Some of the most strongly worded statements from participants were focused on the importance of asking clients whether they believe therapy is working for them and if so, how. Centering the client experience of therapy and how they believe they are benefiting was a commonly recommended way to track therapeutic outcomes. The following exemplar quote from a child welfare worker illustrates this sentiment:
There needs to be a way to check-in, some way to measure and talk about what’s going on, because otherwise things just get lost over time and you kind of lose purpose and sense of why you’re doing what you’re doing. And that might lessen a therapist’s desire to stay involved if they’re not necessarily seeing the positive outcome from what they’re doing. [They might say] ‘I’m just talking to this person every day for the past seven years, why am I doing this? What am I getting out of this?’
Centering client voices and feedback on how well their therapy is working could easily be paired with individually tailored goal setting and attainment scaling. One former foster youth and A Home Within client said:
Well, how do they [the client(s)] feel like it’s a success? Do they feel like anything is changing? Checking in on the work together and telling them to not just say what they think you want to hear. Ask them ‘Do you feel like anything has changed for you?’
The majority of participants, regardless of their stakeholder status in child welfare, mentioned that a client’s attendance and engagement with the therapy process is a way to measure whether or not services are beneficial. This is not a new concept and often clinical intervention studies will use attendance and participation data to determine the acceptability and satisfaction with an intervention. One foster youth identified that a measure of engagement could be to watch the client and see if “they are happy there [at therapy appointments]? Are they thinking about what you are offering them?” Still, other participants suggested that attendance and engagement are distinct and may look different for foster youth, who have often experienced attachment ruptures in key relationships. One participant who has been volunteering with A Home Within for two decades said:
The engagement in treatment as shown by reliance of the therapist. It is not always going to be about showing up for appointments because many of these patients have spotty attendance. And still there is something powerful that is happening in the relationship.
Another long-time A Home Within volunteer and licensed mental health professional took it further by saying:
In addition to engagement and attendance, I would hope to see a deepening of the relationship with the therapist and with the material, especially when relational issues are part of the goals.
Finally, a few participants suggested that it is important to get the perspectives of others in the client’s life as to whether or not they think the therapy they are receiving is beneficial. Participants mentioned that for child clients, gaining others’ perspectives was particularly important.
Individual and Relational Outcomes as Success Indicators
Individual Outcomes
All participants mentioned individual level outcomes that they thought could be key indicators of successful therapy. These responses are clustered into two main subcategories: (a) increases in healthy experiences; and (b) decreases in unhealthy experiences, described further below.
Participants frequently mentioned individual outcomes such as increased stability, as determined by obtaining and maintaining secure housing, an increase in independent living skills and being gainfully employed. Just as often, participants articulated that when current and former foster youth clients contribute to their community, it is a sign of improved wellbeing. For example, one licensed psychotherapist and A Home Within volunteer said, “ …if I see myself as somebody who contributes that means I see myself as somebody who has something valuable to give.” Some participants believed that we should measure clients’ engagement in healthy lifestyle habits such as getting adequate sleep, nutrition, and physical activity.
Another mental health outcome mentioned across about half of the interviews was when a client has increased hopefulness and belief in themselves. One former foster youth and A Home Within client shared how she had made changes in this way:
And I started saying okay well, this happened, or I didn’t do this today, but I can do it again and I’ll get it right the next time. And before I wasn’t able to do that, I’d always be like ‘oh I give up, I don’t want to do it no more.’ But now I’m like, ‘No I did it. I learned from it. And I’m going to try again’. Hope and being consistent and knowing that they’ll be better outcomes. It’s all about believing that things will get better, and things won’t be the same.
A related theme for how we can measure improvements in mental health is by tracking increases in experiences of joy, creativity, and play. Participants mentioned that these are important aspects to measure both in and outside of the therapy room and within the therapeutic relationship. One former foster youth and mental health professional exemplified this idea for measuring success by stating:
Being in the world. So just going out in the world, and being in the world, and then I think making new friendships, having more fun, the ability to enjoy themselves and let go of the past and not victimize themselves anymore. I’ve learned in training that when you know you’re having compassion-fatigue, you lose your sense of humor. So, it’s critical for mental health.
Participants frequently provided examples of outcomes related to decreases in unhealthy experiences. One mental health professional and A Home Within volunteer we interviewed said, “If a person is massively depressed, suicidal, has low self-worth, emotional reactivity, then those are the outcomes to measure.” In addition, one third of the participants mentioned measuring decreases in crisis service use as exemplified by this quote:
A lot of youth that we do work with utilize the higher-level services, like crisis services, and have to be hospitalized, or go into partial hospitalization. So I think measuring those numbers and whether they decrease over time would be a helpful way to measure positive outcomes through therapy.
A foster parent shared:
I know when my kids are doing better because they have less referrals at school. As a foster parent, I get less phone calls from their teachers, we have less referrals, there is less fighting. I have behavioral charts for my kids when they’re struggling, and I have skills builders that I have set them up with that go into the school. Because we have things going on at home, they get used to the structure and the rules here. But when we put them out in the community, the only measurable thing I can think of right off the top of my head would be in the school system. The teachers are very involved with my kids. They know the circumstances.
Not surprisingly, the majority of participants identified decreases in symptoms of mental health distress (depression, anxiety, PTSD/trauma) as a psychotherapy success indicator. Participants said things similar to this mental health professional, “I think it’s a little bit different when it comes to mental health like diagnosis. That is a whole other separate topic --- if you do have depression, obviously you want treatment to help the depression to get better.”
Within this major theme, it is worthy of mentioning that several participants described individual outcomes as being too narrow and thus potentially limiting, but as easy ways to measure successful outcomes if they are considered alongside increases in relational wellbeing. For example, one long term A Home Within volunteer and mental health professional said:
A measure people talk about all the time is socioeconomic status. Because it is one measure of success. It’s numbers. We can count it. Educational achievement is another one. Another thing I would be curious about is on a give and take scale, where do people fall? How much are they just a taker and how much are they a giver. How do you measure that? Another way of putting is, ‘Where do you see yourself in your community? Where do you see yourself in the world?’ Because if I see myself as somebody who contributes that means I see myself as somebody who has something valuable to give.
Similarly, a former foster youth and A Home Within client demonstrates how they see the interconnection of individual outcomes and relational wellness:
It feels like a north star in terms of programs and outcomes for young people. Because at the end of the day it’s important for people to have housing and get into school and meet their tangible life goals and those kinds of things. But I think even deeper than that, beyond just those tangible kinds of things, that it’s the north star of a former foster youth feeling capable, lovable, and worthy. I think it is a really big deal.
Relational Wellness Indicators
Virtually all the participants mentioned ways to measure therapy effectiveness centered on clients’ relational wellness and functioning. Related to goal attainment scaling, about one third of participants suggested that clients could define what healthy relational functioning means to them, develop associated goals, and then use goal attainment scaling to measure progress. One child welfare worker stressed its relative importance with this quote:
We know that if kids go out into the world without having formed an attachment, they’re really disadvantaged and have a difficult road ahead. So, to me, that’s a big one to just know that there is a relational connection and that it is strong enough to sustain them on some level over time. That is probably bigger than anything else, than doing well in school, than being “successful.”
Another frequently offered idea was that if clients were to increase their help seeking behaviors and attitudes, then that would be an indicator of successful therapy. The following quote speaks to this:
Help seeking…I think being able to identify a network of support is a good thing, right? Like being able to identify who you can lean on for different needs, something about the number of healthy non-abusive relationships.
Participants noted that a client’s perception of themselves as loveable and having a sense of goodness in themselves is a facet of maintaining healthy relationships. One licensed mental health professional and A Home Within volunteer said that a measure of improved relational wellbeing is when clients are less likely to dismiss others as potential sources of support or to protect themselves from the “relational knots” that are common in long-term, close relationships. However important, measuring these types of processes can be difficult as one former foster youth pointed out:
I think that the outcome part can be hard because let’s say that you know you have some kind of outcome [of importance]. I have trouble trusting people, right? And that’s kind of the main outcome. How do you measure whether you trust people more? I think that there are some pieces of this that are really hard to measure. Especially if you have no baseline.
Discussion
The aim of this study was to learn more about stakeholders’ thoughts and recommendations related to psychotherapy outcomes, to inform future research, policy, and practice. To this end, this study examined the effectiveness of services through A Home Within, a national nonprofit providing pro bono mental health services to current and former foster youth. Participants included people affiliated with A Home Within, and not, with the goal of understanding perspectives on psychotherapy outcomes broadly.
A major finding of the study was that stakeholders viewed inconsistencies in how researchers, practitioners, and clients see and assess success in treatment. Stakeholders shared a belief that measuring outcomes is important, but that it should not be a focal part of therapy. Participants described psychotherapy as a process that takes time and “success” as something that waxes and wanes. Consequently, findings suggested that measuring progress on specific outcomes risks minimizing the importance of the therapeutic process. Participants offered opinions about the types of outcomes that matter and recommended assessing success through clients’ perceptions of incremental improvements in: (a) their understanding of self, in the context of experience and trauma, (b) individualized outcomes that allow for differences in need across age, time, experience, and identity, and (c) relational wellbeing. Stakeholder participants emphasized the importance of centering client’s perceptions of helpfulness to understand if and how psychotherapy benefits current and former foster youth. Collectively, these findings offer an opportunity to develop a client-centered framework about mental health with and for current and former foster youth.
In this study, stakeholders recommended that progress in psychotherapy be measured incrementally through both process and outcome variables. Stakeholders emphasized that incremental assessments would allow positive change, in and of itself, to represent success. As part, stakeholders recommended sustained curiosity about the how and why of change. They wondered if understanding mechanisms of change could help parse out ways to capture incremental change (e.g., if decreased self-criticism was progress associated with decreased depression). They also thought examining process variables could support help capture “the why” needed to sometimes destigmatize mental health need. Current and former foster youth voice concern about the representation of their mental health needs, explaining that the representations are inaccurate and/or simplistic and create a barrier to accessing services (Jee et al., 2014; Ruff & Harrison, 2022; Villagrana et al., 2018). Assessing incremental change, inclusive of process variables, in research and/or practice may help reduce this barrier to services, and identify ways to maximize therapeutic change (Farmer et al., 2017).
Study participants recommended that assessments of change in research and practice include asking clients/participants directly about their belief as to whether therapy is helpful or not. They noted that a focus on client’s perception of helpfulness may increase a client’s agency as a consumer of services, and bolster treatment efficacy (Huber et al., 2021). And while it may be commonsense, prioritizing this as an explicit conversation is all the more important when working with a population that has typically not experienced agency and collaborative decision-making when interacting with services and systems (Chateauneuf et al., 2021). As discussed by participants in the study, these conversations may also challenge common assumptions about attendance and engagement (e.g., not attending is due to a lack of readiness for change rather than perceived benefit of approach) to ultimately support improved outcome.
In considering change over time, findings underscored the unique importance of allowing time and providing support in the beginning phases of therapy to consider and identify meaningful outcomes. Findings of this study suggest that clients may request support with developmentally-focused tasks (e.g., careers, education), not wanting to have their care center around their foster care experiences or affiliation. Current and former foster youth may also provide initial goals as a way to test the waters of therapy to learn if it will be useful. Consequently, a recommendation for further consideration is for ongoing reassessment of meaningful outcomes over the course of the therapy process.
Mental health needs among current and former foster youth will also differ by age and by proximity of time from child welfare involvement and experience. What one may need one year after child welfare involvement may differ from what is needed four years later. Similarly, what one may need or want at ten years of age in psychotherapy may differ from what one needs at thirty. Participants suggested that psychotherapy for current and former foster youth should be readily available when it is needed, with the recognition that there is often a communicated readiness for therapy at a later point in a youth’s life – when they are no longer in crisis. Naturally, at different points in a person’s development, client goals will differ according to their current context, development, and functioning.
Another major finding of the present study was to individualize assessments. Participants commonly noted that a reduction in mental health diagnosis and/or system involvement was an appropriate focus of treatment, but that it may not always be the most accurate representation of a client’s presenting concern. Participants expressed concerns about being as foster youth first, and individuals second. They recommended curiosity about the differences between clients, not just the commonalities in need. Again, stakeholders recommended assessing therapeutic success by asking about increased understanding of themselves and/or an increased ability to recognize positive changes and growth within themselves. Initial studies (e.g., Kleinrahm et al., 2013) have looked to goal attainment scaling as an opportunity to individualized goals, and understand the range of needs across identity and experience (Krasny-Pacini et al., 2016; Kucheria et al., 2022). This approach may offer the opportunities to learn what clients need from therapy and understand both the collective and individual needs of current and former foster youth.
Last, findings of this study emphasized clients’ relational wellness as both an important change variable and as a significant mental health outcome. This finding was critical in the context of A Home Within, as a nonprofit that facilitates pro bono therapy “for as long as it takes.” A Home Within is focused on reducing barriers to care so as to allow the time needed to develop a therapeutic relationship that supports improved wellbeing (Clausen et al., 2012; Ruff et al., 2022). Participants emphasized that therapy requires that clients and therapists work together to develop a relationship that not only allows for the identification of treatment goals and desired outcomes, but that offers opportunities to challenge and revise these over time (Bickman et al., 2004, Norcross & Lambert, 2018). Participants clarified that the development of a therapeutic alliance, and understanding of the therapeutic work, helps the process of both identifying initial goals, but also their revision. This is particularly evident in trauma-informed care and is supported in research by Villagrana & Lee (2020) that examined the perceptions foster care alumni formed about the interpersonal aspects of the therapeutic relationship while receiving mental health services. These researchers found that positivity in sessions increased when sessions were client-led. Foster youth participants reported negative reactions to a perceived agenda set by a therapist as well as lack of confidentiality, lack of empathy, and lack of authenticity. Youth felt they wanted to forget their past trauma and not relive it, and when mental health providers pushed too soon, it left them emotionally raw (Villagrana & Lee, 2020). Our research underscored the need for clinicians who are well trained in trauma-informed psychotherapy practices and who prioritize the development of a therapeutic alliance that allows for collaborative goal setting and care. Related, stakeholder participants recommended assessing the relationship between therapeutic alliance and the quality of other relationships in a client’s life, to learn if and how improvements in both may bolster and/or represent treatment success (Duppong et al., 2017).
Limitations and Future Directions
While the study findings are informative for building our collective understanding of the meaningful objectives of therapeutic interventions for current and former foster youth, there are several study limitations that are important to mention. We interviewed individuals that were over the age of eighteen and cannot report on the perspectives of younger youth. As is reported in our study findings, participants believed that healing and successful therapeutic outcomes should be viewed as developmental processes. It is important that future research gain perspectives from child and adolescent participants regarding what matters most to them as far as therapeutic services and measures of success.
We also interviewed many participants affiliated with A Home Within and relied on individual interviews with key stakeholders to inform a collective understanding of therapeutic outcomes for current and former foster youth. We realize these foci may influence findings and their generalizability, and we recommend future research on that examines and compares perspectives, parsed out by specific stakeholder status.
In summary, findings from this study offer the perspectives of key stakeholders in foster care to support additional conversations about collaborative goal creation to help frame the expectations of need among foster youth. Findings underscore an important and maybe even necessary process to be captured in the framing and measurement of treatment outcomes with current and former foster youth: goals and desired outcomes are not always evident at the initiation of psychotherapy and if they are, they may not remain static. Policy makers, practitioners, clients, and researchers may consider framing the therapy process as expected to change over time, in relationship to context, and/or in proximity to one’s past, and researchers may consider whether doing so decreases barriers in accessing services for current and former foster youth.
Acknowledgements
The authors wish to thank A Home Within for their long-term, consistent support of current and former foster youth, and the University of San Francisco for funding compensation of the student researchers of the Foster Care Research Group.
Compliance with Ethical Standards
Conflict of Interest
The authors declare that they have no conflict of interest.
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
