Contents
Engaging Older Youth: Program and City-Level Strategies
Click here to download the full report:
Engaging Older Youth: Program and City-Level Strategies
Overview
Out-of-school time (OST) programs represent a vital opportunity and resource for learning and development for children and youth. There is growing recognition that OST is important not just for elementary school students, whose parents need supervision for their children when they are not in school, but also for middle and high school youth, whose participation in OST programs can help keep them connected to positive role models and engaged in their education at a time when many are beginning to disengage from schools.
Further, evidence suggests that once older youth have enrolled in a program, meaningful and sustained participation is a key factor in attaining positive outcomes. However, despite the well-documented benefits of OST participation for older youth, their participation wanes with age. OST programs struggle with how to recruit and retain older youth and continue to look for guidance on how to do so more effectively. There are also real discrepancies in access to and participation in OST programs by location and socioeconomic status. Predictably, youth from lower-income families and neighborhoods have fewer OST opportunities than their more privileged peers, and many low-income and minority families report unmet need for high-quality and accessible programming. The lack of opportunity for some youth is especially problematic given our nation’s increasing dropout rates. If, as research suggests, OST programs have the potential to support graduation and postsecondary success, then better access to quality OST programs may have the potential to help address educational inequalities, particularly in urban areas.
In response to the evidence pointing to the benefits of out-of-school time, coupled with the lack of access in many urban neighborhoods, many cities are creating citywide infrastructures to support networks of OST programs, with one goal being to support participation. With support from The Wallace Foundation and other private and public dollars, these nascent OST city initiatives are attempting to build the capacity of programs to deliver better-quality programming by engaging in one or more of the following efforts: supporting professional development for providers, providing funding, implementing quality improvement efforts, establishing data tracking systems, and connecting OST programs to one another and to other community institutions. All of these efforts can directly or indirectly support improved access to and sustained participation in OST programs.
Given the potential of city-level OST initiatives to support participation, and against the national backdrop of inequitable access to quality OST programs for older youth from disadvantaged communities, The Wallace Foundation commissioned this research study. To understand how to promote sustained participation in OST programs, this study examined the program characteristics—both program practices and structural features—associated with high participation and retention that were employed by OST programs, primarily serving disadvantaged youth, in six cities that have worked toward building OST initiatives. In particular, this report addresses how OST programs keep middle and high school youth engaged over time (i.e., the duration of participation) and how the supports that city initiatives provide can help foster youth participation, with the assumption that programs can have a potentially greater impact if they are able to work with these youth over an extended period of time.
We examined three key questions:
- What are the characteristics of high-participation OST programs that support sustained participation as measured by retention?
- How do these characteristics differ for middle school and high school youth?
- What strategies are city initiatives implementing to support access to programs and sustained participation, and how do OST programs perceive the usefulness of city-level strategies for achieving their participation goals?
Research Strategy and Methods
Using mixed-methods research strategies, the study design brought together both survey data from a large sample of programs and in-depth interview data. This design allowed for both breadth and depth in our understanding of critical issues related to access to and sustained participation in OST programs for older youth. We collected and integrated these qualitative and quantitative data and used an iterative analytic process, weaving together findings from both sets of data to confirm, augment, and challenge our understanding of program characteristics—both program practices and structural features—and support from city initiatives.
The six cities in the study—Chicago, Cincinnati, New York, Providence, San Francisco, and Washington, DC—were chosen because they have an intermediary or government agency coordinating funding and providing services for OST programs, a management information system (MIS) or database to keep track of attendance and participation, extensive programming aimed at middle and high school youth, and a focus on low-income youth and distressed neighborhoods. The initiatives in these cities all provide a set of supports to OST programs in the community, and they are making efforts to raise the profile and increase understanding of out-of-school time in their cities; they are also all relatively new, having been founded between 2004 and 2007.
After we identified the six cities for inclusion in the study, we then identified a large number of programs in these cities with high participation rates among middle and high school youth, based primarily on MIS data gathered by the city-level OST initiatives, and administered a survey to program leaders, asking about program activities and features, staffing, youth participants, family involvement, use of data, recruitment and orientation practices, practices for fostering and supporting engagement, and involvement with the OST initiative in the city. Out of the sample of programs that returned a survey, we selected a smaller subset of programs to interview in depth. The survey sample had an average program-level participation rate of 70 percent, and the interview sample had an average program-level participation rate of 79 percent. We also selected a group of city-level respondents to be interviewed for the study.
Altogether, we analyzed data from 198 program surveys, 28 program interviews, and 47 city-level respondents. Our quantitative analysis focused on the program practices and structural features associated with retention (i.e., duration of participation) of youth in programs. To identify characteristics that were significantly associated with higher rates of retention among older youth participants, we first examined which of the numerous individual program practices and structural features from the survey data were significantly more common in high-retention programs than in lower-retention programs. For this study, we define high retention as retention of 50 percent or more of a program’s youth participants for 12 months or more. We then conducted a regression analysis of retention to isolate which of the many competing practices and features were uniquely associated with the variation in retention rates, even when taking into account other practices and features.
Analysis of our interviews, in addition to document review, enabled us both to identify program practices that respondents cited as relating to greater retention and to create a picture of what it takes in programs and at the city level to keep youth engaged in programs over time, using a grounded theory approach. We focused on the major themes present across programs related to the successes and challenges of achieving high participation and retention rates and what program practices or features were linked to these efforts. We also analyzed program data to understand how programs participate in OST initiatives. Throughout the analysis, we cross-walked findings from the interviews and the survey against each other to refine our understanding of participation.
Major Research Findings
Five program characteristics (two program practices and three structural features) were identified that set apart the programs that were the most successful in supporting high retention:
- Providing many leadership opportunities to youth in the programs
- Having staff keep informed in several ways about youth outside programs
- Being community-based
- Enrolling 100 or more youth
- Holding regular staff meetings
These practices and features explained 38 percent of the variance in retention. Our analyses indicate that among the group of programs serving older youth, the ones that achieve relatively high rates of retention emphasize youth leadership and outperform other OST programs in their efforts to stay connected with youth; they are also more likely to be larger community-based organizations that give staff members regular opportunities to meet about their programs.
There is an additional set of retention and recruitment practices that, while not statistically related to retention when we account for other factors, were consistently reported as being important in engaging older youth. High-retention programs often employ these practices.
Retention practices: fostering a sense of community through connections to program staff and peers, providing developmentally appropriate activities and incentives, and engaging families.
Recruitment practices: using peers and staff as recruiters, using organizational relationships, and matching program attributes to youth needs.
These additional strategies may be associated with engagement and/or participation frequency; more research is needed.
The study found that the same five program features and strategies were significant in understanding how programs retained middle and high school youth, yet program leaders reported that there were also important differences geared toward meeting the needs of each age group.
The factors that were quantitatively linked to retention were the same across the two age groups—keeping informed about youth participants’ lives, providing many leadership opportunities, and the presence of certain structural features. However, our interviews with the 28 high-participation programs allowed us to better understand how these and other practices manifested themselves differently when working with middle or high school youth. Successful middle school programs give youth opportunities to interact with peers, create structures and routines to make youth feel comfortable and safe, and take advantage of their participants’ willingness to try new things, particularly through peer interaction. High school programs focus their programming more on providing formal and informal opportunities to explore and prepare for college and other postgraduation plans; giving youth more responsibility through job-like programming, apprenticeships, and mentoring; and offering the content and the particular skills older teens want to learn.
City-level OST initiatives employ a set of common recruitment and retention supports, but it is less clear that these efforts have made a difference in programs’ abilities to recruit or retain older youth.
City initiatives provide a set of services aimed at increasing OST participation broadly rather than solely for older youth. These supports include:
- Engaging in citywide recruitment efforts
- Coordinating information about programs across the city and helping programs network
- Collecting and using data on OST programs
- Supporting quality improvement efforts
- Providing professional development and technical assistance to programs
They were also beginning to foster relationships with school districts and to work with families on a citywide basis. Based on city-level respondents’ reports, these efforts may be increasing recruitment and participation at the city level.
The data collected for this study, however, provided little evidence that accessing these city-level supports (which were deemed useful by the programs surveyed) was directly related to the retention rates of individual programs. Helping programs to network, providing training in youth engagement, and helping with evaluation were three of the supports used by the greatest number of programs surveyed. Both high- and lower-retention programs, however, reported similar patterns of use of these and many other supports that they were asked about on the survey. In two cases where there were differences, it was the lower-retention programs that were more likely to use the supports.
In addition, programs reported that being part of a city-level initiative created new challenges having to do with data management, program competition, and tying participation numbers to quality within a high-stakes funding environment.
Implications
Our findings can help programs move toward a more nuanced approach to recruiting and retaining older youth and help cities understand their role in supporting participation. In addition, these findings have implications for future investment and policy decisions about OST programming for older youth. Therefore, we offer a set of implications aimed at key decision makers—city leaders, funders, and others—whose goal is to continue to improve access to and participation in OST programs as part of their overall efforts to support learning and development and to create pathways of opportunity for older youth.
The program practices distinguishing programs that achieve high rates of retention among older youth from those that do not can help guide the actions of program directors and city leaders as they try to improve participation within a context of limited resources.
Our findings about the two practices that set high-retention programs apart—providing many leadership opportunities to youth in the programs and having staff members keep informed about youth outside programs in several ways—can give other programs an idea of where to direct scarce resources. Because we know these practices support retention, city initiatives can target professional development and technical assistance efforts to ensure that these practices are implemented effectively.
The other practices that high-retention programs use, even though they did not prove to be significant in the regression analysis, warrant further attention. Although we do not know conclusively whether these practices promote retention in other settings, we do know that they were reported by the programs in our study (both on the survey and in interviews) as being part of an overall “participation package.”
Cities should consider offering a variety of specialized activities for high school youth.
Choice is an important program component and a key feature of youth development, but it seems to matter in different ways for middle school and high school programs. Our interviews with program staff suggested that youth become more focused in their interests as they move into high school, which often means that they are in more specialized or single-focus programs. As a result, while activity choice within programs is developmentally appropriate for middle and high school students, high school students may also benefit from choice across a variety of more specialized programs. Cities can work toward this objective either by providing programs with funding to add specialized activities or by creating a variety of specialized OST opportunities for high school youth.
OST programs’ attention to developmental changes can support continuing youth engagement in OST programs.
Understanding developmental growth can help programs retain youth longer as well as support program participants’ transition from middle school to high school. High-retention high school program providers reported that their participants want programming to help them meet concrete goals, such as taking the SAT. Middle school programs reported that, particularly around eighth grade, youth stop attending because they want a program that feels “older.” OST programs can use this finding as an opportunity to create programming for eighth and possibly ninth graders that includes more responsibility and skills aimed at having a successful ninth-grade year. Cities can support these efforts by bringing OST providers and school staff together to create curricula for transition programs and establish a team approach to the transition. By supporting youth in transition from middle to high school, this collaborative effort could lower the dropout rates for particular schools.
Family engagement matters for older youth participation.
Program and city-level respondents alike clearly understand and value family engagement as a strategy to recruit and retain older youth, but are challenged as to how to implement effective family engagement strategies. Further, though family engagement practices were not statistically related to retention, high-retention programs in this study reported using more strategies to engage families than did lower-retention programs. Our findings have implications for city-level professional development efforts, which could be designed to include training on working with families. They also have implications for recruitment strategies, which should include reaching out to families in a variety of ways to persuade them of the value of OST participation for older youth.
Supporting school-program partnerships can help recruitment efforts.
Initiatives are in a strong position to influence and advocate for partnerships between school and district leaders and OST program leaders. They can increase youth access to programs by actively supporting the establishment and development of these partnerships. The stronger the partnerships between programs and schools, the more energy they can invest in targeted recruitment fairs and strategic marketing efforts during and outside of the school day. City-level initiatives can support partnerships not only by linking and connecting schools with OST providers, but also by helping programs and schools develop mutually beneficial goals and expectations; streamlined tools for data sharing; and clear, two-way channels of communication regarding students.
Resources for organizational capacity are important to support participation.
Our findings suggest that high-retention programs have strong organizational capacity and sound program management. These programs’ staff members have time to go the extra mile, attend meetings and plan programming, network with other providers and schools, and attend professional development opportunities. In fact, many of the programs selected for our in-depth study were supported by large OST intermediaries (like Beacon initiatives and Boys & Girls Clubs) that provide this kind of capacity building. These findings suggest that investments in direct service alone are necessary but not sufficient to improve retention and that resources should be allocated to sufficiently support organizational development, including resources to support the finding that regular staff meetings matter for retention.
Improved data-based decisions can improve participation.
Cities use data in multiple ways to support participation, including data about location of and access to programs, where underserved youth live, participation rates, and quality across the initiatives. Overall, programs reported that the city-level supports that enabled them to obtain and use information were helpful for improving recruitment and retention; they also reported challenges, however, related to data collection and use that cities need to address. Initiatives can work, for example, to ensure that data collection and databases are supporting programs’ work and that programs are spending their time managing data in ways that are helpful for participation and are not sapping organizational resources. City initiatives can support programs’ understanding and use of participation data in order to improve recruitment and retention. The next step in the coordination of data is to link OST data to other data systems, including those of schools, to develop a more comprehensive understanding of participation and outcomes across all the supports, including schools, available to youth in the city.
City-level initiatives should work with programs for older youth to learn how to better support retention goals.
All of the cities in our study employ city-level supports to improve access to and sustained participation in OST programs; few of these strategies, however, appeared targeted toward the participation of older youth in particular. Rather, the strategies were part of cities’ overall initiative-building efforts to support the quality and sustainability of OST programs. Although cities reported using strategies that directly addressed recruitment, such as social marketing, most of the strategies they employed addressed retention only indirectly. Further, none of these strategies supported high-retention programs’ participation goals in a statistically significant way. Therefore, applying what we have learned about the high-retention programs in our study—and with the understanding that recruitment and retention are two sides of the same coin—it is important for cities to strengthen their recruitment and retention efforts, finding out from programs what is needed to promote the sustained participation of older youth.
< < Previous | Next > >