Contents

Engaging Older Youth: Program and City-Level Strategies

Click here to download the full report:
 Engaging Older Youth: Program and City-Level Strategies

By design, the programs in this study were all located in cities that have coordinating bodies responsible for developing and sustaining a city-level OST initiative. Thus, in addition to investigating recruitment and retention characteristics at the program level, as described in chapters 2 and 3, this study explored city-level strategies for supporting participation in OST programs for older youth. This chapter draws on interviews with the 47 city-level respondents to describe initiative-level supports for participation available in the cities and the ways in which cities implement those supports. Survey data and program interviews are used to explore which of these supports programs find the most beneficial and what challenges programs experience related to these supports.

Overview of City-Level Supports

Each OST initiative included in this study reported that it provides a set of supports to programs that are aimed at improving access, enrollment, and sustained participation within and across programs that serve all children and youth.xiv Most of these supports do not solely target the improved participation of older youth; rather, they are part of the initiatives’ overall efforts to build and sustain quality OST programming across the city. Nonetheless, when city respondents were asked to talk about the ways in which their initiatives support the participation of older youth, they reported providing the following five categories of supports to participation:

  • Engaging in citywide recruitment efforts
  • Coordinating/networking OST programs across the city
  • Collecting and using information, including developing MIS
  • Supporting citywide quality improvement efforts
  • Coordinating and providing professional development and technical assistance

In addition, some cities were working toward developing two additional supports that they viewed as critical to improved participation for older youth:

  • Engaging families
  • Fostering collaborative relationships with school districts

Each of these supports is described below, followed by a discussion of programs’ views of the supports the six cities provide.

Engaging in citywide recruitment efforts

All of the initiatives in our study were directly supporting recruitment efforts through market research, social marketing, and/or recruitment fairs. Market research involves directly surveying both youth and parents about their ideas and perceptions of OST programming; social marketing targets youth using clear messaging to encourage youth to take advantage of OST programming; recruitment fairs are community events designed to provide information about OST programming and promote enrollment.

Two of the cities in our study, Providence and Chicago, hired a consultant to conduct market research by surveying youth and parents directly and to develop branding and marketing strategies. As a result of the data gathered, Providence’s social marketing strategy consistently portrays hip-hop culture in order to attract the youth they are trying to reach. Providence, Washington, DC, and San Francisco provide professional development training to OST program leaders on how to market their program successfully to the youth population they serve. For example, Providence provided training on how to “sell the brand” of “it’s cool, it’s hip, it’s Providence”; Washington, DC provided one-on-one training or strategy sessions on how to market or repackage programs or to target a different audience, usually in response to decreases in program recruitment or participation rates.

Market research has also allowed for youth input— another strategy to make programs more attractive. In Washington, DC, Project My Time sites conducted a thorough assessment of what youth want and need from programming. In Chicago, early focus groups with older youth revealed that teens in OST programs were on the whole disappointed: They did not like being mixed into programs with children of all ages, and they felt that they were used to babysit younger children. They also reported that the adults in the program didn’t meaningfully engage them, and they instead wanted to be with adults who were experts in their field. In Providence, youth in focus groups said staff needed to be cool, fun, and interesting, while initiative leaders noted the importance of adult authenticity, caring, and appreciation of youth culture.

City-level supports have helped programs look beyond traditional methods of recruitment (e.g., flyers and newsletters) to employ the technology and social media platforms that middle and high school youth use. San Francisco’s initiative, for example, has marketing material on its website (www.sfkids.org), and New York’s Department of Youth and Community Development has Facebook, Myspace, and Twitter accounts.

All of the cities have formed partnerships with schools to support recruitment efforts. In three of the city-level OST initiatives, programs have the opportunity to recruit in schools during school lunch, during school-based information fairs, and sometimes during class time. Providence After School Alliance (PASA) organizes recruitment fairs early in the school year so youth can learn about the various afterschool activities offered in their AfterZone. Providers have reported that the initial interactions between youth and program staff at these events are a determining factor in students’ decisions to join programs. CincyAfterSchool partners with the city’s Community Learning Centers, which are Cincinnati Public School neighborhood educational resource hubs, to organize and implement social marketing strategies to recruit older youth, and also works with school-based afterschool providers to develop program and activity marketing tools that are age-appropriate and targeted to appeal to older youth. Project My Time promotes recruitment and participation in afterschool programs by organizing recruitment fairs and promotional weeks, distributing information and other marketing materials throughout the community, and working with schools to do outreach through staff meetings.

Within these fairs, providers report that some strategies work better than others. Above all, programs need to have a hook: some tangible demonstration that will engage youth interest, whether it be a video or a piece of jewelry that youth participants would make in a program activity, or a taste of an activity itself such as a soccer demonstration. Successful recruitment also means repeated follow-up with youth, particularly in middle school. And though these fairs are important, many providers said they were recruiting on their own and did not rely solely on fairs to recruit youth.

Coordinating and networking OST programs across the city

Often as a first step toward coordinating and networking OST programs, city-level initiatives in this study have created program locators, many of which are available on their websites. Cities use these tools to do outreach to families and youth and to better match programs with community needs and preferences, which in turn affects recruitment and retention. Families, programs, and youth use program locators to identify the programming options available to them.

Information about where programs are located helps cities address barriers to participation such as gang territories, transportation challenges, and school rivalries. In Chicago, for example, one respondent noted, “There are some times that maybe we have to make sure that several of us are doing the same thing because of gang lines or some type of boundary that we’re not aware of.” Networking among programs builds on the community’s knowledge about challenges to youth participation and enables programs to address those challenges; city initiatives are key to facilitating this knowledge sharing.

Many city-level respondents reported that another benefit of networking OST providers through city initiatives is the opportunity for programs to learn about other organizations’ offerings, share best practices, and solicit help with challenges, all of which in turn can address issues of participation and retention.

Collecting and using information, including developing MIS

A critical component of each city-level effort to connect and improve programs is an MIS used to track attendance and participation in the initiatives’ funded programs.xv These databases have been crucial in understanding participation because they increase knowledge about attendance patterns within programs and across initiatives.

Providence, for example, mandated the use of youthservices.net, a customized participation tracking database, throughout its three AfterZones. This MIS can be updated by multiple users in real time, enabling initiative managers to access up-to-date data on a daily basis and to intervene quickly to provide technical support to programs experiencing drops in attendance. Using its data system, Providence not only enables programs to target recruitment to previous participants but also supports programs in placing individual phone calls to participants to encourage continued attendance. While PASA chose a single data system, other cities face the challenge of coordinating multiple data systems, which can create confusion and capacity problems for providers.

In addition to using citywide management information systems, OST initiatives in this study support and encourage programs to conduct their own evaluations and in some cases broker relationships between researchers and programs. For example, in Chicago the OST Initiative has connected the Chicago School of Professional Psychology with tutoring programs to conduct pre- and postsurveys about socioemotional needs. Some cities, like New York and Providence, are implementing citywide OST evaluations, the results of which can be used to shape and inform programming to support improved participation.

Participation databases*

  • Four cities use customized databases from Cityspan.
  • One city uses a customized database from Cayen.
  • New York created its own database, DYCD Online.

*See Appendix A for more information about city initiatives

 

Supporting citywide quality improvement efforts

Each of the cities in this study is involved in efforts to improve program quality through the development and implementation of quality assessment tools. Some cities use an assessment tool based on existing and valid measures or standards such as the Youth Program Quality Assessment (YPQA). For example, PASA adapted the YPQA based on input from OST stakeholders across Providence to develop its own version of that tool, the Rhode Island Program Quality Assessment (RIPQA). Other initiatives, such as the Department of Youth and Community Development (DYCD) in New York City, are encouraging the use of statewide assessment tools, such as the New York State Afterschool Network (NYSAN) Program Quality Assessment Tool. Washington, DC and San Francisco are developing citywide “minimum standards” for quality programs.

While all of the cities in this study have quality improvement efforts in place, they have developed different strategies to incorporate quality assessment into the life of programs. Some provide incentives to programs for going through the process. For example, Providence has implemented an endorsement system for those programs considered to be high quality; endorsed programs earn an extra 5 percent in program grants from PASA. Chicago and Cincinnati provide targeted support to programs through training in the areas in which the program does not score well on quality assessments, while San Francisco, Washington, DC, and Chicago require action plans or improvement plans from programs to address deficiencies in their quality assessments. Chicago and Washington, DC are planning to use the results of quality assessments to assist in grantmaking decisions.

While quality improvement does not necessarily directly support participation, it can point to areas of improvement for some of the key program features that—as we know from this study as well as prior research—are related to sustained participation. For example, through Chicago’s quality improvement pilot, the city was able to identify three areas for improvement across the participating programs: connections between OST and the school day, program planning, and curriculum. In San Francisco, initiative leaders hope that common program quality measures used across funding streams “will bring quality up, and as a result, bring more kids into the programs.”

Coordinating and providing professional development and technical assistance

Providers reported that professional development and technical assistance offered by city initiatives were indirectly helping programs with recruitment and retention, often by using the results of quality assessments to identify areas to guide staff development. The majority of initiative-level professional development opportunities are organized around the core principles of youth development; they include topics such as classroom management, youth culture, community mapping, and mental and behavioral health. Providence and Washington, DC both use the Building Exemplary Systems of Training (BEST) tool for youth workers, which delivers the national AYD (Advancing Youth Development) curriculum—core training in youth development principles, concepts, and practices.xvi Providence’s training is provided two to four times per year, free of charge to youth workers across the city. New York and San Francisco provide professional development programs that explore how to work with older youth and the developmental differences between middle and high school youth. New York contracts with Partnership for After School Education, a New York City-based organization that promotes and supports quality afterschool programs, to provide comprehensive professional development;xvii San Francisco delivers professional development through the school district and the Department of Children, Youth & Their Families.

Quality improvement tools used*

  • Youth Program Quality Assessment (YPQA)
  • Rhode Island Program Quality Assessment (RIPQA)
  • Strive Six Sigma
  • Search Institute’s Developmental Assets Tool
  • New York State Afterschool Network (NYSAN ) Program Quality Assessment Tool

*See Appendix A for more information about city initiatives.

 

City-level investment in program staff through professional development is designed to support both youth retention in programs and the sustainability of the programs themselves. Respondents from every city noted that staff members who received training were more likely to remain with an organization long term, leading to continuing and successful relationships with the youth in the program; as noted in Chapter 2, providers reported that meaningful long-term relationships support better retention among youth.

Emerging opportunities for cities to support participation

Two additional promising opportunities for cities to support participation were identified during interviews and supplemental discussions with city respondents: engaging families and developing relationships with school and district leadership.xviii

Engaging families

As noted in Chapter 2, program providers reported that family engagement is critical to their work with older youth but difficult to implement well. In supplemental conversations, city-level respondents reported viewing family engagement as critical to OST participation, and some are developing city-level strategies to support and engage families.

Through their social marketing and public awareness campaigns, for instance, initiatives can build families’ knowledge of the benefits of participation in OST programs—in particular, how OST programs for older youth can better position participants for postsecondary success. City initiatives can also help families be “good consumers” of programs by providing information and resources on program quality and ideas on how to help older youth make good choices.

In addition to working directly to engage families, city-level initiatives can build the capacity of program providers to ensure that family engagement is part of the “daily business” of their programs by supporting professional development on family engagement and by embedding family engagement in professional development in other areas. For example, program staff training in how to support learning and/or college preparation in OST programs can and should include the role of families.

Developing school district partnerships to support learning

In interviews, providers and city-level respondents reported the importance of having effective working relationships with schools to support and improve participation. However, these relationships occur on a school-by-school basis, with little city-level coordination. Some city-level respondents indicated that, because education reform agendas in their cities view out-of-school time as a core learning support and strategy, they see an opportunity to form meaningful partnerships with school districts to support expanded learning opportunities.

In all cities in this study, the school district is involved in some way with the OST initiative; in San Francisco, for example, the district is one of the citywide collaborators in the Afterschool for All initiative. In another significant collaboration, New York’s Department of Youth and Community Development developed a memorandum of understanding with the city’s Department of Education (DOE) in which the DOE funds the use of school buildings when they would typically be closed and agrees to give programs access to schools during the school year and summer at no charge. The DOE also provides security, fingerprinting of staff, and snacks for OST programs in school buildings.

But the real benefits for participation are the connections around learning—how to combine the strengths of schools and OST to support youth across the district, how to share data about youth in meaningful ways, and how to get school and OST program staff working together districtwide to support students.

Program Views on Participation in OST Initiatives

While all the cities in this study reported that they are directly and indirectly addressing issues of access, enrollment, and sustained participation by providing the supports described above, this study was designed to learn whether and how the supports actually help programs in their efforts to improve older youth participation in OST programs. Therefore, through interviews and our survey we sought the programs’ perspectives on the contributions of initiative-level supports toward increasing recruitment and retention and asked what they see as both the value-added and the challenges of being part of an initiative and receiving support. We were particularly interested in exploring the degree to which the programs in this study are using these resources and examining whether there are any systematic relationships between initiative-level supports and higher rates of retention in OST programs.

This section reports on (1) the findings from the full survey sample to understand which, if any, initiative supports programs value as important to their participation goals (regardless of their ability to reach those goals) and to determine whether there were any relationships between reported use of initiative-level help and retention rates; and (2) the findings from the interviews with the 28 programs we visited, during which we asked program providers about how they use initiative-level supports and what advantages and disadvantages they perceived regarding these supports. Though largely descriptive in nature, this information can be used to inform future city-level investments in OST and sets the stage for more in-depth exploration of the role of city-level supports in promoting participation.

The programs in our survey sample identified ways in which city initiatives were supporting their participation goals, but none of these efforts was significantly related to higher retention. Interviews with providers did, however, offer insights into the benefits and challenges of being able to access and receive support from initiatives.

 


Value of city-level supports to program participation goals

When asked about the value of city-level supports to their enrollment and engagement goals, 72 percent of programs in the survey sample agreed that the initiative helped increase their enrollment of older youth, and 68 percent agreed that the initiative helped them increase engagement of older youth, suggesting that city initiatives are supporting programs’ abilities to attract youth and interest youth in their activities, and therefore contributing to programs’ participation goals.

The survey also asked about the variety of services that cities might provide to support program efforts to engage older youth. Table 4.1 lists the 10 city-level supports that programs, regardless of participation rates, reported as being the most helpful in aiding their own participation goals. The top three were

  • Increasing connections to other organizations (79 percent)
  • Providing funding (73 percent)
  • Helping with access to participation data (70 percent)

It is notable that of the top 10 supports, 4 relate to getting and using information (access to participant tracking information, training on using the data systems, learning best practices, and involvement in program evaluation). This finding points to the potential of city initiatives in continuing to support programs’ efforts to use data to inform their recruitment and retention strategies.

The supports that providers identified as least helpful to their participation goals, with one-quarter or fewer positive responses, were

  • Recruitment/referrals/interviewing of staff and volunteers
  • Coordinating fundraising or grant writing
  • Helping with budgeting or finances
  • Providing curricula
  • Decreasing competition for funding through coordination of initiativewide efforts

Interestingly, the only significant differences between high- and lower-retention programs’ responses about their views of city supports were cases in which a greater proportion of lower-retention programs reported that certain supports were helpful for participation goals:

  • 26 percent of lower-retention programs responded that the OST initiative was helpful with recruitment/ referrals/interviewing of staff and volunteers, compared with 13 percent of high-retention programs.
  • 43 percent of lower-retention programs responded that the OST initiative was helpful by providing in-kind resources, such as snacks, compared with 23 percent of high-retention programs.

The lower-retention programs may be in greater need of basic help with resources and capacity than the high-retention programs and would therefore find snacks and other donations as well as help with staff recruitment more helpful than high-retention programs would.

Program interviews, however, painted a more nuanced picture of the role of city-level supports in participation. Although the overall survey sample indicated that city-level supports were helpful, many of the leaders of the 28 high-participation programs in our interview sample reported feeling that they must rely more on their own program practices for recruitment and retention of older youth than on supports or services at the initiative level. Characteristics of the program interview sample suggest, in part, why this might be the case. Many of the programs predate the existence of the initiative in their city; about half of the surveyed programs have been in existence for 5 or more years, whereas the initiatives were established within the last 3 to 6 years. Almost three-quarters (71 percent) of programs interviewed have been involved in their respective initiatives for 2 years or less. Since the programs we interviewed were already using many effective recruitment and retention practices when they became involved with the city initiative, they might not report that the initiative supports added value to what they were already doing. Further, many of the programs in this study were already part of a larger OST intermediary such as a Boys & Girls Club or a Beacon initiative and were therefore already receiving the kinds of supports from their parent organizations that the city initiative also provided.

Managing city-level OST supports: benefits and challenges

Programs interviewed for this study were clear that some of the city-level participation supports created both benefits for and challenges to program implementation. This chapter concludes with a discussion of four issue areas that survey and interview data identified as having potential benefits to participation but are associated with inherent challenges: funding, program coordination and competition, data and evaluation, and quality assessment.

Funding and funding stream issues

On the program survey, the vast majority (76 percent) of programs reported that funding from the initiative was helpful to them. It is no surprise that more funding would be helpful to overall program operations; several of the interviewed programs, however, explicitly made the connection between funding and its indirect impact on retention. Providers reported that funding can, for example, influence the number and quality of staff hired (which in turn affects staff–youth ratios and relationships), the training and technical assistance available, the number of programs and/or slots available, and program sustainability through supporting program quality and providing matching dollars.

In addition to the funding itself, program staff in San Francisco articulated the value of the role of initiative staff in helping the programs navigate the complexities of the various state and local funding streams with different requirements. As one program director in San Francisco commented, Afterschool for All has helped “to demystify the convoluted funding for after school by bringing everyone into the same room to unravel it.”

Funding from initiatives, however, comes with its own requirements, which often include goals for participation. Just over two-thirds of the programs surveyed (68 percent) indicated that they receive funding that stipulates specific participation requirements.

Program coordination and competition

As noted in Table 4.1, a full 79 percent of programs surveyed reported that increasing connections or partnerships with other organizations helps support participation goals. According to the providers we interviewed, these increased connections to other programs have yielded several benefits for participation, including the ability to share knowledge about and connect youth to services, increased attention to quality, and greater awareness of weaknesses or needs in the initiative as a whole such as gaps in services or resources.

However, although programs acknowledged the benefits of coordination across the initiative, very few (24 percent) reported on the survey that the OST initiative decreased competition among programs; in fact, during the interviews, providers noted that competition has increased in some ways among providers for youth and for funding. For some, the structure of the initiative sets up competition for participants and funding:

[Program competition] is one of the big criticisms that providers have of being part of [the initiative]. They love the idea of looking very intentionally at this work of engaging older youth. They understand that you need high quality. But they also feel as though our set-up makes them . . . have to compete for the same young people, and so then, in turn, the same dollars at any given site.

For others, the structure of school-based programs makes it difficult to meet attendance requirements because older youth have other conflicting activities or obligations. Having to be at a program at a certain time and attend the academic portion of the program often conflicts, for instance, with sports practice, which can eliminate a large group of youth from enrollment.

Data and evaluation: building knowledge while creating management challenges

Our study showed that the clear benefit of MIS is a better knowledge of participation and of how recruitment practices are working or need to be adjusted. But programs face challenges related to their use of initiatives’ data systems, especially when it comes to training a frequently changing staff. Many programs have to use multiple databases (e.g., for different funding streams, for their larger organization such as a YMCA, and for the initiative) for attendance tracking or grant reporting, creating additional staff training needs and redundancies in data entry. One program in Chicago had five different databases into which staff members were entering information and still was planning to develop a sixth that would provide the data that the program itself needed for program improvement.

Positioning quality assessment

Programs in each city are involved in quality improvement efforts, which, as noted earlier in this chapter, indirectly support participation. On the whole, programs appreciated the validation of their existing practices and procedures as well as the suggested steps for improvement that resulted from the quality assessments.

Programs expressed frustration, however, with the practice of tying program quality to participation numbers. Some program providers felt that their city initiatives equated the notion that youth “vote with their feet” with program quality, when in fact there can be many factors that cause enrollment or attendance to fluctuate, including friends switching programs together, the presence of new programs at a school site, and even the time of year. One concern expressed among programs was that punitive measures associated with participation levels might lead programs to recruit only youth whom they think will be “good” participants. One respondent noted, “They won’t even try to reach out to these kids that are going to be a little bit more difficult.”

Summary

All of the cities in our study employ a set of city-level supports to improve access to and sustained participation in OST programs for older youth, namely

  • Engaging in citywide recruitment efforts, including social marketing
  • Coordinating/networking OST programs across the city
  • Collecting and using information, including developing MIS
  • Supporting citywide quality improvement efforts
  • Coordinating and providing professional development and technical assistance

Some cities recognized the usefulness of their roles in supporting family engagement and developing district-level partnerships for improving participation and retention among older youth. None of these city-level strategies, however, appeared targeted to the participation of older youth in particular. Rather, they were part of overall initiative-building efforts to support the quality and sustainability of OST programs.

When the surveyed programs were asked whether the city-level supports helped with their enrollment and engagement goals, the majority responded that they did, and programs identified a set of supports—chief among them getting and using information—as being important to their participation goals. However, in part because of the nascent stage of development of the city initiatives, and in part because many programs in this sample appeared to already have well-developed infrastructures to support participation, high-retention programs in this study were not more likely than lower-retention programs to have used any of the available supports from the initiative. As the study’s implications presented in our next chapter suggest, future investments to improve participation should consider these findings carefully to ensure that investments are wisely spent on supports that may end up making the biggest difference.

< < Previous | Next > >

References

xiv. Funding in San Francisco comes through the Department of Children, Youth & Their Families (DCYF) or the San Francisco Unified School District (SFUSD), not through Afterschool for All (AFA).

xv. In the case of San Francisco, the initiatives are funded by DCYF or SFUSD.

xvi. See nti.aed.org/NationalBEST.html for more information.

xvii. See www.pasesetter.com for more information.

xviii. These issues emerged most clearly in conversations with the study’s Community of Practice. See Appendix B for more information.