AfterZones: Creating a Citywide System

Click here to download the full report:
 AfterZones: Creating a Citywide System

Based on the need for an efficient and effective way to make high-quality OST programs available to large numbers of low-income youth, cities have begun to invest in building systems that can coordinate, grow, improve and sustain the independent efforts of OST providers. This study examined the structure and operations of the AfterZone initiative, a citywide system-building effort in Providence, RI, that aims to provide high-quality, accessible OST services to the city’s middle school youth. Specifically, the study looked at the implementation of the initiative’s unique multisite, neighborhood campus service delivery model; it also documented the mechanisms that PASA (the intermediary created to lead the initiative) put in place to engage and retain middle school youth, to ensure AfterZone programs are high quality and to sustain the initiative beyond the start-up grant period.

In the five years since its inception, PASA can point to many significant achievements. Chief among these is the fact that Providence now has an accessible, citywide system of after-school programs of solid quality where almost no good middle school programming existed. The AfterZone initiative integrates as many as 100 of Providence’s OST providers into a network with a coordinated schedule and a centralized registration process, and PASA has established a grant application system for distributing program funds. The initiative includes a system for transporting youth to programs outside of the anchor middle schools and then home at the end of the day. Through consistent data collection and an effective use of, PASA keeps close watch over enrollment and attendance—in individual programs, in local AfterZones and at the citywide level—and uses these data to inform planning and decision-making. Finally, the quality of AfterZone programs, as measured by systematic observations and youth feedback surveys, indicates that most programs, especially those focused on the arts, provide youth with high levels of adult support and with opportunities to learn and interact with peers in positive ways.

Other aspects of PASA’s system-building efforts have met with more qualified success, and serious challenges do remain. To date, eighth graders have not been as attracted to the AfterZones as have youth in sixth and seventh grades. PASA’s quality improvement strategies, while thorough and systematic, lack consistent follow-up, limiting their power to produce and document improvement. The transition to CBO management of local AfterZones, which was intended to embed the initiative more deeply into the fabric of the city, has not progressed as smoothly as was hoped, with some CBOs feeling underfunded. Finally, like with most OST programs around the country, PASA’s long-term sustainability plans for the initiative have been hampered by a severe economic downturn that has affected all sectors of the national and local economy.

In this concluding section, we discuss in greater detail the accomplishments of PASA’s systembuilding efforts and identify areas that need further development to ensure youth benefit and the system can be sustained.

Leadership and Management of the Initiative

Active support from the mayor and PASA’s strong leadership shaped the initiative and propelled it forward. The mayor’s active engagement throughout the initiative has been crucial to the progress Providence has made in building its citywide OST system. Riding on a wave of popularity and change following his election, the mayor was able to bring key city players together to plan the initiative, and he leveraged commitments from city departments and the school district to redirect their resources (staff, facilities and funds) to help support it. As an advocate and champion of the AfterZone initiative both within Providence and beyond, he expanded available resources and brought the initiative to the attention of national foundations and elected officials. He worked closely with PASA throughout, making sure the intermediary had the support and cooperation it needed from the city to carry out its mission. He is continuing efforts to secure the initiative’s long-term survival.

PASA’s focused, skillful and strategic leadership has also been vital to the initiative’s progress. By carefully cultivating relationships with providers and focusing on capacity building and collaboration, PASA overcame the provider community’s initial concerns about the intermediary’s effect on their access to youth and funds, brought almost all of Providence’s OST providers under the AfterZone umbrella, established itself as a resource for the city and maintains broad-based support. While keeping a steady focus on the goals of the AfterZone initiative, PASA has made modifications based on changing conditions on the ground, been open to feedback and suggestions, and learned from missteps and mistakes.

The intermediary’s three senior staff members possess a blend of talent, knowledge and experience—in advocacy, strategic planning, youth program management, community relations and regional planning—that helps PASA meet the challenges of its far-reaching mission. PASA has frequently augmented its in-house talent by seeking expert opinions, reviewing relevant research, and hiring various experts to help it develop a three-year business plan and a quality improvement strategy as well as conduct studies of early implementation. Finally, PASA’s project-management and decision-making approach is data-driven, and the intermediary makes good use of its data tracking tool to monitor progress and determine what is working and what is not.

An effort to transition from PASA’s direct management of local AfterZones to management by CBOs was intended to embed the AfterZones more deeply into the fabric of the city, but this effort did not progress as smoothly as was hoped. PASA’s leadership has been crucial to building the infrastructure needed to support a citywide system of high-quality programs. To keep the system going, however, PASA believes that the city’s public and private stakeholders, including its youth-serving CBOs, will need to increase their ownership of and investment in the AfterZone initiative. PASA pursued this goal by contracting with four local CBOs that had experience running programs for children to serve as site management agencies. Supported by 21st CCLC grants, each agency is responsible for a managing a single AfterZone middle school, which entails hiring and supervising school-based AfterZone staff and overseeing all of the logistical, coordination and supervisory tasks that are part of the daily operations of the school program.

However, the transition to CBO management of local AfterZones remains problematic. PASA would like the site management agencies to play a more active role in tracking attendance, developing relationships with the school-day faculty and staff, fundraising and performing other management tasks. It is not clear, however, that the agencies have the organizational capacity to carry out an expanded role. Recent and severe cuts in their own budgets, staff and services have made it difficult for them to commit more staff time and resources to managing AfterZone middle school programs without additional compensation.

Because PASA believes that CBO management of the local AfterZones is necessary for the long-term sustainability of the initiative, it plans to continue searching for funding and technical assistance providers to help strengthen the internal infrastructures of the agencies weakened in the economic downturn. PASA hopes that this type of support will enable the agencies to play the site management role more effectively, while simultaneously strengthening their own internal organizational capacity and long-term financial viability.

Implementing a Multisite Campus Service Delivery Model

Off-site programming is a costly option but may provide youth with enriching learning experiences. The AfterZone multisite service delivery model, where programs are offered in middle schools as well as in the community, presents unique opportunities and challenges. To access off-site programs, youth have to be shuttled from the middle school to the off-site location at the start of the afternoon, and then back to the middle school in time to get picked up by parents or board a school bus home. The fleet of vans and shuttle buses PASA and its partners have patched together is both expensive and logistically complex to run. The campus model also entails funding additional managers—namely, AfterZone managers—to integrate the school-based and offsite programs. However, off-site programs offer youth enriching learning experiences in, for example, an art center or a marina, that are simply not possible in a school setting. Consequently, PASA and the AfterZone Coordinating Councils believe the potential benefits of offering youth these unique opportunities outweigh the costs of transportation and additional time required from staff to carry them out.

The challenge of integrating and supporting program providers operating in multiple locations is met through an effective use of staff and open channels of communication between the field and PASA. The AfterZone multisite service delivery model presents more oversight and management challenges than the typical school-based after-school program. Assigning a site coordinator to each middle school and an AfterZone manager to each AfterZone appears to provide enough field staff to enable daily operations to run smoothly. With the help of assistant staff and volunteers, the site coordinators attend to the many daily logistical details, such as taking attendance, making sure youth get to off-site programs, and ensuring youth and providers leave classrooms and other school facilities clean and in good condition.

The AfterZone managers play a particularly crucial role. In addition to overseeing the off-site providers in the local AfterZone, they provide intensive support to the school-based site coordinators, dispensing advice and guidance for the myriad tasks and decisions that make up their daily routine. In addition, the AfterZone managers make sure PASA is kept abreast of developments on the ground. PASA senior staff’s supervision of AfterZone managers’ work in the field helps ensure successful and consistent implementation of citywide strategies. PASA’s role is also critical in that it assumes time-consuming tasks that can be most efficiently handled at the city level—such as maintaining the citywide youth participation database, organizing professional development trainings and engaging in fundraising.

Engaging Middle School Youth

PASA developed the AfterZone model with a keen sensitivity to the developmental needs of middle school youth. Middle school youth have historically been difficult to engage in OST programs, in part because few programs have been developed specifically for their age group. Based on extensive upfront research, PASA was able to identify qualities known to be important in promoting participation among this age group, such as autonomy and choice. It was also able to tailor AfterZone programming and recruitment and retention strategies to middle school youth’s social, emotional and academic needs and interests. For example, the AfterZone initiative provides youth with opportunities for choice through a multisite service delivery system, wherein youth can attend programs at their own school or at off-site locations in the community. While secure in their own anchor school, youth can safely try other types of programs in less familiar environments. PASA’s strategies—such as incorporating aspects of youth culture into the style and content of programming and giving youth a voice within the activities— appeal to the sensibilities of this age group and help set the AfterZones apart from programs for younger children.

While the AfterZones appeal to middle school youth’s need for more autonomy and choice, PASA and the program providers also recognize youth’s ongoing need for adult support, guidance and encouragement—and design recruitment and retention practices accordingly. For example, recruitment strategies are based on making direct personal contact with youth and building relationships between youth and adults, a critical principle of positive youth development. In addition, PASA seeks to hire young assistant staff the youth can relate to, look up to and look forward to seeing when they come to the program. Such practices are designed to help youth develop personal relationships with staff and activity leaders, which is a key factor in engaging and retaining youth and a crucial aspect of youth development programs.

Successful recruitment practices included face-to-face contact with staff followed by close monitoring and follow-up phone calls. PASA sets targets for enrollment and attendance levels and puts these targets in the Memoranda of Understanding with CBO partners. In addition, PASA closely monitors enrollment and retention data with, which enables it to intensify outreach if necessary or identify reasons for attrition or low attendance. Recruitment practices that allow youth to meet the program providers and sample program materials and projects are particularly successful. PASA doesn’t rely on just one recruitment strategy, such as recruitment fairs, but also incorporates additional face-to-face opportunities. Staff engage in targeted phone outreach to recruit students and place reminder phone calls to all enrolled youth at the beginning of the session. Further, if youth do not attend programs for which they signed up, or are absent from the program, a staff member will call to find out why they are not attending and try to reengage them.

The AfterZones succeeded in enrolling nearly half of the students who attend the seven participating middle schools. Involving the older middle school youth, especially eighth graders, however, proved more difficult than expected. Most AfterZone participants were sixth and seventh graders. Some schools saw enrollment numbers begin to decrease in seventh grade, but in all schools there was a sizable drop in enrollment among eighth graders. The difficulties that PASA has had attracting eighth graders highlights how programming for middle school youth needs to be finely attuned to the rapid developmental changes youth are experiencing during the middle school years. Activities that appeal to students who have recently left elementary school may not appeal to older middle schoolers trying to distance themselves from their younger schoolmates as they prepare to move on to high school. To better serve youth as they move into the higher middle school grades, PASA and AfterZone providers will need to learn more about the interests and concerns of youth in this age group—as well as the barriers that may be preventing them from enrolling (such as increased responsibilities at home)—to inform recruitment strategies and programming.

Improving Program Quality

PASA implements comprehensive quality improvement and professional development strategies and activities; however, voluntary participation and lack of timely follow-up may decrease the power of these strategies to improve program quality. The quality improvement strategies used by PASA with AfterZone providers include an agreed-upon set of dimensions or standards that define high-quality programming; an assessment tool, the RIPQA, used to gauge program quality along each of these dimensions; and a feedback mechanism and professional development opportunities designed to build the capacity of activity instructors to incorporate best practices into their programs. In addition to activity assessments and professional development, PASA implements a set of requirements and incentives designed to keep and further improve their most committed program providers. Programs that maintain a specified level of youth attendance and participate in the RIPQA process are rewarded a financial bonus and given expedited review on their application for inclusion in the AfterZone initiative. Putting these strategies into place citywide by the third year of the initiative represents a considerable achievement.

PASA’s decision to make providers’ participation in the RIPQA assessment process and professional development activities voluntary has the benefit of fostering a culture of collaboration and capacitybuilding between PASA and the providers. However, the power of these systems to effect change is limited by the lack of regular follow-up. While thorough and systematic, the use of the RIPQA-based observation and feedback process is costly and timeconsuming. A team of trained observers conduct a 40–50 minute observation using the RIPQA tool. After the observation, they compare scores, write up an action plan based on their findings and provide feedback to the program’s instructors. Because of the large number of programs to observe and the limited number of staff to do the observations and feedback, little time is left to return to the program for follow-up observations.

Follow-up observations could ensure that the program provider has made suggested changes or, in cases where program quality is quite high (as it generally was in the observed programs), that the strong practices previously seen are being maintained. This would create a true system of continuous program improvement, which would be more effective in raising program quality. In the 2009–10 school year, PASA plans to focus on a small group of providers whose RIPQA scores indicated a need for improvement and provide this group with more intensive coaching and frequent follow-up.

Sustaining PASA and the AfterZone Initiative

Long-term sustainability for the AfterZone initiative is a challenge. The short-term financial outlook for PASA and the AfterZone initiative is good, thanks in large measure to a new grant from The Wallace Foundation; a one-time federal appropriation; and continuing support from the city, the school district and the police department. The grant from The Wallace Foundation will give PASA three more years to search for new, sustainable sources of revenue, and unless the city experiences more severe budget cuts, PASA expects to continue receiving funds and in-kind contributions from municipal sources.

In our experience studying OST and other youth programs, achieving long-term sustainability and finding renewable sources of funding is often difficult, even for proven programs in good economic times. In the AfterZone initiative’s case, the flow of public funding sources, such as the 21st CCLC grant program and state subsidies for childcare, has slowed as a result of the current recession. Similarly, the recession has decreased private foundation endowments, which has slowed private giving.

The effort to make local AfterZones an integral part of the city’s agencies, departments and schools has succeeded to the extent that the police department, the school district and the mayor’s office continue to direct a portion of their own resources toward supporting it. In large measure, the marshalling of city resources has been engineered by the current mayor, for whom building a high-quality OST system in Providence has been a high priority. It is impossible to know whether a new mayor will continue to leverage support for the initiative.

Similarly, it is too soon to predict whether the extended-day learning movement, which advocates integrating school-day and after-school programs into a seamless network of learning opportunities for youth, will open up a new and financially viable future for the AfterZone initiative. With a grant from the Charles Stewart Mott Foundation, PASA has begun to plan for what it hopes will be the next phase of the initiative’s evolution: namely, providing enrichment activities to support school-day learning. The challenge for PASA will be to demonstrate to district leadership and the public that highquality OST programs can improve student learning while maintaining the AfterZone model’s youthoriented, choice-based approach to programming.

Final Thoughts

This report documented the experiences, accomplishments and challenges of Providence’s effort to design and implement a unique citywide system to coordinate, grow and improve its OST programming. Building on its existing but largely independent provider community, and galvanized by committed and effective leadership, the city made enormous progress toward reaching its goal of making high-quality after-school programs accessible to low-income middle school youth in a relatively short period. It has shown that a campus model is feasible and indeed attracts middle school youth. It has also demonstrated that, with a concerted effort on tracking program quality and providing professional development, programs of solid quality can be put in place within a four- or five-year time horizon.

P/PV’s final report on the AfterZone initiative will be an in-depth study of the ways in which youth participate— how frequently they attend, how long they remain involved, the range of programs they select and how they spend their out-of-school time when they are not in AfterZone programs. The report will also examine whether and how youth benefit from participating, in terms of their developmental and school-related outcomes, compared with their nonparticipating peers.

< < Previous | Next > >