Contents

AfterZones: Creating a Citywide System

Click here to download the full report:
 AfterZones: Creating a Citywide System

Increasingly, research has shown that participation in out-of-school-time (OST) programs can lead to improvements in youth’s educational outcomes (e.g., academic achievement, school behavior, attitudes toward school, attendance and educational expectations); enhance social and emotional development (e.g., self-esteem, positive social behavior); and reduce the likelihood that they will engage in risk-taking behavior.1 However, two conditions must exist for these benefits to accrue: The programs must be high quality and youth must participate over a sustained period of time.2

There is compelling evidence that participation in structured organized activities dramatically falls when youth enter middle school. For example, an evaluation of after-school programs that were part of the Extended-Service Schools Initiative found the average attendance rate for youth in grades 6 through 8 was 1.6 days per week compared with 1.9 days per week for youth in grades 4 and 5 and 2.2 days per week for youth in grades 1 through 3.3

Yet, during the middle school years, youth face many new challenges and need the support that high-quality OST programs can provide. Self-esteem tends to drop as youth enter middle school, and they begin to feel less confident in their ability to master academic subjects, at the very time when pressures to achieve are increasing. School-day curricula become more rigorous and demanding, and many youth begin to experience academic failure.4 Their desire to assert their independence and make their own decisions increases, but so does their potential to engage in risky behavior, especially during unsupervised time after school.5 Low-income youth may be particularly vulnerable because their families and communities lack the resources needed to provide quality structured activities during the after-school hours.

Within cities, the rapid growth in OST programs over the past two decades has often resulted in a fragmented landscape of independent efforts with precarious funding and uneven quality.6 Acknowledging the need for an efficient and effective way to sustain and improve OST programs and make them available to more low-income youth, a growing number of cities have begun building systems to support after-school initiatives.

Building on a long history of investments in OST learning, The Wallace Foundation launched an out-of-school learning initiative in 2003. The initiative was created to support citywide system-building efforts that could advance three interrelated goals for the OST field: improving program quality, making programs accessible to youth who need them most, and improving youth participation so more children can realize benefits. The Foundation granted funds to five cities to support their afterschool system-building initiatives: Boston, MA; Chicago, IL; New York, NY; Providence, RI; and Washington, DC.

One of the cities, Providence, RI, developed a citywide after-school initiative for middle school youth called the AfterZone initiative, to be led by the Providence After School Alliance (PASA), a local intermediary. The following pages summarize a report by Public/Private Ventures (P/PV) that documents AfterZone’s implementation. P/PV carried out data collection for this study from February 2008 through March 2009. A second P/PV report that focuses on how youth participated in AfterZone programs, and the relationship of various patterns of participation to youth outcomes, will be published in 2011.

The AfterZone Model

The AfterZone model has two features that distinguish it from other citywide after-school initiatives. First, in contrast to traditional after-school models in which programs are offered in a single school or center, the AfterZone model is based on a neighborhood “campus” structure where services are offered at multiple sites in a geographically clustered area. This provides youth with the opportunity to travel to programs located outside of the main facility—the “anchor” middle school—to local libraries, recreational and art centers, and other community facilities. Second, while many citywide initiatives address program quality, the AfterZone model places a particularly strong focus on continuous quality improvement. In planning the initiative, PASA set out to establish a single set of standards that would define high-quality programming and then incorporate these standards in all AfterZone offerings.

PASA’s mission is to utilize, coordinate and strengthen existing youth programs and community resources across the city to provide middle school youth with easily accessible, high-quality after-school programs. PASA is also responsible for putting mechanisms in place for training and supporting local programs and providers citywide. To carry out this mission and to sustain broad-based support throughout the initiative, PASA has worked closely with the mayor and leaders of the city’s public and private youth-serving agencies.

During the school year, the AfterZones offer twoand- a-half hours of programming four days a week in three sessions that run from September through May. Programs offered in the fall and winter sessions are 11 weeks long; programs offered in the spring session last 6 weeks. In 2008, the AfterZones began offering a four-week summer program, drawing youth from across the city. There are three distinct AfterZone campuses, each with a different menu of specific programs; while the particulars vary, all the campuses offer programs in the arts, life skills/leadership, sports and academic enrichment. Seven middle schools participate in the initiative, providing space for AfterZone programs and support for recruiting students.

The Study Design

Using a combination of qualitative and quantitative research methods, the study examines the implementation of the AfterZones’ unique features and documents the strategies used to: engage and retain the city’s middle school youth, ensure AfterZone programs are high quality and sustain the AfterZones beyond the start-up grant period. The study also examines the extent to which AfterZone programs incorporated practices associated with positive youth development.

The research methods were selected to document the early history of PASA and the AfterZone initiative as well as the mechanisms PASA put in place to manage, coordinate, strengthen and sustain the citywide network of AfterZone providers and to improve the quality of AfterZone programs. P/PV researchers aimed to include the perspectives of all key stakeholders, and data analysis was oriented toward identifying the challenges and successes of the citywide system-building effort. Data was also collected to provide a snapshot of the quality of AfterZone programs during the study. Primary sources of data were:

  • Site visits—Site visits were conducted in May 2008, December 2008 and March 2009. During each three- to four-day visit, P/PV researchers interviewed AfterZone staff, program providers, representatives of governance groups and other key stakeholders to learn about the initiative’s structure and operations and its implementation accomplishments and challenges.
  • Program observations—Observational assessments of a sample of AfterZone programs were conducted over two years by on-site consultants using a quality assessment tool known as the RIPQA.7 The tool provides ratings of a range of youth development practices in such categories as Supportive Environment and Youth Engagement.
  • Youth feedback surveys—Surveys asked youth about their experiences in AfterZone programs.
  • Surveys of instructors—Surveys of AfterZone program instructors gathered information about the training they received through PASA.
  • AfterZone program documents—Researchers reviewed documents generated by AfterZone leaders, such as business plans, annual reports, program brochures, the AfterZone website, and earlier evaluations.

 

Summary of Findings

In the five years since its inception, PASA has built an accessible citywide system of after-school programs and installed a number of mechanisms to coordinate, manage and support this system.

The AfterZone initiative integrates as many as 100 of Providence’s OST providers into a network with a coordinated schedule and a centralized registration process. To support this network, PASA established a grant application system for distributing funds and built a system for transporting youth to programs outside of the middle schools and then to their homes at the end of the day. Through consistent data collection and an effective use of a web-based data tracking tool, youthservices.net, PASA keeps close watch over enrollment and attendance— in individual programs, in local AfterZones and at the citywide level—and uses these data to inform planning and decision-making. AfterZone stakeholders adopted a single set of program quality standards as well, and PASA uses an observation and feedback process to assess the implementation of these standards and help providers incorporate them into their programs. Finally, PASA offers a menu of professional development and training opportunities for AfterZone providers.

Effective leadership was important to building a citywide OST system. Active support from the mayor and PASA’s strong leadership shaped the initiative and propelled it forward.

Providence’s mayor, David Cicilline, was able to bring key city players together to plan the initiative, and he leveraged commitments from city departments and the school district to redirect their resources (staff, facilities, funds) to help support the AfterZones. As an advocate and champion of the AfterZones both within Providence and beyond, he expanded available resources and brought the initiative to the attention of national foundations and elected officials. He worked closely with PASA throughout, making sure it had the support and cooperation from the city to carry out its mission; he is continuing efforts to secure the initiative’s long-term survival.

PASA’s focused, skillful and strategic leadership has also been vital to the AfterZone initiative’s progress. By carefully cultivating relationships with providers and focusing on capacity-building and collaboration, PASA established itself as a resource for the city’s after-school providers and maintained broad-based support. PASA has augmented its in-house talent by seeking opinions from outside authorities and reviewing relevant research; it hired various experts to help develop a three-year business plan and a quality improvement strategy and to conduct studies of early implementation. Finally, PASA’s approach to project management and decision-making is datadriven, and the intermediary makes good use of its data-tracking tool to monitor progress and determine what works and what does not.

An effort to transition from PASA’s direct management of local AfterZones to management by community-based organizations (CBOs) was intended to embed the AfterZones more deeply into the fabric of the city, but this effort did not progress as smoothly as was hoped.

PASA’s leadership was crucial to building the infrastructure needed to support a citywide system of high-quality programs. To keep the system going, however, PASA believes that the city’s public and private stakeholders, including its youth-serving CBOs, will need to increase their ownership of and investment in the AfterZone initiative. PASA pursued this goal by contracting with four local CBOs that had experience running programs for children to serve as “site management agencies.” Supported by 21st Century Community Learning Center (21st CCLC) grants, each agency was responsible for managing a single AfterZone middle school, which entailed hiring and supervising school-based AfterZone staff and managing all the logistical, coordination and supervisory tasks that are part of the daily operations of the school program.

However, the transition to CBO management of local AfterZones was problematic. PASA wanted to see the site management agencies play a more active role in tracking attendance, developing relationships with the school-day faculty and staff, fundraising and other management tasks. It was not clear, however, that the agencies had the organizational capacity to carry out an expanded role. Recent and severe cuts in their own budgets, staff and services made it difficult for them to commit more staff time and resources to managing the AfterZone programs without additional compensation.

Because PASA believes that CBO management of the local AfterZones is necessary for the long-term sustainability of the initiative, it plans to continue searching for funding and technical assistance providers to help strengthen the infrastructures of the agencies that were weakened in the economic downturn. PASA hopes that this type of support will enable the agencies to play the site management role more effectively, while simultaneously strengthening their own organizational capacity and longterm financial viability.

The AfterZone multisite service delivery model, in which programs are offered in middle schools as well as in the community, presents unique opportunities and challenges. Off-site programming is costly but has the potential to provide youth with enriching learning experiences.

To access off-site programs, youth have to be shuttled from the middle school to the off-site location at the start of the afternoon and then back to the school in time to get picked up by parents or board a school bus home. The fleet of vans and shuttle buses that PASA and its partners have patched together is both expensive and logistically complex to run. Despite these difficulties in reaching programs offered outside the middle school, the off-site programs provide youth enriching learning experiences—in, for example, an art center or a marina—that are simply not possible in a school setting. Consequently, PASA believes that the potential benefits to youth of offering these unique experiences outweigh the cost of transportation and additional time required from staff to carry them out.

The challenge of integrating and supporting program providers operating in multiple locations is being met through an effective use of staff and open channels of communication between the field and PASA.

The AfterZone multisite service delivery model presents more oversight and management challenges than the typical school-based after-school program. Assigning a school-based site coordinator to manage operations in each middle school and an AfterZone manager to each local AfterZone appears to provide enough field staff to enable daily operations to run smoothly. The AfterZone managers play a particularly crucial role: In addition to overseeing the off-site providers in their local AfterZone and supporting the school-based site coordinators, they make sure PASA is kept abreast of developments on the ground. PASA senior staff’s supervision and guidance of AfterZone managers’ work in the field helps ensure successful and consistent implementation of citywide strategies.

PASA designed the AfterZone model with a keen sensitivity to the developmental needs of middle school youth.

Based on extensive upfront research, PASA identified program qualities critical to promoting participation among this age group—such as autonomy and choice—and tailored the AfterZones’ programming and recruitment and retention strategies to middle school youth’s social, emotional and academic needs. PASA worked to incorporate aspects of youth culture into the style and content of programming, which also helped set the AfterZones apart from programs for younger children.

PASA and the program providers recognized middle school youth’s ongoing need for adult support and designed recruitment and retention practices accordingly. PASA strived to hire young assistant staff the youth could relate to, look up to and look forward to seeing when they came to the program. Such practices were designed to help youth develop personal relationships with staff and activity leaders, which is important for engaging and retaining youth and well recognized as a crucial aspect of effective youth development programs.

The AfterZones succeeded in enrolling nearly half the students who attended the seven participating middle schools. Involving older middle school youth, especially eighth graders, proved more difficult than expected.

Attendance data indicate that over the two years of the study, the proportion of students from the seven participating middle schools remained at 44 percent of the total student enrollment, ranging from roughly one third to one half of the students in each school. However, most participants were sixth and seventh graders. Some schools saw enrollment numbers begin to decrease when youth reached seventh grade, but in all schools there was a sizable drop in enrollment among eighth graders.

The difficulties the AfterZones experienced attracting eighth graders highlights how programming for middle school youth needs to be finely attuned to the rapid developmental changes occurring during these years. To curb program attrition as youth move into the higher middle school grades, PASA and AfterZone providers will need to learn more about the interests and concerns of youth in this age group as well as the barriers that may prevent them from enrolling (such as increased responsibilities at home).

PASA relies on several recruitment strategies, all of which are based on making direct personal contact with youth. Recruitment fairs designed to allow youth to meet AfterZone instructors and see examples of each program’s materials and projects are the most successful strategy. Staff also engage in targeted phone outreach to recruit students and place reminder phone calls to all enrolled youth at the beginning of each session. If youth do not attend programs for which they signed up, or are absent from the program, a staff member will call to find out why they are not attending and try to reengage them. PASA sets targets for enrollment and attendance levels and closely monitors enrollment and retention data, intensifying outreach as necessary.

PASA implements comprehensive quality improvement and professional development strategies and activities; however, the lack of timely follow-up decreases the power of these strategies to improve program quality.

The main quality improvement strategies used by PASA with AfterZone providers include: an agreedupon set of dimensions or standards that define high-quality programming; a research-validated observation tool, the RIPQA, used to gauge program quality along each of these dimensions; a feedback mechanism to discuss the findings from a one-time quality assessment using the RIPQA; and professional development opportunities designed to build the capacity of activity instructors to incorporate best practices into their programs. Putting these strategies in place citywide by the third year of the initiative represents a considerable achievement.

However, the power of these systems to effect change is limited by the lack of regular follow-up. While thorough and systematic, the use of the RIPQA-based observation and feedback process is costly and time-consuming. A team of trained observers conducts a 40–50 minute observation using the RIPQA tool. After the observation, team members compare scores, write up an action plan based on their findings and provide feedback to the program’s instructors. Because of the large number of programs to observe and the limited number of staff to complete the observations and feedback, little time is left to return to the program for followup observations.

Follow-up observations could ensure that the program provider has made suggested changes or, in cases where program quality is quite high (as it generally was in AfterZones), that the strong practices previously observed are being maintained. This would create a true system of continuous program improvement, which would be more effective in raising program quality. In the coming year, PASA plans to focus on a small group of providers whose RIPQA scores indicated a need for improvement and provide this group with more intensive coaching and frequent follow-up.

Most AfterZone programs, especially those focused on arts, provide youth with high levels of adult support and opportunities to learn and interact with peers in positive ways.

Systematic observations of 76 AfterZone programs measured four dimensions of quality: adult support, the physical and emotional safety of the environment, the quality of adult and peer interactions, and opportunities for youth to plan and make choices. The highest scores were on scales measuring positive adult support and emotional safety. Opportunities for youth to plan and make choices received relatively lower scores, suggesting that despite the AfterZone initiative’s consistent emphasis on the importance of choice, program instructors are still not fully engaging youth in making plans and decisions during activities.

In surveys, youth participants reported that they enjoy AfterZone programs and find them to be supportive learning environments. Youth reported relatively high levels of adult support and lower levels of youth choice, mirroring the RIQPA data.

Ensuring long-term sustainability for the AfterZones is a challenge.

The short-term financial outlook for PASA and the AfterZones is good, thanks in large measure to an extension grant from The Wallace Foundation; a one-time federal appropriation; and continuing support from the city, the school district and the police department. Finding renewable funding to sustain PASA and the local AfterZones over the long term, however, presents a major challenge.

The flow of public funding sources, such as the 21st CCLC grant program and state subsidies for childcare, has slowed as a result of the current recession. Similarly, the recession has decreased private foundation endowments, which has slowed private giving.

Believing that public funds will only become available to support high-quality after-school programs if they are seen as an integral part of a child’s education and not as something “extra,” part of PASA’s long-term strategy for sustaining the AfterZones is to integrate after-school programming into the fabric of the student’s school day, creating a seamless transition from one to the other. This strategy is based on an “extended-day learning” model, in which students apply the academic concepts and skills they learn in their classrooms in experiencebased after-school activities. PASA recently received a three-year grant from the Charles Stewart Mott Foundation that will enable it, in conjunction with the mayor’s office and the school district, to develop plans for more closely linking AfterZone programming and school-day learning.

Although the extended-day learning model has received increasing attention from policymakers and funders and has been endorsed by the Obama administration as a promising approach for enhancing student learning, it is still too new to predict whether it will help foster a financially viable future for the AfterZones. Over the next several years, at least, making sure the AfterZones are securely financed is likely to be an uphill battle.

A Final Thought

Building on its existing, but largely independent, provider community and galvanized by committed and effective leadership, PASA has made enormous progress toward reaching its goal of making highquality after-school programs accessible to lowincome middle school youth in a relatively short period of time. It has shown that a campus model is feasible and indeed attracts youth in this age group. It has also demonstrated that, with a concerted effort to track program quality and provide professional development, programs of solid quality can be put in place within a four- or five-year time horizon. In all, the AfterZone experience provides useful insights for the many cities across the country seeking to build systems to expand and support their after-school offerings.

< < Previous | Next > >