A Place to Grow and Learn

Click here to download the full report:
 A Place to Grow and Learn

So where should cities begin? Research and the early experiences of the five cities in our initiative suggest six elements need to be in place if such coordinated efforts are to be built and sustained.

Element one: Committed leadership

Broad-based, committed public and private leadership is “the price of admission” for getting started on citywide OST improvement. Such leadership must begin at the earliest stages and needs to include the mayor (or whoever has executive municipal authority), given his or her unique ability to forge connections among the leaders of public and private agencies and to use the bully pulpit to increase public support for OST. In short, committed leadership is fundamental to securing necessary public funding and influencing policy affecting OST.

Experience has shown that strong mayoral leadership is a powerful driver for progress in OST improvement. In New York, Mayor Bloomberg led a reorganization of the city’s OST programs for increased quality and more accountability, and substantially increased spending on OST. In Providence, Mayor Cicilline’s backing has been “essential,” says Hillary Salmons, director of the Providence After School Alliance (PASA), the organization created by the mayor with Wallace’s support to plan and manage that city’s OST improvement efforts. Cicilline deemed the development of quality after-school activities a top priority during his first mayoral campaign in 2002, and he has personally participated in planning meetings for his city’s OST initiative. His support led to a first-time OST municipal appropriation in 2007 and to unprecedented partnerships between OST programming and key city agencies, including the recreation department, to develop a new neighborhood-based system of OST opportunities for middle-school youth.

School superintendents and principals also play a key role in coordinated OST efforts, which often depend on public education to provide everything from after-school space and personnel to bus transportation and exchanges of ideas and information. School involvement is also important if the OST goals are to make learning in and out of school more seamless. “We feel like we can’t not be at the table,” says Erica Harris, who represents the local public school system in Chicago’s initiative.

Other leaders are needed as well. In Providence, the chief of police helped make possible a new program that organizes officers to work as coaches and mentors for youth. Chicago’s OST efforts, which are targeted at high school students, enjoy the support of a leading business group that is also working for expanding citywide coverage.

The essential point is that public and private leadership support for OST improvement has to go beyond a single individual so that it is broad enough to survive transitions in city administrations.

Element two: A public or private coordinating entity

To plan, implement and sustain a citywide effort, a coordinating entity, called an “intermediary” by some, has proven to be essential in the cities participating in Wallace’s initiative. The coordinating entities vary widely. For example, given the relatively large sums of municipal funding involved, New York City planners decided it made sense to keep the coordinating function within a government agency, the Department of Youth and Community Development. A citywide OST effort can also be coordinated by a not-for-profit that receives private support as well as varying degrees of public funding. Examples include the DC Children and Youth Investment Trust Corporation, which oversees Washington’s Project My Time out-of-school time initiative, and PASA, which coordinates Providence’s neighborhood-based “AfterZone” strategy.

What do these coordinating bodies do, whether or not they are governmentally or privately run? Initially, they are charged with planning. They also gather the data necessary to inform important decisions by city leaders, such as where program dollars can best be spent. In some cases, such as PASA, they actually allocate funds based on program performance and family needs. Equally important, because they are promoting high-quality programming, they lead the development of quality standards. They then determine the necessary steps to help programs achieve better quality and participation.

The coordinating bodies also serve a key communications role. They inform parents and children about OST options and program locations, and oversee efforts to build and sustain broad support for OST. “The hardest part,” says Greg Roberts, who leads the initiative in Washington, “is being the entity that’s responsible for a lot of moving parts.”

Element three: Multi-year planning

Multi-year planning is more than just a document. To be sure, it does commit to paper a rigorously thorough description of the OST effort’s goals, a roadmap for getting there, specific roles and responsibilities of key players, funding needs, and hoped-for results. It also provides a means for organizers to know at each stage whether they are succeeding or going off-track.

But to serve its true function, planning has to be a continuing, multi-year process that keeps public and private leaders engaged – and coordinating bodies need to factor in the time investment it will require of themselves and city leaders they wish to involve. Experience suggests that planning documents should generally plot a three- to five-year course and plans should be regularly updated as initial assumptions change or the unforeseen arises. New York revisits its plan monthly. And Salmons recently found she had to adjust budget estimates in the Providence plan after Rhode Island tightened requirements for child care subsidies.

Washington’s 43-page planning document has enabled Roberts to look beyond day-to-day needs of Project My Time. It details a six-year work schedule beginning with a 22-month planning and early implementation period, and continuing with a four-year period of “rollout and scaling.” By 2012, the goal is to make programming available to children in all of the District’s public middle schools. “Business planning takes an organization through a process in which you have to do some deep strategic thinking,” Roberts says. “Typically, we raise money for a year, at best 15 or 18 months out. Now we have to be four, five years out with incremental benchmarks. It makes you think about the outcomes you have set out to accomplish and what it will take.”

Element four: Reliable information

Accurate information about OST programs is indispensable for planning, but in most cities, decision-makers and funders have relied more on anecdote than fact. As a result, they have often lacked credible information about such issues as which neighborhoods are underserved, how often children are attending and whether programs are of high enough quality to attract children and be effective. The five cities in Wallace’s initiative have taken a range of steps to fill those knowledge gaps. Several have carried out first-ever “mapping” projects to determine which neighborhoods need OST as well as where OST resources are plentiful or scarce.

Several have also conducted market research to identify the OST preferences of children and families so that resources can be directed to address them.

Beyond collecting those information basics, all five cities have been establishing “management information systems” designed to gather reliable, up-to-date OST program attendance data which, in some cities, can be combined with school records to determine whether after-school programs are affecting school attendance, student attitudes and academic performance. Such systems for tracking OST are far from the norm among U.S. cities.

Mapping citywide OST service distribution

To identify underserved neighborhoods or populations, several of our innovation sites juxtaposed the location of OST programs with demographic and other data. The results were telling. Chicago researchers found that because programming had failed to keep pace with residential changes, including a burgeoning Hispanic population in some neighborhoods, the city had more teen OST activities in areas that were losing population than gaining.xii New York discovered an abundance of programs in relatively affluent areas, such as lower Manhattan, and a dearth in low-income neighborhoods in the Bronx and elsewhere. Using that information, New York began to allocate more than half of its funding to areas identified by zip code that needed OST most.

Determining what families want

Attendance at OST programs is generally voluntary, yet cities have rarely collected reliable information for determining if existing programs meet the needs and wishes of children and parents. Through focus groups, surveys and other market research, the five cities in Wallace’s initiative have been listening to, and been influenced by, what the “customers” say. Such market research has revealed, for example, that safety is a bedrock concern for parents. Washington, D.C.’s market research revealed powerful parental support for arts and culture along with homework assistance. Providence, on the other hand, has decided to put more emphasis on sports programming based on its market research.

Tracking participation

Attendance data can provide crucial insights into the responsiveness, accessibility and quality of OST programs, but in most cities, there is no means for centrally collecting or analyzing this information. Wallace and its partners cities have invested heavily in computer systems that can, for the first time, gather attendance figures and related information into a single data base where participation in programs throughout entire cities can be monitored and assessed. Why does such information matter? For one thing, attendance data show which programs are drawing children consistently, and which are not. This can be an indicator of program quality, or the lack of it, and can help cities better direct OST resources.

We have learned, however, that data gathering can be a significant challenge for cities and OST providers. Data entry can be daunting to OST organizations that are short-staffed or that juggle the reporting requirements of numerous funding agencies that may not use the same software. Chicago has had to consider the lack of computer skills among some of that city’s OST organizations as it has implemented a new management information system.

Still, success in data gathering could eventually help answer important questions about the effectiveness of out-of-school time programming for both the city and the OST field in general. For example, several cities have worked out agreements allowing researchers access to school data that, when correlated with program and attendance data, might eventually show whether coordinated OST efforts have an impact on graduation rates or school attendance.

Element five: Expanding participation

Increasing participation has long been a goal of OST providers and advocates. It is also one of the most difficult and frustrating challenges if participation means something more than having children occasionally show up. Indeed, some OST programs have such low attendance that they over-enroll to ensure that a sufficient number of children are served daily. The goal, however, is not simply to add enrollment, but to have children attend often enough to realize learning or developmental benefits. And the experiences in Wallace’s partner cities and elsewhere show that this challenge only gets harder as children grow older and have competing job or family responsibilities, other possibilities (good or bad) for occupying their time, or little idea that OST programming can be engaging.

The challenge facing cities that want to expand participation that results in benefits to children is two-fold: “intensity,” or the number of days and hours per week or year that children attend programs, and “duration,” or the span of time over which they take part.xiii Given the differing challenges among various age groups, this has led some OST planners to conclude that they should “shift the focus from unbridled growth to promoting participation of targeted youth, and at levels sufficiently high to benefit them,” as Making Out-of-School Time Matter, a Wallace-commissioned study by RAND, puts it. xiv

In fact, this is what most of the five cities in Wallace’s initiative have done. Chicago, Washington, D.C. and Providence are focusing primarily on expanding participation among middle-school students or teens. Boston’s initiative aims at reaching struggling elementary students. New York City, by contrast, is working to build participation across the entire age spectrum. (Click here to enlarge the graph below on A Citywide Approach to Building Sustainable, High-Quality Out-of-School Time Opportunities: Elements and Outcomes.)

Cities are testing a variety of strategies to promote attendance. Providence, for example, removed a key barrier by securing additional late-afternoon school bus transportation for OST participants. Chicago’s After School Matters teen apprenticeship program pays stipends to all participants and they are required to regularly attend – resulting in unusually strong weekly attendance rates for that hard-to-retain age group.

The citywide information made possible through market research and management information systems is opening new avenues for building participation. New York City monitors participation in programs receiving city funds and reduces funding for those that fail to meet targets. With market research revealing that teachers are major influences on Washington teens, OST organizers have asked faculty members not only to work in programs, but to publicize them in their classrooms. And making sure parents and children know about available programming is so important that cities are using new websites, online program locators, print materials or other means to get the word out.

Finally, all five cities are beginning to take steps to increase program quality on the premise that this will entice more children to attend more frequently. A New York City OST official put it this way: “It can’t just be babysitting, especially for kids older than second or third grade, and it can’t be just school either – and that’s not easy.”

Element six: A commitment to quality

One of the pillars of our working hypothesis is that lifting the quality of OST programs is crucial if they are to attract children frequently enough to realize benefits. But achieving that goal on a broad scale faces continuing, long-standing obstacles. The first is that quality costs – and many OST organizations are chronically strapped for resources. As a result, many programs have facilities problems, limited management expertise, and part-time or under qualified staff who may get along well with children but aren’t well-prepared to maintain structure or discipline. A second common obstacle is that the field has tended to invest more in increasing program enrollments than in ensuring consistent enough attendance for children to benefit.

A commitment to quality has to begin with some understanding of the program characteristics that are likely to contribute to benefits for children. Drawing on existing research on youth development, education and other relevant areas, RAND’s Making Out-of-School-Time Matter listed some of those basic conditions: a clear mission; high expectations and social norms; a safe environment; supportive emotional climate; small total enrollment; stable, trained personnel; appropriate content and pedagogy; and frequent assessment.xv Existing research still leaves many unanswered questions about the attributes of OST quality and how they relate to the realization of benefits for children.

Notwithstanding the obstacles and knowledge gaps, the stakes for raising OST quality have gone up significantly in recent years. State and federal funding have increased sharply, and with those added tax dollars have come higher expectations for ensuring that children benefit, accompanied by more calls for standards and accountability to ensure that OST providers deliver on those expectations.

Those higher stakes were laid bare in a series of evaluations of the federal 21st Century Community Learning Centers programs by Mathematica Policy Research, Inc.xvi The researchers found that despite rapid growth in funding for such programs nationwide, the 21st CCLC programs they studied had few impacts on student achievement among elementary and middle-school children in low-performing schools. Additionally, participants were slightly more likely to engage in negative school behaviors.xvii A likely explanation is that many of the programs studied suffered from critical problems in quality, including content and staffing matters.

Still, the question remains: if quality is a must for the ultimate goal of achieving benefits for more children, how might a more coordinated, citywide approach help many more programs reach that objective?

An important initial step Wallace sites have taken to date is codifying the meaning of quality through standards, adopting program assessment tools, and then working to get programs to meet the standards. Providence’s standards, for example, concentrate on five areas: health and safety; relationships among staffers, volunteers, young people and their families; programming and activities; staffing and professional development; and administration.

Wallace-funded cities are also beginning to provide a range of support to programs to enable them to meet quality guidelines. To date, these have included training for both program managers and frontline staffers on issues ranging from staff retention to the emotional development of adolescents. Washington, D.C. organizers have been working with small groups of providers to help them meet the standards of, and then receive accreditation from, the National AfterSchool Association. In New York City, youth workers can receive scholarships for college courses leading to a youth worker certificate. Further steps will undoubtedly be needed to address the range of common organizational capacity weaknesses.

New citywide data collection systems have made it possible to pinpoint and address quality problems more precisely. In Providence, a drop-off in attendance in one performing arts program led to the discovery, and replacement of, an uninspiring teacher.

It’s too soon to say whether such efforts will result in substantial and widespread improvement in OST programs. But one Providence student, Ben-Oni Jean-Pierre, neatly summarized the stakes: “I’d walk a mile for a quality program,” he said. “But I wouldn’t walk across the street for a bad one.”

< < Previous | Next > >