Contents

Hours of Opportunity

Click here to download the full report:
 Hours of Opportunity

The comparative case study approach yielded rich details and increased understanding of the pathways, processes, and hypotheses that can be tested in the future. This analysis provided useful comparative information about what cities can do to address shortfalls in access or quality of after-school provision and how some cities have built MI systems and strived for sustainable funding. Our analysis showed that the context of each city mattered in what it chose to focus on. It also confirmed much of the literature in terms of what would prove to be important for progress. It provided evidence on very specific actions that mayors could take to push their efforts forward. The companion monograph on the building of MI systems, Hours of Opportunity, Volume 2: The Power of Data to Improve After-School Programs Citywide (McCombs, Orr, et al., 2010), makes clear how strong leadership manifests.

The descriptions herein, and those in McCombs, Orr, et al. (2010) and McCombs, Bodilly, et al. (2010), provide concrete examples for others to consider based on the approaches of The Wallace Foundation grantees, their reasons for taking these approaches, and the proximate result—the immediate effect on OST provision, structure, access, quality-assurance processes, information for planning, and sustainability. We now summarize some themes from the analysis that other cities might consider.

Coordinated system-building efforts can work to improve access and quality. The analysis showed that these cities’ coordinated attempts at system improvement were effective in meeting several goals. Through their efforts, four cities increased the number of students served by OST programs. For example, in Providence, OST program enrollment increased from 500 to 1,700 middle schoolers under this initiative, and New York City increased the number of slots from 45,000 to 80,000. Programs were located in all DCPS schools in Washington, D.C., and, in Boston, five schools began to offer after-school programs where none had existed before. In each case, these efforts targeted high-need student populations. Essential to this progress were early needs assessments, development of program locators for use by parents and students, and student tracking information to determine program demand and student locations.

While we cannot at this point determine whether quality improved, each of the cities used the investment funds to begin or put in place quality-assessment systems, including developing and promulgating standards, vetting and assessing providers against the standards, offering professional development to improve staff expertise and programming, and using contractual clauses to ensure that participation goals were met. Crucial to these efforts was the development and use of MI systems to track student demand for programs and the use of student and parent surveys to obtain opinions about quality.

While the sites made progress in obtaining more sustained funding for example, by winning 21st Century Community Learning Center awards), unfortunately, this study took place at a time of great national financial upheaval. The sites struggled with ways to ensure steady funding, but uncertainty remained. Nevertheless, the respondents thought that their efforts to improve system building before the economic downturn put them in better positions to argue for sustained funding by allowing them to show progress toward outcomes, and more efficient use of resources was already under way.

This initiative provided a proof of principle—that organizations across cities could work together toward increasing access, quality, data-based decisionmaking, and sustainability. The final impact, however, remains unknown until the evaluations undertaken by the sites are published.

Each city has a unique context that should drive what is attempted. City context influenced the focus, scope, and lead for the system-building work. Early planning efforts revealed different needs and challenges in the cities and influenced some cities to focus on a particular target population, such as middle school students.

Cities varied in organization of the effort, with some being led by intermediaries and others a government agency. In cities in which an agency provided significant funding for OST, a city agency was designated as lead. In cities with a low level of city funding for OST, an intermediary took on the lead role. These few examples do not lead us to view one approach as preferable to another. Instead, it seems that, again, context matters. The lesson for other cities is that the decision about who will lead the effort and the structure of coordination needs to take into account the assets at hand, the locus of control, and the skills and talents of leaders. It seems unlikely that the Providence intermediary-led model would have worked in New York City with its strongly independent, multiple, and uncoordinated city agencies. But neither would the New York City agency-led model work in Providence, which lacked city agencies involved in after-school programming.

In summary, each city’s initiative differed due to its unique circumstances. Other cities will need to consider their own circumstances before deciding what might best propel their efforts forward.

Investments in early planning and management information system development paid off. These sites were given a unique opportunity because The Wallace Foundation investment allowed them to carefully consider what needed to be done across the city for improvement to take place. They deliberated the specific assets in place, the organizations involved, the challenges faced, and the funding available. Investments in the early planning phase paid high dividends in clearly identifying targets for improvement and beginning to develop a means of sharing information to promote better decisionmaking across the city.

Similarly, investments in MI systems and evaluations helped the actors understand whether progress was made and allowed them to argue more effectively for additional funding. Furthermore, all this work brought together different actors, often for the first time, to discuss how to build a better OST system. While building information systems was a major goal of the effort, these systems also became a major enabler of further progress on access and quality as well as the glue that led to cooperation and coordination in a couple of cities.

Cities should definitely consider early data gathering to help inform their efforts. These sites offer examples of the types of information collected and how it could be used to propel efforts forward. The major caveat is that it must be shared across organizations and stakeholders to improve system-building efforts.

Cities can consider an array of approaches to improving access and quality. The sites we studied found an array of ways to meet their goals to increase access and improve quality. Some part of successfully improving access had to do with identifying underserved areas and students and finding the mechanisms to provide convenient access, such as placement of programs in neighborhood hubs, providing transportation to and from the programs, program locators, and free programs.

Cities attempted to improve quality through the adoption of standards, the use of the standards to assess program quality, provision of professional development, and evaluating their own efforts. A major difference among the grantees was whether the lead chose to use contractual means to hold the providers accountable for improving quality as in New York City, with DCPS in Washington, D.C., possibly following suit) or whether the lead used more collegial means, such as significant professional development or joint reflection on quality, as in Providence, Chicago, and Washington, D.C., under the Trust. Again, this is an important choice and depends on city context. Importantly, Providence chose this path, as did the Trust, because its early planning efforts showed a scarcity of providers. Planners in these organizations thought that developing better existing resources was a more viable pathway to quality and access than driving poor providers out of the system.

Again, the major lesson is that context is important and should be considered carefully when developing approaches to increasing access and improving quality.

Cities can consider an array of mechanisms for increased coordination. The sites used an array of mechanisms to improve coordination. Efforts included early planning that brought multiple organizations together, engaging stakeholders to build shared goals, restructuring and consolidating roles, establishing coordinating committees or steering committees, and other regular means to share information and decisions. One used the appointment of mayoral envoys to ensure interagency cooperation or the development of interagency MOUs. It was in the instance of Boston, which did not undertake these types of activities to the same extent in early years of the grant and which changed lead organizations, that coordination occurred in fits and starts. Several of these steps proved to be most important from the interviewees’ point of view, and we describe them as enablers in the next section.

Several enablers were important. Interviewees agreed on several important enablers of collaborative efforts. They were the building of a common vision across stakeholders in the early planning period, effectively collecting and using data and information, supportive mayoral actions, the buy-in of the schools, and investment funding.

Wallace Foundation staff clearly recognized these potential enablers as they developed the initiative. The Foundation provided early planning grants to encourage sites to conduct early needs assessments and establish a shared vision for the work that informed their business plans. It required the adoption of MI systems to create an ongoing source of data for the cities. Indeed, cities with strong needs assessments, a strong vision shared by stakeholders across the system, and MI systems made significant progress toward their goals.

In addition, The Foundation selected cities based, in part, on evidence of mayoral support. Mayoral support was key to the progress made in these cities, but it took on forms far beyond simple encouragement and bully pulpit statements. Getting a mayor actively involved will be challenging in many cities. Educating the mayor early in the process about how he or she can affect the outcomes by reorganizing agency responsibilities or realigning funding sources and by demanding data on progress might be an additional strong investment with a high payoff later.

Ensuring the support of the schools appeared to be a complex process and one that was ongoing, taking significant time and resources. Not only was it necessary to ensure the cooperation of the central office to allow access to schools free of charge, it was necessary to ensure that principals and staff actively supported the programs and encouraged children to attend. This process took concerted effort and was aided in several cities by a school site coordinator whose job, among other tasks, was to actively engage the school staff. The capabilities of these coordinators were crucial in enabling effective program offerings and operations. Thus, a solution was found, but it was dependent on further resources.

Finally, the funding provided by The Wallace Foundation was an essential ingredient for supporting cities as they developed their OST systems. Whether other cities can move forward effectively without this degree of outside support remains an open question, as does cities’ ability to maintain progress in the face of an unrelenting squeeze on funding. Some cities were challenged to weave together different sources of funding while trying to build more coherent programming—a challenging task in flush times but one far more difficult in the midst of budget cuts.

While The Wallace Foundation funding pushed progress forward and the lack of it would constrain progress toward the initiative’s goals, there was nothing in these case studies that indicated that progress was impossible without it. For example, the market research was not a significant expense and could be undertaken by many cities. Strong actions by mayors can lead to significant restructuring and consolidation, as was shown in Washington, D.C., and New York City. Mayors control funds that can be used to build MI systems, they can appoint special advisers, and they can demand accountability—all without adding significant financial burden.

Thus, other cities should consider what actions they can take within the confines of their specific environment. Small steps forward can add up over time to significant improvements for underserved children.

< < Previous | Next > >