Contents

• Getting to Work on Summer Learning, 2nd edition​
  
• About This Report
  
• Executive Summary
• Recommendations
• Teacher Selection and Professional Development
  
• Sufficient Time on Task
• Student Recruitment and Attendance
• Enrichment Activities and Their Implementation
• Positive Summer Climate
• Summer Cost and Funding
  

ABOUT THIS REPORT

This study was undertaken by RAND Education and Labor, a division of the RAND Corporation that conducts research on early childhood through postsecondary education programs, workforce development, and programs and policies affecting workers, entrepreneurship, and financial literacy and decisionmaking. This study was sponsored by The Wallace Foundation. Based in New York City, The Wallace Foundation is an independent national philanthropy whose mission is to foster equity and improvements in learning and enrichment for young people, and in the arts for everyone. Current areas of interest include school leadership, expanding and diversifying audiences for the arts, social and emotional learning, summer learning, arts education, and afterschool. Wallace aims to help solve problems facing the fields in which it works, benefiting both the organizations it funds directly and the broader field by developing credible, useful knowledge to inform policy and practice nationwide. Research commissioned by and produced by the foundation is available without charge from the foundation’s Knowledge Center at www.wallacefoundation.org.

The full report, Getting to Work on Summer Learning: Recommended Practices for Success, 2nd Ed., by Catherine H. Augustine, Jennifer Sloan McCombs, Heather L. Schwartz, and Laura Zakaras (2018), can be found at www.rand.org/t/RR366-1 or www.wallacefoundation.org. More information about RAND can be found at www.rand.org. Questions about this report should be directed to Heather Schwartz at heather_schwartz@rand.org, and questions about RAND Education and Labor should be directed to educationandlabor@rand.org.

This is part of a series of reports stemming from the NSLP. The series consists of the following titles:

1. Catherine H. Augustine, Jennifer Sloan McCombs, Heather L. Schwartz, and Laura Zakaras, Getting to Work on Summer Learning: Recommended Practices for Success, Santa Monica, Calif.: RAND Corporation, RR-366-WF, 2013 (www.rand.org/t/RR366).
2. Jennifer Sloan McCombs, John F. Pane, Catherine H. Augustine, Heather L. Schwartz, Paco Martorell, and Laura iv Zakaras, Ready for Fall? Near-Term Effects of Voluntary Summer Learning Programs on Low-Income Students’ Learning Opportunities and Outcomes, Santa Monica, Calif.: RAND Corporation, RR-815-WF, 2014 (www.rand.org/t/RR815).
3. Catherine H. Augustine, Jennifer Sloan McCombs, John F. Pane, Heather L. Schwartz, Jonathan Schweig, Andrew McEachin, and Kyle Siler-Evans, Learning from Summer: Effects of Voluntary Summer Learning Programs on Low-Income Urban Youth, Santa Monica, Calif.: RAND Corporation, RR-1557-WF, 2016 (www.rand.org/t/RR1557).
4. Catherine H. Augustine and Lindsey E. Thompson, Making Summer Last: Integrating Summer Programming into Core District Priorities and Operations, Santa Monica, Calif.: RAND Corporation, RR-2038-WF, 2017 (www.rand.org/t/RR2038).
5. Heather L. Schwartz, Jennifer Sloan McCombs, Catherine H. Augustine, and Jennifer T. Leschitz, Getting to Work on Summer Learning: Recommended Practices for Success, 2nd Ed., Santa Monica, Calif.: RAND Corporation, RR-366- 1-WF, 2018 (www.rand.org/t/RR366-1).
6. Catherine H. Augustine and Lindsey E. Thompson, Getting Support for Summer Learning: How Federal, State, City, and District Policies Affect Summer Learning Programs, Santa Monica, Calif.: RAND Corporation, RR-2347-WF, 2020 (www.rand.org/t/RR2347).
7. Jennifer Sloan McCombs, Catherine H. Augustine, John F. Pane, and Jonathan Schweig, Every Summer Counts: A Longitudinal Analysis of Outcomes from the National Summer Learning Project, Santa Monica, Calif.: RAND Corporation, RR-3201-WF, 2020 (www.rand.org/t/RR3201).

EXECUTIVE SUMMARY

This executive summary of the second edition updates guidance first published in Getting to Work on Summer Learning: Recommended Practices for Success.¹ It is intended for district leaders and their partners across the country who are interested in launching or improving summer learning programs. In this summary, we distill lessons about implementation that we have gleaned from a six-year study of voluntary summer programs in the five urban districts participating in the National Summer Learning Project (NSLP).

The second edition presented the best available guidance on how to develop high-quality summer programs, summarized here. From summer 2011 through summer 2014, researchers from the RAND Corporation collected more than 1,200 surveys of summer instructors and 10,000 surveys of elementary grade students, conducted 900 interviews, and observed more than 2,000 hours of classroom and enrichment activities. We are confident that these recommendations are based on the most-comprehensive data currently available about voluntary, academic summer programs.

The successful summer programs we observed delivered several sound educational and youth development practices: teachers with content knowledge using rigorous academic curricula, high-quality enrichment experiences, a high level of engagement between adults and students even during transitions and time outside of class, and an emphasis on consistent attendance.

Putting these elements in place required careful planning. Without it, programs suffered from logistical problems and poor instruction. For that reason, our first recommendation as presented at the end of this section is that a summer program director with at least half-time devoted to this work start actively planning the program in January.

Although many school districts offer mandatory summer programs to students at risk of grade retention, fewer districts offer summer learning programs to a broader population of students as a means of stemming summer learning loss and boosting academic performance. To expand summer program opportunities for students in urban districts and create knowledge that would benefit the field, The Wallace Foundation launched the NSLP in 2011, providing support to public school districts and community partners in Boston; Dallas; Duval County, Florida; Pittsburgh; and Rochester, New York. Each of these districts offered a five- or six-week full-day summer program that served students rising from third into fourth grade; most districts served other grade levels as well. The programs all focused on reading, mathematics, and enrichment activities (such as arts, sports, and science exploration).

As part of the overarching project, the RAND Corporation assessed the effectiveness of the five districts’ summer programs. We found strong evidence that voluntary summer learning programs can produce short-term gains in mathematics. We also found promising evidence that, after two consecutive summers, students with high attendance (20 or more days per summer) outperformed their peers in mathematics and English language arts (ELA) and displayed stronger social-emotional competencies.

To help these districts strengthen their programs so that they could be tested for their effectiveness, The Wallace Foundation asked RAND to conduct formative evaluations of the programs over two summers (2011 and 2012) so that districts could make successive improvements to their programs before 2013, when RAND launched a randomized controlled trial. The randomized controlled trial involved slightly more than 5,600 students who had applied to attend two consecutive summers (2013 and 2014) of these five districts’ programs. The study examined the shortand long-term effects of voluntary summer programs and the characteristics of the programs that were effective. RAND also continued to provide formative feedback to each district during summers 2013 and 2014.

The recommendations in this summary are informed by both the student outcome findings and the data we gathered during and after the four years of program implementation.

RECOMMENDATIONS

Planning

Launching a summer program is akin to starting a new school year, but with less time for planning and execution. A good planning process might be the most important characteristic of a strong program: It reduces logistical problems and increases instructional time for students.

1. Conduct early, robust planning

• Commit in the fall to having a summer program. Program leaders who decided on a summer program in the fall and began planning no later than January ran a smoother summer program with less disruption to instruction.
• Dedicate a director to manage summer program planning who has influence, authority, and committed time. The quality of summer programs was better when there was a director who had decisionmaking authority, project management experience, and at least half of his or her time reserved for summer program planning and management.
• Determine which students to target and plan accordingly. Decide whether the summer program will be open to all grade-appropriate students enrolled in the district or restricted to certain students, such as English language learners or those in high-poverty schools.
• Consider a cross-departmental planning team. The district summer programming lead should work with staff from relevant departments, including curriculum, transportation, facilities to host summer sites, human resources for hiring staff, procurement for enrichment partners, information technology for attendance-taking and for use in the summer program, and communications for recruitment materials.
• Create a calendar that stipulates task deadlines. Set clear deadlines to anchor planning and use a shared calendar to signal who is required at particular meetings. This promotes efficient use of staff time.
• Use meeting time wisely. Hold regular meetings and use the meetings to focus on developing clear procedures and logistics for the summer program.
• Engage both community-level and site-level staff in the planning process. Planning worked best when a summer program director in the district central office ran the planning and involved site-level leads in some of the decisionmaking, such as creating site-specific master schedules or conducting site-specific professional development.

2. Plan for both enrichment activities and academics

Enrichment and district partners should jointly plan staff hiring, training, and curriculum and behavior policies. During the planning phase, establish which organization has ultimate responsibility for overseeing the quality of instruction and managing the instructors.

3. Engage in a continuous improvement process

Plan to administer pre- and post-tests, observe instructors, collect staff views about the summer program, and share evaluation data after the summer ends to improve the program over time and to reinforce community stakeholders’ commitment to retaining the summer program.

Teacher Selection and Professional Development

According to the research, teaching quality has the largest school-based impact on student outcomes of any factor. Hiring effective teachers and giving them the support they need are critical steps to maximizing student achievement.

1. Recruit and hire the district’s most highly effective teachers

• Advertise attractive program features and encourage promising teachers to apply. Encourage promising teachers to apply by advertising the benefits that prior summer program teachers have experienced.
• Hire teachers with relevant content knowledge and grade-level experience. Prioritize hiring teachers with this experience because in the NSLP they typically had deeper content knowledge and more-varied teaching strategies for the grade level of students they were teaching.
• If possible, hire based on staff motivation and performance rather than seniority. Write a summer teacher job description that sets performance expectations for teacher attendance, hours, and engagement with students throughout the day and not just during their class times. Create a selection process that includes principal recommendations and, if feasible, interviews. If relevant, seek hiring exemptions with the local teacher’s union to hire teachers based on their performance rather than seniority.
• Hire experts to support to students with special needs. Programs that serve a substantial number of English language learners or students with Individualized Education Programs should hire bilingual and special education teachers to help serve those students. The NSLP districts that did so saw student behavior and learning improve.

2. Provide teachers with sufficient professional development prior to the program

• Familiarize teachers with the summer curriculum and how to teach it. Provide professional development on the summer curriculum and give the curriculum materials to teachers prior to or at the training. Model curriculum use in the session, so that teachers get hands-on experience prior to the first day of the summer program.
• Train teachers to avoid common culprits for classroom instruction time loss. NSLP programs lost between 11 and 28 percent of their intended math and ELA instructional minutes in summer 2014 because of a combination of classes that started late, ended early, or had prolonged disruptions. Explain during training how instructional minutes are typically lost and model activities to help teachers teach “bell to bell.”
• Emphasize that engaging academic work is a part of summer fun. Highlight during professional development that protecting instructional time is a key way to achieve rather than hinder the goals of the summer program.
• Train teachers to effectively check for student understanding. Through modeling, make clear during training that teachers should circulate among all students during their independent practice to ensure each understands the material.
• Engage all instructional support staff in academic training sessions. If the summer program includes instructional staff who support lead teachers, those support staff should be involved in the curriculum and classroom management training.

Sufficient Time on Task

Students in the NSLP study who received at least 25 hours of math and 34 hours of language arts instruction in summer performed better on subsequent state exams. Yet intended time for summer instruction is easily lost. Summer program schedules and staff training can be designed to maximize the time allocated for instruction.

1. Operate the program for five to six weeks with three to four hours of academics per day

If possible, operate a program for five to six weeks with 90 or more minutes of math and 120 or more minutes of ELA per day. This length is to allow a typical student who attends 75 percent of program days to obtain the 25 hours of math and 34 hours of ELA instruction that we found was correlated with improved achievement on subsequent state exams.

2. Provide time for transitions in the master schedule

Create a master class schedule that builds in the amount of time it realistically takes for student and staff to get to the class locations so that classes can start and end at the intended time, even if that means lengthening the program day.

3. Schedule academic classes to occur in one continuous block

Avoid scheduling classes to have multiple parts (e.g., a session before and after lunch); multipart classes typically lost more time than single, continuous class sessions.

4. Minimize the loss of instructional time by attending to summer site logistics

Late ordering of supplies or materials, buses, and meal delivery led to less instructional time. These logistical challenges were common, but they did improve over time.

5. Communicate the importance of maximizing instructional time to site leaders

Communicate to site leaders and to school and enrichment instructors the importance of maximizing instructional time and how instructional minutes are typically lost. Also explain the desired number of instructional hours by subject or activity that site leaders should be targeting.

6. Provide teachers with strategies for maximizing instructional time

Model activities during training or coaching to minimize time loss at the start and end of class, during the afternoon “slump,” and during independent practice time.

Student Recruitment and Attendance

The NSLP study revealed that students needed to attend at least 20 days over the course of the summer program to experience academic benefits. In addition to offering enrichment activities, accurate recruitment materials and incentives can help maintain good attendance.

1. Acknowledge that consistent attendance is possible in different types of summer programs

Across the four summers, the two districts with the highest average daily attendance rates had quite different designs. One was structured like the school year and the other was designed like a camp.

2. Develop accurate, timely recruitment materials

Develop recruitment materials that accurately explain both the program requirements and the attractive features of the program. Communicate several times with parents and students before the program starts.

3. Personalize recruitment of students and their families

The most effective recruitment process we observed paired recruitment materials with some personalized recruitment, such as letters from teachers to students encouraging them to sign up.

4. Establish a firm enrollment deadline

Set a date after which a student cannot enroll in the summer program. Having a deadline enables districts and program sites to finalize staff hiring, class schedules, and bus routes in time for an orderly summer program.

5. Establish a clear attendance policy

In summer application and orientation materials, make clear that the district expects students to attend every day of the summer program. Enact a policy that students who miss more than a set number of days might be asked to leave the program.

6. Track the number of initial enrollees who never attend, as well as summer attendees’ daily attendance

Track both because this will enable the school district to hire the right number of staff in future summers and to ensure that students attend enough of the program to benefit.

7. If resources allow, provide incentives to parents and students for attendance

Attendance incentives are most effective when provided to both parents and students, but they are costly. Anecdotally, summer program leaders felt that a mix of field trips and weekly prizes for students helped increase attendance.

Academic Curricula and Their Instruction

Summer programs are short and often provide little time for teachers to plan their lessons. To maximize the effectiveness of instruction, teachers should have both high-quality curriculum materials that are matched to student needs and small class sizes.

1. Engage experts to anchor the program in written curricula that align with school-year standards and student needs

• If purchasing curricula, adapt them to fit student needs and available instructional time. When purchasing curricula, a district curriculum expert should adapt them—before they are distributed to teachers—to fit the amount of instructional time available in the summer program, align with district school-year standards, and meet the district’s student needs.
• Curricula developed in-house should be created by district curriculum experts over the course of several months. If self-developed, a district curriculum expert should start months before the summer program starts so that the curricula are coherent, comprehensive, and align with or extend the school-year curriculum. Teachers should not write their own curricula.
• Provide strategies for differentiation in curriculum materials. Differentiate activities within lesson plans, particularly for independent practice. This allows students who quickly complete tasks to extend their learning and students who struggle to get additional support.

2. Encourage instructional leaders to observe instruction of the curriculum and provide feedback

Encourage site leaders to observe teachers’ classes, provide them with feedback, and build in time for teachers to confer with one another about practices. If the site leader lacks instructional expertise, we also recommend encouraging curriculum designers and coaches to observe instruction.

3. Serve students in small classes or groups

Cap class size at 15 students per adult if possible. Small classes allowed teachers to get to know students’ needs, establish norms, and launch instruction on the compressed summer schedule.

Enrichment Activities and Their Implementation

All districts featured fun and engaging enrichment activities, such as art, sports, and science exploration, to differentiate their programs from traditional summer school and to attract students and promote attendance. Some good practices characterized the most well-organized and engaging activities we observed in the districts.

1. Select a model for providing enrichment activities

We found that hiring district teachers, contracting directly with enrichment providers, or establishing partnerships with intermediaries all worked well as long as they were implemented by qualified enrichment staff.

2. Ensure that enrichment instructors have strong content knowledge

As with academics, prioritize the content knowledge of enrichment teacher applicants. Those with strong content knowledge more frequently demonstrated and modeled skills, corrected student techniques, and built on student strengths.

3. Train enrichment instructors in behavior management strategies and monitor their implementation

Model behavior management strategies during training and create written rules that align the enrichment and school portions of the day. We observed higher rates of student misbehavior during enrichment than we did during academic classes.

4. Plan lessons to include sequenced activities

Require that enrichment have preplanned and sequenced activities because good enrichment classes included activities that were organized and engaging and allowed the majority of students to actively participate for the duration of the activity.

5. Plan carefully if enrichment is integrated with academics

Not all enrichment activities need or should be linked to academics. But if integrating them, offer specific curriculum guidance and additional training for enrichment teachers.

6. Keep class sizes small

As with academic classes, cap class size at 15 students per adult if possible.

Positive Summer Climate

Positive site climate drives student daily experiences and enjoyment of the program and is correlated with higher student attendance. The quality of staff-to-student interactions was the item most strongly and consistently related to whether students appeared to enjoy the day.

1. Train all staff on the importance of positive adult engagement with students throughout the day—not only in classes

Train staff to interact with students not just during class time but also during transitions, arrival, departure, and mealtimes.

2. Develop a clear, positive message about the summer site culture and ask staff to convey it consistently to students

To promote a coherent culture and consistent application of behavior management techniques, develop an explicit message about the values of the program and how students are to be treated, and train staff on both.

3. Ensure that site leaders observe instructional and noninstructional periods

Convey the expectation that summer site leads should not only routinely observe academic and enrichment activities but also observe transitions and lunch periods to ensure that staff are sending a consistent message about the site’s values and behavioral expectations.

4. If resources allow, consider hiring staff to support positive student behavior

To address some student behavior, such as bullying or fighting, consider staff roles, such as social workers or behavior management specialists, to provide one-on-one support to students.

Summer Cost and Funding

The cost per student who attended at least one day of the summer 2014 program ranged from $1,070 to $1,700, with an average of $1,340. Districts can minimize costs—and maximize value from an investment in summer learning—by following these recommendations.

1. Hire staff to achieve desired ratios based on projected daily attendance, not the initial number of enrollees

About one-half of summer program expenditures was for academic and enrichment teacher salaries. Factoring in no-show rates and average daily attendance rates will help districts hire the right number of staff to achieve desired adult-student ratios.

2. Consider cost-efficiencies in the design of the program, but weigh them against potential impacts on program quality

• Partner with community-based organizations. These partnerships not only exposed students to enrichment activities they might not otherwise have experienced but also saved costs because enrichment staff typically earned lower wages than district instructors.
• Reduce the number of summer facilities. Reducing the number of campuses where the summer program was hosted saved on some campus costs, such as program directors.
• Centralize planning activities. It is often less costly for a small centralized team to develop policies and content than to expect each summer site to create its own.
• Continue the summer program over time. The up-front costs of developing policies, procedures, and materials for summer programs can stretch over multiple summers.
• Extend the school-year curricula. District curriculum designers can use the schoolyear curricula to develop additional lessons for a five- or six-week summer program.

This executive summary is from the fifth report in the following series about the National Summer Learning Project. All reports in the series can we found at www.rand.org:

1. Getting to Work on Summer Learning: Recommended Practices for Success, by Catherine H. Augustine, Jennifer Sloan McCombs, Heather L. Schwartz, and Laura Zakaras
2. Ready for Fall? Near-Term Effects of Voluntary Summer Learning Programs on Low-Income Students’ Learning Opportunities and Outcomes, by Jennifer Sloan McCombs, John F. Pane, Catherine H. Augustine, Heather L. Schwartz, Paco Martorell, and Laura Zakaras
3. Learning from Summer: Effects of Voluntary Summer Learning Programs on Low-Income Urban Youth, by Catherine H. Augustine, Jennifer Sloan McCombs, John F. Pane, Heather L. Schwartz, Jonathan Schweig, Andrew McEachin, and Kyle Siler-Evans
4. Making Summer Last: Integrating Summer Programming into Core District Priorities and Operations, by Catherine H. Augustine and Lindsey E. Thompson
5. Getting to Work on Summer Learning: Recommended Practices for Success, 2nd Ed., by Heather L. Schwartz, Jennifer Sloan McCombs, Catherine H. Augustine, and Jennifer T. Leschitz
6. Getting Support for Summer Learning: How Federal, State, City, and District Policies Affect Summer Learning Programs, by Catherine H. Augustine and Lindsey E. Thompson
7. Every Summer Counts: A Longitudinal Analysis of Outcomes from the National Summer Learning Project, by Jennifer Sloan McCombs, Catherine H. Augustine, John F. Pane, and Jonathan Schweig

Commissioned by The Wallace Foundation

​ 

References

1. Catherine H. Augustine, Jennifer Sloan McCombs, Heather L. Schwartz, and Laura Zakaras, Getting to Work on Summer Learning: Recommended Practices for Success, Santa Monica, Calif.: RAND Corporation, RR-366-WF, 2013. For the second edition, see Heather L. Schwartz, Jennifer Sloan McCombs, Catherine H. Augustine, and Jennifer T. Leschitz, Getting to Work on Summer Learning: Recommended Practices for Success, 2nd Ed., Santa Monica, Calif.: RAND Corporation, RR-366-1-WF, 2018.