Contents

Central Office Transformation for District-Wide Teaching and Learning Improvement

Click here to download the full report:
 Central Office Transformation for District-Wide Teaching and Learning Improvement

Central office administrators across the three districts used evidence to varying degrees to inform how they participated in central office transformation, as suggested schematically by Figure 6 below.

 

As part of their use of evidence, central office administrators routinely examined schools’ standardized test results, but even more commonly, they collected evidence from principals’ and other central office administrators’ experience with the central office transformation process and incorporated that evidence into their decision-making. Research on educational and other organizations highlights the importance of this form of evidence use, what researchers have called “learning from experience,” or “working knowledge” to helping organizations realize their goals (Honig, 2003; Kennedy, 1982; Levitt & March, 1988). As that research would suggest, and as indicated by our observations and interviews, such evidenceuse activities were an important dimension of central office transformation.

Both the collection and incorporation of evidence from experience into ongoing decisions about the central office transformation process proved challenging for most central office administrators. As one central office administrator described, the “frenetic” pace of the central office transformation effort left hardly any time to document and examine their work. Another wanted to be spending time with staff discussing “how it’s going, and I don’t feel like we … have time to talk about that.” Likewise, a third reflected, “There’s so much information … and a real … data collection opportunity … . And I just haven’t had the capacity” to be able to incorporate those data into decisions. However, these and some other central office administrators generally reported that they were, in the words of one, “always working” to make such evidence use a routine part of their practice, including hiring additional staff to help with that aspect of the work.

In this chapter we summarize our findings about how central office administrators in the three districts collected evidence from experience and worked to incorporate that evidence into the ongoing development of each of the previous four dimensions of central office transformation: assistance to principals, support to those who were assisting principals, the reorganization and reculturing of the rest of the central office, and stewardship of the transformation effort.

Use of Evidence to Support Direct Partnership Relationships with Principals (Dimension 1)

The individual ILDs, whose work offered support for principals’ instructional leadership, routinely collected and used evidence about the effectiveness of that work to inform their continued work with principals. Just as excellent classroom teachers routinely assess their students’ learning and use that information to differentiate and improve instruction, many of the ILDs in our sample approached their work with principals in a similar way. For example, as described, several Oakland NExOs and virtually all executive directors in Atlanta regularly gathered and used evidence about their principals’ development as instructional leaders. This evidence included school performance on standardized state-mandated tests but also their own observations, for example, of how principals prioritized their work on teaching and learning, and principals’ ability to observe and analyze teachers during classroom observations. These ILDs used that evidence to ground decisions about prioritizing visits to particular schools and how to group expert and more novice principals during network meetings.

In some districts, ILDs also routinely discussed with each other what they were learning about how to support principals and how they might use those lessons to inform their practice. For instance, the twice-monthly network leader meetings in New York City featured frequent examples of Empowerment Schools Organization (ESO) central staff facilitating conversations about how the network leaders typically handled certain situations with principals and the pros and cons of various approaches. As one ESO staff member reflected on those meetings, “It’s been a long time since I’ve been in a place where people invite that kind of criticism and see it as an avenue for your own growth.” Also in this district, we observed how senior central office staff used meeting time to share results from regular principal satisfaction surveys with network leaders and to facilitate small and whole group discussions about lessons they might take from the survey findings. Executive staff also followed up on these discussions by meeting with individual network leaders to discuss results and the implications for their own practice.

Use of Evidence to Inform and Strengthen Professional Support for Instructional Leadership Directors (Dimension 2)

In New York and Atlanta, senior central office staff intentionally used evidence to inform and continuously improve how they worked to support ILDs’ practice. That is, rather than using this evidence mainly to evaluate ILD effectiveness, these central office administrators used various data to inform their own practice in supporting the ILDs. To illustrate, a central office staff person in New York reflected publicly that, in examining feedback from network leaders about their experience working with him/her,

    I often found myself stretched too thin. I know I was giving short shrift to things that must be done more thoroughly, from certain people who could benefit from more support. I didn’t make the time to work closely with them. As I learned and got deep into operations issues I wasn’t being as focused on instructional issues for a period this year. In general my weakness in terms of supporting folks is I’m not that good … at positive feedback. I display my sense of respect usually by critique [rather] than by applause. That usually doesn’t work for everyone.

This comment, to which network leaders responded with applause and praise, captures how those assisting the central office–principal partnerships were using evidence from experience, including feedback, to improve their own practice in providing such assistance.

Those providing professional support to ILDs routinely solicited and used evidence from various sources to inform their assistance for ILDs. Particularly in New York, where professional development for ILDs was also the best developed of all three districts, feedback from ILDs significantly shaped how other central office administrators designed and implemented opportunities for network leaders to improve their practice. For instance, executive central office staff in New York routinely facilitated extended discussions during their twice-monthly network leader meetings about how well the meetings and central office staff were working for participants and how both might be improved. At several of these meetings, central office staff used data from written end-of-meeting reflections and evaluations to kick off and otherwise ground those discussions. As one central office staff member framed one of those discussions,

    Most of what came back [in the recent written meeting evaluations] wasn’t new information, but [it is] useful to have [it] in this way. I want to throw open the floor for people to talk about what their thoughts are. [We] asked three questions at the end, including, “What’s working for you?” [That’s] important, but not the main reason [we are] having the conversation. [We’re] more interested in what’s not working, since we are interested in making adjustments. So that’s the second question, “What’s not working?” … That’s the point of this—to solicit that information. So the floor’s open.

Multiple staff people within the ESO described and demonstrated that their professional responsibilities specifically included capturing input from network leaders and translating it into terms that others throughout the ESO and central office system might use to inform and improve their supports for network leaders. For example, one of these staff people jumped into a conversation and reminded the group,

    I’m working hard to capture a lot of rich and useful conversation, and trying to distill it into three large strands. The broad stroke is we talked about how we need to differentiate our work, and do that in [a] way that builds capacity at the network level, school leadership, and classroom level. How do we find ways to identify strengths … within networks … and between schools? Second, how do we provide collegial resources, in that, making sure network teams recognize the needs they have, and have the time to go deeper to come up with multiple grouping strategies to figure out areas of needs and strengths. Collect tools so schools don’t have to replicate. This could be housed electronically and include inside tools, best practices in teaching and learning and operations … . Others talk about the importance of [having the time for] sharing best practices … Anything else?

The attempt by central office administrators to ground support for network leaders in feedback marked a distinct shift and improvement in the functioning of the central office between 2006–07 and 2007–08. In the spring of 2007, network leaders generally indicated that ESO central staff talked at them and did not provide opportunities for them to participate in the development of the ESO. However, starting in the fall of 2007, their comments more typically included remarks such as the following, “I think [ESO central office staff are] listening to us a lot more now than they did say last year, and so that’s helped.” Another confirmed, “I’ve worked with [ESO central office administrator] in [other capacities] and I know he does listen, and he does take feedback, and he listens to and wants input from everybody.”

These central office staff not only collected feedback but they also actually used it in the design and implementation of supports for ILDs. For example, New York network leaders had conveyed that as the ESO had grown—from around 15 network leaders in 2006-2007 to 22 by the end of 2008 (and closer to 100 network team staff attending some ESO meetings)—network leaders were losing a sense of collegial professional support within the ESO. In response, senior ESO staff developed a new model for organizing network leaders into smaller clusters of three to five network teams, providing a format for doing work within and across clusters while streamlining network leaders’ points of contact for issues such as business services, rather than have different people serve different clusters. One ESO central office staff member responsible for the professional learning of network leaders described that her position came about when she was a network leader because she had been “nagging” the head of the ESO for such a position. That leader responded by creating such a position and hiring the network leader and others.

In another example, one ESO central office administrator described,

    A lot of network leaders didn’t know how to enter into conversations with schools around instructional support … . I have been thinking about how to address that need, so that schools begin to see instructional support as something they would want to go to their network team for. The thing that tipped me off to this was when we did the network leader survey [a survey of principals about their network leaders] two and a half months ago. The lowest rating in terms of network leader satisfaction [i.e., principals’ satisfaction with their network leaders] was around instructional PD [professional development] we had been doing.

This person went on to describe how they then used such evidence to inform not only the conversations they prompted in network leader meetings but also in their development of tools for network leaders, including a common instructional framework much like the 26 Best Practices tool in Atlanta.

In fact, one reason the 26 Best Practices tool came about in Atlanta was that senior central office administrators reviewed various sources of evidence about principal performance and how the executive directors worked with their principals. Senior central office administrators subsequently used that evidence to inform the development of the tool and support for executive directors in using it to ground their work with principals. As one central office administrator recounted,

    So I just kept asking principals when I’d visit their schools and they kept saying – well first they didn’t know how to articulate it [high-quality teaching]. They were saying, “Well … everybody’s focused on data … All the SRT’s help us understand that. But they do it differently. They look at things differently.” So I’m hearing that. ... So I’m listening and I’m visiting classrooms and I’m saying, you know what, the things I’m talking with the principals about, [executive directors] didn’t pick up [in observing classrooms]. So [the executive directors] don’t know what to look for [to gauge the extent to which principals are responding aggressively to low test scores]. So I had the executive directors bring in examples of principal feedback to teachers. They didn’t know what they were seeing—they couldn’t give feedback to it, right? So I said to [two consultants] who have been my partners in crime through this, we have to do something. I need some APS teaching expectations that go further than just the performance evaluation instrument that really details what teachers do, alright? What do teachers do at high levels? So we worked on it … . For three years we figured this out. ...

Use of Evidence in the Reorganization and Reculturing of the Rest of the Central Office (Dimension 3)

As discussed above, the reorganization and reculturing of the rest of the central office fundamentally involved the use of evidence. For instance, project management by design demanded that central office administrators use evidence from their own experience, internal assessments and audits, and other sources to help them address specific problems with their support for teaching and learning improvement. For example, as one central office administrator described the centrality of evidence use to the ongoing project management focus of the Oakland Operations Support unit,

    I think one of the things that’s successful [about what we do] is that the whole is greater than the sum of its parts. That we really capitalize on [our] … shared learning [so the work can] … evolve and be what it needs to be at the time.

We found that across all three systems, central office administrators also routinely collected and tried to use evidence, mainly from principals and, occasionally, from ILDs to inform their understanding of their progress with reorganizing and reculturing other central office units and adjustments to their change strategies.

One key strategy for collecting feedback from principals related to changes in central office work was to convene principals for discussions of their interactions with the central office. For instance, in Atlanta, the superintendent met regularly with small groups of principals to discuss their experience with various aspects of the central office, including those aspects that reflected the district’s reorganization and reculturing efforts. These conversations provided either the feedback or direct recommendations back to central office units to inform their change efforts. Sometimes, the superintendent asked staff from other central office units to sit in on her principal meetings to hear the feedback directly. As one central office administrator described these meetings,

    When Dr. Hall meets with those principals and they bring it to her attention that maybe [one department] is creating a barrier for them … . And the [department] person was able to hear that and she went back and really worked with her department, reorganized the entire department so that they could be closer to the school when they assigned the specialist to the schools.

Similarly, the senior central office administrator overseeing the reorganization and reculturing process in Oakland regularly convened several principal advisory groups that provided input on their efforts. The leadership of the ESO in NYC convened principals in a quarterly meeting of Principals Council where they discussed various changes in the central office and solicited principals’ feedback on them.

Central office staff in each district also developed and implemented a regular survey to capture principals’ feedback on various aspects of their reorganization and reculturing efforts. For instance, in Oakland, central office administrators created the Use Your Voice survey, based in part on a long standing model from Edmonton, Alberta (Canada), which asked principals and community members detailed questions about their experience with the central office and, in the case of community members, their experience with schools. Central office units also surveyed principals and other relevant constituencies about their performance. Likewise, in New York City, central office administrators in the Office of Assessment and Accountability, with participation from other units, developed surveys to gauge principals’ satisfaction with central office services.

Principals reported that particular central office administrators took informal opportunities to collect information from them and how they used that information to effect central office changes as part of their reorganization and reculturing efforts. For instance, one principal described how, whenever he ran into one of those central office staff people in the hallway of the central office, she always stopped and asked how things were going,

    … And I really tell her … Last year [for example] we had a really hard time and I think this is district-wide, attracting and retaining quality qualified teachers. So I refuse to keep people who should not be with kids. So I keep dismissing people at the end of the year. But I get the same thing back. The same quality … .

This person went on to describe how last year the teachers were not only low quality but they were also new to teaching and not teaching well.

    So I was really struggling with how to support so many new people … So [the central office staff person] e-mailed me saying, “Glad we had that conversation [in the hallway]. I’m going to look at how we can support schools that are going through this.” And I believe from that came this idea of intensive support for teachers. So they sent somebody out to do intensive support with one of the teachers and that was a direct response to that conversation.

As another principal corroborated, “I think [two central office staff people leading the reorganization and reculturing effort] have really helped streamline a lot of things at the district level that need a lot of work … . And I think they’ve really done a great job of keeping their eyes and ears open, having meetings, having focus groups … really putting ideas into action.”

As the above examples show, central office administrators in these systems not only collected data, they also intentionally used those data to change how the central office operated. For example, in Oakland, central office administrators used findings from the conversations on the principal advisory groups as well as the Use Your Voice survey to develop a Service Scorecard for each central office unit. As its main architect described, the Scorecard “highlights their key services and the standard to which those service will be delivered, and progress toward those goals.” This administrator then used the Scorecards in meetings with central office staff to engage them in challenging conversations about their progress with improving their service to schools. Data from the Scorecords was the starting point for productive retreat conversations, among other instances, that identified and set in motion projects aimed at improving the functioning of central office units concerned with facilities maintenance, management of teacher substitutes, and payroll accuracy, among other targets. In subsequent work on these projects, staff continued to use evidence to monitor their progress.

Respondents throughout the central offices generally reported that these efforts to systematically and regularly collect and use multiple forms of evidence to inform their reorganization and reculturing efforts were fundamental to their progress. As one central office administrator in Oakland explained, “If you really just listen to principals you would think there’s never a sub [substitute teacher] in any class at any day. But when you say … —‘We’re at 77 [percent rate of filling principals’ requests for substitutes] this month. We’re trying to get to 80 next month and here’s what we’re doing’—those data shift the conversation from blame to problem- solving.” The administrator went on to describe that one of the problems with the old central office was the “lack of being data-driven … . And I think … central office has suffered from that just as much as schools have suffered” in the sense that the absence of data fueled a sense that nothing was working and a culture of blame. The administrator went on to explain,

    And the more we have data to tell our story about, well here’s how we’re really doing and you can perceive it however you want. But actually what happens is when you start using data you start changing your perceptions of people. So that’s why I’m training my managers on how to do that and be very data driven. And be almost like a coach of a baseball team that’s using their stats all the time.

As the comment immediately above suggests, use of evidence strengthened the reorganization and reculturing efforts not only by infusing the change process with input and new ideas but also by helping create a feeling among central office staff that they were being listened to and acknowledged for their work—a key resource for reculturing in organizations where staff may have felt unrecognized, criticized, or outright demoralized.

New York City/Empowerment Schools Organization (ESO) stood out for efforts to use not only school performance and principal satisfaction feedback but also feedback from network leaders as key evidence for information their reorganization and reculturing efforts. Certain central office staff people described that part of their job involved collecting information about network leaders’ experience and passing it on to senior central office staff to help guide the reorganization and reculturing effort. In the words of one,

    I … work with the Network Leaders … . I interact with them a lot again and particularly in this air traffic control sense … . Where there are things that are not working for them – helping surface that up to [ESO leadership] in places so that we can come up with things [we should change], like an aspect or a component of this performance management system that we’ve been working on.

Central office staff also used feedback from network leaders to inform their decisions to expand the scope of the ESOs work to absorb some of the central office functions that network leaders reported were not being carried out well by others— that is, to reorganize and reculture the rest of the central office by assigning certain lines of work to their own staff who might perform the work at a higher level of quality. In one instance, network leaders shared with executive central office administrators that the central office unit responsible for special education placements had been sending students to schools that had no openings. In one network leader’s words, “They were sending 15 kids to a school that had only 12 openings for special ed … . Nobody seemed to know what was going on. So as empowerment leaders, we expressed this all to our empowerment organization.” This network leader went on to describe meeting with ESO leaders who decided to shift responsibility for special education placements in ESO schools from the regular central office unit that served all the school support organizations to the student services managers on their network team. Under the new system, network team staff members were to work within their network to “juggle it around within your own network. And then if you’ve exhausted all that, then you could go to your sister network or your brother network.” Network leaders reported that this arrangement solved the special education placement problem but created the new problem of taking excessive staff time away from working with schools on other matters. In response, the network leaders developed a plan to hire someone to help with the placements and the ESO leadership agreed.

In another example, central office staff routinely collected feedback from network leaders that the Integrated Services Centers (ISCs), the central office units established to handle basic operational transactions for all the schools in certain geographic regions, were not serving their schools well. Executive ESO staff responded, in part, by piloting the Children First Network (CFN) that added eight other staff members to carry out the operational functions that otherwise fell under the ISCs’ umbrella. Our observations of this pilot revealed that ESO central staff routinely consulted with CFN staff members to understand their experiences and inform their decision about whether and how to expand the pilot. As one member of the network pilot described,

    We had a lot of meetings with [ESO central staff] … . We met with the Chancellor and with [the head of budget] … . The Chancellor had an audience with us to ask us whether we think the CFN is terrific for us or not and if we would recommend it getting larger, or developing new CFN models … And … after … they agreed to do it and so obviously they … heard us [that]… there are not enough hours in the day or days in the week [for us] to spend time doing things four times over. You have to do things one time and get it done well and so as a principal there are so many things that happen that you don’t have time to work on … . You should be spending your time doing what’s important which is the leadership in the school and the instructional leadership.

The experience of central office leaders in Atlanta reinforces that such evidence use processes are ongoing and fundamental to the work of reorganization and reculturing central office units, even nine years into the work, in part because the nature of the work is not to implement a fixed model but to continually adapt. As one central office administrator described,

    What’s different [over these years of our transformation effort] is I’m more in tune now on my improving as a leader to better support the people who count on me… than ever before. I think for a couple of reasons. I think because in Atlanta Public Schools we do the technical work at a very high level, but now it really is about the adaptive work … especially since we’ve done what all school systems say they should be doing, but very few have at a system-wide level. And that is almost narrow or flatten the achievement gap between us and the State at the elementary. And then when you really look at it, a school system that has eight years of continuous movement in a positive way. That trajectory is very rare, because you usually see peaks and valleys, no matter who the superintendent is, even if they happen to have a superintendent that stays longer than 2 or 3 years. So given that, there aren’t very clear packages of what to do next. And I really paid attention to this piece.

This administrator went on to describe that he continually consults with evidence of how well the reorganization and reculturing process is working to inform his own participation in the process. The administrator reflected, “That really was part of my learning.”

Use of Evidence to Ground the Stewardship of Central Office Transformation (Dimension 4)

Use of evidence also appeared essential and consequential to the stewardship of the central office transformation effort. For example, central staff of the Empowerment Schools Organization (ESO) in New York City routinely used their meetings with network leaders, particularly during the second half of the 2007–08 academic year, to present new draft models of ESO organization and engage network leaders in extended conversations about the pros and cons of the models for the central office transformation effort moving forward. As one ESO central staff person framed one of those conversations at a network leader meeting, the purposes of such conversations included, “to get a sense of what you have learned this year and what you are thinking about for the future and what you think I need to be thinking about for the future.” Similarly, another central office staff person described convening smaller groups of network leaders to inform the development of the ESO’s overall strategic direction around instructional improvement. As one staff person described this effort, the emerging instructional model, came from…

    … the best of the knowledge of the people in these five networks … . And so as a result of me coming together with them with their practical knowledge and my calling it more research-based knowledge and more policy-based knowledge, it’s been very exciting. It fuses together some of the best of those two worlds and it really brings people to the table.

Network leaders generally reported that, especially compared to the previous year, central office staff, not only in the ESO but throughout the system, seemed to be trying to learn from their experience to inform the ongoing development of the central office transformation effort. In the words of one, “listening going on” is “so astronomically different” and “better.” As one network leader described, “Joel Klein [New York City Public Schools chancellor ] is one of the best listeners I’ve ever met in my life.” This person explained that though rarely in direct touch with him, “indirectly we know that what we say goes through [one ESO staff person], goes to [another ESO staff person], goes to Joel, and I think that’s pretty cool. So I might not feel directly listened to, but I think indirectly I feel that the kind of things that we’ve been able to push are the case.”

Here, too, central office staff did not just collect information but they actually used it to inform their stewardship of central office transformation—and their way of doing so reflected the notion that stewardship implies ongoing learning by leaders within the system about how to design, implement, and support the central office transformation effort. As one central office administrator from Atlanta reflected, that shift in orientation in working with evidence was…

    … a growth piece for me because living urban education as long as I have and knowing that usually you’re at the brunt of people’s criticism, it’s hard not to just naturally want to defend [what we are doing] or think that … they [those outside the central office] just don’t trust us, they don’t think we know anything [and not consult with them] … But over time … I do far less of the defense and getting angry … What I [now] tend to do [if I receive negative feedback] is say, “Mmmm. We probably weren’t clear or it sounds like the parent is really frustrated about the change [and] they need some more hand holding.”

In Oakland, we found evidence that executive central office staff solicited feedback on their stewardship of the central office transformation effort. Those staff also seemed to use that evidence, mainly to inform their decisions to communicate about their theory of action for central office transformation. For example, the East Bay Community Foundation funded and staffed a major effort to convene a series of more than 40 “community engagements” across the district. At the engagements, a facilitator from the host organization engaged community members in providing feedback to the school district on their central office transformation effort and other aspects of their “Expect Success” initiative. We observed how one central office staff person generally attended these meetings and spent most of the time listening to the conversation and occasionally urging participants to provide honest and direct feedback, promising the group that the input would be brought back to the rest of the central office. Likewise, two central office staff people convened the principal advisory groups mentioned earlier in part to solicit their feedback on various strategic initiatives within the central office. Executive central office staff reported that they learned from those meetings as well as the Use Your Voice survey that principals and community members generally did not understand the theory of action underlying the central office transformation effort, particularly when it came to the new results-based budgeting system. Nor did people outside the central office know who the staff members on the executivelevel strategy team were or what they were responsible for. In response, Strategy Team members increased the frequency of school visits.

However, we found limited to no evidence that the high-level central office staff people in Oakland used the evidence collected through the engagements, advisory groups, or surveys to inform other dimensions of the stewardship of the central office transformation effort. The Oakland example, corroborated with evidence from New York City, reinforces that evidence use is important to stewardship not only for the information it provides to central office staff but also for the feeling it creates among some staff that they are valued participants in the central office change process, fundamental to creating a culture of change and improvement with the central offices. To elaborate, NExOs generally reported that they believed they had been hired to participate in the stewardship of the central office transformation effort but that they generally were not involved in or consulted about those aspects of the change process. As one NExO commented,

    I would like to be at the table with Strategy Team … to really discuss the big picture. And that’s just not going to happen because … that’s not their belief system. And so that’s been a big reality check I think for all of us. And some people are saying, “well now I know that this might not be the job for me.” I’ve heard people say that.

Another leader expressed similar frustration and gave the example of senior central office staff convening a lunch for their principals but not taking the opportunity to engage people in a conversation about how the work was going.

    Either you want us on your team and either you want to build that loyalty or you don’t. And that’s not to say we need to be involved in every decision. But there’s a process of information transfer and input solicitation that’s not hard to do. [Our relationship is] not structured in a way that it’s co-reflections. That we are partners. That’s not good.

A significant number of NExOs attributed the turnover of NExOs (half of whom had left their position at the end of our data collection period) to their limited opportunities to participate in discussions of stewardship. Similarly, some attributed the turnover of some network leaders in NYC to disagreements with the direction of the central office transformation effort, particularly the design of the accountability system, and their limited influence on those aspects of the central office transformation process.

Summary

In sum, central office administrators in all three districts intentionally collected evidence about their experience with the central office transformation process and worked to incorporate that evidence into their ongoing decisions about how to strengthen their work. We documented that these use-of-evidence processes were part of all four dimensions of central office transformation. Because these processes were such a prominent and promising aspect of central office transformation, we report them as a dimension of central office transformation in and of their own right.

< < Previous | Next > >