On October 7, MCPS sent a letter from Phil Kauffman the President of its Board of Education to the Maryland State Superintendent of Schools requesting the state to reconsider plans to use the new annual test called the Partnership for Assessment of Readiness for College and Careers (PARCC) assessment as end-of-course exams for purposes of fulfilling high school graduation.  For several years the Maryland State Department of Education (MSDE) has been preparing to use these new tests developed by a consortium of states as a replacement for the Maryland State Assessment (MSA) and High School Assessment (HSA) tests that have been used for the No Child Left Behind (NCLB) law’s accountability requirements.

These new PARCC tests hold much promise to improve the information to schools.  They are developed to be aligned with the new Common Core State Standards (CCSS) for literacy and math.  They use state of the art technology for an adaptive testing experience.  They are also unproven and given that they are likely more stringent than the MSA/HAS tests they replace, many students may not pass them, which would require many students to take a transitional class prior to graduating.  The state is considering a two tier approach where there is a criteria (cut-off score) for being considered “college and career ready” and a lower score to allow students to graduate.  This tiered approach has been advocated for by MoCoEdBlog editorial member, Rick Kahlenberg in a piece titled “Hold Students Accountable and Support Them.” 

MCPS is requesting that MSDE delay the implementation of these requirements and enter into a discussion about how to move forward.  Kaufman’s letter posed several important questions:

“If a college-ready cut score differs from the graduation cut score, what is the most meaningful indicator for institutions of higher education or employers? What messages do tiered cut scores send to students? Maryland now requires all students to be assessed for college and career readiness, and those found not ready, must be enrolled in transitional courses. Given this new paradigm, is there benefit added to continuing the requirement to pass end-of-course exit exams to receive a diploma? Moreover, if, during this period of transition from HSA to PARCC, it is appropriate to prohibit use of PARCC for purposes of personnel evaluations, why is it not equally appropriate to delay the use as a high stakes test for students?”

These are important questions.  I will focus on the last one about using the tests for one purpose only: graduation.  I believe that within this question lays an important systemic consideration that Kauffman’s letter only hinted at: if the tests are not ready to be used to evaluate teachers and principals then why should they be used for students?  I believe this is a strong argument in support of MCPS’ request.  While the change from one testing system to another may seem a matter of upgrading the measurement approach, the reality on the ground is that these kinds of changes are not “plug and play.”  There are many interdependent and moving parts in a school system and high-stakes tests impact many of them, including teachers and students alike.  Students are dependent upon the instruction they receive and the instruction is shaped by both the ability of the teachers and the rewards, incentives, and constraints they work within.

To use the tests for one part of the system (student graduation) but not for another part (educator evaluation) would, in my view, create a tension in the system.  It would have part of the organizational focus in one direction and another part in another.  High school achievement, like the achievement gap and many other big problems in education are systemic.  They have multiple interrelated causes and one of the reasons they are so hard to address for MCPS and the rest of the country is that independent solutions rarely address the combination of factors that underlie the problems in complex systems.  Using PARCC for only student accountability is a partial approach.  And, as Kauffman’s letter says, it puts students in the unfair position of being the ones getting the shorter end of the stick as these new tests are tried out on their future first.  For this reason alone, I believe the MCPS request deserves support from both MSDE and Montgomery County’s elected officials.

There are more questions that can be asked of MCPS about their readiness to support PARCC across the system.  Below, I will sketch out some other important factors and end with some questions that could be included in the conversation that MCPS in Kauffman’s letter requests.

Some Observations about PARCC

PARCC Will Initially Be Disruptive

The implementation of the new tests will be disruptive.  How much they will disrupt the work that goes on in schools is not clear.  But, history has shown, including with NCLB, that most large scale changes in schools can “shock the system” and take some time to become assimilated into the routine.  The day to day work of schools is so labor intensive and what teachers do especially is so often based on what they have done in the past that any change such as a new curriculum will take some time to assimilate.  The fact that PARCC will be aligned to the CCSS will help as MD schools will have a couple of years of experience with these standards. Still, the new tests under the best of circumstances will require some adjustment, at least at least in the first year.

The impact on the schools has the potential to be even bigger and more difficult on schools.  If the PARCC results are tied also to high-stakes consequences such as teacher evaluations or school performance.  One of the lessons of NCLB (and there is a lot of research on this) was that schools with greater challenges suffered much more collateral damage than schools with better circumstances.  So, if the PARCC tests will be high-stakes then they will be higher stakes in the schools with the most needs.  The tests can still provide much valuable information and the information should be used, but tying these results to high-stakes for educators would likely be disproportionally absorbed by high-needs schools.  MCPS needs a robust plan that addresses the impacts of PARCC on the system.

Implementation Questions with Technology-Rich Assessments

PARCC Assessments are designed to be delivered on computers rather than paper and pencil. However, not all school systems or school buildings have the same technology and so there are alternative testing approaches that have raised some questions about how PARCC will work when the rubber meets the road. For example, national education expert Rick Hess raised three big issues earlier this year:

1. Testing under different testing conditions (some in classrooms, some in media centers, some offsite at different locations.
2. Testing using different devices (ex: computer vs. paper and pencil)
3. Testing windows that can vary from school to school so that the tests may be taken at different times by different students

None of these issues fundamentally compromises the value of the tests both as well designed instruments and even more being aligned with the CCSS.  But they all can impact the scores in ways that will be really hard to know until after the tests have been administered.  Will the impact on the scores vary based on the kind of school in the same way high-stakes impacts high needs schools more than others?  Quite possibly they will.  MCPS should look at its implementation options and try as best as possible to standardize testing approaches across schools.

Will the Tests Perform as Designed Initially?

When we read that the testing of the tests has gone smoothly, it is important to remember that these are reports from the people who are administering the tests and that smoothly may mean different things to them than to educators.  For example, if the field trial occurs where and when it was planned and the results are able to be tabulated by PARCC, the field trial is smooth from a technical perspective.  This doesn’t mean, however, that the tests were measuring the same things that were taught or that they did as good a job with different populations as the designers hoped.  Larger amounts of real data and more time are required to know this.  Again, it probably will not be until after the first year of full administration that these issues will be clearer.  Also, scuttlebutt from behind the scenes at PARCC has for a few years now has been that the amount of money they began with was less than they needed so don’t be surprised if the quality of the tests is not even; that some parts of the curriculum test more reliably than others.  MCPS should be careful about making inferences based on the results of any part of the curriculum until the broad strengths/weaknesses of the test quality are known.

Some Important Questions for MCPS’ Implementation of PARCC

While it is important to support MCPS’ request, some questions could be asked of them about their plan going forward.

1. Professional Development and Support. With the recent adoption of CCSS curricula, MCPS along with just about every school district has found the need for professional development was more urgent than expected.  How are the plans coming to train MCPS educators in how to use PARCC?  What lessons have been learned from the pilots thus far about the technology needs as well as the performance of the tests beyond the fairly positive accounts MSDE and PARCC have provided?
2. PARCC Impact on Technology Budgets. How will PARCC impact the spending decisions throughout the school system?  One of the biggest criticisms of the CCSS has been it is an opportunity for companies to make even more money from education.  School principals, teachers, and even some families are getting inundated with many offers of products that will help prepare students to do well with CCSS and PARCC tests.  Most of these claims are unverified. There is no body that will certify that a product is 100% or 50% CCSS compliant.  There will in the future probably be ways of rating these products this way by the people who use them; but not today.  MCPS would be wise to not to spend too much public money on materials to help prep for first round of tests if it can be avoided.  Much of what is on the market now has been rushed to market and is full of errors. Reviewing materially centrally and making recommendations to schools for how to purchase makes a lot of sense as does working with partner districts to assess the quality of materials and technology.  While MCPS tends to defer a lot to individual schools (site-based management) rather than centrally manage and direct, in this case it may be useful for MCPS to take stock of the products that are out there and provide good technical support to schools.
3. Accountability Options. One of the driving reasons for high-stakes tests is that not all schools perform as they should and not all schools perform equally well with all groups of students.  Even with all of the many problems with implementation, policies like NCLB have been important ways to see educational differences and also to shift the conversations for many in education towards hard outcomes.  As the sanctions of accountability are even temporarily lifted, what will MCPS be doing to ensure all students are getting the kind of education they deserve?  Will the PARCC test results be combined with other forms of evidence to ask about where there are areas that need improvement and additional attention? While delays in using PARCC for HS graduation make sense, what other external accountability options will MCPS use to ensure all children receive appropriate education.

Summary

As Kauffman’s letter spells out, the issues surrounding the use of PARCC tests for high school graduation are complex and consequential.  The MCPS request to delay implementation of the state’s plan is reasonable.  Whether the state will listen is unclear.  Whether more information about how MCPS is getting ready for PARCC and the new testing and standards paradigms it is part of will help MSDE in their decision is unclear.  For those closer to MCPS—parents, teachers, local elected officials—this kind of information is probably important to have.  For the sake of MCPS’ management thinking and capacity to deal with the difficult and complex problems of student achievement, it is probably important to develop.  MCPS, like pretty much all districts, has traditionally been dependent on state policy and so there may be a tendency when in this role to wait and respond rather than taking the lead and driving the discussion.  MCPS is no ordinary district.  It has not only broad needs but many financial and intellectual resources so it is in a better position than most to lead rather than respond.  The tone and message of Kauffman’s letter suggests this is what MCPS is trying to do.  Let’s hope the state is ready to meet them in a discussion about this difficult issue.