Is your assessment system Progress-ready?
22 Sept 2014
"We already measure progress," many schools say. But progress isn't what it used to be.
Teachers say (via various Schoolzone research consultations) that they welcome the changes to school accountability - it's fairer to compare schools on the basis of value-added than on straightforward SAT level or GCSE grade comparisons. Previous attempts to use value-added haven't made it through to the league tables, but this time it's built into the new secondary Progress Eight floor standard and is the most-favoured accountability measure for primary schools. While SAT scores, EBacc and number of A* to C (or, presumably will be 9 to 4 or eventually 5) will still feature in reported measures, the main measures are based on progress.
This will have major repercussions - for some secondary schools in particular. If you have a high-attaining intake, obviously your P8 measure will be lower than for similar-attaining schools (five A* to C) with lower-performing intakes. Ditto for schools who have done well out of EBacc through careful manipulation of options, playing the early entries game or other strategies which will not work with P8. Grammar schools are likely to be hardest hit on the basis of value-added - not only for these reasons, but because the old style SATs had a ceiling of level 5, which high proportions of their intakes might be comfortably above: new SATs will incorporate the equivalent of level 6, so grammar school value added will be calculated from a higher baseline.
Anyway, whatever your school intake, the basis on which your school will be held accountable is changing and your assessment system will certainly need to change to accommodate it. Here are just some reasons why you shouldn't be complacent.
National curriculum levels
Scrapped. If you plan to continue to use them, they'll need revising for the new curriculum - ie they are no longer standardised and hence not much use for assessment. Plus, if you use them for longer than the next couple of years, you'll find it Ofsted taking a dim view.
Benchmark attainment against expected national performance
Schools tend to read SATs scores as benchmarking even if, on a day to day basis schools use NC levels simply to check understanding and so on - it's built into the thinking. However, as the new NC beds in and secondary schools prepare students for new GCSEs, the validity as a benchmark will fall away. So how will you know that your students are progressing as well as those in other schools?
Monitoring attainment is not enough
At present, your assessment system will probably tell you how well your students are doing, but it probably won't tell you, day to day, how much progress they have achieved from the end of the previous key stage. Schools tend to look at this retrospectively. Even if you do have this built it, it will need adjustment - read on.
Setting targets and monitoring progress towards them
Probably, if you use target setting, you base it on NC levels (defunct) or GCSE grades (reforming), so the basis for the definition of targets will no longer be relevant. But, just as importantly, your school is to be judged on how much progress pupils have made, so targets need to be more ambitious - simply achieving targets means zero progress by the DfE definition and your school will be below floor standards.
Use of previous internal assessment outcomes for monitoring progress
Generally, assessment systems work at the subject level and they compare one set of assessment data against a previous set generated by tests (etc) set by the school. But the new progress measures will be based on external (SAT) tests - in English and maths only. So your assessment systems, if they are to be any use in checking that the school is on track, will need to take these into account too.
Monitoring progress between years or within a whole key stage
This is how most schools currently look at progress: "this year, the average was x; last year/key stage it was y", but if you've read this far it's hopefully already clear that this is probably not enough to be sure that the cohort is making sufficient progress. All assessment grades will, ideally incorporate progress.
Use of KS2 data
Secondary schools routinely distrust KS2 SATs and teacher assessment data and often use their own baseline assessments instead. However, P8 is firmly based on SATs, so this will have to change. The DfE have promised to tighten up on this (by asking secondary teachers to moderate), but there is a cultural shift required here. Furthermore, KS2 SATs scores will be scaled each year so that pupils will have a numerical score, where 100 is the average. No-one yet knows what this will look like for their own school, so current predictions of P8 scores are based on shaky evidence and current assessment systems will somehow have to adapt to this new basis. Also, current systems are unlikely to be able to accommodate KS2 cohort fluctuations - ie the average score of 100 is not necessarily the same, at school level, year on year.
In summary, if you use an assessment system based on NC levels predicting GCSE grades, its using a short-lived measure, predicted a short-lived target, on the wrong basis, with the wrong outcome, using the wrong kind of assessment based on the wrong data.
That's phrased deliberately provocatively, with the intention of encouraging schools to think very carefully about whether you need to change your thinking to accommodate the new definition of progress.
Recent blog posts