25 March 2015
The DfE have been playing with performance data in the same kind of way that schoolzone do, with our educational intelligence data. Like us, they try to find patterns in schools data that reveal something meaningful. Recently, presumably in order to support Ofsted's new freedom to inspect (but not judge) academy chains, they've been looking at how well chains perform compared to LAs.
Last week it was that less than half of academy chains achieve better than average results (think about this mathematically). This week the DfE observe generally that chains aren't doing so well, compared to LAs, but that this is probably because chains tend to have mopped up failing schools. They illustrate the differences thus:
Best performance is upper right, worst is lower left. On the face of it there are more LAs than chains in the best quadrant. But there are 100 LAs in the sample and only 19 chains. If we ignore the clump around the centre, being pretty much average and look at those close to (but outside) the central clump, we find that:
in the outer reaches of lower left
- 43% of chains are poor () and 57% are very poor.
- 64% of LAs are poor, 36% are very poor.
So when you compare "poorness" chains are less successful than LAs.
If you apply the same reason in the opposite direction: looking at distance from the centre:
in the upper right quadrant:
- 100% of chains are very good
- 56% of LAs are very good
So, like most stats, it depends on how you look at the data. And the basis for comparison is dodgy here because the improvement score (y axis) is based on the raw score, not a scaled one, meaning that we need to consider whether schools should find it easier to add value to low prior attainers or to high: low attainers have more room to improve and are more likely to have under-performed at KS2; high attainers are more likely to have over-performed, especially in KS2 maths, and so on. Improvement measures need to be based on a prior attainment that strips out this uncertainty.
The reason we talk about playing with data is that (apart from the fact that it's fun), there are many ways to look at it. In this report the DfE struggles with trying make analyses based on a too-small data set - just 19 in the case of chains.
So are chains less successful than academies? Too soon and too few of them tell.
A more important question at the moment might be: why do LAs have the responsibility for finding school places in their regions, but have no power over chains to force them to do anything about it? While chains are free to reap any rewards gained from the freedoms they have over admission numbers. And so on.
(Apologies to General MacArthur for mangling his words in the title)
Recent blog posts