My previous six posts have examined the position on educational purpose taken by Professor Biesta. I have concluded that when he (like many of his child-centred colleagues) says that we should focus more on purpose, he does not mean to clarify but rather to obfuscate that purpose. He means to place responsibility on individual teachers to decide what their various, implicit and often meaningless purposes should be. This leaves no possibility of taking systematic action to achieve such objectives or of giving any clear account to the rest of society on how effectively this has been done. It is a model that sits uncomfortably with Professor Biesta’s professed desire to improve democratic accountability. In this post, I turn to the reasons why Daisy Christodoulou also opposes the explicit description of educational purpose.
Many would rate Daisy Christodoulou, ideologically speaking, as the polar opposite of Gert Biesta. Yet when it comes to stating explicitly the purposes of education, Biesta and Christodoulou share the same position. Professor Biesta does not see a need to state the purpose of education in meaningful terms because he does not think teachers should be accountable for delivering purposes imposed on children from without. Daisy Christodoulou does not see a need to state the purpose of education because she thinks such purposes are not only legitimate but also obvious. Even if we might have some trouble defining exactly what we mean by “soft skills”, Christodoulou argues, no-one could object to our students becoming numerate and literate—so why do we not just get on with achieving these obvious goals?
To attain an objective, it helps if it is precisely defined. If on a military exercise, you share with your colleagues only a vague notion of the location of the enemy position to be captured, you should not be surprised when your midnight attack ends up in a shambles. Bill Schmidt’s work on “curriculum coherence” suggests that American children are taught “long laundry lists of seemingly unrelated, separate topics” which are “highly repetitive, unfocused, unchallenging, and incoherent” instead of what we should be teaching which is “really a handful of basic ideas”. Even when it comes to something as apparently clear-cut as “numeracy” or “maths”, we seem to have very different ideas of what this involves. If we are to ensure that teachers are aiming at the right target in the first place, we need to define our objectives at more granular level: to describe in detail what a numerate person looks like, breaking down high level objectives into their constituent parts.
Such a description of granular learning objectives is what was attempted by the model of criterion referencing that was formally introduced in the 1988 Education Act and formally abandoned in 2013. Daisy Christodoulou has played a significant part in attacking criterion referencing as a member of the Commission for Assessment without Levels and in her blogs and presentations.
Our model of criterion referencing was certainly flawed—but the problem lay not with the principle that we should attempt to define our educational objectives. The problem lay with the particular way that it was done.
The first problem was the brevity and consequent lack of clarity of the rubrics by which criteria were defined. In his evidence to the 2011 Framework for the National Curriculum, Professor Paul Black, the Chairman of the original 1987 Task Group for Assessment and Testing (TGAT), suggested that attainment targets should be described in longer, discursive paragraphs rather than short phrases (p.43).
A second problem was frequently a lack of clarity in the underlying concept that the rubric described. Daisy has quoted the example given by Dylan Wiliam regarding the comparison of fractions. A criterion that required that students should be able to “compare two fractions to identify which is larger” would not indicate a clear level of attainment because the difficulty of this task depends very largely on the particular fractions that are to be compared. Research by K M Hart in 1981 suggested that 90% of 14–15 year olds could tell which of 3/7 or 5/7 was the larger while only 15% could do the same for 5/7 and 5/9. Although this particular criterion was invented by Dylan Wiliam for the specific purpose of illustrating what a bad criterion reference might look like, it is fair to say that many statements of attainment in the National Curriculum were drafted in a similar way, giving very little idea of the difficulty of the performance that was being required.
This problem could be solved by the provision and analysis of exemplars, as was specifically recommended by Professor Black’s original TGAT report: “We recommend that attainment targets be exemplified as far as possible using specimen tasks. Such tasks can then assist in the communication of these targets” (para.56). This was poorly done, if at all. Each SoA in the 1988 curriculum was accompanied by a single example, which was brief, included no student response to give an impression of the standard of performance expected, and was in many cases inconsistent with the rubric of the SoA.
As well as communicating the meaning of a criterion, exemplars may also be used to verify whether the standard of work expected is clear. The sort of inconsistency of anticipated performance that is identified by Professor Wiliam would immediately become apparent if not only exemplars but also the application of the criterion to the work of real students was monitored and correlated. Needless to say, this was never done.
A third problem was the use of binary measurement metrics: either a student had “got it” or they hadn’t—but it was not allowed that they might lie anywhere between. As Daniel Koretz says in Measuring Up, such binary criteria “are just cut scores on a continuum of performance” (Kindle location 3362). As teachers naturally wish to see their students do well, the latter tended to be credited with mastery on the basis of the most superficial performances, aggravating grade inflation and trivializing the curriculum.
A fourth problem was that relationships were asserted between different criteria, which were organised into scales of ascending difficulty, without any empirical research being used to establish that, even if a consistent understanding could be assumed for what the level 3 Statement of Attainment really meant, it was at least more difficult than level 2.
The fifth and perhaps the most serious problem of all, is what I would term the logistical problem. The 1988 National Curriculum introduced 297 Statements of Attainment for Maths and 407 for Science. Teachers could not realistically track student performance against so many different criteria and so it was no surprise that the 1993 Dearing Review, complaining about the “meaningless ticking of myriad boxes” (p.62), drastically cut the number of attainment targets.
In solving the fifth problem, Dearing introduced the sixth, which was one of over-simplification. The new high-level criteria—what became known as “levels”—no longer operated at an appropriate level of granularity to support teachers who needed to manage formative assessment. Instead, student performance had to be rated using a “best fit” methodology, which often credited students with levels of attainment that did not reliably represent their true standard of performance.
The recommendations of Professor Black’s original TGAT report were largely ignored in the subsequent implementation of the National Curriculum. The whole concept of criterion referencing was misapplied and subsequent attempts to mitigate the original problem only succeeded in compounding the mess. Twenty-five years later, the approach has been abandoned, apparently on the basis that that because we did something really badly, it is therefore not worth trying to do it well. When I mentioned criterion referencing at the Commons Education Select Committee conference, a delegate from Ofqual gave me a look of pity: criterion referencing has become a dirty word.
No-one seems to have analysed what went wrong (as I have done above) or how it might be fixed (as I shall do in a future post). No one appears to be interested in why we still need to describe our educational objectives with precision and at a granular level. This will be the subject of the ninth part in this series.