I have intended to build up a carefully stepped argument in this blog, only progressing to look at specific policy issues when I had covered some important background first. But in view of the speed with which the current debate around education technology is progressing (and in particular, the opportunity this week presented by the #AskGove Twitter campaign), I have decided to publish ahead of schedule a summary of reasons why the DfE should cancel it’s ill-conceived Information Management and Learning Services framework.
Background
On 11 November the DfE published an Invitation to Tender for a framework agreement for school software, with the deadline for responses on 22 December.
The framework will produce a short-list of suppliers, simplifying EU rules for local authorities which subsequently make procurements worth more that £150,000. It is irrelevant for schools, which invariably make procurements worth less than £150,000.
The framework has two lots: lot 1 covering management information systems (MIS); and lot 2 covering learning services (also known as learning platforms).
Lot 1 is intended to address a lack of competition in the MIS market. This strategy follows a rationale produced by Becta in its September 2010 report, School MIS and value for money[1].
Lot 2 replaces Becta’s 2006 learning services framework, now expired, which was intended to push new forms of education technology into schools.
Both lots aim to improve standards of technical interoperability. The current inadequacy of these standards reduces market competition and restricts opportunities for innovative SMEs and open source providers.
Reasons for cancellation common to both lots
The centralised procurement of education software by local authorities is wasteful, bureaucratic, tends to ignore genuine user requirements, and is inconsistent with the government’s desire to increase school autonomy.
The framework undermines competition, tending to exclude SMEs, which:
- cannot provide a “one-stop-shop” for all the services required for each lot;
- had difficulty in dealing with the complex 500 page tender within tight deadlines;
- are disadvantaged by the tender’s focus on supplier capability rather than product functionality (estimated to cover only 16.5% of the ITT’s requirements[2]);
The ITT was poorly drafted, resulting in 279 requests for clarification, with many answers being delayed or having to be drafted by the questioners themselves.
The provisions for technical interoperability are vague, unclear and unenforceable, so that, like previous initiatives, the framework will fail to deliver better interoperability, while the DfE continues to ignore the need to take effective action to address this problem[3].
Signing contracts under the framework will require companies to absorb extra legal costs and make provisions for draconian penalty clauses, sometimes doubling or tripling the price of products, which may originally be worth less than £5,000.
Most bidders expect that there will be very few call-offs against the framework (there was a total of one call-off on Becta’s two last frameworks); but still feel constrained to bid in order to avoid reputational damage as a result of not being on the short-list.
Specific reasons to cancel lot 1
The lack of competition in the MIS market is not caused (as Becta argued) by the lack of a formal procurement mechanism, but by the control over procurement exercised by local authorities, which the framework seeks to perpetuate. A market study by Atkins, on which the Becta report was based, showed that local authorities suffer a conflict of interest in this regard:
- most local authorities are only able to support a single MIS;
- inefficient technical support of school MIS by local authorities costs more to schools than the MIS licences themselves[4].
Becta’s arguments against procurement by individual schools (now being cited by ex-Becta employees in the DfE) were never credible:
- Becta discounted procurement by schools on the basis that each secondary school would spend 19 man days selecting an MIS[5]—while a straw poll of head teachers by the SSAT found that in practice this took a maximum of 2 man days;
- Becta claimed that aggregated procurement saved 20%[6], without providing any justification for this figure and failing to consider the likely impact of a more competitive market, which would result from disaggregated procurement;
- Becta warned that procurement by schools would “swamp the MIS marketplace”[7], ignoring the trend towards cloud services, specifically targeted at schools, requiring no local hosting and minimal installation and maintenance;
- Becta was itself not impartial, its main argument for its own existence resting on the money it claimed to save by so-called “aggregated procurement”[8].
Specific reasons to cancel lot 2
The learning platforms procured by Becta in 2006 were widely criticised and, according to separate reports by BESA in 2008[9] and Ofsted in 2009[10], little used.
In the year following the 2006 procurement, Becta’s approval rating amongst head teachers, instead of meeting its target of 70%, fell to under 20%[11].
No effective measures have been taken to address the failure of interoperability standards, which were largely responsible for the failures of the previous learning services framework and of the BBC Jam initiative (2003–2007).
Neither Becta nor the DfE have provided any rationale to justify the new learning services procurement, or revealed who is asking for it to happen.
Conclusion
The government should cancel both lots of the IMLS framework, showing support for an open market in education technology, led by schools and not local authorities.
The government should suspend the DfE’s dysfunctional Information Standards Board[12] and consult with suppliers on how to take effective action to improve interoperability standards.
[1] School management information systems and value for money, September 2011 at http://www.school-portal.co.uk/GroupDownloadFile.asp?GroupId=162416&ResourceID=3480985.
[2] As estimated by “schoolpack”, a contributor to the EduGeek forum at http://www.edugeek.net/forums/mis-systems/76154-something-keep-eye-dfe.html. A full analysis of the IMLS requirements will be conducted in a future post on this blog.
[3] A full analysis of the interoperability requirements in the IMLS ITT will follow on this blog.
[4] Atkins estimated the cost of local-authority support at £65 million per annum (ibid, para 1.41, page 8) compared to the total cost of MIS licenses, compared to the cost of MIS licenses at £38–£44 million per annum (ibid., para 1.40, page 7).
[5] Op cit, para 10.22 on page 37.
[6] Ibid, para 10.23 on page 37.
[7] Ibid, para 10.19 on page 37.
[8] See Becta’s response to the announcement of closure, reported for example at http://www.pcpro.co.uk/news/358156/becta-bites-the-dust-in-government-cuts.
[9] “Whilst most head teachers at recent BESA focus groups have confirmed that they have learning platforms, response on their effectiveness is so far disappointing and interoperability difficulties are part of the issue”, http://tinyurl.com/7dabcy2, BESA Policy Commission, 2008, page 62.
[10] “Our survey found evidence of a great deal of development work being carried out by individual institutions and local authorities…However, in most of the provision surveyed, the use of VLEs to enhance learning was not widespread”, Virtual learning environments: an evaluation of their development in a sample of educational settings, Ofsted, January 2009, http://www.ofsted.gov.uk/resources/virtual-learning-environments-evaluation-of-their-development-sample-of-educational-settings.
[11] Chief executive’s report to the board, October 2007, the relevant page of which is copied to http://www.saltis.org/orphans/Becta-annual-report-2007-10.gif. Becta reacted to this collapse in confidence by changing the way that they collected its approval rating data and by launching a major PR campaign the following January.
[12] A full examination of the DfE’s Information Standards Board will be conducted on this blog. In the four years of its existence, it is not responsible for a single standard which has been implemented by anyone. It has produced a suite of specifications which have never been piloted and which have been developed without any consultation with the industry, and even though it is acknowledged that they are unimplementable, are now being imposed as a mandatory requirement in the IMLS framework. The ISB allows no industry representation on its board and no details of meetings, minutes or attendees or agendas of SIGs or working groups are published on their website at http://www.education.gov.uk/escs-isb.
Ed Tech Now 
A main reason that the MLS market is uncompetitive is because SIMS has a the market share and does the job satisfactorily. It would be unwise for my school to move to another system.
The one size fits all functionality of learning platforms meant that everything was done but nothing well. At our school we use Google Apps, WordPress and Edmodo to do the jobs for free that previously cost us thousands and more efficiently.
Thanks for your comment, Gocollide.
Re. MIS – I have no opinion on which is the best system. My point is only that schools should be able to select whatever they want, rather than having a system imposed on them at the behest of the local authority which, right or wrong, is often an interested party.
Re. your school’s use of free, generic web apps, I would be interested in your comments on my argument about education-specific software, in “Aristotle’s saddle-maker”, at https://edtechnow.net/2012/01/25/aristotles-saddle-maker/.
While not wishing to sound disloyal to ex-Becta colleagues I fully agree with all the points that are made in this blog post. I think the facts speak for themselves but I also have a perspective that comes from a history that includes being a business development manager for Capita SIMS (a long time ago), being both successful and unsuccessful in gaining accreditation on Becta’s frameworks and recently supporting suppliers in meeting the requirements of the IMLS framework.
Areas where I would make slightly different arguments are:
Interoperability standards
Rather than a fixation on particular set of interoperability standards more needs to be done to improve the trust and commercial frameworks in which different suppliers operate. The standards would then emerge as tools to fix a problem from supplier led initiatives rather than being dictated by central government. Arguably the vagueness with the IMLS specifications allow for this, however, this driver will only work if the framework provides an appropriate procurement route for the majority of schools. (As Crispin points out such frameworks do not have a good record of call off and so will have little impact on the way in which suppliers operate…)
School involvement in procurement
IMLS are a set of core functions that underpin everything about how an educational institution is managed and how good teaching makes an impact on children’s learning. I would expect a school to spend at least 19 man days in making sure that the right systems and services were being purchased (particularly if the whole staff are engaged in the process as part of change management). If schools can make such a decision based on just 2 man days they are a) using the wrong approach, b) unlikely to embed and get value for money from their purchases. Please note that I’m not making an argument for the IMLS framework as schools using the framework may spend a disproportionate amount of time reading the general contractual documentation (the original contract was over 200 pages long….). This time that could be more usefully spent evaluating suppliers solutions against their own specific requirements.
Dear Simon, It is good to hear from you – thanks very much for your comment.
I entirely agree with you that standards need to “emerge as tools to fix a problem from supplier led initiatives rather than being dictated by central government”. But this begs the question, “why is this not happening?”.
My answer is that commercial companies will only invest in interoperability when it makes a difference to their bottom line. And it won’t, so long as sales are driven by aggregated procurement (whether this means products being specified at the level of the LA or central government). In these circumstances, the government-written specifications become the only standards that matter.
I think we would both say that we are living in a no-mans land at the moment, where the reality is a depressed but increasingly decentralised market, while the DfE continues Becta’s tradition of central procurements, pretending that it has an effect just as Beafeaters pretend to defend the Tower of London. In a situation in which the IMLS framework is irrelevant, you may say that I am making too much of a fuss. The problem is that even without call-offs, the framework continues to affect reputations and thereby delays what is a painful transition from a centrally planned to a fully commercial market. It sows confusion and disinformation where people are looking to the DfE for clarity and leadership, and it also hides, at least from the perspective of officials in Whitehall, their own policy vacuum in this area.
I do not agree that the vagueness of the IMLS specs encourages innovation (though this is precisely the argument that was made to us by DfE officials). Some people are on the list and others are not, so there is nothing vague about the outcome of the selection procedure. That is what matters, not the specificatons which mean very little and are read by almost no-one. While I do not think that granting particular companies such market privileges is ever likely to be justified – defenders of the practice need at least point to some kind of pay-back – a set of conditions which the privileged suppliers are required to meet. And it is in this asymmetric way that the vagueness comes in. As far as interoperability is concerned, there are virtually no enforcable conditions at all.
Thanks for highlighting the importance of interoperability. I agree that we should not have “a fixation on particular set of interoperability standards”, though I am not entirely sure I understand what you mean by “trust and commercial frameworks”. If this translates as good market information, then I would agree. As soon as I can get round to it, I shall be posting more on this blog about interoperability standards, which I think are critical to the whole application of education technology.
On your second point, “School involvement in procurement”, I think you raise an interesting point regarding the relationship between *choosing* a system and then being prepared to *use* it productively. And I think we are both agreeing that, if you are to spread a sense of ownership of the new system across the staff, aggregated procurement by the Local Authority is not helpful.
But I disagree with you about the 19 man-days. Not because I disagree that this is what would be required to specify the system in detail and to consult properly with all likely users of the system, but because I do not believe that such detailed specification is either necessary or desirable for the selection of an appropriate product.
We have two procurement models.
A.1. A large number of authorities or organisations each go through a lengthy deliberation to specify a product, and then go to the market and see if an existing product meets these specifications, or more likely, to choose a supplier to create an individualised product. This is the model the IMLS framework assumes.
A.2. Suppliers each come up with their own standard products which provide what they decide are a desirable set of features. These products start to attract their own reputation through the grapevine and product reviews. Schools then buy those products – not primarily on the basis of a detailed set of specifications – but rather on the basis of market reputation. They most likely only discover the full range of functionality (or begin to percieve its need) when they start to use the products.
The first model makes sense for large, one off projects – when the MOD decides to build a new jet fighter, for example (though even in these cases, the success rate of government-written specifications is extremely low). But for multiple procurements addressing very similar needs, the first model is wildly inefficient.
B.1. There is the duplicated effort of at least hundreds of detailed specification exercises (this disadvantage may indeed be mitigated by a central framework – but would only be fully avoided by a single, national procurement). Does School A really have such a radically different requirement for its MIS to School B?
B.2. The specifications are drawn up by people who are not generally competent to do the job. It would be like a group of users saying to Steve Jobs, “I will only buy your new tablet if it meets these specifications which we have drawn up”.
B.3. The people drawing up the specifications (particularly at central, bureaucratic levels) are not accountable for the failure of their specifications, but carry on doing the same thing, year after year, even though everyone knows what a fiasco it was last time. In an efficient market, business that develop poorly specified products quickly go under.
B.4. People who develop really innovative products (and innovation is nearly always the product of unusual and original thinkers) have no chance of marketing those products, because everyone will be buying on the basis of bureaucratic specifications produced by unimaginative non-specialists, parroting the current orthodoxy.
That is why, in commercial markets, model A.2. is used. You buy an iPad because you hear on the grapevine that it is cool, and you discover what it does after you have bought it. This may sound haphazard from the point of view of a central planner – but it isn’t. A properly regulated market, working as a system, is the most efficient way we have of picking winners. It encourages the introduction and testing of new ideas, even in a highly conservative institutional environment.
That is why, in my view, it is anyone who spends *more* than two days choosing a new MIS who is using the wrong approach – because they are taking on themselves a responsibility for which they are not qualified and which they are sure to screw up.
As for your point about buy-in, I have two points.
As far as a school-wide system like the MIS goes, I don’t think you need buy-in from the Maths teacher. You need buy-in from the SMT – that is all – and an SMT that is worth its salt will ensure that the system is used (if it is fit for purpose, which, if it has been specified by the school, the LA or the DfE, it almost certainly won’t be).
As far as Maths-specific tools are concerned, you should get buy-in from the Maths teacher not by *consulting* the Maths Department but by letting them buy the thing themselves. The Maths software needs to plug-in to the MIS automatically, which is where interoperability becomes essential. But the IMLS framework not only aggregates procurements, it also aggregates services into unwieldy one-stop-shops, increasing risk and bureaucracy, increasing the threshold of entry to the market for SMEs, decreasing flexibility and buy-in from the front line.
I guess you will say that teachers are generally poorly informed customers – and I would agree. But I think the answer to this problem is neither to make teachers’ decisions for them, nor to give them a blank sheet of paper and get them to write down in legally watertight detail what they really really want and need. Much better to give them an authoritative catalogue, the (currently non-existent) “What Hi-Fi?” magazine or “TripAdvisor” of education technology, and let them make their own choices. If they make some mistakes, then let them be small ones and ones that they will learn from. Let entrepreneurs do the specifying and let teachers do the buying.