Action Research in the Evaluation of a National Programme 1
7 Action Research in the Evaluation of a National Programme 1
What kind of contribution can teachers’ action research make to the evalu- ation of a large-scale national innovative programme? Is this one way of giving the teaching profession a voice in the policy-making process? What are the methodological implications for ‘insiders’ carrying out action research in ‘a blame culture’? This chapter is about the evaluation of the ICT Test Bed project, starting in April 2003 and projected to end in December 2006, two years after the time of writing this chapter. The ICT Test Bed Evaluation, funded by the UK government’s Department for Education and Skills and managed by the British Educational Communications and Technologies Agency (Becta), is a joint project of Manchester Metropolitan University and Nottingham Trent Universities. Writing about work in progress, rather than a completed project as in the previous chapters, means that I can invite readers to share my thinking about methodological issues that are currently engaging me. The chapter will end with both reflections back on the first two years of the evaluation and forward to possibilities yet to come. At a time when evaluation contracts are rarely for more than two years, and often for a considerably shorter period, the ICT Test Bed evaluation, covers a period of almost four years. This offers a unique opportunity because, for the first time in my experience, the evaluation will have two years to focus on the impact of the investment after the installation of infrastructure and equipment – parts of which have taken almost two years – has been fully completed (Somekh et al. 2005a and b).
Reading the invitation to tender (ITT) for the ICT Test Bed Evaluation in November 2002 I was amazed to find that, in addition to measurement of gains in pupils’ test scores, action research was a required component of the research design:
1 The action research work reported in this chapter has been carried out jointly with Andy Convery, Cathy Lewin and Diane Mavers. I would like to thank them for their enormous
contribution to the ideas contained here. I would also like to thank Tim Rudd and Di Matthews-Levine of Becta for their valuable advice and support.
ACTION RESEARCH
The main evaluation will also identify and demonstrate, through action research, how the appropriate deployment of ICT may impact upon standards, improve added value, refocus teacher workloads and enable greater internal and external collaboration across the Test Bed institutions. (emphasis added)
Seeing action research specified in a government ITT, as far as I knew for the first time since I had completed work on the PALM project in 1990, I felt a rush of adrenalin in my blood. This was a contract that I very much wanted to win.
I had been inducted into evaluation of government policies for IT in education through working with Barry MacDonald at CARE/UEA in 1987 on a retrospective evaluation of the Department for Trade and Industry’s Micros in Schools initiative, 1981–84 (MacDonald et al. 1988). Evaluation was always very highly regarded at CARE because of its focus on critical engagement with policy and the operation of power (Wildavsky 1993). Researchers at CARE believed that education, as an essential liberating asset of a civil society, should not be left in the hands of politicians and bureau- crats without scrutiny. Evaluation, in the tradition of MacDonald (1974) and House (1974; 1993), who was a regular visitor from the USA, focused on critical scrutiny of both policy formation and bureaucratic control of policy implementation. Its purpose was to hold government and its agents to account for the spending of public money. Evaluation itself should operate democratically to serve the needs of all stakeholders, rather than merely serving the bureaucratic needs of government administrators or taking upon itself autocratic power. My interest in the special problems of ICT initiatives grew during the course of conducting evaluations under gov- ernment contract both at CARE and later at SCRE in Scotland. I became increasingly interested in means of maximizing the educative function of evaluation studies, and in the necessity for evaluators to engage in what I have characterized as supportive evaluation in order to ensure excellence in innovative ICT programmes (Somekh 2001). Immediately on my arrival at Manchester Metropolitan University, at the end of 1999, I had established the Centre for ICT, Pedagogy and Learning with my colleague, Diane Mavers, and we had already developed a track record for evaluation work. The contract for the ICT Test Bed Evaluation was, therefore, doubly attrac- tive to me, as an evaluation of a major national ICT initiative and one that offered the opportunity to develop a research design incorporating action research.
ACTION RESEARCH IN THE EVALUATION OF A NATIONAL PROGRAMME