Principles of the evaluation

Principles of the evaluation

The evaluation should: • build trust with the team(s)

• remain independent of the team(s) and other stakeholders • operate within agreed ethical guidelines.

Evaluators are in a position of power in relation to the programme director and team(s). Their access to information and the quality of the information they collect is dependent upon establishing trust. They need to form good working relationships with the director and team(s) so that they have access to the informal, as well as the formal, working processes of the programme. Evaluators need to understand problems. In my view their privileged position places them under an obligation to provide advice and assistance where they can, within the limits of their remit. The evaluators’ active engagement with the purposes and working practices of the team make this approach somewhat akin to the kind of ‘participatory evaluation’ recommended by Cousins and Earl (1995) in which ‘trained evaluation personnel (or research specialists) and practice-based decision makers (work) in partnership’ (p. 8). However, I am not here suggesting that the project participants take on any of the function of the evaluators, because a large part of the evaluators’ usefulness results from their independence. They need to maintain a distance from the programme director and team(s). Striking the right balance between independence and informality is important. In my experience you need both, with some formal procedures to distinguish between different purposes for different occasions. Once trust and informal relations have been established it can be helpful to formalise some of the data-collection events and meetings, perhaps by linking them to formal written feedback.

140 Research methods for ICT in education The evaluation must operate within clear ethical guidelines which give all

concerned control over access to information and its clearance for publication. A good starting point is the codes of practice that set ethical standards for evaluation practice that have been produced by the American Evaluation Association and the British Educational Research Association (AEA 2004; BERA 2004). However, each evaluation needs to develop its own code of practice customised to the institutional and political context of the programme. It is best to negotiate these guidelines with the programme director and team(s) early in the life of the project and produce an agreed written statement. The team(s) needs to be assured that any written reports will be fair and accurate, that they will be asked to comment on draft reports and play a part in improving their fairness and accuracy, and that where they disagree with an interpretation made by the evaluators they will have the right to have their own alternative views included in the report. An example of guidelines of this kind, developed for an action research project, are given in Somekh (1997). The guidelines have at least three purposes:

• to ensure that the evaluation is able to report fully and fairly on the work of the

programme, which is important to ensure accountability for public spending; • to protect the rights of the programme director and team(s) and place reasonable limits on the power of the evaluators, which is important to ensure social justice;

• to ensure that the evaluators have access to full information, without the team(s) feeling the need to conceal anything potentially damaging, which is important to ensure quality.