Monday, January 5, 2009

Unit 5 Using Monitoring and Evaluation findings

Introduction
As you will now appreciate from your study of Units 1 - 4, evaluation is a systematic, planned enquiry undertaken in order to enable senior leadership teams to make judgements concerning the quality of learning and teaching, school policies and their implementation, projects or programmes of study. Monitoring and evaluation can be a lot of effort but, when done efficiently, vigorously and with the intention of improving educational provision, it is worthwhile for the information gained about the school. Thus, it can be fed back into the system to improve the effectiveness of the institution.

But, quite often, the findings of evaluation are not implemented. In this unit you will be asked to consider how evaluation findings can be disseminated and to examine how we can ensure that the evaluation findings are used by all concerned to enhance school effectiveness.

Individual study time: 2 hours

Learning outcomes
By the end of this unit you should be able to:

State the importance of reporting the results of monitoring and evaluation programmes to those who need to know
Integrate the analysis of evaluation findings into the school decision making process
Relate the quality of evaluation findings to the level of effectiveness of the school leadership.

The importance of evaluation findings
The host of educational problems besetting schools today, such as pupil wastage in terms of those not completing the course or underachieving and the low standard of education found in some schools, all point to the need to overhaul our educational systems and programmes. But before this can be done, we need reliable and objective data about their current status. Through regular monitoring, evaluation and reporting we will know much better where we are and thus be able to decide what changes are needed to bring about improvement.

As we have seen in the previous units, monitoring is the regular collection of information about ongoing projects and programmes within the school system with a view to determining the nature and level of their performance. A report from a monitoring team should be an objective account of progress in carrying out plans, which provide a baseline against which to judge the impact of inputs onto the system under study. School self evaluation should be done mainly by school heads, SLTs and middle managers. External monitoring and evaluation in Guyana takes the form of visits by the officers of the Regional Departments of Education and the MERD Unit from Central Ministry.

Evaluation concerns judgements made for the purpose of improvement or accountability. It is also a formal process which should be formative or summative in nature and is always designed for a particular educational purpose.

Activity 5.1
In your experience, are reports from internal monitoring and supervisory visits from the Regional Department of Education and the MERD Unit, made widely available to Headteachers, schools and parents?

What is your view on the dissemination of this information? Give your reasons.

Comments
There is now a requirement that the report of the MERD Unit about the region is made public. Indeed it is sent to the national press for publication. Whatever may have been the practice in the past, it is essential that all who have the power to improve education – REDOs, DEOs, Heads and teachers are kept fully informed of the judgements that are made about their work. Otherwise, how can they know what they need to do to improve?

The steps which need to be taken to rectify the situation include:

Writing reports which can be read (that is, they are brief and to the point and use intelligible language)
Improving the means of production and reproduction.
Recognising that knowledge and power are for sharing
Ensuring accuracy and reliability in our reports, through setting and demanding high standards.

Evaluation reports
In order to appreciate how to improve the dissemination and use of evaluation findings, it will be helpful to recall the steps which lead up to the planning and implementation of an evaluation project and the preparation of the report.

We have seen in Unit 3 that before embarking on evaluation there are certain points that we need to clarify, including:

For what purpose is an evaluation required?
Who will organise the evaluation, who will participate and in what ways?
What are the possible modes of evaluation?
What are the appropriate tools of evaluation?
How will the data be recorded and analysed?
Who will use the information?
What are the best ways of giving the information to them?
The information gathered should be seen as relevant to the purpose of the evaluation. The credibility of an evaluation process and its findings can be established in relation to the following:

It has to pass scrutiny amongst all those concerned with the overall quality of education.
It needs to be seen to be credible in the school, for example, in respect of what is being evaluated.
It has to satisfy people that any conclusions drawn derive from good quality information and are valid and reliable.
It must lead to development and ultimately to raised standards.

Activity 5.2
Identify the qualities of a good evaluation report. Use your school experience and knowledge of reports by the region or the MOE to answer this question.

Comments
You will probably have included in your list, qualities such as: systematic, comprehensive, relevant, usable, valid, reliable, quantifiable, based on evidence, objective.

It must also be readable!

If the purpose of an evaluation is absolutely clear, it is more likely that the correct information will be gathered to enable conclusions to be drawn and recommendations made as a basis for decision-making. It is therefore very important that, after an evaluation is done, one or more meetings between the evaluation team and the rest of the staff are held, during which the findings can be discussed before the release of the evaluation report. Results could also be disseminated, for example, to parents, through other means, such as a newsletter or a briefing at one of the PTA meetings.

Evaluation findings by external agents, such as the MERD Unit or the Region, can help to justify the huge amounts of money spent on education every year, as well as to clarify the progress of major innovations in education such as the introduction of a new curriculum. This highlights how important it is for school heads to be able to analyse evaluation findings and decide whether, for example, the advantages claimed for a particular innovation which formed the focus of an evaluation, are valid enough for them to continue to use it or perhaps to adapt it more precisely to the school’s needs.

The relationship between evaluation and effective leadership
We can illustrate the relationship between evaluation and the use of evaluation data or findings for effective school leadership by considering a number of examples.

Pupils’ enrolment
Often school enrolments fluctuate from year to year. Many factors may account for this. It might be difficult for a head to explain such a phenomenon, let alone suggest solutions, unless some evaluation is conducted and the findings disseminated. Such an evaluation might enable factors like school performance in public tests or examinations, distance of the school from the village, safety and security of pupils, availability and quality of transport, movement or transfer of parents to be taken into account in an analysis of the situation. There are also the issues of early leavers and fluctuations and movements. A school may wish to evaluate the influence of some of these factors to try to rectify the situation. On the other hand, the MOE might decide to evaluate the situation in a sample of primary schools, the results of which may provide useful conclusions and solutions which may be applied to all schools.

Activity 5.3
Using your school experiences, show in a table the enrolment of pupils for the last two or three years by class. Indicate the following as a percentage of the total:

§ early leavers
§ casual intake or repeaters
§ graduation rate
§ test or examination success at a level commensurate with the age of the pupils

Discuss this with a trainee in another school.

1) How do these figures compare with those for other schools in your locality?
2) To what extent would you say the issues you have discovered are common or unique?
3) Where there is a problem, what would you choose to do about it?

Comments
You may have difficulty in completing this activity if your school records and data are not well kept or up-to-date. You may not have been aware of the importance of such records nor how to keep them correctly maintained, in which case, could it be that you and your staff should try to obtain more training in record-keeping? You may also feel that training is required in the use of evaluation instruments and in the interpretation of data and in the use of evaluation findings.

Leadership effectiveness
Evaluation can be used to assess leadership effectiveness in a school. We first looked at this in Unit 4. We will say a little more about it now. Firstly, you may need to consider what leadership effectiveness is, and the extent to which it is determined by:

Personal characteristics of the leader
The context of the school
Personal characteristics of the staff

Next you would need to develop evaluation criteria in order to assess your own leadership effectiveness together with that of your other senior staff. The chart below suggests ten criteria for judging leadership and you might like to develop this by adding more evaluation criteria. If you were to undertake such a self-evaluation exercise, how might such findings be useful to you as a school head in improving your leadership practice? You may wish to answer this for yourself and get others to comment on your own leadership skills as stated below. Remember, in all cases, there must be evidence.

Assessing leadership effectiveness

INDICATORS LOW HIGH

1) Clear vision for the school
2) Ability to delegate
3) Thinks strategically
4) Has a drive for improvement
5) Holds people accountable
6) Uses initiative
7) Develops potential
8) Creates teams
9) Shows respect for others
10) Challenges and supports


We will complete this unit and module by being quite controversial. It could be argued that the effectiveness of a leader should be judged solely on the effectiveness of the school. In other words, if the school performs well, the leaders must also be doing a good job. Conversely, if school performance is below par, the leaders are less than effective.

Activity 5.4
Read the above statement a second time and give arguments for and against the propositions made. Support your answer with evidence and examples.

Comments
You may agree or disagree with this statement but it is never that simple. Some schools, although rarely, have proved to be effective despite their leaders, although they rely heavily on the ability and experience of the teachers as well as the compliance of the children. On the other hand, a school head may be very effective in that he / she is implementing strategies for school improvement for which it will take time for the school to see the effects. This is why monitoring is so important so that we can see the changes which result from such plans.

Summary
Your studies of this unit and of others in this module, will, we hope, have convinced you of the importance of monitoring and evaluating and of using the findings to bring about school improvement. We have provided below just a few examples. There are, of course, many areas where an evaluation exercise is likely to produce findings which could inform the school decision-making process and contribute towards improved school effectiveness, for example in:

teaching and learning
school-community relations
financial management
effectiveness of the curriculum
social development of the children
behaviour management
resource management
leadership skills

You should have noted that monitoring school effectiveness involves the regular review of all the services of your school and especially, its prime function – teaching and learning. Through the collection and analysis of relevant information and the setting of appropriate expectations, criteria and standards, you may draw conclusions about the extent to which the mission, objectives and targets of your school are being achieved.

The diagram below relates monitoring and evaluation to decision-making, which is the essence of any effective leadership activity.

Monitoring, evaluating and decision-making

Monitoring
- A systematic and regular overall collection of of qualitative and quantitative information about performance
Decision-making
The process of using management information to inform collective plans and make decisions about future development
Evaluating
The targeting of questions, carrying out and specific activities and drawing conclusions which lead to development
Review of Policy
The practice of appraising the quality of policy decisions which is carried out by further monitoring and evaluation
Reviewing the success of developments

Further monitoring which leads to the continuation or adjustment of the programme

As you can see, the whole process is a cyclical one and continuous. A head can never sit back and say “We have finished our work” but must constantly respond to the changes and challenges which he / she faces. But, of course, it is also a good idea to pat oneself on the back from time to time to recognise a job well done.

No comments:

Post a Comment

NCERDLEADERS will be happy to receive any comments or suggestions about this website. It is also an effective way of communicating with the Education Management Specialist. However, any comments you make will be seen by others. We have restricted comments only to people with Google Accounts because of the amount of inappropriate.spam which spoils the site. All comments will be moderated and spam will not be published. If you prefer to send a private message, please contact us - see sidebar.