1.1 The need for effective schools
1.2 Effectiveness and efficiency
1.3 Quality in education
1.4 Methodology of school effectiveness research
According to Haddad et al (1990, p3), empirical evidence suggests
that in both developed and developing countries, educational investment has been one of the most important factors contributing to economic growth; that expenditures on education contribute positively to labour productivity; that the economic payoff to spending on education - from both a private and public standpoint - is high, in absolute terms and compared to other investments; and that increased education of parents - especially mothers - has an important impact on child health and reduced fertility at all levels of economic development.
They argue that improving access to and the quality of basic education is a priority in almost every low-income and middle-income country, and point out that, in many countries, particular attention has to be paid to girls' education.
Quality is important. Referring to work by Creemers, Peters & Reynolds (1989) and by Raudenbush & Willms (1991), Lockheed & Verspoor (1991, p19) state that
recent research on the effect of schools on learning provides clear evidence that variations in the characteristics of schools are associated with variations in student outcomes.
They argue that to increase the pace of economic and social development in developing countries, schools must teach most school-age children the essential skills targeted by the primary curriculum, which include literacy, numeracy, communications and problem-solving skills. One of the aims of this report is to review research evidence as to how schools can best do this, and hence to indicate possible priorities for education aid. It is clear that developing countries must concentrate their resources on those improvements that are known to enhance student learning (Lockheed & Verspoor, 1991).
Haddad et al (1990, p50) quote several sources to confirm that there are some consistent general findings from the research.
Variation in school inputs, such as teacher experience, teacher motivation, the presence of textbooks, homework, and time spent in school during the year do contribute to varying pupil achievement, even when family background differences are accounted for.
Bacchus (1991) identifies three major thrusts in efforts to improve the quality of basic education:
* raising the academic performance of students in the various subjects offered in schools with the currently available resources. Such efforts are often referred to as attempting to improve the 'internal efficiency' of the schools.* providing children with education that is most likely to help them improve the quality of their lives when they become adults.
.... This approach is often referred to as attempting to raise the 'external efficiency' or the 'effectiveness' of schools.* increasing the rate of school enrollment by providing more school places and reducing the inequalities which currently exist between the sexes and between different regions in a country.....
(Bacchus, 1991, pp 5-6)
There is a danger of confusion when defining the terms 'efficiency' and 'effectiveness' and their descriptors 'internal' and 'external'. According to Lockheed & Hanushek (1988, p22),
Efficiency refers to a ratio between inputs and outputs. A more efficient system obtains more output for a given set of resource inputs, or achieves comparable levels of output for fewer inputs, other things equal..... The output of education refers to that portion of student growth or development that can reasonably be attributed to specific educational experiences.
Inputs are conceived in broad terms, to include the complex interactions of students and teachers, as well as text-books, teachers' salaries, and so forth. Lockheed & Hanushek restrict the term 'efficiency' to monetary inputs and use 'effectiveness' for non-monetary inputs. Outputs expressed in non-monetary terms (e.g. learning) are 'internal' and outputs expressed in monetary terms (e.g. earnings) are 'external'. Hence external efficiency is equated with a cost-benefit ratio. Unfortunately, the classification is marred by equating internal effectiveness with 'technical efficiency', and internal efficiency with 'cost-effectiveness'!
There are obvious difficulties with these definitions. For example, it is arguable as to exactly what should count as inputs, and how inputs might best be measured, even if all inputs are measurable. Similar difficulties apply to outputs. However, whatever precise definitions are adopted, it is clear that there are various policy options for attempting to improve the output-input ratios, for example
1) achieving existing output levels for cheaper or fewer inputs (e.g. by selecting low-cost building alternatives)2) increasing outputs for the same inputs (e.g. by reducing absenteeism)
3) reallocating existing resources to new inputs which increase outputs (e.g. perhaps by spending less on teacher training and more on textbooks).
It must be kept in mind that efficiency is not the only criterion for policy-makers, who must take account of a range of social, political, economic and educational considerations in arriving at judgements as to priorities for objectives and methods of achieving them. Nevertheless as Lockheed & Hanushek (1988, p21) point out
When there are limited resources - as there always are - those resources should be used to promote society's objectives as fully as possible.
They identify three important constraints on improving internal efficiency: (a) inadequate knowledge about internal effectiveness, (b) inadequate knowledge about costs in inputs, and (c) difficulty in obtaining appropriate information. Evidence on which to base decision-making is limited by these constraints.
It is noteworthy that, while the World Bank has invested over $10 billion in education projects, research necessary to answer questions about the internal efficiency of education has been conducted in fewer than half a dozen instances.(Lockheed & Hanushek, 1988 pp 27-28)
In view of the paucity of research evidence, there is a strong case for both multilateral and bilateral agencies to support more research studies related to the effectiveness and efficiency of alternative educational inputs. Lewin with Ross (1992) recommends enhancement of the ODA educational research programme support, including the generation of new knowledge of the effectiveness of intervention strategies.
The concept of quality in education is not easy to define. Hawes & Stephens (1990) believe that quality is characterised by three inter-related and inter-dependent strands: (i) efficiency in meeting its goals; (ii) relevance to human and environmental conditions and needs; (iii) "something more", that is the exploration of new ideas, the pursuit of excellence and the encouragement of creativity. If this is accepted, there might be debate about the relative importance of each strand, about what the goals should be, and about what is meant by "relevance". In particular one major goal might be equity, and it can be argued that equity considerations should be part of a broad view of effective schooling.
Urwick & Junaidu (1991, pp l9-20) distinguish two contrasting orientations towards quality, which they describe as "technical efficiency" and "pedagogic".
The 'technical efficiency' orientation focuses on the provision of basic school inputs (especially teachers, educational materials and learning time), their effect on academic achievement and the consequent priorities for investment. This orientation is characterised by positivist assumptions and by attempts to measure production functions through large-scale surveys.... The 'pedagogic' orientation towards the quality of education does not give much emphasis either to physical inputs or to their 'effects', but rather sees teaching skills, patterns of school organization and curricular content as the essential components of 'quality.
In practice, there is a danger of over-emphasis on efficiency and on the use of quantitative indicators. Tipple (1990) points out the tendency to restrict to what can be measured, and argues that 'the measurable thus assumes unwarranted importance'. Wilcox (1990, p39) warns that
Performance indicators.... will seldom if ever tell an unambiguous tale. Quality of educational experience will always be an elusive entity which evades precise delineation.
Wilcox suggests a range of possible indicators, including client satisfaction (using questionnaires) and qualitative indicators based on observation by advisers or inspectors in addition to achievement scores. The notion of 'value added' is important in making comparisons of assessment test results. In other words, the effectiveness of a school depends not just on the final results of the students, but on what improvements in performance have been achieved by those students while at the school. Indicators should take account of the social and economic contexts of the schools.
Vulliamy (1987, pp 220-221) takes the view that in discussing effectiveness we should always ask "effectiveness for what and for whom?", and argues that
The notion of effectiveness presupposes a consensus on the desired outcomes of schooling, which tends to disembody schools from their wider social, political and economic context...... In the Papua New Guinean context, for example, it may be that a school with relatively poor examination results is providing a relatively better preparation than other schools for those of its students who are likely to return home to their villages.
It is true that most school effectiveness research uses examination or other test results as the indicator of quality. Nevertheless, according to Singh (1991, p70),
The accumulation of research findings and evaluations of development projects have brought together a wealth of knowledge about the essentials for quality schooling.
Singh quotes Throsby & Gannicott (1990) that the following statements encapsulate the state of thinking on quality in education:
* trained teachers make a difference* class size is not relevant
* the provision of instructional materials is one of the most cost-effective ways of raising the quality of education
* education is most effective if initial instruction uses the mother tongue
* lavish buildings and equipment will not raise quality
* curriculum reform will not necessarily raise educational quality
* examinations are a useful way of monitoring school quality
* healthy well-fed children learn better
* amount of learning time affects educational outcomes
* quality depends on good decentralised education management
These points are discussed in more detail in later sections.
Various aspects of research methodology need to be taken into account when evaluating the evidence to be presented in this summary report, and when applying the evidence in policy formulation. Firstly, a narrow range of output variables is used in much of the research, with test and examination results featuring prominently. It could be argued that these proximate outputs do not necessarily give an accurate prediction of more important remote outputs, such as whether school leavers become good citizens, or make a positive contribution to national development. It might also be argued that more attention should be paid to process variables such as the quality of school management and that an input-output model is too simplistic (Willms, 1992).
If these arguments are accepted, it might be felt that there is a need for more qualitative research. Some examples of predominantly qualitative studies are quoted in this report (e.g. Vulliamy, 1987; Levin & Lockheed, 1991; Urwick & Junaidu, 1991), but the majority of the cited literature is based on quantitative work.
Not everyone would feel that the scientific paradigm (within which much quantitative work is located) is always appropriate. There are methodological difficulties, although it is claimed that the new multilevel models overcome many of them (see Riddell, 1989 and Heyneman, 1989 for a discussion). There are also difficulties of interpretation: for example, establishing a statistically significant correlation between two variables does not establish causality. Another point is that some studies (especially transnational studies) have problems of aggregation.
A further point made by Keith Lewin (personal communication) is that
very often the various studies treat factors as separate entities where variance is unproblematic. It makes a great deal of difference how school size, for example, varies from place to place. Treating things like the existence of textbooks or science laboratories dichotomously often loses important dimensions of school quality related to how they are distributed. A further problem related to this at the policy level is that, whatever the difficulties of disaggregation, it will generally be the case that selected inputs on the margin have far more effect than flat rate increases.
None of these caveats mean that existing research evidence should be rejected; it is the best evidence we have. However, it is important that it should not be applied blindly or prescriptively, but rather used to assist decision making, with due attention to context. On the other hand, it can be said with confidence that policy-makers who ignore the research evidence take a considerable, and unjustifiable, risk.
