Learning outcomes assessment in primary schools

Learning outcome assessment has been the holy grail of primary school education. How do we assess, on an objective measure, the learning outcomes of each student? As a corollary, how do we assess the performance of teachers?

New York City has aggressively pioneered standardized testing to assess student learning outcomes for nearly a decade now. Such tests have been the centerpiece of Mayor Mike Bloomberg's school reform efforts. Now, in an attempt to directly assess teacher performance, it recently announced plans to develop up to 16 additional standardized tests to cover science, math, social studies and English for 3rd through 12th grades.

This is part of a new state law, wherein all school districts in New York State must evaluate teachers on a scale from "ineffective" to "highly effective", with potential firing for those rated ineffective for two years in a row. These tests would be in addition to the standardized state English, math and Regents tests students in the City's 1700 odd schools already take.

The new law, part of the nationwide "Race to the Top" initiative, mandates that 40% of a teacher's grade will be based on standardized tests or other rigorous, comparable measures of student performance. Half of that should be based on state tests, and half on measures selected by local districts (like that being proposed by New York City). The remaining 60% is to be based on more subjective measures, including principal observations.

However, as the Times report writes, "the prospect of more tests, particularly ones that will have a direct influence on teachers, is causing dismay among those who believe that students already spend too much time preparing for exams and not enough on the broader goals of education, like social and emotional development".

Nobody disputes the need for a system that assesses the outcomes of education, both at the student-level and cumulatively at school-level. However, any evaluation system has to be fair, rigorous, trustworthy, and reasonably reflective of the efforts put in by teachers and students to be acceptable to all stakeholders.

Excessively rules-based, formulaic, and centralized testing models are vulnerable to being gamed by stakeholders. Teachers, school administrators, students, and parents, all develop an incentive to subvert the evaluation mechanism. This trend gets amplified if the results are used for high-stakes personnel related decisions.

In countries like India, where there is no objective yardstick of comparing learning outcomes across different primary schools and where penalizing, leave alone firing, teachers is impossible, learning outcomes assessment may have three important uses.

1. It can be a very useful instrument to spot class-wise collective learning deficiencies among children and administer the required training for the teachers. It is commonplace to find that a major share of children in certain schools or classes have not understood certain concepts (say, the concept of carry forward addition) and it may be useful to spot such weaknesses and provide focused training on this concept to the particular teacher.

2. Class room instruction currently involves teachers teaching a defined syllabus on a single-track mode to all the students in a particular class. This approach fails to appreciate the widespread differences in the pre-existing learning levels among children. For example, since a vast majority of children in Class V do not have the learning competency of even Class III, teaching them Class V Math straight away without bringing them upto speed with Class IV learning levels is a recipe of widening the already existing learning gaps.

In the circumstances, it is critical that the children be divided into groups based on their learning levels, and classroom instruction structured to close the learning gap through some form of remedial education. Assessment scores can be a useful instrument in this.

3. Currently we have no system to track the learning trajectories of students across their schooling career. How much improvement has the student achieved in a year with respect to the benchmark for that particular class? What has been the improvement in learning levels in Math from Class III to Class V? What is the learning level of a particular student vis-a-vis his best friend?

This information is necessary to not only improve the quality of monitoring and tracking, but also to generate demand-side pressures among both parents and the child itself. If properly presented, such information can be used to generate intrinsic motivation among the child itself (subtle comparison of marks of all children in a friendship group) and among their parents. Currently, till the tenth class, parents have no reliable and objective mechanism to assess the learning levels of their child.

My argument is that, whatever its flaws, an objective and standardized mechanism to measure learning outcomes is a pre-requisite for making any meaningful improvements in our primary schooling system. Once in place, we can then worry about how to keep such testing mechanism dynamic enough to prevent it being gamed by stakeholders.

Update 1 (8/7/2011)

An investigation into Atlanta’s public school system has uncovered evidence that teachers and principals have been secretly erasing and correcting answers on students’ tests for as long as a decade. A state investigation found that 178 educators at 44 of the district’s 56 schools engaged in cheating. The report is a huge blow to an urban school district that for years was hailed as one of the country’s most successful due to increased student performance.

This is the second high profile teacher cheating example in US, following a USA Today investigation found evidence of teacher cheating among some of Washington DC's highest-performing public schools. See this Freakonomics post.