Tuesday, June 28, 2011

Learning outcomes assessment in primary schools

Learning outcome assessment has been the holy grail of primary school education. How do we assess, on an objective measure, the learning outcomes of each student? As a corollary, how do we assess the performance of teachers?

New York City has aggressively pioneered standardized testing to assess student learning outcomes for nearly a decade now. Such tests have been the centerpiece of Mayor Mike Bloomberg's school reform efforts. Now, in an attempt to directly assess teacher performance, it recently announced plans to develop up to 16 additional standardized tests to cover science, math, social studies and English for 3rd through 12th grades.

This is part of a new state law, wherein all school districts in New York State must evaluate teachers on a scale from "ineffective" to "highly effective", with potential firing for those rated ineffective for two years in a row. These tests would be in addition to the standardized state English, math and Regents tests students in the City's 1700 odd schools already take.

The new law, part of the nationwide "Race to the Top" initiative, mandates that 40% of a teacher's grade will be based on standardized tests or other rigorous, comparable measures of student performance. Half of that should be based on state tests, and half on measures selected by local districts (like that being proposed by New York City). The remaining 60% is to be based on more subjective measures, including principal observations.

However, as the Times report writes, "the prospect of more tests, particularly ones that will have a direct influence on teachers, is causing dismay among those who believe that students already spend too much time preparing for exams and not enough on the broader goals of education, like social and emotional development".

Nobody disputes the need for a system that assesses the outcomes of education, both at the student-level and cumulatively at school-level. However, any evaluation system has to be fair, rigorous, trustworthy, and reasonably reflective of the efforts put in by teachers and students to be acceptable to all stakeholders.

Excessively rules-based, formulaic, and centralized testing models are vulnerable to being gamed by stakeholders. Teachers, school administrators, students, and parents, all develop an incentive to subvert the evaluation mechanism. This trend gets amplified if the results are used for high-stakes personnel related decisions.

In countries like India, where there is no objective yardstick of comparing learning outcomes across different primary schools and where penalizing, leave alone firing, teachers is impossible, learning outcomes assessment may have three important uses.

1. It can be a very useful instrument to spot class-wise collective learning deficiencies among children and administer the required training for the teachers. It is commonplace to find that a major share of children in certain schools or classes have not understood certain concepts (say, the concept of carry forward addition) and it may be useful to spot such weaknesses and provide focused training on this concept to the particular teacher.

2. Class room instruction currently involves teachers teaching a defined syllabus on a single-track mode to all the students in a particular class. This approach fails to appreciate the widespread differences in the pre-existing learning levels among children. For example, since a vast majority of children in Class V do not have the learning competency of even Class III, teaching them Class V Math straight away without bringing them upto speed with Class IV learning levels is a recipe of widening the already existing learning gaps.

In the circumstances, it is critical that the children be divided into groups based on their learning levels, and classroom instruction structured to close the learning gap through some form of remedial education. Assessment scores can be a useful instrument in this.

3. Currently we have no system to track the learning trajectories of students across their schooling career. How much improvement has the student achieved in a year with respect to the benchmark for that particular class? What has been the improvement in learning levels in Math from Class III to Class V? What is the learning level of a particular student vis-a-vis his best friend?

This information is necessary to not only improve the quality of monitoring and tracking, but also to generate demand-side pressures among both parents and the child itself. If properly presented, such information can be used to generate intrinsic motivation among the child itself (subtle comparison of marks of all children in a friendship group) and among their parents. Currently, till the tenth class, parents have no reliable and objective mechanism to assess the learning levels of their child.

My argument is that, whatever its flaws, an objective and standardized mechanism to measure learning outcomes is a pre-requisite for making any meaningful improvements in our primary schooling system. Once in place, we can then worry about how to keep such testing mechanism dynamic enough to prevent it being gamed by stakeholders.

Update 1 (8/7/2011)

An investigation into Atlanta’s public school system has uncovered evidence that teachers and principals have been secretly erasing and correcting answers on students’ tests for as long as a decade. A state investigation found that 178 educators at 44 of the district’s 56 schools engaged in cheating. The report is a huge blow to an urban school district that for years was hailed as one of the country’s most successful due to increased student performance.

This is the second high profile teacher cheating example in US, following a USA Today investigation found evidence of teacher cheating among some of Washington DC's highest-performing public schools. See this Freakonomics post.


sai prasad said...

I agree that some degree of standardized testing is needed to evaluate the LLs of students as well the effectiveness of teachers.

I dont think we need to fear that chiildren might end up spending too much time preparing for tests. In fact, there is severe shortage of testing in the case of government school children.

Do we need to start with a state law first ?

sai prasad said...

"" If the purpose of evaluating teachers was to help improve instruction, we wouldn’t be testing children overtime and further narrowing curriculum to figure it out. We would be developing tools to measure the quality of the instruction teachers provide every day, and we would focus on only those qualities of sound teaching that are critical to student learning. ""

Have any efforts been made in this direction anywhere ?

Anonymous said...

Hi, There are 2 points that needs to be considered here -

1.tracking a teacher

it all depends upon the infrastructure and the resources he/She is provided and the Objectives setting is done based on above parameters.
In a country like India, especially when the schools are under state governments, its really important to set a objectives considering above points.

2.tracking a student -

everything is dependent on the teacher and he/she is a more expert than anybody in understanding/tracking a student learning through out his primary school career.

jyothikraja said...

Hi, I really appreciate the fact your continuous strive for the primary education and its importance.

but, when we consider from India's perceptive, there should be some reforms to be made in primary educational system on a whole when it comes to the affordability of middle class children.

1. the entire marks/percentage system need to be debated
2. 20-30 students need to be mentored under each teacher and his/her objectives should be set accordingly.
3. the classes should be more interactive rather than a mere a teacher show of an hour because this is main issue where actually the expression/communication skills of a child is implanted.
4. the tagging/existence of this board exams need to be debated.

and not the least, make use of IT.

gulzar said...

the point about "tools to measure the quality of the instruction" is spot on, though i am not aware of any intervention anywhere in the world. the process of teaching-learning is so qualitative that my hunch is that it would be difficult to quantify its quality beyond a point - some reasonable thumb rules about effective teaching-learning outcomes.

yes sir, the role of testing is critical. this is contrary to what RTE and some experts think when they criticise and even advocate bans on testing.

RIAZ UDDIN said...

Really Helpful Information About. How to be a Primary Teacher and what is it like to be a primary school teacher in the 21st century?