Tuesday, December 7, 2010

Limits of technology - the case studies with education

In the hype surrounding rapidly proliferating information and communications technologies, it is easy to get carried away by technology-based silver-bullet solutions to development and governance issues. In this context, Kentaro Toyama introduces a dose of much-needed realism by cautioning against excessive optimism that these technologies can exercise magic-wand properties to eliminate poverty.

He writes that technologies are merely instruments to be deployed by human beings involved in the implementation of development programs. He describes them as "magnifier of human intent and capacity", and writes (and I quote extensively),

"But as we conducted research projects in multiple domains (education, microfinance, agriculture, health care) and with various technologies (PCs, mobile phones, custom-designed electronics), a pattern, having little to do with the technologies themselves, emerged. In every one of our projects, a technology’s effects were wholly dependent on the intention and capacity of the people handling it. The success of PC projects in schools hinged on supportive administrators and dedicated teachers. Microcredit processes with mobile phones worked because of effective microfinance organizations. Teaching farming practices through video required capable agriculture-extension officers and devoted nonprofit staff...

technology—no matter how well designed—is only a magnifier of human intent and capacity. It is not a substitute. If you have a foundation of competent, well-intentioned people, then the appropriate technology can amplify their capacity and lead to amazing achievements. But, in circumstances with negative human intent, as in the case of corrupt government bureaucrats, or minimal capacity, as in the case of people who have been denied a basic education, no amount of technology will turn things around.

Technology is a magnifier in that its impact is multiplicative, not additive, with regard to social change. In the developed world, there is a tendency to see the Internet and other technologies as necessarily additive, inherent contributors of positive value. But their beneficial contributions are contingent on an absorptive capacity among users that is often missing in the developing world. Technology has positive effects only to the extent that people are willing and able to use it positively. The challenge of international development is that, whatever the potential of poor communities, well-intentioned capability is in scarce supply and technology cannot make up for its deficiency."

Ironically, techonology as a magnifier of human capability also results in exactly the opposite outcomes when capability and intent are absent or very weak. Echoing the digital divide voices, he writes,

"The greater one’s capacity, the more technology delivers; the lesser one’s capacity, the less value technology has. In effect, technology helps the rich get richer while doing little for the incomes of the poor, thus widening the gaps between haves and have-nots."


There are two recent stories about the use of technology in education that are classic examples that overlook the wise words of Prof Toyama. First, the NYT reported about the use of cameras to videotape classroom instruction and use them to both remotely assess teachers and help them improve. Supported by the Bill Gates Foundation, a number of American school districts are videotaping classes and score teacher performance remotely by independent assessors. The teachers will also be provided feed-back and training to improve the quality of instruction.

In a Mint op-ed Harvard Professor Tarun Khanna pointed to the success of South Korean on-line learning website, Megastudy, with improving education quality. Megastudy's core idea is that good teachers are videotaped, and then others can pay to subscribe to their lectures, via online access to the videos. Prof Khanna reasons that the incentive structure in Megastudy's model (teachers get a share of the revenues) encourages the good teacher and "the underperforming ones understand how they must improve". Further, it also enables access for all children to the best available teachers.

Both these are typical examples of technology-driven euphoria glossing over the real-world challenges of replicating such models. Especially if the objective is to use these technologies to improve the quality of instruction and increase student access to the most powerful learning resources in developing country environments.

The videotape model to assess teachers is simply too fanciful to even cross the first stage of laboratory trial. While theoretically attractive, it becomes simply impractical when viewed through the lens of implementation on scale. The exorbitant installation cost and maintenance itself should be enough to junk such ideas (at $1.5 million for a district with 140 schools. it would cost nearly $12 bn to instal in 1.1 m Indian schools).

Even overlooking the cost and the very relevant questions about whether a videotape can capture all the different dimensions of classroom instruction, there still remain issues of implementation. Who will watch the videotape? How do we ensure quality of their assessments? How can the subjective opinions of the thousands of assessors be standardized? How do we administer the teacher feedback mechanism? We could easily end up subsituting an ineffectual assessment system by principals or supervisors with an equally inefficient system involving third-party assessors sitting at remote locations!

And then there are the real world issues. Any such technology and approach to assessment is vulnerable to being subverted. Cameras will develop problems and some will be made to develop problems and fail to record! And even when they do record, some of them would certainly end up recording something else (or without sound)! The possibilities are too numerous to be addressed to any level of satisfaction.

The Megastudy model offers little towards improving learning outcomes and teacher performance in any meaningful manner. Given the massive scale involved (1.1 million schools and nearly 200 million students in India), such websites will always have marginal reach. The small sliver of students who can afford it will benefit and increase their learning gap with the rest. The good teachers (and here too the chances are that there will be very few from government schools) would of course increase their bank balances.

In fact, on-line learning websites, admittedly not exactly Megastudy-model based, are already widely available in India. The sheer size of the education market in countries like India mean that firms like Educomp (which offer such services) can expand rapidly for a long time to come by merely concentrating on the top 5-10% of the market. It would be decades before the aforementioned models start penetrating the mass-market, involving government schools. In any case, why go for paid Megastudy material, when you can get the Khan Academy for free?

The businessman in me (and I am not in that profession!) will be excited by the commercial possibilities of videotaping and on-line classroom content distribution. However, the public official in me (and I am one!) sees limited possibilities for its scalability.

5 comments:

KP said...

Dear Gulzar,

I happened to read this column by Prof Toyoma earlier and was impressed by his candour.

The reason : I think the use of technology and data is overrated. Tarun Khanna and others of his tribe have a pattern of research that effectively is "measure and correct". Period. Very little insight.

Apparently trail blazing research from management theorists is simply about being a raconteur with data that dazzles and cliched pitches of their constituency (incentive + technology in this case)

So every research paper is about statistics overdosed with Engish.The more incentive / more technology / better outcomes has been explored ad nauseam - but is there anything new there.

Heres a report by McKinsey on education that uses 126 pages

"How the world's most improved school systems keep getting better"

To say this ::

"...Many systems in our sample have created a pedagogy in which teachers and school leaders
work together to embed routines that nurture instructional and leadership excellence.

....Our findings indicate that six interventions occur with equal frequency across all the improvement journeys, though manifesting differently in each one. These six interventions
are: revising curriculum and standards, ensuring an appropriate reward and remuneration structure for teachers and principals, building the technical skills of teachers and principals,
assessing students, establishing data systems,and facilitating the improvement journey through the publication of policy documents
and implementation of education laws...."

----

The first part of the quote was the plain obvious obfuscated with jargon. The second part was obvious and merited a shorter report.

---

It takes a person like Prof Toyoma to tell us the truth like it is, insight we can relate to with / without data. Impressive! and an honest assessment.

There is nothing in the regular research reports on education that cannot be produced by an administrator with an average level of insight /competence and an excel sheet ( I hope you agree!) - and we may be spared the exaggerated attempt at scholarship. Systems produce averages - and mosly broad distributions along a bell curve - those at the negative extreme can learn from those ahead of the curve.

A more nuanced view on teacher assessment can be found in the following columns by Stanley Fish.

http://opinionator.blogs.nytimes.com/2010/06/21/deep-in-the-heart-of-texas/

and a follow-up

http://opinionator.blogs.nytimes.com/2010/06/28/student-evaluations-part-two/

In the self-obsessed world of academics ...Prof Toyoma..is all about research... once more with feeling ...and insight.

regards,KP.

KP said...

Dear Gulzar,

I happened to read this column by Prof Toyoma earlier and was impressed by his candour.

The reason : I think the use of technology and data is overrated. Tarun Khanna and others of his tribe have a pattern of research that effectively is "measure and correct".Period.

Trail blazing research! from management theorists is simply about being a raconteur with data that dazzles and cliched pitches of their constituency (incentive + technology in this case)

So every research paper is about statistics overdosed with Engish.The more incentive / more technology / better outcomes has been explored ad nauseam - but is there anything new there - and even there research on motivation does not agree with these simplistic findings.

Heres a report by McKinsey on education that uses 126 pages

"How the world's most improved school systems keep getting better"

To say this :

"...Many systems in our sample have created a pedagogy in which teachers and school leaders
work together to embed routines that nurture instructional and leadership excellence.

....Our findings indicate that six interventions occur with equal frequency across all the improvement journeys, though manifesting differently in each one. These six interventions
are: revising curriculum and standards, ensuring an appropriate reward and remuneration structure for teachers and principals, building the technical skills of teachers and principals,
assessing students, establishing data systems,and facilitating the improvement journey through the publication of policy documents
and implementation of education laws...."

----

The first part of the quote was the obvious obfuscated with jargon.The second part was obvious and merited a shorter report.

---

It takes a person like Prof Toyoma to tell us the truth like it is. Impressive! and an honest assessment.

There is nothing in the regular research reports on education that cannot be produced by an administrator with an average level of insight /competence and an excel sheet (I hope you agree!)- and we may be spared the exaggerated attempt at scholarship.

All systems produce averages, and mostly bell curve distributions. Those at the negative extreme can and should learn from those ahead of the curve.

A more nuanced view on teacher assessment can be found in the following columns by Stanley Fish.

http://opinionator.blogs.nytimes.com/2010/06/21/deep-in-the-heart-of-texas/

and a follow-up

http://opinionator.blogs.nytimes.com/2010/06/28/student-evaluations-part-two/

In the self-obsessed world of academic reseach...Prof Toyoma..is all about research ...once more with feeling...and insight.

regards,KP.

KP said...

Dear Gulzar,

Shanghai Test Scores Stun Educators

http://www.nytimes.com/2010/12/07/education/07education.html?src=fbmain

A nice article from NYT in context..and some talk about "motivation" .. primarily.

regards,KP.

Jayan said...

We attempted in similar direction; much smaller scale though. http://nptel.iitm.ac.in/ - is used by many students and professionals.

Having video/animation as an educational tool has limited use. It has to be dynamically adjusted to suite market needs- say a change in syllabus. Also the content needs to be available in multiple languages - Tamil, Hindi and English for example. This need a professional system to support the content - best done by private companies. There are number companies this - Cost is rather high for the school. As you wrote it the digital divide is going to get 'magnified'.

Helping teachers to improve their teaching with help of training and technology may be more suitable for govt schools. Kerala's IT@School is doing some thing similar --http://www.itschool.gov.in/

gulzar said...

thanks KP for the comments and the links. the Mckinsey report extract is an excellent example of complicating the simple. i completely agree with your observations.

thanks jayan for your links. i see great potential in computer aided learning in the coming years.