Substack

Monday, April 17, 2023

Edtech and student learning outcomes

I have thought about blogging on Edtech, but never managed to get around. This post has a set of observations on the sector, triggered by an article in The Ken which draws attention to some trends in India's Edtech landscape. 

It points to declining innovation in India's Edtech space,

A useful way to think about edtechs is their evolution(?) from inventions to machines. As inventions, they were allowed to experiment with different technologies and business models. The novelty of using tech to solve learning or teaching problems was still… novel. The possibilities for scaling were unmatched. The price points, for quality education, were unbelievable. We may not have invented new ways of learning—it was still a digital lecture—but we had hit upon something new. I would argue that these fledgling inventions were coaxed into becoming big machines partly by how much money they raised. Everyone was in a hurry to see the big edtech flagship industry come out of India—Byju Raveendran even made it a personal goal—and churn out machine after successful machine. And you needed lots of staff to turn all the cogs. But yeah, this kind of stuff is surely gone now. What got lost in the mad rush was the invention, and the inventors. And because no one’s building anything new, the need for creators, designers and even product managers has been slashed by approximately 70%... the roles now left behind in B2C edtech are mostly ones that keep the sputtering machines running: sales people, category heads, content developers, engineers. Definitely essential work, but no one’s looking around for a chief innovation officer. It’s all rinse, repeat.

The article also cautions with an example on placing too much expectations on AI in Edtech,

Nirmal Patel is an education entrepreneur based in Gujarat. Patel experimented with GPT-4 to teach a polynomial sum, and took a counterintuitive approach. Patel wasn’t interested in generating only the right answer for (2y+5)*(2y+5), "I asked GPT to tell me wrong ways to solve the problem and it was not able to reproduce any wrong ways. Then I gave it one of the wrong answers, and asked how to go from question to wrong answer and it kept failing... so I felt that it lacks data on how people make mistakes." ... clearing misconceptions are actually a key pathway to learning. Especially with maths, where you could carry misconceptions across grades, and keep making the same mistakes.

Some observations:

1. The point about declining innovation requires some qualification. I don't think Indian Edtechs were ever innovative in expanding the frontier of our knowledge about child learning. Their primary success was in accumulating voluminous digital content (especially questions), organising them, and delivering in a user-friendly manner. Innovation was limited to the superficial aspects like gamification, and not in the science of child learning. For example, personalised adaptive learning, which examines how learning happens (and, more importantly, does not happen) never took off. 

As a reflection of this prioritisation, India's Edtech space was driven by technology professionals guided largely by the anecdotal wisdom of a few teachers, and rarely by education researchers with deep insights into the way learning happens. It would be very interesting to get a break-up of the various categories of professionals employed in the big Edtechs - IT programmers, gaming experts, Education (not economics) researchers and PhDs, teachers, marketing and sales professionals, finance professionals etc.

2. This lack of focus on innovation and R&D (beyond brute force piling content and presenting them) is emblematic of India's startup eco-system. Too many copycat entrepreneurs attracted by the misleading perception of a continental sized market and the attendant possibility of instant gratification from massive growth and high valuations. They follow the boiler plate business model of copying the idea, capturing subscribers with discounts and promotions, boosting valuations, attracting more investors...

Edtech was an area where the entrepreneurs and financiers, as well as their mentors and boosters, had the opportunity to strike out differently, and design and develop a world-class cutting-edge set of products and solutions which truly expanded the frontier of general student learning and thereby addressed arguably the biggest economic development problem facing India and many other developing countries. This has proved elusive and looks likely to remain so. 

3. The binding constraint on learning deficiencies in developing countries is that of basic literacy and numeracy skills among children in lower grades, which gets carried forward to higher grades. In its absence children end up sleep walking through higher grades or drop out. For Edtech to make a dent on this requires content and pedagogy (or digital instruction pathway) which can address deficiencies in foundational learning. This, in turn, requires understanding of how children make mistakes on basic literacy and numeracy or fail to understand basic concepts, so that content can be designed and delivered appropriately to address those deficiencies. 

The better-off children don't suffer anywhere as much from basic literacy and numeracy deficiencies. And they being the primary consumers of Edtech, the developers have limited incentive to develop content and pedagogy that responds to the problem of basic literacy and numeracy. There is also a limited library of answers/responses of the common mistakes made by children in basic literacy and numeracy, and focus on how they can be addressed. 

4. In the absence of good pre-existing data on basic literacy and numeracy (student responses), any AI solution like Chat GPT will struggle to churn out meaningful solutions to the problem. I struggle to imagine how, in the absence of a mechanism to address this data gap, Chat GPT and its likes can do much in the foreseeable future to address the issue of basic literacy and numeracy.

5. If one were to do a balance sheet of the Edtech wave, it can be argued that it has expanded the learning tools available to those already cognitively better-off kids who form a small minority of the kids. However, I don't think it has made much, if any, difference to the massive learning deficits problem that afflict the overwhelming majority of students in India. In fact, the Edtech wave may actually have strengthened the Matthew Effect in life outcomes between the rich and poor by marginally improving the learning levels of the better-off children while doing little or nothing at the margins to the rest. 

I think the same divide applies to teachers too. The teachers in the good private schools (and a few interested teachers in the public schools) may have benefited at the margins from the Edtech materials. For these reasons, I'm not hopeful that this current Edtech wave will have any significant impact on improving the general quality of school education in India. It would have been a missed opportunity. 

6. The boom in the Edtech space happened only because it was marketed effectively as an innovation to improve learning and fed into the most important priority of Indian households, the education of their children. It effectively expanded the off-school (or at-home) learning space and also cannibalised the tuition market (though there may be a re-cannibalisation now happening).

7. Finally, while the jury is still out on whether the venture capital in particular and private equity in general have been net positive for the world economy, this Edtech boom is a great example of the distortionary effects of financial markets and the need for some form of market guidance. The VC bubble of the last few years, fuelled by ultra-low interest rates and fear of missing out (FOMO) investing incentives of VCs, have resulted in a massive misallocation of resources towards short-term business growth and valuations without focus on quality and outcomes (sustainability). 

A few examples of innovation does not mean that, in general, the wild-west of entrepreneurship and VCs can work everywhere and on everything, especially in solving deep-rooted and critical development problems. It should be a strong cautionary reminder to those who think naively that digital technologies can disrupt and solve chronic problems like student learning deficiencies. 

No comments: