Smith had written that self-interested actions, working through the now famous invisible hand, often leads to socially benign outcomes. Darwin's insight was that natural selection favours traits and behaviours according to how they affect the success of individuals, not species or other groups; and while sometimes individual and group interests coincide, often they conflict. In other words, traits that help individuals are often harmful to larger groups.
Prof Frank gives examples from animal species of such evolutionary changes that conflict with group interests - the large size of male elephant seals helps them in the competition (by giving them an edge in battles) to attract mates, while making them as a group far more vulnerable to sharks and other predators; a mutation for larger antlers served the reproductive interests of an individual male elk (again giving them edge in battles with other males for mating), but this becomes a fatal handicap when predators pursue males into dense woods etc.
The incentive structure in financial markets that favors imemdiate pay-offs, as against longer-term returns, means that the resultant competition favors those money managers who have a "nervous system biased in favour of short-term relative reward". But such an evolutionary development has adverse consequences for the market. Prof Frank writes,
"Anyone disinclined to seize immediate gains at the risk of having to incur costs in the future would experience low relative rewards in the short run. And when competition was intense and immediate, such individuals often didn't survive to see the long run.
In market settings, a nervous system biased in favour of short-term relative reward is a recipe for disaster. When the price of an asset like housing is rising steadily, unregulated wealth managers can create leveraged investments that generate enormous rates of return... But investors faced a tough choice - they could earn high returns by continuing to invest in them, or they could move their money elsewhere. Many rejected the latter strategy because it would have required watching friends and neighbours pass them by. Wealth managers felt compelled to offer the risky investments, since many customers would otherwise desert them. Managers also knew there would be safety in numbers when things soured, since almost everyone had been following the same strategy. The resulting collapse was inevitable."
He elaborates with more examples,
"To make their funds more attractive to investors, money managers create complex securities that impose serious, if often well-camouflaged, risks on society. But when all managers take such steps, they are mutually offsetting. No one benefits, yet the risk of financial crises rises sharply.
Similarly, to earn extra money for houses in better school districts, parents often work longer hours or accept jobs entailing greater safety risks. Such steps may seem compelling to an individual family, but when all families take them, they serve only to bid up housing prices. As before, only half of all children will attend top-half schools. It’s the same with athletes who take anabolic steroids. Individual athletes who take them may perform better in absolute terms. But these drugs also entail serious long-term health risks, and when everyone takes them, no one gains an edge."
And about the need for regulation, he writes,
"By calling our attention to the conflict between individual and group interest, Darwin has identified the rationale for much of the regulation we observe in modern societies — including steroid bans in sports, safety and hours regulation in the workplace, product safety standards and the myriad restrictions typically imposed on the financial sector."
Mark Thoma points to a talk given by Paul Krugman to the European Association for Evolutionary Political Economy on what economists can learn from evolutionary theorists.