Is commodity futures speculation driving oil prices?

Back in 1997 the then Malaysian Prime Minister Mahathir Mohammed was scoffed at and virulently denounced when he blamed speculators like hedge funds for being responsible for the East Asian economic crisis. Today the roles have changed and influential voices in the American policy making and academic circles are accusing commodity futures speculators, fleeing the sub-prime mess, of being responsible for the rising oil and commodity prices. They claim that expectations of a higher future price and/or investment in the futures market by institutional investors are pushing up the current price. However, as then, the evidence in favour of speculation as a major contributor to rising commodities prices appears thin.

Economists like Guillermo Calvo argue that the rise in prices cannot be explained solely by the growth in China and India led emerging Asia. They point to the concurrent increase in transaction volumes in the forward markets, and claim that these speculative expectations are determining the current spot prices. Futures market therefore ends up determining the spot prices. Prof Calvo claims that soverign wealth funds and other sovereign investors have been shifting to higher return investments in commodities futures, thereby driving up transactions by a large quantity.

In a controversial paper, Jeffrey Frankel attributed the rising commodity prices to the low interest rates. He argues that high rates increases the supply of storable commodities by increasing the incentive for extraction today rather than tomorrow (extract and sell them, and then invest proceeds at these high interest ratees); by decreasing firms’ desire to carry inventories; and by encouraging speculators to shift out of spot commodity contracts, and into treasury bills. A decrease in real interest rates has the opposite effect, lowering the cost of carrying inventories and incentivizing inventory build-up and thereby raising commodity prices. So it is argued that the low prevailing interest rates have led to build up in inventories and therefore higher prices.

Paul Krugman, here (Mark Thoma has explanation to the graphs here), here, and here, has been one of the most vocal opponents of the speculation driven oil price hypothesis. For this theory to be correct, Krugman argues, speculators have to indulge in buying up excess supply in the expectation of higher future prices. In other words, somebody has to indulge in hoarding. If this were true, there would have to be a build up of inventories and futures prices have to be atleast higher than spot prices, so as to justify the expectations.

Further, as can be seen from the graph below, oil supplies remain within recent historical ranges, which would mean that current prices reflect fundamental forces rather than speculative excesses.



Further, if the speculative pressure is driving up oil prices, then the long positions ("call option", which speculates that the prices will go up, as opposed to "put options" that seek to profit from prices falling) have to be large in number and rising. But as Alan Reynolds of the Cato Institute writes, "The "net long" position on the New York Mercantile Exchange fell from 113,307 contracts on March 11 to 25,246 by June 10 - so nearly as many traders are now shorting oil as are going long."

In fact, the long positions have dropped sharply since oil crossed $100. Even the exchange-traded US Oil Fund, which tracks the price of West Texas crude, has had a 140% increase in short positions since January 2008, outnumbering long bets by two to one.

But in reality, both there has been no build up in inventories (if anything inventories are at their lowest) and futures market has been characterized by "backwardation" - spot prices ruling over futures. Whenever the futures prices have gone above the spot prices (contango), the difference has been too weak to justify any speculative exuberance.

What makes the speculation story even less credible is the similar trend with other commodities. This USDA data show stocks of wheat and other grains declining in recent years. The trend, as can be seen below (from Mark Thoma), is same with other foodgrain commodities.






Some commentators argue that the hoarding of inventories may be occurring by way of oil producing countries with-holding their reserves from full exploitation. In other words the inventories are being hoarded under the ground. But this explanation comes into conflict with the similar trend in prices of foodgrains which cannot be left under the ground for too long.

It is also possible that hundreds of millions of oil consumers are hoarding small quantities of oil in their car tanks or their basements in expectation of rising prices. The cumulative effect of such multiple hoardings can also be substantial and exert significant price pressures, especially in tight market conditions. But again, this cannot be used to explain rise in the prices of minerals and metals. And in any case, this cannot be interpreted to blame speculation for the high prices.

Evidence from the Indian market too goes against speculation driving up prices. The ban on futures trading in rice, wheat and pulses have had little impact on their prices, which have continued to rise unabated. In contrast, sugar and potatoes which are traded in the futures market, have experienced only very small rise in prices.

Update 1
From Paul Krugman, the spot and futures prices


Update 2
Mark Thoma explains the Krugman model in discrete and continuous time series here and here.

Update 3
NYT has this neat summing up of the more common reasons against the speculation-is-driving-up-prices hypothesis.

Update 4
Big Picture outlines ten fundamental reasons why the oil prices rose in the first instance. He argues that major bull market moves begin on fundamentals, but shift towards the end of its life into a speculative frenzy.