Prediction markets

In 2006, Google decided that it wanted to obtain better forecasts of what the company and its competitors were planning to do. As with most firms, better predictions of product launch dates, office openings and so on are of significant strategic importance to Google. But instead of using the standard approach - forecasting the future by asking designated experts - Google opted to set up a market, in which any Google staff member could bet on the chances of an event coming true.

If a Google staffer had strong information that an event would come true, then regardless of her rank in the company, she could bet on that outcome. The results - based on the aggregated bets of thousands of Google staff members - were strong predictors of the actual outcomes. Where the trading volume was high, the forecasts were even more precise. And as closing time drew closer, the markets became steadily more accurate.

This example describes an increasingly important information-aggregation tool, known as “prediction markets”. Analytically, these are markets where participants trade in contracts whose payoff depends on unknown future events - just as in any financial or betting market. The defining feature of a prediction market is that the price of these contracts can be directly interpreted as a market-generated forecast of some unknown quantity.

Much of the enthusiasm for prediction markets derives from the efficient markets hypothesis: in a truly efficient market, the price of a financial security or prediction market contract reflects all available information. Thus, efficient prediction market prices hold the promise of yielding efficient and unbiased forecasts.

A number of successes in these markets, both with regard to predicting public events and corporate outcomes, have generated substantial interest among social scientists, policymakers and the business community.

We begin by outlining the types of prediction markets currently on offer, and then discuss their accuracy. We review what is known about how to design an effective market, and the constraints of the present legal regime. Finally, we conclude with our own prognostications on what the future might hold.

Prediction markets - an overview

Perhaps the best-known type of prediction market are election-betting markets. The Iowa Electronic Market, established by political scientists at the University of Iowa in 1988, is perhaps the world’s best known election market. These academics operate an electronic market in which traders can purchase “futures contracts” that consist of a promise to pay $1 if the candidate wins the popular vote. Thus, the price of this contract reflects the probability of a candidate winning the election.

Assuming market efficiency, these prices should yield assessments that reflect all available information - including polls, the state of the economy, and recent policy pronouncements.

For United States presidential elections, the Iowa Electronic Market has tended to be more accurate than opinion polls. Research has suggested that this is because the betting market focuses on the underlying dynamics of the race, and is therefore better able to parse out events that occur several months before the election, but will not change the outcome.

Conversely, the betting market responds rapidly to occurrences that affect the underlying dynamics of the race (such as the appointment of a new campaign manager), even if these events elicit relatively little response in the media or polls.

In Australia, looser regulation of sports betting in the Australian Capital Territory and the Northern Territory has led several bookmakers - including Betfair, Centrebet, International All Sports, SportingBet and SportsAcumen - to offer punters the ability to bet on election outcomes.

True to national traditions, Australian gamblers wagered more than $1.5 million on the 2001 federal election, and more than $2.6 million on the 2004 federal election. While betting markets do not “look” like financial markets, the betting odds yield the same directly interpretable forecasts offered by the Iowa Electronic Markets.

Outside the political sphere, there are a growing number of web-based prediction markets, often run by companies that provide a range of trading and gambling services. Some prominent examples include Tradesports.com (also known as InTrade.com), Betfair.com, and pseudo-markets (in which participants trade virtual currency) such as Newsfutures.com and Ideosphere.com.

Some prediction markets focus on economic statistics. For instance, Goldman Sachs, Deutsche Bank, and the Chicago Mercantile Exchange have launched markets on the likely outcome of future readings of economic statistics, including employment, retail sales, industrial production, and business confidence.

How accurate are prediction market forecasts?

Arguably the most important issue with these markets is their performance as predictive tools. In the political domain, the Iowa Electronic Markets has both yielded very accurate predictions for United States elections, and also outperformed the major pollsters.

In a comparison of election-eve forecasts, the Iowa markets predicted candidates’ vote shares with an average absolute error of 1½ percentage points, while the average forecast error for Gallup polls was 2.1 percentage points.

In Australia, election-eve betting markets correctly predicted that the Coalition would win the 2001 and 2004 federal elections. By contrast, a Coalition win was predicted by the election-eve polls of only two out of three major pollsters in 2001, and only two out of four major pollsters in 2004.

Perhaps more interesting in terms of how well prediction markets can aggregate information is the performance of markets at the level of the individual district. Typically districts are sufficiently small that there is little interest (or funding) for local polling, yet when Australian bookmakers started betting on district-level races, we found that they were extremely accurate.

In 2001, the Centrebet favourite won in 43 out of 47 marginal seats. In 2004, the Centrebet favourite won in 24 out of 32 marginals.

That said, comparing the performance of markets with a mechanistic application of poll-based forecasting may not provide a particularly compelling comparison. A more relevant starting point might be to compare the predictions of markets with those of independent analysts.

For example, a survey of ten experts published on the Sunday before the 2004 federal election found that three thought Mark Latham would win, while seven thought John Howard would win, but with a smaller majority than in 2001. None forecast the true result - a Howard victory with an increased majority. By contrast, the betting market was more confident of a Howard victory in 2004 than in 2001.

In a corporate context, the Hollywood Stock Exchange predicts opening weekend box-office success, and these predictions have been quite accurate. Further, this market has been about as accurate at forecasting Oscar winners as an expert panel.

Some firms have also begun to experiment with internal prediction markets. An internal market at Hewlett-Packard produced more accurate forecasts of printer sales than the firm’s internal processes.

Gerhard Ortner described an experiment at Siemens in which an internal market predicted that the firm would definitely fail to deliver on a software project on time, even when traditional planning tools suggested that the deadline could be met.

While the Hollywood markets have drawn many participants simply on the basis of their entertainment value, the HP and Siemens experiences suggested that motivating employees to trade was a major challenge. In each case, the firms ran real money exchanges, with only a relatively small trading population, and subsidised participation in the market, by either endowing traders with a portfolio, or matching initial deposits.

The predictive performance of even these very thin markets was quite striking. Yet as the Google example discussed in the opening paragraph of this article suggests, internal market with hundreds of traders are likely to have even greater forecasting power.

In another recent prediction market, traders in “Economic Derivatives” predict the likelihood that economic data released later in the week will take on specific values. The traditional approach to aggregating forecasts is to simply take an average or a “consensus estimate” from a survey of 30 or so professional forecasters.

Comparing average market and consensus forecasts for four variables (non-farm payrolls, retail trade, unemployment claims and business confidence), shows that the market-based predictions of these economic indicators are not only extremely close to the corresponding “consensus” forecast, but also typically closer to the actual outcome, outperforming the experts on average.

Market design

The success of prediction markets, like any market, can depend on their design and implementation. For a prediction market to work well, contracts must be clear, easily understood, and easily adjudicated. This requirement for clarity can sometimes turn out to be complex. For example, in the course of Siemens’ internal prediction market on whether a software project would be delivered to the client on schedule, the client changed the deadline.

One intriguing question is how much difference it makes whether prediction markets are run with real money or with some form of play money. Legal restrictions on gambling have led some groups like NewsFutures.com to adopt play money exchanges, with those who amass the largest play-fortunes eligible for prizes.

One suggestive experiment compared the predictive power of the prices from a real-money and play-money exchanges over the 2003 NFL football season, finding that both yielded predictions that were approximately equally accurate. Interestingly, both sets of prices also outperformed all but a dozen of 3,000 people in an online contest, and also easily outperformed the average assessments of these “experts”.

On “play money” exchanges, such as Foresight Exchange, one often sees quite loosely worded “contracts” such as that a “scientific study will conclude that astrology is a statistically significant predictive method to describe an individual’s personality traits”.

Some prediction markets will work better when they concern events that are widely discussed, since trading on such events will have higher entertainment value and there will be more information on whose interpretation traders can disagree.

Ambiguous public information may be better in motivating trade than private information, especially if the private information is concentrated, since a cadre of highly informed traders can easily drive out the partly informed, repressing trade to the point that the market barely exists. Indeed, attempts to set up markets on topics where there are insiders with substantial information advantages have typically failed.

For instance, the Centrebet market on who will lead the Liberal Party to the 2007 federal election has generated very little trade despite the inherent interest in these questions.

Legal constraints

In the United States, the legal environment has forced onshore prediction markets to make compromises that have limited their attractiveness. The Iowa Electronic Markets agreed to limit positions to $500 in order to receive a “no action” letter from the Commodity Futures Trading Commission. This compromise limits the scope and depth of their markets, and possibly their efficiency.

Platforms such as NewsFutures, the Hollywood Stock Exchange, and the Foresight Exchange operate using play money, albeit play money that can sometimes be converted into prizes. Thus these platforms evade anti-gambling legislation by operating as games of skill.

The consulting firms running corporate prediction markets have taken the same approach as experimental economists on campuses: they have subsidised participation, allowing everyone to leave a winner, albeit to varying degrees.

Finally, two markets that we are aware of, Economic Derivatives and Hedgestreet, have obtained regulatory approval to offer trading of innovative futures contracts. Hedgestreet started in 2004, and Economic Derivatives, a joint venture by Deutsche Bank and Goldman Sachs that runs markets in economic numbers, has been operating since 2002.

In Australia, the legal environment has prevented prediction markets from establishing themselves in most states. In addition, there is a concern that for major bookmakers such as Centrebet, trading on economic derivatives would bring them into conflict with the Australian Stock Exchange and the Australian Futures Exchange.

Since Australian betting agencies already handle significant sums of money for elections and major sporting events, relaxation of the regulation governing such markets would bring little risk, but a significant public benefit.

Future applications

Prediction markets are extremely useful for estimating the market’s expectation of the probability of a particular event occurring. Markets on election outcomes serve a useful public purpose in providing an up to the minute barometer of the probability that a given party or candidate will be elected. For firms whose business is exposed to a large degree of political risk, our advice is to follow the betting markets, not the opinion polls or the televised talking heads.

Prediction markets doubtless have their limitations, but they may be useful as a supplement to the other relatively primitive mechanisms for predicting the future like opinion surveys, politically-appointed panels of experts, hiring consultants, or holding committee meetings.

With the exception of a short-lived attempt by a United States defence agency to create information markets, governments have shied away from directly operating them. This may be a sensible policy stance - yet politicians should also recognise that if the valuable information generated by trade in these markets is not fully internalised into the profits earned by these private firms, prediction markets will be underprovided. In the area of economic derivatives, it might well be in the interests of government departments to fund the creation of markets that aggregate traders' expectations of future economic releases.

Indeed, the Reserve Bank of Australia already closely tracks the difference in yields on inflation-indexed bonds and non-indexed bonds - effectively a prediction market on future inflation.

More generally, the highly restrictive stance that governments have taken towards regulating prediction markets does not appear to take any account of the informational benefits of such markets. Would there be a public benefit from liberalising gambling regulations to give prediction markets a freer hand? You bet.