Prediction Markets

From The Internet: Issues at the Frontier (course wiki)
Jump to navigation Jump to search

Topic Owners: Matthew, Elisabeth

back to syllabus

Precis

Our seminar this week aims to discuss novel and/or controversial applications of prediction markets. Our study will be organized into three parts. First, before class everyone will get acquainted with prediction markets as they now exist and understand the basic economic theory behind their success--this page and the first required reading (the Wolfers/Zitzewitz article) will cover that background. We also tried a (failed!) experiment with creating our own prediction markets on intrade.net that inadvertently taught a lesson about how difficult it is to get efficient information markets (more below). Second, we will examine a few proposals for how prediction markets might be used at the frontier. The readings cover Professor Wolfers' crime prediction market, the DARPA project, and the Google Flu Tracker, as well as a series of short ideas from Professor Abramowicz. Professor Wolfers will appear by videoconference to discuss these questions. Third and finally, we will try to figure out what prediction markets can and cannot do well as applied to law or government. At this point, we will consider skeptical arguments, both the set of (weak but popular) moral objections and more sophisticated practical/theoretical worries. Ultimately, the class will come to some conclusions about what types of uses prediction markets might have, perhaps inventing some ideas of our own.

What Prediction Markets Do Now

The most high-profile examples of prediction marketsthe Iowa Electronic Markets and Intradestarted by focusing primarily on predicting election outcomes and related political and financial events. Now they have expanded to cultural (Oscars) and technological (X Prize) events as well. The status of the commercial markets is uncertain; for example, Tradesports announced recently that it is closing. And questions remain about the legality of prediction markets, whether the CTFC will regulate them, and whether they will be taxed. The basic idea is that prediction markets are more accurate than experts or traditional public opinion polls, because among other advantages they require participants to offer predictions backed by some conviction (money) and because they aggregate knowledge held by many minds. Private companies, like Google and Hewlett-Packard, are thus increasingly employing internal and external prediction markets to better forecast sales or to predict which products are likely to be most popular, among other things.

II. Novel Applications of Prediction Markets

More central to our purposes are increasingly prevalent proposals to use prediction markets to help guide law and policymaking. One of the earlier and more controversial applications was DARPA's planned terrorism futures market, which would have helped the Pentagon predict terrorist attacks, assassinations, and the like but was killed following political outcry. Somewhat less controversial are proposals to use prediction markets in areas with less potential for moral hazard. For example, some scholars have argued that they would be useful for corporate governance, for example as an alternative to disclosure requirements aimed at insider trading and other problems -- the idea is that prediction markets in which corporate insiders participated would provide the desirable information, at lower cost and greater accuracy. Professors Wolfers, Henderson, and Zitzewitz, in one of the required readings, describe how prediction markets might be useful to policymakers allocating resources to combat crime. Michael Abramowicz's recent book Predictocracy offers an extended analysis of the usefulness of prediction markets in law and decisionmaking by courts and legislatures (among other institutions).

III. Skeptical Concerns

Prediction markets offer a lot of promise but of course are far from a perfect tool, perhaps especially as applied to policymaking. A prediction markets skeptic would point to myriad problems with applying them to law. For example, they aren't very helpful when information is secret. If government changes policy in response to the market results, it might be difficult to identify winning conditions. Is there still room left for optimism?

Class Plan

We aim for the class to proceed as follows:

  • Professor Wolfers will lead off with an overview of how prediction markets work, their potential for use by private and public policymakers, and the most cutting-edge or salient issues that prediction market proponents face today
  • Professor Fisher will offer a brief response
  • We will then turn to questions and class discussion
  • Finally, we'll discuss and critique the policy applications that class members themselves came up with

Readings

Required Readings

Background

Justin Wolfers and Eric Zitzewitz | 18 Journal of Economic Perspectives 107 (2004) | (19 pages, essential background--only one equation for the math-phobic)

Novel Applications of Prediction Markets

Miguel Helft | New York Times | November 11, 2008 | (1 page, useful diagram)

M. Todd Henderson, Justin Wolfers, and Eric Zitzewitz | Olin Working Paper No. 402 | April 2008 | (5 pages, just read abstract and introduction)

Michael Abramowicz | Excerpts from Predictocracy | A series of short suggestions on novel applications of prediction markets to law and government

Carl Hulse of the NYT and Wire Services | July 2003 | (~6 pages)

Skeptical Concerns

Discussion of the DARPA Markets in the Senate | July 28 and 29, 2003 | (2 short comments, mostly for humor value)

Student Note | 122 Harvard Law Review 1217 (2009) | (21 pages, a short but good overview of defenses and criticisms of prediction markets, especially as applied to law)

Michael Abramowicz | Excerpts from Predictocracy | Professor Abramowicz's discussion of some advantages and problems of Prediction Markets

Optional Readings

  • Professor Sunstein's Infotopia
  • Cass R. Sunstein, Group Judgments: Statistical Means, Deliberation, and Information Markets, 80 N.Y.U. L. Rev. 962 (2005)

Class Participation

Creating a Market on Intrade.net

Several weeks ago the class started an experiment with its own prediction markets. Here was the plan: "Everyone will create their own prediction market contract via intrade.net. Once you set up an account there, you can then create you own contract at http://www.intrade.net/market/create/start.faces. Your contract can be about anything you like (doesn't have to be legal), but it should conclude before our class date, April 27, 2009. The idea is (1) for everyone to get a feel for how prediction markets operate; (2) to see what kinds of contracts get enough volume to be successful and what kinds don't; (3) to see how accurate the predictions are, how they change over time, etc.

"Once you've signed up and created a contract -- please do so by Friday, March 20 -- you can invite other people in the class to bet on your contract to give it some initial starting volume. (Intrade.net gives everyone $10k in play money upon signing up, so obviously if people want to bet on the wider world's contracts, that's great too). Everyone should list the contracts they've created below to facilitate class participation on intrade (and so that we can generate a variety of different kinds of contracts). At the actual class session on April 27, everyone should come in prepared to briefly discuss what happened with their contract (feel free to create more than one)."

Here were the contracts that the class created:

The experiment was, like Apollo 13, a successful failure. The markets got no traffic, and the predictions did not move. There are several explanations for this result, some of which definitely played a role and some of which may have. First, intrade.net is a fake money site. The lack of a financial incentive is, of course, a substantial disincentive to would-be bettors. But this explanation cannot be sufficient, because the whole intrade.net site--not just our class-constructed markets--is based on the same fake money system. In fact, what is remarkable is how active and accurate some of the fake money markets are. Second, though, our markets were placed in the user-created market ghetto of the intrade.net site. Traffic there is generally lower than on the rest of the site, and activity is correspondingly reduced. Could this be an example of a coordination problem? Third, some of the topics chosen were obscure; an efficient market would need enough investors interested in the subject to get trading volume sufficiently high. Fourth, and probably most devastating, the betting mechanism only allows an up-down vote. This limitation takes away one of the essential comparative benefits of prediction markets over voting: it does not account for intensity of preference. The bettor who thinks the going rate is 1% off sends the same signal as the bettor who thinks the rate is 50% off. A well-designed prediction market would never be designed this way.

Since our first effort didn't work out, we then asked class members to try a second round of experimentation with prediction markets.

Class-Generated Ideas for Applying Prediction Markets to Law and Government Policymaking

The Task. As part of our discussion of novel uses of prediction markets, we hope not only to entertain critical perspectives but also to be constructive. To that end, we're asking everyone in the class to come up with a suggestion for how to use prediction markets in law or government. The suggestion can be subject-specific (e.g., using prediction markets to guide crime policy or to help the DOJ and FTC analyze the risks of proposed mergers) or can cut across subjects and focus on institutions or modes of decisionmaking (e.g., using prediction markets in small claims court adjudication). But it should be specific enough that we can analyze drawbacks or advantages (e.g., not just something like, Congress should use prediction markets to help determine legislative priorities). See the Applications section of http://predictocracy.org/index.html for models and ideas. Please send your (brief) ideas to Elisabeth and Matt by Sunday the 26th at 5 pm; we'll likely discuss some in class and will definitely post them on the site later on.


The Ideas, Grouped in Loose and Overlapping Categories

  • Predicting Whether Policies Will Be Successful.
    • My idea is for the government to use prediction markets to gauge public opinion on how likely it is that a given statute, rule, or action would accomplish its purpose. For instance, prior to Congress voting on a bill a prediction market could analyze whether the bill would have significant positive effect in the real world. The results could inform Congressional debate on whether to pass the bill, modify it, or reject it.
    • I'm not a political scientist, but I easily believe that public confidence is a meaningful and desirable quality in any government program, and I can easily believe that confidence-inspiring programs are, all things being equal, better than those that don't inspire confidence. With that in mind, the FDIC, the the US Dept. of Treasury, and others, could use these markets to predict the confidence-inspiring factor of proposed bailout packages. Among those that roughly tend to distribute moneys in the same ways, we might prefer the ones that the public votes "will fix the financial crisis by the end of YYYY". This concept can broadly be applied to any proposed program. E.g., "If immediately implemented in full, Pickens plan will improve our economy, national security, and environment in a cost-effective manner by 2012." Granted I would hardly advise *anybody* to rely on the prediction market to choose between alternative programs except in the extremely narrow case that the programs' expert-predicted outcomes, and their methods of achieving their results, are highly similar.
    • U.S. government agencies fund a small galaxy of projects, some of which are overlapping, and many of which never come to fruition. One person I've talked to spent more than 15 years working for contractors for a sub-cabinet agency, and not one of the half-dozen or more projects this person was assigned to was completed. These contractor staff knew which projects were more likely to be completed than others. Therefore we propose a market aimed at predicting which projects will be completed, versus many others that are cancelled after wasted spending.


  • Deciding Whether Particular Policies or Decisions are Politically Viable.
    • I wonder if there's a prediction market for nominations to federal government positions, like federal judgeships and cabinet appointments. If the public could make their views known that candidate X has sufficient support from the public, perhaps the current, overly-hostile environment surrounding confirmations could be alleviated somewhat. Of course, the extent this is true is dependent on something I don't quite understand, which is the prediction market's ability to alter the very prediction it is "about." In other words, will the bettors just be betting on whether or not the current, hostile Congress will be able to confirm potential nominee X? If so, that's not very useful. But if the prediction market could alter things somewhat, so that X is more confirmable since members of the general public have more input, then it could be useful.
    • Have an internal prediction market limited to Congress and congressional staffers that would sell contracts on the likelihood of passage of particular pieces or legislation, judicial nominations, or other legislative actions. For instance, a market could be put in place for just the Senate and its staffers as to whether Judge Hamilton will be voted out of the judiciary committee within a specified timeframe (e.g. 1 month) and whether he will be confirmed to the 7th circuit within a specified timeframe (e.g. 2 months). In contrast to a public market, such internal markets will likely be more accurate since virtually all the relevant information is held by these few individuals. Such markets would ideally assist legislative leaders in setting legislative agendas by providing information as to whether it is sensible to even schedule a vote on a particular legislative action.
    • A second prediction market idea would be to have a market identical to the legislator market proposed above except that it would be limited to members of the political media. Having both markets would enable (1) a comparison of the reliability of media (i.e. how much media market prices differed from legislator market prices), and (2) the direction of causation between media stories and legislator perceptions (i.e. whether media market prices lead legislator market prices or whether legislator market prices lead media market prices). If there were also public nationwide markets, further comparisons could be made as to how far apart public perceptions are from media and political perceptions. The information that can be gleaned from comparing prediction markets covering the same subjects but with different participants could prove quite interesting.


  • Aggregating or Obtaining Information Useful as an Input in Policymaking.
    • I think we should use prediction markets to determine the solvency of the various financial institutions for which the government is currently performing stress tests. By utilizing these markets (which can more accurately reflect the binary condition of solvency than can equity markets which include "option values" for scenarios where solvency emerges through some other means), the government can use the markets to evaluate whether these institutions are solvent. Any "stress test" must make a number of assumptions about the future state of the world and financial markets and a liquid market of educated investors would be more effective at making these assumptions than any one body.
    • The government should use prediction markets to assign an initial price on carbon dioxide under a cap-and-trade regime. By canvassing the market through a prediction methodology, the government would reduce initial price volatility and mitigate the amount of money that it "leaves on the table" in selling the first batch of credits.
    • The current outbreak of swine flu caught many people, especially the Mexican government, off guard. Would it be possible to use prediction markets to help determine when a pandemic were likely to occur? Could they help determine where to devote resources in the most efficient way, and to the areas that need it most? Or is health care fundamentally different than other fields, making government intervention fruitless?
    • My proposed use of prediction market is to fix common misconceptions about crime rates. It was shown that usually people over-estimate the current crime-rate (thus supporting a "law and order" policy suggestions). I think we could use prediction markets to ask people and policy makers to assume how the crime rates will increase or decrease in their area. I think it will flash out misconceptions (if such do exist) and will contribute to promoting better-informed policies.
    • Maybe we could use prediction markets to predict what the next stage of our energy market will look like? For a question on energy efficiency, we could ask what the most popular/wide reaching advancement in energy efficiency will be in 5,10, etc. years OR which technology will be responsible for cutting the most carbon emissions in the year 5,10, etc. years from now? hybrids? electric cars and the infrastructure to support them? zero energy or highly efficient homes? other? For a question on alternative energy, we could bet on which source of energy will be of highest use in 5,10, etc, years: wind, nuclear, hydroelectric, solar, etc.?
    • Prediction markets could be used to predict threats to cyber-infrastructure. The cyberwarfare idea is overhyped, but cybercrime is very real, and often, specialists and hobbyists both can see which technologies are exploitable without actually having a concrete answer as to how to exploit them. If the spread of new technologies were accompanied by a market in which people could make bets as to where the next major virus, worm, or targeted installation were to take place, the public and private sectors could concentrate their research dollars on addressing concerns with that code. The major problem: ensuring accurate reporting of "success" cases in order to properly reward winners: companies who have fallen prey to attack are rarely eager to share that information.


  • Prediction Markets as a Partial Substitute for Elections.
    • So here's my use of a prediction market by gov't - as an input for a "middle ground" solution between appointing and electing state judges. AFAI understand it, the basic impetus for electing state judges was that it was thought (god knows why) that the judicial system should better reflect prevailing public sentiment in the state and/or that state judges should be more accountable to the people. Yet as the recent Caperton case shows, there are some problems with judicial elections. My proposition would be to use an appointment system combined with a prediction market, which would give the governor (he's the one that appoints state judges, right?) a way of determining whom the public would like to see in office. If he rejects the public choice, he would have to face them at the next election, giving the public some credible clout (ie. the prediction market would not be meaningless) in determining the judicial choices. This would follow a standard winner-takes-all market structure.


  • Deciding Whether to Investigate Internal Government Wrongdoing.
    • I think there should be markets for investigations of government officials for corruption. I think Ted Stevens provides a good example of this. Despite being ultimately vindicated, I think quite a number of people believed that he was crooked and would have been willing to wager that he’d eventually be investigated. The idea being that there exists wisdom amongst the electorate and contractors of which politicians are ultimately more crooked than others. The main reason why people might not be willing to bet against someone like Stevens is that he’s too powerful to be taken down. Yet were Inspectors General and others willing to take advantage of prediction markets, that might change. So a market that said “I believe that charges will be brought against elected official X” would reflect more than anger from party opponents, it would be a way for a distributed understanding amongst the electorate that some official is corrupt to have an impact on information gathering that could lead to a prosecution. It would also make conspiracy theorists pay for their ideas. A market for “I believe Obama was not born in the US and ought to be investigated” would result in a wealth transfer from the crazies to those willing to bet against them.
    • Using a normative prediction market to determine the desirability of investigating or prosecuting members of the Bush administration (or any administration) for war crimes. A prediction market might be desirable because it would present a picture of how the average legislator would view such actions, isolating the extent to which they would be perceived as purely ideological.


  • Prediction Markets for Law Firms and Litigants.
    • Law firms have a hard time making human resources decisions. Sometimes their summer classes are oversubscribed, sometimes undersubscribed. Sometimes they are too litigation-heavy, sometimes too corporate-heavy. Sometimes a lot of people in a class take a year for a clerkship, other times only a few. What about using prediction markets to help with recruiting decisions. There are many different possibilities. You could think of a law firm opening a prediction market to its summer class to get a sense of how many may end up doing a clerkship, etc. You could have a prediction market open to all of a firm's associates to get a prediction on what the firm's attrition rates will be in the upcoming years. You could have a prediction market open to all law students from top 20 schools to get a sense of whether litigation or corporate law is in vogue, whether students are leaning towards firm jobs or government jobs, etc.
    • How about a prediction market for law firm layoffs? The real decision-makers would never participate, but lowly associates probably would, and in aggregate would have a lot of useful predictive information. It would work particularly well for the firm world since it's component firms are so readily comparable, but in principle it could apply to any industry.
    • Using prediction markets in high-stakes commercial litigation to help parties determine both outcome and expected damages in potential jury trials. This scenario would be limited to big cases, where a critical mass of people who are knowledgeable about the facts of the case (and perhaps who could also be jury members absent this knowledge - which would exclude people with substantive ties to either party) can participate in the market.


  • Why Prediction Markets Won't Work.
    • I'm very skeptical of the idea of using prediction markets in law and government. I'm fixated on two main objections: (1) if everyone knows that government decisions will be based on a prediction market, an interested party could and in most cases would simply invest lots of money in his desired outcome and thereby effectively buy policy; and (2) even apart from the issue of interest group influence, I don't see how these markets can be depended upon to generate clear (how to translate a price or a percentage into a yes or no?) or reliable (how to attract informed betters and exclude uninformed ones?) answers. Working from this perspective, I was not able to come up with any aspect of law or government in which I could imagine use of a prediction market being a good or even feasible idea. I have therefore decided instead to give an example of a bad idea for use of prediction markets in government: FDA approval of pharmaceuticals. I think this example illustrates both of my objections. Drug companies desiring approval would use their considerable financial resources to move the market in their favor, and other potential market participants would lack both the information and the expertise necessary to weigh in in any meaningful way. I have trouble imagining casual betters reading through pages and pages of witness statements as suggested in the Predictocracy small claims adjudication example, and I find it even less likely that a better would be interested in perusing the lengthy and technical records of clinical trials. Though this example is perhaps extreme (USPTO issuance of patents would play out similarly), I think the problems it illuminates are endemic to any attempts at basing law and government decisions on prediction markets.

Class Session Recap

The class got underway with a brief presentation by Professor Wolfers, setting up discussion, and a response by Professor Fisher. After a response to that by Professor Wolfers, we opened up discussion and questions to the class. Below is a summary and outline of some of the major themes and issues that drove the discussion.

Empirics and Design

As Professor Wolfers noted, and discussed more fully here, whether and when prediction markets generate accurate forecasts is an empirical question. Their efficacy should be measured comparative to the available alternatives, not comparative to a baseline of perfection. And they should be measured in a context-specific way. The markets are unlikely to be very effective at predicting low-probability events, for example, but may be better as a corporate management tool.

Many of the toughest questions surrounding prediction markets are questions of design. Though design problems can be resolved, it may be easier or less costly to resolve them when applying prediction markets to some circumstances but not to others. A few of the design problems the class discussed are as follows. First, it may be difficult to create a contract with winning conditions that map onto desired policy outcomes. A betting market about which team will win the World Series is easy to formulate; it may be harder to reduce predictions about crime patterns to a contract, especially because of difficulties of measuring the underlying data or outcomes. Second, the markets may be subject to manipulation by the most interested parties. Third, markets rely on the participation of the uninformed, and so attracting those people is key to success. But since the markets also rely on punishing the uninformed, it becomes harder to recruit participants. Prediction markets may work best where biases that might lead the uninformed to bet are common, but mild.

Finally, some noted that given the recent failure to regulate economic markets, with attendant disastrous consequences, it seems especially questionable today to increase reliance on market mechanisms like prediction markets or other kinds of futures markets.

Applications to Legal Questions

We intended the class to focus in a rough way on how prediction markets might be useful for government or law. The upshot of much of the discussion was that those uses might be limited. Prediction markets work poorly in situations where information is secret and the information holders have strong reasons to keep it secret, where participants are subject to many biases, and where the existence of the market may itself affect the policy outcome (Professor Fisher likened this last problem to the Heisenberg uncertainty principle). These characteristics may obtain with respect to lots of legal issues. At the same time, problems like policy feedback loops may be solved with the right kind of design.

Tradeoffs With Democratic Values

More generally, even if prediction markets work well, they may sacrifice other characteristics of decisionmaking that we generally value. Professor Michael Abramowicz has proposed using prediction markets as a substitute for legal decisionmaking by judges or juries, but adjudication may be a desirable mode of decisionmaking for non-instrumental reasons. Prediction markets may crowd out public deliberation and other types of decisionmaking that promote community rather than just the aggregation of individual bits of knowledge. At the same time, it may be possible to come up with ways to use prediction markets that retain the benefits of discussion -- the class suggested that the deliberative aspects of making a decision might center on what values or metrics the community wishes to maximize, and the markets will help them determine whether they've reached their goal.

A recurring example throughout the class was the possible use of prediction markets as a replacement for hiring discussions at faculty meetings. The market would predict, for example, the likelihood that a candidate would become a productive scholar; the decision could then track that or other metrics directly (or, alternatively, it might be a useful data point at the faculty meeting). On one hand, prediction markets are unlikely to be able to predict all the information relevant to faculty hiring, like whether some would be a good colleague. And the information about potential productivity, especially in small fields, might be relatively thin. On the other, prediction markets in such a context may actually help avoid some of the recurrent problems with deliberative systems -- namely that they tend to marginalize voices who are afraid to speak publicly. That problem may have unwelcome distributional consequences, and so in some sense prediction markets may do even better than democratic deliberation, on its own terms. In the corporate governance context, it was pointed out, prediction markets help give voice to lower-level workers whose ideas or views are crowded out by middle management. Of course, prediction markets may tend to over- and under-represent certain types of people as well.

Teacher's Guide

Evaluation of the Class

Suggestions for Future Iterations

I. Conceptual Basics First

How prediction markets work may not be intuitive to the non-economists in the room. For example, the idea that, in a market with sufficiently high volume, random bets or bets attempting to rig might actually make the market more accurate (by serving as a subsidy and creating a financial incentive for investors with actual knowledge to bet even more) seemed difficult to grasp to many students. So before class--and, most importantly, before any simulations, the students need to be briefed on how a simple market like Intrade actually works. Here are some basic principles that students should understand before getting into the articles or simulations. First, prediction markets differ from voting! They allow the investors to indicate their confidence level (on the analogy to voting, you might think of it as strength of preference, but for an outcome's likelihood of occurring rather than the desirability of it occurring). In fact, it is important to emphasize (as obvious as this might seem) that a rational investor will be just as willing to bet on outcomes she says as revolting as outcome she sees as favorable. Second, prediction markets avoid the bottleneck problem. That is, information does not have to be screened through any layers of where decision-makers may have a non-truth-tracking agenda. This is especially important in creating internal markets in organizations. The whole advantage is getting unfiltered access to people "on the ground," which explains (as Prof. Wolfers pointed out in our discussion) why they can be threatening to hierarchy. Third, prediction markets are only as good as the information that goes into them. Prof. Wolfers had to emphasize this point repeatedly. The relevant, comparative question in evaluating prediction markets is always, is this market better than the other means we have available for collecting this information? Often all prediction markets can do is give an unbiased account of the conventional wisdom, but that is still useful! When markets predicted Obama to win the New Hampshire primary, it was not a fatal criticism of their usefulness. The relevant question was, did they outperform polls? The answer to that is yes, they generally do (see confidence level explanation above).

Retrieved from "http://cyber.harvard.edu/iif/?title=Prediction_Markets&oldid=2922"