Summary
This paper presents the results of an experiment comparing the effects of social, financial, and hybrid incentive schemes on worker performance in an online labor market (Amazon's Mechanical Turk). Overall, workers performed better than chance, but performance varied considerably depending on task difficulty. With regard to the incentives, we find that asking workers to prospectively think about the responses of their peers -- when combined with financial incentives -- produced more accurate performance than a control condition. In other words, workers performed best when their payment was tied to their ability to anticipate the answers their fellow workers would give to the same questions. Notably, workers in India and workers with lower web-browsing skills performed worse than their peers irrespective of the treatment condition. We attribute these variations to a combination of the behavioral and institutional factors.
Acknowledgments
This project benefitted from the participation of a number of people at the Berkman Center. In particular, Professor Yochai Benkler provided early guidance and support throughout. Jason Callina and Anita Patel designed the Not-a-Number web application that handled the data collection for the study. Eszter Hargittai provided several survey questions on Web skills and permission to use them. The authors also thank the Ford Foundation, the Ewing Marion Kauffman Foundation, and the Law Lab for support.