Day 3 Predictions

Daniel: My guess is that three issues will be focused:

1- labor rights – workers in UHC are not attached to a safe work environment, do not receive any fringe benefits, health care, etc., and as of yet there are no unions for Turks and the like. It is quite easy to see homeworkers as nonworkers, and to build digital sweatshops.

2- workers’ new expectation of complete anonymity, that go way beyond privacy demands in regular work environments. Hopefully ethical issues concerning this faceless workforce will be discussed, as well as its potential identity and community feelings (taking into account that, unlike bearers of formal jobs, UHC workers have shifting numbers, not social security ones). Still on this topic, I expect debates about people willing to perform otherwise shameful tasks, and about the opportunities for children, sick or unfit workers in general to work / be worked.

3- the use of UHC for complex, creative tasks, analyzed in conjunction with a look at the economics of commoditized labor pools. Resulting discussions could examine quality control and its costs, and proper design, necessary to unleash creativity and demand more than repetitive, boring tasks from fellow anonymous humans. On that note, it is nice to see that, as scientific experiments with Mechanical Turks become more popular, academic attention is drawn towards the problematic incentives in the platform’s most common setting (low payment + repetitive tasks), which encourages Turks to finish HITs as fast as they can, at the expense of proper comprehension of the tasks.

Andrew: Since at least some of our guests tonight are "creatives", I hope to hear some discussion about the relationship between full-time freelancers and websites that crowdsource complex, creative tasks (e.g. Worth1000, iStockPhoto). At a Berkman lunch last spring, Jeff Howe cited a study that showed only 4% of iStockPhoto sellers derived their primary income from the site. As the site and its peers begin to dominate the market for stock photography, what happens to the livelihoods of those who depended on stock photography for a living? Protectionist worries like this parallel those about outsourcing more generally and are vulnerable to the same counters about progress and efficient markets; I hope some of those arguments play out tonight.

My wish list for the session: discussions of solutions / tools such as Turkopticon, a Firefox application designed to identify and expose “shady employers”.

Michael: Since two of our guests have used UHC for artistic projects, I expect one or both of them will respond to some Cult of the Amateur-style criticisms. I am especially interested to hear whether our speakers think UHC improves the quality of creative/design work that can be accomplished or aggregated from UHC or whether it represents a possible step backward from looking to established professionals for this kind of work. Based on the backgrounds of the speakers, I would imagine Bjoern Hartman and Aaron Koblin will argue that UHC presents the possibility for improvement over the previous paradigm of established professionals. I imagine one or more of the speakers may believe that UHC doesn't really represent a change in quality of such higher level work, but just a difference in kind.

I don't know if any of our guests plan on discussing this topic, but I would be especially interested to hear their perspectives on UHC's effect on the morality of work done. Anonymity on the internet can sometimes remove users' filters of social convention and politeness. So UHC might make it easier for users of Mechanical Turk or CrowdFlower to do work (either voluntarily or unknowingly) that would be morally dubious. There also seems to be a lot of noise in the Mechanical Turk system especially -- a lot of scams rather than true HIT tasks. A possible solution would be to use the existing Mechanical Turk or CrowdFlower functionality for users to rank the morality of the various tasks and provide marginally greater pay or benefits for tasks with higher moral/utility rankings. But I would be interested to hear to what extent our guests think this is a problem.

Ramesh: I predict that the founders of human computing websites will be more focused on the technology and potential of the websites and may have a blind spot for the legal issues that may be raised by UHC (applicability of minimum wage and other laws) while as law students, we may naturally focus on the legal issues implicated.

Tyler: I agree that Lukas is obviously very excited about the technology and the potential of his website and similar websites. However, I believe that the "blind spot" that Ramesh predicts will be more of a "worrisome spot". I think Lukas will be extremely worried about the legal implications concerning products in his space. I think a lot of this worry is justified since it is likely unsettled or unclear law that even an experienced employment lawyer could only guess about, and it I think Lukas may be kept up at night by the potential for new or clarified law to derail his project. I suspect that Lukas will have attempted a risk analysis of his business plan with respect to legal risks, but have had great difficulty because of the challenges associated with assigning probabilities and magnitudes to the risks that he faces.

Alternatively, perhaps the founders of UHC websites will see them simply as a continuation of current trends, especially the increasing numbers of contractors in the labor force of large companies and governments and the outsourcing of call-center (and increasingly higher-skilled) jobs overseas. Does UHC present any problems that are different from the current trends? What role can employment and labor law play in a world where increasing numbers of workers are "independent contractors" or even Mechanical Turks? Will technology re-enact Lochner?

Franny: Given the guest list, I diplomatically disagree with Daniel (and agree with Ramesh) and would expect these guests to address the positive potential and advantages of human computing applications into business, arts and culture, as well as the benefits available through this new type of labour force with built-in autonomy. As libertarian as I may be in my views, I agree with Daniel that there is a real possibility that UHC can develop into a last resort for unskilled workers to earn income in order to survive. I just don't think that the negative aspects will be the focus of today's session.

I would also be interested to hear our guests' thoughts on whether UHC can be applied to tasks in which sensitive information is involved, and if so, how could private content be protected?

Jason: Totally agree with Franny here. I was at first somewhat surprised that in the talk that Lukas gave at TechCruch 50 there was zero discussion of any of the legal aspects of this (no one asked, "Um, do you have to withhold taxes from the workers?" or "What if it turned out that someone was a child?" or "Won't your business model be ruined if it turns out you have to pay taxes for not providing health insurance to these people?" or anything along those lines) - but, of course, I forgot that I'm a law student and that's not the lens through which they are viewing this technology. Faced with a room of (mostly) lawyers, these questions will obviously come more the fore than they were there, but I suspect that the considerable advantages and potential of this type of work will dominate the discussion.

Reuben: I wonder how feasible it is to actually make any real money as a mechanical turk. All of the tasks I tried took me at least a few minutes to read the instructions and then a little bit of time to actually perform the task. I suppose if someone did the same repetitive task over and over it would cut down on the amount of downtime and you could more quickly make some money, but at $.02 a task (which seems to be a going rate), even if you spent as little as 30 seconds on each HIT, it works out to $2.40 an hour. You need to pay about $.07 a task to add to the basic minimum wage. I see how the employers/taskmasters benefit, but aside from the novelty, I don't see any real benefit for the turkers. I'd love to hear from Lukas what the average worker makes doing CrowdFlower tasks.

Hector: Yes I'm also very interested in seeing some numbers. Do they have any system for flagging usage? First, setting aside fair wage issues, does anyone do this for 40 hrs a week? If so, then many of our worries can be concretely identified and addressed. Check out this article showing relative sweatshop rates in different countries: Sweatshops and Third World Living Standards

Elisabeth: This isn't scientific, but here's a personal finance blogger on making minimum wage on AMT. He thinks it's possible if you take qualification tests and can write quickly. In my limited experience, I tend to agree that it's possible (and recall that commuting/wardrobe/etc expenses are nonexistent).

Amanda: I think the incentive structure around these sites (non-monetary incentives) is really interesting because I still don't know where to draw the line between UGC and HITs - for example, when I post a review on Yelp, I don't own it and they are making money off of what I post (indirectly), and yet they don't pay me and I don't expect to be paid. I saw the Crowdflower presentation live this year and the response I think they'll have to the minimum wage question is that this work is all on-demand, you can do the work whenever you want to - that opt-in element makes it somewhat different than a traditional employer-employee relationship.

Elisabeth: A related question is whether the popularity of paid-crowd sourced material will drive out unpaid work like Yelp. I don't think Yelp could be replaced by a series of HITs (what entity would pay for it?), but if people can immediately monetize their time, they may be less willing to do free tasks.

Juan:

By doing quality control and tracking the quality history of workers, Crowdflower moves one step closer to a real employer. How will it and other human computing websites deal with labor law issues, such as employment relationship, jurisdiction conflict, non-compete agreement, anti-discrimination, disability, leave time, wage and hour requirements, and etc. Also, building up workers' career path, balancing between monitoring and privacy intrusion, disclosing information for workers to evaluate the moral value and giving them the opportunity to opt out, shall be new problems in the cyberspace. Besides, this paid work on-line may have an impact on those contributions without payments. How will we address this issue to make sure people will have incentives to embark on free works.

Another thing I want to hear is whether UHC will develop verticals like the traditional industries. How will it develop those verticals not suitable for on-line outsourcing per its nature?

Sheel: The 'quality history of workers' that Juan alluded to is particularly interesting to me. Crowdflower seems to integrate the 'reputation' of workers in ways that Mechanical Turk doesn't do for its HITs. I'd like to hear Lukas's thoughts on this mechanism---how easy is it for users who have a low reputation to just start all over, taking the good (long history with website) away from the bad (similar to the online reputation possibilities that JZ mentioned in his book)? Does CrowdFlower only track user names in this manner? Also, would Crowdflower ever consider having tasks cost a higher amount for the vendor with the stipulation that only those with a certain reputation will be able to perform them?

My prediction is that the reputation system will be naturally brought about in conversation when Lukas is describing his company as his 'competitive advantage'. He will probably say that there isn't a need to have tasks cost a higher amount for the services of those with a good reputation (if the question is posed) because the service is inherently reliable as it can recognize spammers from truly competent workers.

Sharona: I agree with Franny and Ramesh - I think the speakers will generally focus more on the positive contributions these types of sites can offer - the innovation from crowd sourcing, the efficiency, the specialization - and less concern over the legal issues. One thing I would like to hear is whether they think these tasks will continue to be performed by US residents, or how quickly they will also be outsourced to English speaking (or non-English speaking) people across the world looking for menial labor especially. Another thing to consider is how or if people could actually make a career out of doing tasks online, or whether it is just something to supplement another job. How will things like health benefits or insurance policies come into play for these kinds of workers?

Yosuke: I agree with the majority opinion, I think guests will not really focus on negative aspects of ubiquitous human computing, such as potential problems of child labors in developing countries, as JZ stated in this paragraph in his book. While I guess they will address some amazingly positive aspects of UHC.

Vickie: My guess here is that the "creative types" as Andrew called them are going to be psyched about the creative potential of mechanical turk but less satisfied by the "mundane tasks" that are fueled through the programs (as stated by Aaron Koblin in this video http://vimeo.com/3199933). I believe Koblin and Hartmann are not going to be happy to hear as much about CrowdFlower. Koblin, specifically said in his presentation in this video that he limited the amount of times his contributors could use HITs for his art projects. Unlike CrowdFlower Koblin isn't looking to use Turkers as steadfast blinded workers. He seems very conscious of all the Marxist qualms with the program.

Hector: From my layman's perspective, Marxist qualms can often be settled by magical economic arguments, but you're right about certain moral problems stemming from the mundane nature (if you begin talking about the dignity of labor as a human experience). On a less lofty note, I wonder if these fellows will present (explicitly or between the lines) a picture of what they imagine the average turker to be. Is it a net-savvy hipster from Portland who likes to submit cat stories? A stay at home mom? A child in a third world internet cafe? Ruben's note above raises some concerns, I'll talk more up there.

Lien: I also agree that the speakers will try to keep it positive. I'm however interested to hear whether these "human computing companies" ran into law suits already because of legal issues, both in the EU and other continents. Further, did they already got (negative?) reactions from governments, human rights organisations, labor organisations, unions, etc. . Also, if a human computing company would be sued by a turk, does it involve the "requester" (as the "employer"?) in the lawsuit? Speaking to my colleagues in Europe, none of them had really heard of this mechanical turk fenomenon. However, I'm sure that taking into account our very strict labor law that highly advantages employees, and strong influence of labor unions, labor regulations will become a huge issue.

Emily: I hope all of our guests address issues of transparency. Why didn’t I know about mechanical turking until I was required to read about it for this class? There’s information about the program on Amazon’s website, but how would anyone know to look for it? We can hold journalists responsible (and hopefully, we won't need Jacob Riis again) for this stuff, but apart from existing laws, how much can (or should) companies be doing to bring these relatively new labor practices out in the open?

Bruno: I would like to hear Lukas' opinion about the possibility of CrowdFlower's workers having the same labor rights that traditional workers in the industry have. Would this mean the end of such service - both b/c of the potential excessive bureaucracy that could be required and an eventual increase in the service's cost? My guess is that probably not, b/c the system seems to present advantages even with such regulation - such as easy scalability and increased flexibility of working hours for instance.

Elisabeth: CrowdFlower collects huge amounts of data on its users. I'd be interested to know whether the users can take that data with them if they decide to work on say, Odesk (a higher-end UHC service). In other words, to what extent can workers build a CV and advertise the quality of their work? One of the things that struck me about Amazon Mechanical Turk is that they give the workers almost no access to their own data. I'd also be curious more generally to know how much contact Lukas has with CrowdFlower workers--are they individual relationships, or anonymous statistics? And what are their demographics (for instance, this strikes me as a potentially hugely popular among stay-at-home-moms), what do they like about the service, and what are their main complaints?

I'd like to hear everyone's opinion on whether companies ("requestors" or whatever else they're called) should be able to remain anonymous. This seems to me to allow a lot of shady practices, but it might be key for, say, InnoCentive's scientific questions. It's a concern on the "creative" front, too. As I understand it, the sheep project didn't disclose the purpose in the initial requests, and some Turkers were startled and/or upset to find their sheep being turned into someone else's art project. So I'm more interested in disclosure than whether AMT is a "good" way to produce art--I just see it as part of a busy ecosystem, and it's not like it's driving out professional artists (although istockphoto may be).

One thing I notice on this page and our mailing list is that we're lumping a huge variety of sites under the UHC label. It would be valuable for the class, or maybe just the UHC group, to develop more of a taxonomy.