CrowdConf Brainstorm page

Use this page to discuss the best practices reading we did not have time for in class, and brainstorm questions and topics that we might present as a class at the CrowdConf Future of Work Conference next week.

Crowdsourcing (Human Computing) is one of the most promising technologies, which has already been successfully in many different areas (Examples: X-Prize, 99designs, Amazon Mechanical Turk) and we believe has a huge potential for the future. It has the potential to significantly shape and change the way the labor market works. That said, it also creates challenges which need to be addressed. We would love to hear your thoughts on how technology could be leveraged to solve some of these challenges:

• (1) Preserving Confidentiality in Complex Tasks. As the best practices document notes, some tasks require worker exposure to proprietary information. The Best Practices mention contracts as a way of dealing with this issues. Do we think that contractual relationships can assuage companies' fears of workers disclosing propriety information? Does the sheer volume (and potential geographical spread) of workers on a given task make enforcing such an agreement impossible?
  • Is there a way the technology can account for this problem?
  • Could the problem be solved potentially by drafting specific tasks to specific information, the disclosure of which would make the individual who divulged the info identifiable?
  • What are the costs of drafting such complex contracts?

• (2) Feedback Mechanisms and Dispute Resolution. When there is little or no direct contact between employers and employees and when tasks are completed on a one-off basis, it can be tough to encourage fair feedback or to verify a potential worker's competence in advance. Workers themselves face portability problems; a good rating on Mechanical Turk doesn't necessarily carry over to other crowdsourcing sites or offline careers.
  • Could the technology facilitate a cyber dispute-resolution forum? (What if the dispute-resolution process was, in turn, crowd-sourced?!).
  • Could the platform have a rating system that suggested a fair rate based on the type of tasks requested? There could be a "survey" that each employer fills out before submitting the task, which would calculate a suggested rate. Perhaps it could be based off of past rates, as tracked by the platform operator? (Does Amazon's "recommended" technology do this in a different form already?)
  • Is there any way to use technology to prevent abuse of feedback systems, or at least encourage people to use the feedback system in good faith?
  • Have platforms set up features to facilitate the creation of online worker unions? (See SECTION BELOW for more questions on Online Worker Unions)
  • Both this issue and (1) above involve tracing identity to an individual person for a variety of reasons (so that only those with certain characteristics can be selected, any agreements can be enforced, feedback can be given, reviewed, and taken onto other crowdsourcing sites, etc). Has the crowdsourcing community discussed ideas or otherwise brainstormed with the online identity / data portability communities? If so, are any ideas that have come out of that applicable to these issues?

• (3) Disclosure. The anonymity of cyber-space and the possibility to divide a large project into a a large number of small tasks so that the ultimate product is unidentifiable raises a number of ethical concerns. Have companies, clients and platforms alike, explored setting up or mandating an ethical commission investigating these concerns? What about a voluntary code of conduct created and agreed on by the industry as a quality management system to prevent black sheep from ruining the reputation of the entire industry in case of misconduct and as a preemptive action towards governmental regulation? --> How do you prevent a private-run "Manhattan Project" implemented through crowdsourcing and sold to the highest bidder?
  • (Zara) If project identification is important to the worker, would it help if cloudwork websites rated projects by their degree of macro-transparency and distinguished accordingly in their user interface? The ethical disclosure problem has at least three dimensions: 1) a worker's subjective discomfort with not knowing the purpose of their HIT; 2) the decision a worker might have made had they been able to make an informed choice; and 3) objective, ethical standards that should prevail in the industry. Although users can already select a HIT based on their assessment of the project, including the degree of information disclosed, organizing HITs by level of transparency could facilitate user choice that selects specifically for this criterion. By creating two portals, or databases of HITS, classified according to levels of transparency, disclosure would be highlighted as a desirable HIT characteristic as to which workers can and should exercise choice. If the database of high-disclosure HITs consistently generated higher quality workers and a reputation for legitimacy, normative pressures could encourage companies to offer more information about a task’s role in a larger project and the corresponding ability to compete in the high-disclosure market.

---

Follow-Up Questions / Further discussion points

• Online Worker Unions. Crowdsourcing's success is dependent on finding ways to engage its labor pool, whether it be through offering money or gamesque points. However, as mentioned in class and in the best practices document, there are many ways for these laborers to become dissatisfied with their work, whether it be through a lack of transparency, stress, low wages, etc. Is there a potential for a crowdsourcing labor movement in response to these dissatisfactions? As an inherently digital workforce, these individuals' attempts to share discontents and act upon them are facilitated by their familiarity with and access to online communities. However, how far will this unity go? Do you feel that workers will only offer critiques of certain employers to others or could there be the formation of unions and similar entities in the crowdsourcing world?
  • A labor movement forming around crowdsourcing would likely find itself with a least two major problems. The first is one of organization. How does one organize a very loose collective of online workers? This is difficult enough to do with workers in meatspace who have social relationships with others in the workplace. In an environment where the workers do not directly interact each other and whose only shared experience is performing the same group of $0.03 HITs, can any meaningful organization occur? A second issue relates how such a group of workers expect to avoid replacement if they do organize. These distributed computing tasks require a very low level of skill to complete and therefore nearly anyone can complete them. The workforce is designed to be made up of "whoever shows up", which makes it very easy for unsatisfied and vocal workers to be replaced. While a strike would be the typical response to a breakdown in negotiations over conditions in meatspace, can we say that an equivalent exists in virtual space? In light of these difficulties, do tradition models of worker unionization map onto digital workforces at all? If they do, will completely new organizing models and strategies be needed to improve working conditions?

• Worker alienation'. Crowdsourcing is particularly worrying in how it separates a worker from the work that she performs. In today's workplaces, whether one looks at a blue-collar assembly line or a white-collar office where discrete office tasks are divided among a staff, one still has a sense of "ownership" over a piece of a work by way of her contributions. In the virtual workplace, the worker is completely separated from the end product. The lack of transparency prevents a worker from understanding what she is creating. Quality control systems that use redundancy to ensure accuracy can make it very unclear whether a worker's contribution was ever used and therefore whether the final document reflects her work. See discussions about transparency and quality control in this document for related ideas.
  • Would it be good social policy to ensure that workers have the ability to make some sort of psychological ownership claim (which I want to separate out from legal ownership claim) to a project to which they contributed? What would such a claim look like? How can technology assist with facilitating such ownership claims? Alternatively, does the incredible granularity of the HIT make the difference in scale between the worker's contribution and the final product so dramatic that a sense of ownership would be impossible?

• Overlap of legal frameworks. Some countries have a state pension fund that is financed by a tax deducted from a worker's salary. How are these legal requirements adhered to in the realm of crowd-sourcing? How is the location/jurisdiction of the worker determined? If the company's location is chosen, how measures are taken for the worker to have access to that respective legal system?

• Compensation. Crowdsourcing appears to rely on monetary compensation, a gamelike points system, or personal gratification to motivate people to participate in these tasks. Which of these compensation forms is the most effective in ensuring a large labor pool and the best results for employers? Which (if any) of these forms will be the most prominent system of compensation in the future and which do you think would be the most ideal compensation structure for crowdsourcing in the future?
  • Virtual Currency Concerns
    • A growing trend is for companies to pay "cloudworkers" in virtual goods, rather than real money, and this raises some interesting concerns. For example, CrowdFlower (in parternship with game maker Zynga) pays around 100,000 workers in virtual goods rather than real money -- which the gamers can then use to buy things such as virtual seeds for Farmville. CrowdFlower CEO Lukas Biewald told BusinessWeek [1] that demand for virtual rather than real wages skews young: "There will be a whole new generation of kids growing up who won't really see the difference." Yet what is the difference?
    • Does payment in virtual goods attract more children to cloudwork? Does payment in virtual goods circumvent any regulations governing work for "real" currency? Do game makers who offer cloudwork risk harming players? For example, if Zynga offered a weapon in Mafia Wars for performing 10 hours of cloudwork, yet made this same weapon difficult to otherwise obtain, one can imagine that players might feel pressured (or exploited) into performing cloudwork. Have there been any such instances?

• Employment relationship". Are all cloudworkers to be considered independent contractors? Does this qualification pass the test of current labour laws and tax laws?
  • (Zara) The IRS definition of an employee versus an independent contractor, which tries to codify the common law, seems inapplicable to the cloudworker-employer relationship. See Independent Contractor (Self-Employed) or Employee? The page on independent contractors states: "The general rule is that an individual is an independent contractor if you, the person for whom the services are performed, have the right to control or direct only the result of the work and not the means and methods of accomplishing the result." [2] As Davis observes in his discussion of scientific research, cloudsourcing problems tend to be idea-based. Applying the common law definition to idea-oriented micro-tasks, which don't require specialized instrumentalities or complex services, is difficult. It could be argued that requestors control the means and methods of accomplishing the results by virtue of the nature of the task; the task can only be accomplished in readily identifiable ways. But if the "means and method" of accomplishing a result is the process of thinking, then an employer will have a hard time establishing this level of control!
  • In searching the internet I found this interesting blog post (http://behind-the-enemy-lines.blogspot.com/2009/07/is-amazon-mechanical-turk-black-market.html). I do not know if the information provided is accurate, but it raises some interesting questions: how do crowdsourcing platforms respond to the accusation of fostering a black market?
  • Is it appropriate for crowdsourcing platforms to construe a relationship between the worker and the requester? Or should any responsibility be allocated to such platforms?
  • Should the role of the platforms be more active in ensuring fairness of the working conditions and salary?
  • Crowdsourcing platforms have sometimes been depicted as virtual sweatshops. Do platforms feel any pressure from this criticism? Will increasing awareness of this perception in the general public prompt crowdsourcing platforms to adopt more stringent policies for the monitoring of working conditions?
  • Should minimum wages be granted to full-time cloudworkers? Should the maximum working hours be monitored and controlled by the platform? It has been argued that the Fair Labor Standards Act should apply to crowdsourcing platforms (Working For (Virtually) Minimum Wage: Applying the Fair Labor Standards Act In Cyberspace; 60 Ala. L. Rev. 1077). What is the platforms' response to this?
  • How crowdsourcing platforms address the issue of labour conditions in developing countries? Is the age of the cloudworker actually checked in order to avoid children labour exploitation? Should it be? And how?
  • (Anna) As a consequence of Zara's question on whether the worker will be considered an employee or independent contractor - can the worker claim some sort of intellectual authorship on the final outcome? Will that authorship suffice for copyright or patent claims?
  • Another thing is that crowdsourcing attracts different types of workers right now - the hobbyists who work for fun, but as it gets bigger, it may attract, as the posts here hint at, full-time workers. Is there a way for the current technology to make distinctions like this and farm out tasks accordingly?

• Mobile Online Devices. Mobile and closed platforms with constant connection to the Internet have been supplanting sedentary workplaces in popularity.. How has crowdsourcing taken advantage of this change or has it struggled to do so? What advantages/challenges do these mobile devices offer workers, employees, and crowdsourcing agencies?

• Recommendation System / Performance Tracking. It seems like sharing information about workers as suggested in the Best Practices document is a bit invasive for my taste, and perhaps something would have to be written into workers' contracts to explicitly allow this type of information sharing? (I'm not exactly what sure this law would implicate, but I seem to remember that when a potential employer calls a jobseeker's former employers, the employers can only share information that the person worked there, but can't reveal performance evaluations, etc.) Perhaps it's just me, but I'd feel more comfortable if companies enabled performance-tracking software, but didn't go so far as to share it with all other similar companies. (Question from Jenny)

• Encryption? In some cases, the employer might want to protect the large picture of the project from workers while still makes each piece of task workable. For example, a writer might want his article to be shortened or proofread, but not necessarily want the workers to get the idea of the article before it gets published. How to make sure that they would not be able to do so (by e.g. divide the project into extremely small pieces, or, cut off possible communications among co-workers)?
  • Secrecy in projects, however, also implicates the downsides with making discrete jobs that cover up the underlining project purpose. The Best Practices discussed some of these with micro-tasks (e.g. moral qualms with the project and lower quality of performance). Also, the employees will be isolated by such secret tasks since they will both lack a physical place to work with other employees and be cut off from discussing it in crowdsourcing chat rooms. One thing then to think about then is how do you balance those two objectives to get the best overall outcome for a specific project (thinking about desired secrecy and worker performance/satisfaction almost as two axes on a graph)? Again, as mentioned above, would developing contractual relationships to deal with this suffice?

• Quality control. As the demonstration showed last week, the author still needs to use his own judgment to eliminate many inappropriate revisions suggested by the crowd. Is it possible to develop certain methods to have different pools of people to whom different projects can be assigned? Without collecting too much private information, we can use demographic standards to differentiate the crowd (age, sex, education, etc). Then the quality of collaborative product can be raised and author’s final intervention can also be reduced.
  • In thinking about pulling from different pools of crowdsourcing employees, I’d be curious to ask about employee information verification. Basically, when recruiting crowdsourcing employees, especially when seeking out a certain educational background, how difficult is it to verify that they in fact have that background? Is misrepresentation of that kind of information more prevalent in the crowdsourcing context since it is largely faceless without personal interaction to help catch people in a lie? If so, would it be (or has it been shown to be) helped by the improved work environment ideas discussed in the Best Practices, such as chat rooms for employees that simulate a virtual water cooler?
  • A related issue is how crowd sourcing plaforms and their clients will be able to take enforcement action against workers who have engaged in some sort of misconduct (e.g. a breach of confidentiality), given the worker may well not be in the United States. Even if the contract specifies that US law(or the law of some other jurisdiction) applies, there could be real practical issues with taking any sort of enforcement action beyond termination of the worker's employment in the particular crowd sourcing platform. Are there likely to be situations where either the platform or the end client would want to take further action? What sort of further action could be taken - traditional legal mechanisms, which may be difficult in a cross-border context or are there other mechanism thst may invovle use of technology?

• Capability of Crowdsourcing to facilitate intellectual discussion/speculation. So I thought it would be interesting to take our questions about crowdsourcing (What is its future? Potential problems/dilemmas? Unforeseen benefits? etc) and crowdsource them. I put together a prompt on Mechanical Turk and solicited people to give me 1-2 paragraph responses. The results have been interesting, but disappointing. Despite being explicitly prompted to focus on unforeseen or novel consequences/views of crowdsourcing, very few responses said anything beyond a boilerplate of initial ideas. Lots of very similar responses that basically covered only what we discussed in about the first 5 minutes of class last week. What this implies to me is that the crowdsourcing mechanism I used (and perhaps crowdsourcing generally) is not necessarily well suited to thoughtfully brainstorming new intellectual issues. I suppose this may be intuitive given that I'm paying someone $0.20 for a paragraph of their thoughts. It should be noted that the average time spent on this task was 4-5 minutes. But I expected that this community of crowdsourcers would come up with at least a few unforeseen issues that us, a room full of mostly crowdsourcing novices, had missed. Here are the only real highlights, out of 12 responses, and two of them came from the same person:
  • "As you will probably find with this micro-experiment, you only get what you pay for. Which means you will not get two paragraphs of extremely well thought out ideas from anyone, simply because it is not worth their time."
  • "One problem that is already being run into is by article writers who write on mturk and then have their articles reviewed by another user. Because the proofreading only pay a tiny fraction of the actual article writing, there are a lot of blanket statement and unhelpful critiques which then get forwarded on to the writer, who feels frustrated."
  • "Crowd-source workers are increasingly being hit by scams. This includes "website testing" that is actually signing up for a legally binding service contract or services that require a credit card number, putting them at risk of phishing. Getting rid of the scams will increase the confidence of crowd-sourcing workers to take on more complex tasks."

(Sorry, forgot to log in, this is Erin) OK so to keep in mind what our goal is — we're supposed to put together some sort of agenda to talk about with the people who think about this day-in and day-out, and we have about half an hour? So it seems like we should try to focus in on a particularly compelling angle. The list above is good but can we prioritize? I really liked the point made in class last week that pointed out that if we can identify some sort of way for interesting technology to "fix" the "problems" that we see arising out of crowdsourcing, we'll have a much more receptive audience. Nothing jumps out at me from any particular section of the Best Practices document, but if we combine some of it maybe we can come up with something interesting?

• Maybe combining some of the aspects of portability and reviews with the identity movement more generally would be interesting?
• Frankly a lot of the best practices aren't super interesting in terms of the required technology — is there some other way to get them excited about a particular angle on something?
• Is there a way to frame a problem that we're particularly concerned about that will speak to them? Don't mean to make this an "us-against-them" thing — but the way that technologists think about technology is a little different from the way that lawyers do, so we want to be able to frame the issue in a way that will resonate with the audience...
  • What about praising the technology (maybe have a few specific examples), but then asking them if they've met any resistance from, or thought about, any of the potential actors who may block or alter the technology's use? Have you (the technologist) talked with local/state/federal government? Have you discussed potential roadblocks with companies that will implement the technology?
  • Maybe framing some of the 'legal' issues we've identified above in a way that demonstrates why they are important not jsut for the workers but for crowd sourcing platforms and their clients - e.g. while the welfare of the workers is clearly very important from a ethical perspective, it is also beneficial from a public relations perspective and may of the potential 'worker' protections go both ways - in some cases they are protective of the 'employer' as well.

---

Crowdsourcing Science

Jenny here: I've been reading some blog posts on crowdsourcing, and one comment from a scientist (found in the comments section here: http://money.usnews.com/money/blogs/outside-voices-small-business/2009/01/27/using-social-media-and-crowd-sourcing-for-quick-and-simple-market-research ) got me thinking that scientific research and development could suffer if companies move from hiring a dedicated team of scientists to farming all of their scientific problems out to a crowdsourcing lottery payment system (ie, first one to do this gets all of this money; the others get nothing). Honestly, worst case scenario, we'd have even fewer people going into sophisticated scientific fields than we do now, because there wouldn't be any guarantee of a stable living, and I wonder if this could really hinder the development of solutions to scientific problems or if it would limit the scientific fields to scientists who are business-savvy enough to be connected to venture capitalists, etc. Either way, the outcome could be scary.

I'd be interested to hear from crowdsourcing experts about how they think crowdsourcing scientific problems affects the quality of scientific research, and if there could be any safeguards implemented to prevent the aforementioned problems from occurring (ie -- could the crowdsourcing community fund a dedicated pool of scientists, with extra prizes going to those who successfully complete R&D tasks, or would this go against the very core of the crowdsourcing movement?)

• (Heather): I'm not sure this is a novel problem in the crowdsourcing context. I've never worked in a lab or been involved in research, but this concern about stable living and a consistent funding source seems to be pretty common (at least in the academic context, where it seems like a lot of time and energy is spent chasing down grants and competing for funding). Obviously the decentralization of oDesk and similar websites exacerbates the problem by breaking down competition into individual tasks and individual workers instead of across entire projects or teams, but it doesn't seem to me to create an entirely new economic incentive structure. Then again, I'm not an expert on this and we'll be talking to people who are.

• (Davis): I question whether crowdsourcing has the potential to displace much scientific research. Most commercially viable research projects (such as pharmaceuticals) require significant capital investments in sophisticated experimental equipment, or access to tightly regulated materials (such as dangerous chemicals or radioactive sources). There is simply no way to crowdsource around the need for a spectrometer or a chromatograph. The types of scientific problems that are readily solved through crowdsourcing will tend to be idea-based (rather than experiment-based), and correct solutions must be easily verifiable. These criteria alone suffice to tightly restrict the class of problems that are amenable to solution through, say, Innocentive. Moreover, companies will need to employ scientific experts simply to know what questions to ask (and how to divide larger problems into smaller ones), and so a significant amount of centralization will still be necessary even with distributable projects.

• • And of course much scientific research is basic research, and therefore not (immediately) commercially viable. Thus we're unlikely to see a large category of research go the way of Innocentive. Something like the Large Hadron Collider is the very antithesis of crowdsourcing; such large collaborative projects seem to be the direction physics research will be headed for some time to come. The same will probably become true for other scientific fields as they mature.

• • (As a side note, I also feel the need to add that the concern we often hear about having an insufficient number of people pursuing the hard sciences is overblown. In most fields, there simply aren't enough jobs out there to absorb the quantity of science PhDs we produce.)

• (Rachel): As an offshoot of what Davis said, I'm curious as to whether we can discern general principles of when crowdsourcing is or is not viable -- not so much in terms of Rene's question about public acceptance of crowdsourcing, but rather in terms of when it can actually be done or not, as in Davis' Large Hadron Collider example. Or what about litigation? Doc review is being outsourced more and more to contract lawyers working as independent contractors, both within the U.S. and abroad, so that seems like fair game for crowdsourcing (assuming we can get 'specialized' crowds), but it does not seem like the same could be said for actual trial or appellate practice in court. Is it merely a question of skill and experience? Given projects like Innocentive, where scientific issues requiring a lot of skill and experience are crowdsourced, that does not seem to be the case, and yet what is it about things like Large Hadron Colliders and trials that appear to be resistant to crowdsourcing?
  • (Alissa) While doc review certainly does lend itself more to getting outsourced and potentially crowdsourced, there is no reason why other more technical tasks even in the legal field cannot. The more you thinly slice a complex task, the easier it can become for a larger group of people to do. Say you are assigned to research a particular issue in the law for a memo. While the task undoubtedly requires a unique set of skills, a crowd of workers with some knowledge can identify the key words or phrases that need to be searched on Lexis or Westlaw. Then other workers can go through a list of cases that turn up and eliminate all those cases that are completely unrelated. There will come a point wherein crowdsourcing will no longer be vialbe to complete the entire task, but it could further it along quite a bit.
    • Furthermore, a downward economy lends itself to crowdsourcing tasks that may only be appropriate for people with specific skills. With the mass amount of attorneys that law schools graduate every year, it is likely many attorneys will graduate unemployed with a unique set of skills that could be implemented in specific crowdsourcing platforms. Is that the future of crowdsourcing? Where instead of hiring a firm and paying hundreds on the hour for one attorney to do work, clients can go to a crowdsource firm, where there are hundreds of attorneys looking up one issue of a case for a client and the cases that they most turn up as relevant to the issue are then the cases that the next group of turkers decide or favorable or unfavorable to the case at hand, and the next group determine why and so on and so on. It could be a similar cycle that globalization followed that began with the manufacture of basic goods and led to the the outsourcing of detailed service industries.
  • (Davis) I think one major boundary that can determine whether a project is crowdsourcable is the the degree to which a project is knowledge- and idea- focused. My examples of infeasible science projects all required some sort of "capital" good: expensive equipment, restricted materials, or massive installations. Given the need for integrity in collecting and reporting experimental data, along with the need for centralization in order to synthesize data into a result, the experimental process itself can also be seen as a sort of capital "good". Trial practice presents very similar issues in this regard.
    • On the flip side of this would be theoretical science research, which would lend itself well to a crowdsourcing approach. Commercial viability then limits the available funding, but perhaps an organization such as NIH or the NSF could distribute funds through a crowdsourcing approach with ex post awards, rather than relying upon their ex ante application systems.
  • Another boundary might be the extent to which useful contributions to the project require a broader understanding of the context of the project/links to be made across the material - this may not be an absolute boundary but there may become a point at which a project that might be technically crowd sourceable is not practically so because the time that would be needed to be invested in getting the crowdsource workers up to speed and then spent amending and editing their work at the end of the process makes it inefficient when compared to using a worker who already has background knowledge and/or who is working on several aspects together rather than one very narrowly focused task.

---

Case Study: oDesk (Rene)

• I have used oDesk a lot of times over the summer to outsource smaller programming projects for my startup to developers, mainly in India and Southeast Asia. For those who haven't used oDesk, you post a job with a budget, oDesk workers apply for the job, you can interview them and then hire one; a small portion of the overall payment might be upfront, the rest is paid at completion of the project at discretion of the employer. oDesk has standard terms (NDA, etc.) to facilitate the transactions, but I have asked the developers I hired to sign additional documentation. The biggest issue is quality control; despite the fact that there is a rating system, it is quite difficult to evaluate whether someone is able to get a certain job done or not. I really like Jenny's question around recommendation systems / quality control as and extension of point (2) above and would like to hear what technologist have in mind to address this important challenge.

I would like to hear a discussion about the general public's acceptance of crowdsourcing. As mentioned in class, our knowledge and opinions of crowdsourcing is very much a minority viewpoint. Although to us it presents a really novel and theoretically interesting development, I imagine different entities (investors, crowdsourcing employees, workers outside of the field) would view this new practice through the lens of their own interests. I would like to hear these crowdsourcing leaders discuss their interactions with these groups, either through an open question or a directed one.

• Can a parallel be made to outsourcing? When jobs are outsourced, domestic workers feel outrage and a sense that they are being cheated or dealt with unfairly. One can imagine a similar response on a micro-level to crowdsourcing, where formerly dedicated employees are let go in favor of crowdsourcing, but employers are happy because it is more economically efficient.
• If people have made parallels to outsourcing, is that even a fair parallel to make? One can certainly make an argument that the small tasks being accomplished by 100s of people are not necessarily displacing jobs on a 1 to 1 level. Also, these jobs are not disappearing from the United States entirely. If your job has been replaced by crowdsourcing, you could just become one of the crowd at that point and do tasks for a wider array of companies. Are claims of outsourcing just knee jerk reactions to something that actually has the potential to offer the same amount or more opportunities for workers?
• (Jason here): Good points Rene. Building on the question of outsourcing: is there data about the number of "displaced" jobs vs. creating of entirely new types of jobs? Projects like Soylent could introduce a new paradigm for "micro freelancing" where workers could put in just a few hours or fractions of hours per week. Yet indeed a crowd or swarm of workers could chip away and eventually replace what could be a full-time job.

---

Lessig Framework

What if we approach the best practices document with a view to Lessig's four modes of regulation, and frame our discussion of crowdsourcing in terms of which combination of modes could best achieve the desired outcomes? For example, assume a crowdsourcing application that has an architecture in place forcing disclosure pursuant to the best practices model. With such a system in place, norms may then provide the best solution to the fairness problem: workers would share information about employers who are known to violate users' sense of fairness in worker forums, and discourage others from doing the work. Or workers could "strike" by making a concerted effort to accept all that employer's tasks and intentionally perform poorly, thereby obstructing completion of the disfavored company's assignments (sort of like 4Chan meets Mechanical Turk).

• (Alissa) Norms would probably be the single best regulatory mode. In addition to having crowdworkers review and give feedback on the quality of submissions provided by other crowdworkers, they could also give feedback on the assignor on categories, such as amount of information they are given about the project they are completing, the quality of the directions, and their sense of how their work factors into the larger project. A rating system could develop wherein employers receive an x amount points for their ability to communicate to the worker what the “whole” of the project is and how his part factors into that whole. Depending on the architecture of the platform. Previous turkers that worked on the project could leave feedback of the kind of work they have already done to give new turkers an idea of what has happened thus far in the project.
• Law could also be a powerful regulator. Platforms with a best practices document in their terms & conditions that list crowdsourcers rights could be enforced as contracts and when certain provisions are violated, such as nonpayment, crowdsourcers could sue for breach of contract. Other regulatory laws could form ensuring fair wages, disclosure of commissioning party, etc.

On a related question, could a crowdsourcing approach solve any of the crowdsourcing best practices problems? For example, is there a way to implement a feedback and monitoring system whereby the quality of a submission is judged by crowd workers?

---

Mobile Online Devices

This probably won't appeal to the tech crowd as much, but I'm interested in how we will deal with the mobile devices problem mentioned above. If you're working from your phone, which jurisdiction's labor laws apply? Is it the state where your phone was issued, the state where you live, or the state where you're using the device?

• How can companies protect themselves when they want crowdsourcing work done, but want to avoid certain states?
  • i.e. if I want to avoid CA labor laws, I can block CA IP addresses, but that wouldn't necessarily prevent me from taking my TX iPhone and working for a crowdsourcing company when I'm in CA.
• What about international workers?
• Should we use architecture by forcing workers to log in through a mobile application that requires assent to certain terms (i.e. I'm not working in CA)?

---

International Issues

This may ultimately be moved to the employment and labor section, but I wonder if it would be worthwhile to think about how our views change depending on the country. Also we can apply all the issues under mobile online devices to the international context.

• Could/should companies take it upon themselves to pay higher wages to workers in poorer countries? If they're not legally responsible, are they morally obligated to do so?
  • Is there a way to structure the technology to encourage or incentive fair labor practices? This may play into feedback mechanism we choose?
• What about international labor standards? ie rights of the child
  • Could companies require workers in other countries to certify that they are acting freely and are not underage?
    • Now were are moving to a place that seems to fit squarely in Lessig's framework. How do we identify individuals on the computer?
      • I know crowd-sourcing is powerful and fascinating because of the ability to draw on power from all over the globe, but might we face legal restrictions that would require sourced material to be done in specific locations? You might imagine a "crowd-sourcing lounge" where people could come and have coffee, surf the web, chat, and work (on all different projects or not at all). This place would allow a reliable verification system while possibly retaining many of the benefits of crowd-sourcing and possibly providing benefits of a work environment. (Think: "crowd-sourcing cafes.")
• If other countries' governments start developing tasks, should the US government take an interest in their activities? How does this fit into normal US government involvement in cybersecurity? (considering current issues, this seems like it would be low on the list of priorities)
  • This raises questions of how the technology is used? Are there norms in place among technologists about how this stuff should be used? Are there code-based restrictions that could reduce the possibility of "nefarious" uses?
• Are there any lessons that we can learn from the controversy over outsourcing to countries like India? Is this different than traditional outsourcing?

(Julia)