Class 3

'The Human Element'

• Value of Editing. In Soylent, the authors ipse dixit assert that the waning phases of the editing process--where the author cuts her work--is of little to no value. That assumption seems assailable. There certainly is value--both for the author "personally" and the quality of the work--in performing these last edits. How would we assess a similar claim in art or music? Do we think an algorithm or individuals unfamiliar with the work or artform generally could 'edit' music or paintings in the same way? What other issues are at stake? Why might the process be similar, different?
• Expertise. The authors of Soylent touch on expertise, noting that one problem was "a lack of domain knowledge." (p. 8). This raises two issues. First, is there a way that we can solve the expertise problem? Perhaps we could have roll-over definitions for complex terms, but it seems some kinds of expertise command working knowledge of more than just definitions. Second, even assuming we could solve the first problem, could we convince specialists to use such a service?

Cloudwork Best Practices

CLOUDWORK: BEST PRACTICES

This guide is designed to increase communication between (1) designers and managers of cloudwork platforms, (2) creators and managers of applications running on those platforms, (3) companies that use those platforms and applications, and (4) cloudworkers. It suggests a set of basic principles that parties can refer to in using cloudwork.

INTRODUCTION

Cloudwork (also called cloudsourcing, labor-as-service, ubiquitous human computing, virtual work, and more) occurs when a company uses a large and amorphous group of people, connected by the internet, to find workers and get work done. Cloudwork falls into three main categories. First, a large group of workers may do microtasks to complete a whole project; the best-known platform in this arena is Amazon Mechanical Turk. Second, companies may use cloudwork platforms to connect with individual workers, or a small group of workers, who then complete larger jobs (e.g., Elance and oDesk). Finally, a company may run “contests,” where numerous workers complete a task and only the speediest or best worker is paid (e.g., InnoCentive and Worth1000). In some contests, the company commits to picking at least one winner; in others, there is no such guarantee.

At the most basic level, cloudwork requires a worker and a company that needs work. A company may simply post a request for work on its own website and handle applicants directly. More frequently, however, the interaction is mediated by a cloudwork platform that organizes tasks and potential workers. Some platforms are simply listings of tasks or workers; others play a more active role in matching pairs or resolving disputes. Additionally, various applications (e.g., to measure quality, speed, or recurrence of tasks) run on top of platforms. For purposes of simplicity, in the best practices below, we regard platforms and applications as the same.

These practices are intended to create voluntary solutions, not legal ones. American employment law was not written with cyberspace in mind, although some of the concepts can be translated. CDA 230 was written to address different problems, but likely forecloses platform liability unless it is amended. Because current legal doctrine is so ambiguous with regard to many of the burgeoning businesses using cloudwork, a best practices guide seemed most immediately useful; however, it is not meant to preclude an investigation of promising legal solutions.

Some of the problems we identify below are not unique to cloudwork. For example, it is not new that many workers are ignorant of the ultimate purpose of their work. But, as Professor Jack Balkin writes, reflecting on the emerging digital world requires a focus not just on novelty, but on salience, too:

What elements of the social world does a new technology make particularly salient that went relatively unnoticed before? What features of human activity or of the human condition does a technological change foreground, emphasize, or problematize? And what are the consequences for human freedom of making this aspect more important, more pervasive, or more central than it was before?

Our approach to this document reflects our belief in the value of answering these questions.

It is clear that we share this belief with many platforms, companies, and workers, since some of the solutions we propose below have been implemented on some cloudwork sites. Moreover, profit maximization alone may drive platforms independently to adopt some of our solutions in the future. These facts only add to the value of a best practices document. Indeed, our aim here is not just to push participants in the nascent cloudwork market toward an ideal, but also to codify, reinforce, and develop the good norms that have sprouted in that market already.

GUIDING PRINCIPLES

DISCLOSURE

• What is the problem? Workers are frequently given micro-tasks with no sense of what the whole project is, who commissioned it, or what it will be used for. Workers could end up contributing to a project they find problematic, like search engine optimization; worries about this could cause workers to leave cloudwork altogether. Additionally, anecdotal evidence indicates that workers produce higher-quality results if they know and approve of the ultimate goal of the work; conversely, anonymous micro-tasks can produce work of more dubious quality.
  • Solutions:
    • Companies:
    • Although there are valid reasons for preferring anonymity, companies should identify themselves whenever possible. This makes it easier for workers to decide whether to accept a given task, and it promotes healthy public accountability for the company. For the same reason, companies should explain the scope and goal of a task whenever possible.
  • Platforms:
    • Platforms should provide an obvious way for companies to identify themselves, and to make it clear to a worker whether a company has chosen to self-identify. Similarly, platforms should provide an obvious way for companies to identify the scope and goal of a task.
    • Platforms should provide for limited contractual relationships between workers and companies if that will allow companies to be more open about their identity or mission. The contractual regime should be commensurate with the scope and value of the task. For large tasks, companies should consider creating their own provisions for limited contractual relationships.
  • Illustrations
    • Through Platform Y, which is best practices-compliant, Company X offers workers the opportunity to perform a complex task. Company X does not want the task to be known by its competitors, but would receive a better product if the workers thoroughly understood the task and X’s needs. Y facilitates the establishment of limited contractual relationships between X and any interested workers, which then receive the full version of the task and a brief description of X.
  • Several different companies use platforms to conduct tasks. The first, a market research company, is investigating the popularity of its brand. Revealing its identity would likely change the responses, so it does not; however, it explains that the purpose of the task is market research. Another company runs a contest asking for assistance creating a certain chemical compound that it hopes to use in a cleaning product. It does not want its competitors to realize it is investigating this compound, because its use in cleaning products is novel. The company could identify the task as relating to “consumer products,” or it could reveal its identity to researchers in a limited contractual relationship. A third company uses cloudworkers to review restaurants in different cities in order to compile a guide. This company should give its name and explain the task, because that does not create competitive advantage for other companies and will not affect the work product. Finally, a university psychology department runs surveys over a cloudwork platform. Unless there is significant value in disguising the fact that the questions are part of a psychological survey, the university should identify itself. If the research thinks anonymity is valuable, it should consult its research guidelines panel about whether anonymity is acceptable.

FAIRNESS

What is the problem? Workers are sometimes paid late or not paid at all for their work. Although this problem is not unique to the digital environment, anecdotally, it appears the problem is more prevalent there. If workers assume they are at risk of not being paid, this can be problematic in that it can deincentivize workers and create poor employee-employer relationships. Additionally, even when payment happens, cloudworkers sometimes do not receive fair wages – they are underpaid relative to the value of their work contributions and their time spent.

• Solutions:
  • Companies:
    • Companies should set wages at a rate that fairly reflects the time and skill required to complete the relevant task.
    • Companies should issue payment (and, if applicable, approve tasks) as promptly as possible.
    • Companies should comply with US and international labor and child labor laws. Since platforms may be better-positioned to identify violations o those laws, companies should seek out platforms that have mechanisms for compliance.
  • Platforms:
    • Platform operators should build their systems to expedite payment to workers and any preceding approval process; limit the amount of time during which work may go uncompensated; and effectively pursue worker claims of nonpayment.
    • Platform operators should ensure there is a mechanism in place and a venue for dispute resolution, affording workers clear and reasonable due process rights. Dispute resolution mechanisms should be commensurate with the scope and value of the task, and they can include customer support representatives, online dispute forms, and/or the option to speak with dispute assistance representatives. Additionally, for large projects, the parties should be able to go to arbitration if they so desire. . Companies should be required to participate.
    • Platform operators should monitor, to the extent possible, requests for tasks that violate US and International labor and child labor laws.
    • Where the tasks hosted by platforms may be highly granular, platforms should offer guidelines and/or tools that help companies calibrate their pricing toward a fair hourly wage.

 Platforms should make use of escrow and requiring employers to have validated credit cards.

• Illustrations
  • An employee is hired to do a specific task with the understanding he will be paid for it, but midway through the task the company realizes that it is unhappy with the work. Just as an employer in the offline world would pay for a day’s work before firing an unsatisfactory temporary employee, companies that use cloudwork must also pay for already-completed portion of the task. If the company monitors the work as it is being done and does not approve, the company should be able to stop the work and pay the worker for properly logged time until that point. Companies should be upfront about how they will make decisions and then let workers select the opportunities that suit them..
  • A company is concerned that its simple tasks may be performed by child workers. The company should seek out a platform that tries to verify that workers are adults—for instance, by entering a credit card number or giving intermittent “qualification” tests. In the case of work done internationally (e.g., Crowdflower’s “give work” program in refugee camps), companies should seek out platforms that have on-the-ground representatives monitoring conditions.

EFFECTIVE FEEDBACK AND MONITORING

• What is the problem? Workers often have to choose tasks without any way to judge the quality of the task or the company; conversely, companies often accept work without knowing anything about the worker.
• Solutions:
  • Platforms
    • Platforms should build their systems to enable symmetric feedback between workers and companies. Performance-related statistics about workers accessible to all companies should be matched by equivalent statistics about companies accessible to all workers. (For example, workers could be monitored on accuracy and speed, while companies would be monitored on payment speed, worker satisfaction rate, and use of SEO/spam tasks.)
    • Platforms should institute consistent policies for dealing with repeated negative feedback and make those policies easily accessible on their platforms. These policies should include an opportunity for alleged wrongdoers to appeal any actions taken against them.
  • Workers and companies should make frequent and good-faith use of the feedback mechanisms available to them.
• Illustrations
  • Worker A competently completes a series of ten short audio transcriptions for Company B on Platform C and promptly submits her work. B does not pay A until three weeks later, and B rejects three of A's transcriptions without explanation. Since Platform C is best practices-compliant, A is able to express her dissatisfaction with Company B by giving him a "thumbs down" rating for timeliness and fairness. A's evaluations are added to B's overall ratings, which are viewable to all workers.
  • After enabling symmetric feedback in order to become best practices-compliant, Platform A receives a wave of complaints from companies. Now, whenever companies give negative reviews of workers, those workers retaliate with often-false negative reviews of the companies. Platform A attempts to solve the problem by making reviews of a given party invisible to that party until she has reciprocated that review.

A HEALTHY WORK ENVIRONMENT

• What are the problems?
  • Repetitive stress injury. There are no centralized resources for healthy posture and computer use guidance, and if a worker is injured, neither the platform nor the company has to pay workers’ compensation.
  • Alienation and isolation of the worker. The physical separation, the varied work schedules, and language and cultural boundaries all combine to separate workers. Lack of interaction breeds dissatisfaction and distrust, both of which have residual negative effects on the work product.
  • Addiction: Some types of cloudwork are especially prone to addictive behavior.
• Solutions:
  • Platforms should provide prominent links to pages explaining the risk of ergonomic stress injury and the best low-cost ways to avoid it.
  • Companies should implement any of a variety of social features connecting both the co-workers and supervisors, including social boards, rating systems (top-down and bottom-up), options for face-to-face meetings, profile-making abilities, and IM features. (Examples: Gwap's Livechat; Beextra's Activity Feed; oDesk's Work Diary).
  • Platforms could acknowledge the nature of these tasks through a warning about their addictive quality. An effective form warning might include a plug-in that underscores the amount of time a worker has put into working on the platform.
• Illustrations
  • A middle-aged man has taken up a full-time cloudwork job as a source of income and has begun to stay up late to accomplish more tasks. Over time he has become reclusive and simultaneously become disgruntled with his payment procedure and rude clients. Companies should create social features to allow this worker to release his frustrations around the proverbial water cooler, in cloudworker chat rooms. Moreover, in addition to the complaint and feedback systems discussed above, live forms of communication such as IMs should be available to discuss work problems.
  • A sixteen-year-old boy has spent over 100 hours doing a variety of small paid cloudworking tasks because he finds them incredibly gratifying. Despite his upcoming finals project, he has begun to feel symptoms of addiction. In addition to suggested age minimums, companies or platforms should make it visible to the cloudworker how much time he has spent on the service, either as a warning every time he logs on or through a mechanism that keeps a timer as he works.

REPUTATION PORTABILITY

• What is the problem? A worker may do hours (or years) of high-quality work, yet have no record of that work available when he or she decides to seek work on a different platform or offline. Indeed, although many platforms and companies keep detailed records on worker productivity, some platforms and companies will not even confirm any connection to the worker.
• Solutions:
  • A platform or company should be available to confirm, at a minimum, that a worker has done some task through or for it. In cases where the worker does microtasks for dozens or hundreds of companies, the platform should bear the primary responsibility for confirmation; where the worker devotes a substantial amount of time to one company, that company may confirm instead.
  • If a platform or a company chooses to collect information on a worker, the worker should have access to it. The worker should be able to use that data in applying for other work, and the platform or the company should be available to confirm that the data are correct.
  • Platforms should make reputation data machine-readable, so that it could be immediately transported to other cloudwork platforms.
• Illustrations:
  • Worker A has completed over 1000 five-minute tasks on Platform B for several different companies. 100 of the tasks have been monitored by the platform and 98 have received positive ratings. A should be able to note in her resume that she worked part-time for platform B and received 98% positive feedback on her work. Platform B should be willing to confirm both figures, although it need not discuss what is done with the data internally. The platform may wish to give the worker the option to drop a small number of low feedback scores, perhaps after a waiting period, to reduce skewed results. Worker A may, of course, choose not to include the information on her resume—just as workers outside cyberspace may elect to leave off certain jobs, especially short-term ones. Having the option, however, ensures that cloudwork fits smoothly into a worker’s lifelong employment trajectory.
  • An author submits a request for a book cover at a design contest site and receives 35 submissions. The author should confirm, if requested, that a particular designer submitted the winning cover. The author need not confirm or offer feedback on any other contestants, although he may choose to do so.

PRIVACY PROTECTION

• What is the problem? As cloudworkers become more prevalent a concern exists about companies sharing their employees' private information. This may include a variety of sensitive material including financial statements, social security numbers and email correspondences.
• Solutions:
  • Platforms and companies should protect the private information about their workers to the fullest extent possible. They should not sell or release the information without the express permission of the individual workers.
• Illustrations:
  • Worker A has taken surveys in a cloudsourced psychological study, and her answers express a preference for risky behavior. In pursuit of just such information, an insurance company has offered the researchers, as well as the platform, a fiscal reward for that worker's data. In accordance with these practices, neither the company nor the platform should provide the information or any worker's identity. In addition, the company or the platform may share the insurance company's name with other best-practices companies and platforms.