Shaping Consumption

Event Description

Narrative manipulation issues - such as manufactured consensus, brigading, harassment, information laundering, fake accounts, news voids, and more - are increasingly well-documented problems affecting the entire social ecosystem. This has had negative consequences for information integrity, and for trust. This talk examines the ways that these same manipulative tactics are being deployed on Amazon, which is now the dominant product search engine and a battlefield for economically and ideologically motivated actors.

Notes from the Talk

The issue of narrative manipulation online is a major concern for social media platforms like Facebook and Twitter. In her recent talk, Renee Diresta explained how such behavior has become equally alarming on Amazon. In Amazon’s increasingly crowded marketplace, reviews have become more important for sellers. As such, companies now look for ways to “game the reviews.”

Narrative manipulation can take many forms, including manufactured consensus, brigading, harassment, sock puppet accounts, and news voids. In her research, Diresta has found the creation of positive consensus about products on Amazon is sophisticatedly organized. For example, potential reviewers belong to Facebook groups (often with names like “deals” to obscure their true purpose) in which they can be contacted by sellers. Sellers offer them a free product, or sometimes even pay, in exchange for positive product reviews. In order to not draw attention to these incentivized reviews—which have already been banned—sellers instruct reviewers to do things like add similar products to their wish lists, and pay upfront, but accept reimbursement via PayPal at a later date. They also seek out reviewers with long-term Amazon accounts, as this adds credibility to their reviews.

The implications of such practices reach beyond just Amazon. Diresta referred to Amazon as the “next SEO battleground” and noted that their search results are closely mirrored by Google’s—another reason why positive reviews are so important to sellers. However, as concerning as these tactics are for business practices, they have even more frightening cultural and ideological consequences. Diresta described how conspiracy theory groups use similar tactics to those described above to sell products, like anti-vaccination documentaries, for example, that align with dangerous agendas. The algorithms for “product suggestions” and “frequently bought together” can also be weaponized. For instance, Diresta shared how after searching for a book that claims to “cure autism,” she was recommended products to perform a harmful, non-medically advisable procedure suggested in the book.

Research into narrative manipulation in places outside of traditional social media platforms is still in its early stages. Thus, Diresta concluded with two questions for the audience’s further consideration. First, should we consider separating curatorial services, such as down-rankings and reviews, from the platforms that host them? And more broadly, can we use this issue to think about reforming recommender systems across the entire web?

_{Notes by Donica O'Malley}

About Renee

Renee DiResta is the Director of Research at New Knowledge, and Head of Policy at nonprofit Data for Democracy. Renee investigates the spread of disinformation and manipulated narratives across social networks, and assists policymakers in understanding and responding to the problem. She has advised Congress, the State Department, and other academic, civic, and business organizations about understanding and responding to computational propaganda and information operations. In 2017, Renee was named a Presidential Leadership Scholar, and had the opportunity to continue her work with the support of the Presidents Bush, President Clinton, and the LBJ Foundation. In 2018, she received a Mozilla Foundation fellowship and affiliation with the Berkman-Klein Center for Internet & Society at Harvard University to work on their Media, Misinformation, and Trust project. She is a Founding Advisor to the Center for Humane Technology, and a Staff Associate at Columbia University Data Science Institute.

Previously, Renee was part of the founding team of venture-backed supply chain logistics technology platform Haven, where she ran business development and marketing, and a co-founder of Vaccinate California, a parent-led grassroots legislative advocacy group. Renee has also been an investor at O’Reilly AlphaTech Ventures (OATV), focused on hardware and logistics startups, and an emerging markets derivatives trader at Jane Street Capital. Her work and writing have been featured in the New York Times, Politico, Slate, Wired, Fast Company, Inc., and the Economist. She is the author of the O’Reilly book “The Hardware Startup: Building Your Product, Business, and Brand”, and lives on the web at http://reneediresta.com and @noUpside.

Download original audio or video from this event.

Subscribe to the Berkman Klein events series podcast.