Current Berkman People and Projects

Keep track of Berkman-related news and conversations by subscribing to this page using your RSS feed reader. This aggregation of blogs relating to the Berkman Center does not necessarily represent the views of the Berkman Center or Harvard University but is provided as a convenient starting point for those who wish to explore the people and projects in Berkman's orbit. As this is a global exercise, times are in UTC.

The list of blogs being aggregated here can be found at the bottom of this page.

October 13, 2017

Berkman Center front page
Will Wikipedia exist in 20 years?


Featuring Katherine Maher, Executive Director of the Wikimedia Foundation, in conversation with Harvard Law School Professor Yochai Benkler


Join us for a stimulating conversation highlighting different perspectives of the question, "Will Wikipedia exist in 20 years?"

Parent Event

Berkman Klein Luncheon Series

Event Date

Oct 17 2017 12:00pm to Oct 17 2017 12:00pm
Thumbnail Image: 

Tuesday, October 17, 2017 at 12:00 pm
Berkman Klein Center for Internet & Society at Harvard University
Harvard Law School campus
Wasserstein Hall, Milstein East B, Room 2036 (HLS campus map)
RSVP required to attend in person
Event will be live webcast on this page at 12:00 pm

Please join us to hear the Executive Director of Wikimedia Foundation, Katherine Maher, in discussion with Harvard Law School Professor Yochai Benkler on the topic, "Will Wikipedia exist in 20 years?"

About Katherine

From Wikipedia: Katherine Roberts Maher is the executive director of the Wikimedia Foundation, a position she has held since June 2016. Previously she was chief communications officer. In addition to a background in the field of information and communications technology (ICT), Maher has worked in the non-profit and international sectors focusing on the use of technology to empower human rights and international development, specifically improving communities, promoting inclusivity and transparency, and deepening participation.

About Yochai

Yochai Benkler is the Berkman Professor of Entrepreneurial Legal Studies at Harvard, and faculty co-director of the Berkman Klein Center for Internet and Society. Before joining the faculty at Harvard Law School, he was Joseph M. Field '55 Professor of Law at Yale. He writes about the Internet and the emergence of networked economy and society, as well as the organization of infrastructure, such as wireless communications.



by candersen at October 13, 2017 07:35 PM

Justin Reich
101 Apps NOT Mentioned in this Post
How might we create the ideal conditions for professional learning and promote conversations about deeper learning and systemic change?

by Beth Holland at October 13, 2017 05:39 PM

October 12, 2017

Cyberlaw Clinic - blog
Massachusetts Considers Digital Right to Repair

On September 26, 2017, the Massachusetts Joint Committee on Consumer Protection and Professional Licensure heard testimony on proposed digital “right to repair” bills H.143 and S.96. The two proposed bills would require manufacturers of digital devices to provide diagnostic, repair, and service information to independent technicians and owners of devices, information that is currently only available to technicians selected and authorized by the manufacturers. The bills would further require manufacturers allow independent technicians and owners to purchase replacement parts and service tools at a reasonable price. The bills by their terms relieve manufacturers of the obligation to reveal any trade secret; however, they do not address the practicality of providing service manuals and diagnostic information without exposing trade secrets, particularly for manufacturers who rely heavily on trade secret protection.

Massachusetts has tackled right to repair before. In 2012, Massachusetts became the first state to pass right to repair legislation for motor vehicles. Rather than face future legislation from other states, auto manufacturers agreed to make the Massachusetts law their national standard.

The right to repair bills discussed on Tuesday shift the focus to digital devices. The bills cover only “digital electronic product[s],” defined as a device that “contain[s] a microprocessor,” but expressly excluding class III medical devices (high-risk implantable devices) and vehicles. This definition conceivably includes consumer electronics, medical devices, and any other digital device containing a microprocessor.

Senator Michael D. Brady, presenting the Senate bill, explained that both bills are designed to protect consumers from manufacturers who currently control the price of repair. The lack of competition, according to Senator Brady, leads to decreased innovation in service repair and prevents local repair business from performing repairs. However, Senator Brady indicated that he was open to the Joint Committee making minor changes.

Proponents of the bills argued that allowing independent repair and granting access to repair manuals, diagnostic information, and security patches ultimately improves security and device safety. Rachel Kalmar, a research fellow at Harvard University, explained that the device manufacturers’ current method of “security through obscurity” does not work. In the long run, Kalmar explained, published systems are ultimately more secure because researchers are able to find and fix security flaws. Kalmar also explained that safety is improved when individuals have access to updates. When companies stop supporting old devices, those devices may not get the latest software patches and can pose a threat to the larger network. The network is only as secure as its weakest link.

Proponents furthered argued that the current monopoly manufacturers hold on repair hurts consumers by inflating the cost of repair. Manufacturers, motivated to sell more new products and to protect repair as a revenue stream, are incentivized to make repair more expensive than replacement. Adam Fullerton, a manager at MEGA MOBILE, a Boston company specializing in unlocked smartphone repairs, testified that he has watched the cost of repair parts for Apple products skyrocket over the past few years, when the parts were even available to independent technicians at all. In the context of medical devices, Scot Mackeil, a Boston-based medical repair technician, explained that he could make more repairs on critical equipment, without sending the equipment to the manufacturer or waiting on an authorized technician, if he had access to repair manuals and parts.

Finally, the bills’ supporters argued that the monopoly on repair also hurts the secondary market for electronic devices. Rohi Sukhia of used computer wholesaler TradeLoop explained that when a consumer attempts to patch a device she bought used, manufacturers will often refuse to sell the patch or offer it at the cost of a new device, rendering the used hardware worthless.

The bills were primarily opposed by representatives of the manufacturers, especially technology and medical device manufacturers.

Opponents argued that the bills presented a threat to IP and trade secrets, despite the language that had been included to accommodate this concern. Sarah Faye Pierce, who represented a home appliance trade group, was the first to point out that releasing service manuals and repair information may require a release of sensitive IP. Matthew Mincieli of TechNet, a group representing technology companies and their executives, agreed, arguing that these bills will hurt the “tech ecosystem” and may disincentivize technology companies from coming to Massachusetts. Tim Johnson of the Entertainment Software Association agreed that the bills threaten IP and argued that the bills are unnecessary because there is an incentive for manufacturers to fix game consoles: “No one buys games for broken consoles.” Johnson also pointed out that the Copyright Office is currently conducting a review of a possible national right to repair exception.

Representatives of the medical device manufacturers were primarily focused on the potential threat to the quality of repair. Representatives of the Massachusetts Medical Device Industry Council (MassMEDIC) believed that the legislation would allow unqualified third parties to perform repairs on critical medical equipment without adherence to regulations imposed by the FDA. The FDA currently regulates repairs performed by manufacturers, but third parties may be exempt. Tom Tremble of the Advanced Medical Technology Association thought this threat was particularly acute for class II medical devices, such as infusion pumps, which present a moderate risk to patients.

Opponents were also concerned about potential security risks of exposing service manuals and diagnostic information. Faye Pierce explained that many home appliances are now “smart” appliances and that exposing service information could make these devices more vulnerable to hacks. Mincieli pointed out that releasing software repair information would allow bad actors to more easily identify vulnerabilities.

The Joint Committee raised a few questions of its own, for example, whether a car like the Tesla was more like a motor vehicle or a digital device, and seemed receptive to arguments from both sides. If the bills were to pass, they would be the first of their kind nationwide.

Alex Noonan is a second-year student at Harvard Law School and current clinical student in the Cyberlaw Clinic.

by Alex Noonan at October 12, 2017 01:54 PM

Harry Lewis
Professor Allen's Puzzling Motion, Part 2
When I introduced my motion, I referred to Professor Allen’s motionas “astonishingly sweeping.” Yet I did not appreciate how sweeping it was until a student asked me a question. Before I report her question, let’s back up and parse the motion as best we can.

Opinion is divided among my colleagues about the significance of the Allen motion. Some think it is tautologous, therefore meaningless, and therefore harmless. Others see it as threatening and disingenuous. Given the confusion, after more than a year of debate and with an impending vote, it is safer to respond to the motion’s intentions and to assume that the text is just badly drafted—in spite of being a third draft.

And in spite of having been supported by a 17-0 vote of the Faculty Council. That vote suggests that the Council may have thought the motion uncontroversial. But that only adds to the puzzle, since Dean Smith, who chairs the Council but is not a voting member of it, seems uncertain what its impact would be:
“Personally I don’t see all the clear next steps,” Smith said of Allen’s motion. “I’m not a lawyer, so I’m not even going to try to play one here.”
What sort of group dynamics resulted in a 17-0 vote by a faculty committee whose chair won’t opine on the motion’s meaning?

Adding to the irony of an ambiguous motion being voted up 17-0 by the Faculty Council is that the same body (with the same chair and significantly overlapping membership) refusedto take a vote on the motion I filed last year, “citing uncertainty about whether a vote for the motion would impact the policy,” as the Crimson paraphrased the body’s reasoning.

In any case, let’s assume that the intention is, as Professor Allen explained in supporting materials, to require all student organizations to which Harvard students belong to adhere to the nondiscrimination and other rules Harvard requires of recognized student organizations. A student belonging to a noncompliant organization would have three choices: force the organization to change, resign from it, or be “suspended or expelled.” (Professor Allen uses the term “expel,” so I too am using it and its cognates. Again, it is unclear whether what she says is what she really means—in Harvard parlance, to “expel” is more severe than to dismiss, which is more severe than requirement to withdraw. In modern times very few students have been expelled or dismissed. Expulsion is the equivalent of capital punishment--permanent separation with no possibility of return.)

The motion uses the Massachusetts anti-hazing statute to define “student organization.” That criminal statute applies to
every student group, student team or student organization which is part of such institution or is recognized by the institution or permitted by the institution to use its name or facilities or is known by the institution to exist as an unaffiliated student group, student team or student organization.
That broad definition is of course meant to hold colleges’ feet to the fire. I am not a lawyer either, but it’s plainly an anti-hazing statute, not a nondiscrimination statute. It says that colleges have to communicate with off-campus fraternities and the like. In fact, communication is all the statute actually requires colleges to do: to inform the organizations of their responsibilities under the law, to inform all students of the statute, and to attest to the Commonwealth that it has done so.

Professor Allen’s motion puts this category to an entirely different use: to force Harvard’s nondiscrimination rules on them. That is why the Allen motion is such an astounding assertion of power over private associations, far beyond anything Dean Khurana or the Clark-Khurana committee proposed. As I observed in my previous blog post, it would cover ROTC. Indeed, ROTC is exactly the sort of organization the state might want warned about hazing. The statutory definition would also cover the interuniversity fraternities and sororities, including the ones to which many African-American and Hispanic students belong. Again, it makes perfect sense that the anti-hazing statute would apply to them—if we accept the intent of that statute as legitimate, then such organizations and the student bodies from which they draw their members shouldn’t miss out on those warnings just because the students aren’t exclusively drawn from one school.

Like it or not, the definition of “student organization” in the anti-hazing statute makes sense for an anti-hazing statute. But that definition has no rational applicability to the membership policies of organizations Harvard students join. By what legal or ethical reasoning can that definition be used to threaten their student members with expulsion if the clubs they join aren’t co-educational?

And so to the student’s question. I was asked about the impact of the Allen motion on the Harvard Knights of Columbus and the Harvard Daughters of Isabella. Until I got the question, I had no idea these organizations even existed. They aren’t recognized student organizations, and indeed they run afoul of several of the requirements for recognition. They are single-sex organizations. They are under outside control. They may even impose a religious test on their members. They use the Harvard name, probably without permission. In compliance terms, they are worse than final clubs.

Will the Faculty of Arts and Sciences, by passing the Allen motion, assert its authority to force the Knights of Columbus and the Daughters of Isabella to go co-ed, and to expel their members if they do not?

Doubtless there are other student organizations which will unexpectedly fall under Harvard’s authority if the Allen motion passes. Harvard students are both diverse and creative. They are constantly forming new groups, recognized and unrecognized, in response to common interests and commitments.

If the text of Professor Allen’s motion is not further revised as such absurdities become apparent, those supporting the motion may point to the vague language about writing rules that will balance freedoms and rights. But that would simply authorize the administration to use its best judgment to decide whether to sanction individual clubs and organizations, without any unambiguous direction from the faculty about the meaning of its standards of nondiscrimination. The way out of this snarl is not to get into it, which is the idea behind my motion.

In retrospect, the Verba reportnow seems such a masterpiece. It took Harvard’s nondiscrimination rules seriously, and concluded they applied even to ROTC. Some of us gasped at that, but its logic was inescapable. But the report also recognized the illogic and illiberality of punishing student members for what Harvard might regard as an organization’s shortcomings. It combined moral clarity with humility. Not so the Allen motion, which lacks both.

(Corrected 10/14 as to the use of expulsion in response to a comment.)

by Harry Lewis ( at October 12, 2017 02:28 AM

October 11, 2017

Programming the Future of AI: Ethics, Governance, and Justice
How do we prepare court systems, judges, lawyers, and defendants to interact with autonomous systems? What are the potential societal costs to human autonomy, dignity, and due process from the use of these systems in our judicial systems? Harvard Law School Clinical Professor and Director of the Cyberlaw Clinic Chris Bavitz, along with Harvard's Cynthia Dwork, Christopher L. Griffin, Margo I. Seltzer, and Jonathan L. Zittrain, discuss the evolution of artificial intelligence, with an emphasis on ethics, governance, and criminal and social justice. Drawing from the research, community building, and educational efforts undertaken as part of our Ethics and Governance of Artificial Intelligence initiative, leading experts in the field share and reflect on insights from ongoing activities related to the judiciary and fairness. Find out more about this event here:

by the Berkman Klein Center at October 11, 2017 03:36 PM

Berkman Center front page
International students team up with Berkman Klein mentors to learn open source development


This summer six students from around the world remotely worked on the development of open source projects with mentorship from members of the Berkman Klein community through the Google Summer of Code (GSoC) program.

by Daniel Oyolu

This summer six students from around the world remotely worked on the development of open source projects with mentorship from members of the Berkman Klein community through the Google Summer of Code (GSoC) program. Designed to introduce students to the development of open source software (or software whose code is openly available to be viewed, altered, or improved), the program pays selected students a stipend to work as interns at organizations supporting open source software projects.

While separate from the Berkman Klein Center’s flagship on-site Summer Internship Program, the GSoC program is similar in that it aligns with and furthers the Center’s commitments to education, diversity, and network-building.

“The GSoC initiative matches the Berkman Klein Center’s mission of building out into cyberspace and making tools freely available to the public and researchers,” said Ellen Popko, the staff technologist who oversees the Berkman Klein Center’s involvement with GSoC. “The program allows students to acquire real world working experience as they learn about workflows, development cycles, and working with developers.”

The students worked on several projects under the guidance of Berkman Klein community members:

  • Dongge Liu, a master’s student of software engineering at the University of Melbourne in Australia, used machine learning to implement topic creation in Media Cloud, a joint project of the Berkman Klein Center and the MIT Media Lab that is an open-source platform for studying media ecosystems. Liu was mentored by Linas Valiukas, a former GSOC student who has continued to work remotely with the Berkman Klein Center from Lithuania on Media Cloud for the past five years.   (Read more about this work on GitHub.)

  • Gaurav Koley, obtaining his master’s in information technology at the International Institute of Information Technology in Bangalore, India, worked on the Berkman Klein Center’s BookANook project with support from Berkman Klein mentors Jessica Yurkofsky and Justin Clark. BookANook is an open source tool that makes it easier for users to reserve a room in community spaces, like a study room in a library. Gaurav worked to improve the tool’s overall user interface. (Learn more about this project on GitHub.)

  • Morgan Gangwere, a communications student at the University of New Mexico, worked under the mentorship of Berkman Klein affiliate Jason Griffey on the LibraryBox project. The LibraryBox website describes itself as an “open source, portable digital file distribution tool that enables delivery of vital information to individuals with no internet.” (Read more about this work on GitHub.)

  • Kumar Shubham, a computer science and engineering student at VIT University in Chennai, India, worked on the Teem project, an application that allows potential collaborators to connect with community projects they would like to support, under the supervision of Berkman Klein fellow Samer Hassan and Antonio Tenorio-Fornés. This summer Kumar helped make Teem more open and connected to other popular online services by implementing a contextual link preview, embedding functions, and sharing functions across various social media platforms. (Read more about this work on GitHub.)

  • David Llop, a master’s student studying security in ICT at the Open University of Catalonia in Barcelona, Spain, worked with the SwellRT team with direction from Berkman Klein fellow Samer Hassan and Pablo Ojanguren. He helped provide end-to-end encryption to SwellRT in order to enhance user privacy. (Read more about this work on GitHub.)

To participate in the GSoC program, a potential mentor organization applies to and must be accepted by the Google Open Source Office. Students then propose projects directly to the mentor organization, which then selects the students it would like to work with. Students must be currently enrolled in a university program and eligible to work in their home country.  If they meet all the obligations over the course of the internship, they receive a stipend from Google.

This summer, the program accepted more than 200 mentor organizations, of which just a handful are academic institutions.

"We are excited to provide an opportunity for rising coders to work on academic research with open source principles,” said Sebastian Diaz, Director of Technology at the Berkman Klein Center. “Academic groups are building and sharing cool things, too.”

The Berkman Klein Center first began its involvement with Google Summer of Code in 2010, and since then a handful of GSoC students have continued to work with the Center on a variety of projects. In addition to Linas Valiukas, who works on Media Cloud, Miriam Marrouf, a 2016 GSoC student from Egypt worked on Internet Monitor and other Berkman Klein projects throughout the past academic year. Chaitanya Choudhary, another 2016 GSoC intern, came to Cambridge this summer from India as a member of our onsite summer internship program. Choudhary continued his development work on the DotPlot tool.

“This program allows us access to student we wouldn’t normally be working with,” said Popko. It’s rewarding to be able to work with students from so many different backgrounds and on a global scale.”

GSoC student Morgan Gangwere said he has enjoyed the mentorship aspect of the program. “Jason [Griffey] is a fantastic mentor, giving me a huge amount of support on keeping me on track and getting things done,” he said.  Morgan worked to move LibraryBox over to a new kind of hardware, the Raspberry Pi.

“This will do a huge number of things for the project,” said Griffey. “It frees us from the whims of commercial hardware that we've been using and that has become increasingly locked-down and hard to customize, it gives users a more powerful platform to build new and exciting customizations on, and it gives the project itself the ability to start fresh with a new architecture that we can use in different ways to meet the needs of our users.”

You can read more about the GSoC Summer 2017 projects from the students’ perspectives on the Geek Cave blog.


by gweber at October 11, 2017 03:04 PM

October 10, 2017

David Weinberger
[liveblog][bkc] Algorithmic fairness

I’m at a special Berkman Klein Center Tuesday lunch, a panel on “Programming the Future of AI: Ethics, Governance, and Justice” with Cynthia Dwork, Christopher L. Griffin, Margo I. Seltzer, and Jonathan L. Zittrain, in a discussion moderated by Chris Bavitz.

NOTE: Live-blogging. Getting things wrong. Missing points. Omitting key information. Introducing artificial choppiness. Over-emphasizing small matters. Paraphrasing badly. Not running a spellpchecker. Mangling other people’s ideas and words. You are warned, people.

They begin with brief intros of their interests:

Chris Griffin: One of the big questions for use of algorithms in the justice system is what: is the alternative? Human decision making has its own issues.

Margo Seltzer: She’s been working on transparent models. She would always prefer to be able to get an index card’s worth of explanation of how a machine learning system has come up with its output.

Cynthia Dwork: What is our definition of fairness, and how might we evaluate the fairness of our machine systems? She says she’s not that big a fan of insisting on explanations.

Jonathan Zittrain: What elements of this ought to be contracted out? Can we avoid the voting machine problem of relying on a vendor we don’t necessarily trust? Also, it may be that expalantions don’t help us that much. Also, we have to be very wary of biases built into the data. Finally, AI might be able to shed light on interventions before problems arise, e.g., city designs that might lower crime rates.

Chris Bavitz: Margo, say more about transparency…

Seltzer: Systems ought to be designed so that if you ask why it came up with that conclusion, it can tell you in a way that you can understand. Not just a data dump.

Bavitz: The legal system generally expects that, but is that hard to do?

Seltzer: It seems that in some cases you can achieve higher accuracy with models that are not explicable. But not always.

Dwork: Yes.

Zittrain: People like Cynthia Rudin have been re-applying techniques from the 1980s but are explainable. But I’ve been thinking about David Weinberger’s recent work [yes, me] that reality may depend on factors that are deeply complex and that don’t reduce down to understandable equations.

Dwork: Yes. But back to Margo. Rule lists have antecedents and probabilities. E.g., you’re trying to classify mushrooms as poisonous or not. There are features you notice: shape of the head, odor, texture, etc. You can generate rules lists that are fairly simple: if the stalk is like this and the smell is like, then it’s likely poisonous. But you can also have “if/else” conditions. The conclusions can be based on very complex dependencies among these factors. So, the question of why something was classified some way can be much more complicated than meets the eye.

Seltzer: I agree. Let’s say you were turned down for the loan. You might not be able to understand the complex of factors, but you might be able to find a factor you can address.

Dwork: Yes, but the question “Is there a cheap and easy path that would lead to a different outcome?” is a very different quesiton than “Why I was classified some particular way?””

Griffin: There’s a multi-level approach to assessing transparency. We can’t expect the public to understand the research by which a model is generated. But how is that translated into scoring mechanisms? What inputs are we using? If you’re assessing risk from 1 to 6, does the decision-maker understand the difference between, say, a 2 and 3?

Zittrain: The data going in often is very reductive. You do an interview with a prisoner who doesn’t really answer so you take a stab at it … but the stabbiness of that data is not itself input. [No, Zittrain did not say “stabbiness”].

Griffin: The data quality issue is widespread. In part this is because the data sets are discrete. It would be useful to abstract ID’s so the data can be aggregated.

Zittrain: Imagine you can design mushrooms. You could design a poisonous one with the slightest variation from edible ones to game the system. A real life example: the tax system. I think I’d rather trust machine learning than a human model that can be more easily gamed.

Bavitz: An interviewer who doesn’t understand the impact of the questions she’s asking might be a feature, not a bug, if you want to get human bias out of the model…

Seltzer: The suspicion around machine algorithms stems from a misplaced belief that humans are fair and unbiased. The combination of a human and a machine, if the human can understand the machine’s model, might result in less biased decisions than either on their own.

Bavitz: One argument for machine learning tools is consistency.

Griffin: The ethos of our system would be lost. We rely on a judicial official to use her or his wisdom, experience, and discretion to make decisions. “Bias could be termed as the inability to perceive with sufficient clarity.” [I missed some of this. Sorry.]

Bavitz: If the data is biased, can the systems be trained out of the bias?

Dwork: Generally, garbage in, garbage out. There are efforts now, but they’re problematic. Maybe you can combine unbiased data with historical data, and use that to learn models that are less biased.

Griffin: We’re looking for continuity in results. With the prisoner system, the judge gets a list of the factors lined up with the prisoner’s history. If the judge wants to look at that background and discard some of the risk factors because they’re so biased, s/he can ignore the machine’s recommendation. There may be some anchoring bias, but I’d argue that that’s a good thing.

Bavitz: How about the private, commercial actors who are providing this software? What if these companies don’t want to make their results interpretable so as not to give away their special sauce?

Dwork: When Facebook is questioned, I like to appeal to the miracle of modern cryptography that lets us prove that secrets have particular properties without decrypting them. This can be applied to algorithms so you can show that one has a particular property without revealing that algorithm itself. There’s a lot of technology out there that can be used to preserve the secrecy of the algorithm, if that were the only problem.

Zittrain: It’d be great to be able to audit a tech while keeping the algorithm secret, but why does the company want to keep it secret? Especially if the results of the model are fed back in, increasing lock-in. I can’t see why we’d want to farm this out to commercial entities. But that hasn’t been on the radar because entrepreneurial companies are arising to do this for municipalities, etc.

Seltzer: First, the secrecy of the model is totally independent from the business model. Second, I’m fine with companies building these models, but it’s concerning if they’re keeping the model secret. Would you take a pill if you had no idea how it worked?

Zittrain: We do that all the time.

Dwork: That’s an example of relying on testing, not transparency.

Griffin: Let’s say we can’t get the companies to reveal the algorithms or the research. The public doesn’t want to know (unless there’s litigation over a particular case) the reasoning behind the decision, but whether it works.

Zittrain: Assume re-arrest rates are influenced by factors that shouldn’t count. The algorithm would reflect that. What can we do about that?

Griffin: The evidence is overwhelming about the disparity in stops by race and ethnicity. The officers are using the wrong proxies for making these decisions. If you had these tools throughout the lifespan of such a case, you might be able to change this. But these are difficult issues.

Seltzer: Every piece of software has bugs. The thought of sw being used in way where I don’t know what it thinks it’s doing or what it’s actually doing gives me a lot of pause.


Q: The government keeps rehiring the same contractors who fail at their projects. The US Digital Service insists that contractors develop their sw in public. They fight this. Second, many engineering shops don’t think about the bias in the data. How do we infuse that into companies?

Dwork: I’m teaching it in a new course this semester…

Zittrain: The syllabus is secret. [laughter]

Seltzer: We inject issues of ethics into our every CS course. You have to consider the ethics while you’re designing and building the software. It’s like considering performance and scalability.

Bavitz: At the Ethics and Governance of AI project at the Berkman Klein Center, we’ve been talking about the point of procurement: what do the procurers need to be asking?

Q: The panel has talked about justice, augmenting human decision-making, etc. That makes it sound like we have an idea of some better decision-making process. What is it? How will we know if we’ve achieved it? How will models know if they’re getting it right, especially over time as systems get older?

Dwork: Huge question. Exactly the right question. If we knew who ought to be treated similarly to whom for any particular classification class, everything would become much easier. A lot of AI’s work will be discovering this metric of who is similar to whom, and how similar. It’s going to be an imperfect but improving situation. We’ll be doing the best guess, but as we do more and more research, our idea of what is the best guess will improve.

Zittrain: Cynthia, your work may not always let us see what’s fair, but it does help us see what is unfair. [This is an important point. We may not be able to explain what fairness is exactly, but we can still identify unfairness.] We can’t ask machine learning pattern recognition to come up with a theory of justice. We have to rely on judges, legislators, etc. to do that. But if we ease the work of judges by only presenting the borderline cases, do we run the risk of ossifying the training set on which the judgments by real judges were made? Will the judges become de-skilled? Do you keep some running continuously in artesinal courtrooms…? [laughter]

Griffin: I don’t think that any of these risk assessments can solve any of these optimization problems. That takes a conversation in the public sphere. A jurisdiction has to decide what its tolerance for risk is, what it’s tolerance is for the cost of incarceration, etc. The tool itself won’t get you to that optimized outcome. It will be the interaction of the tool and the decision-makers. That’s what gets optimized over time. (There is some baseline uniformity across jurisdictions.)
Q: Humans are biased. Assume a normal distribution across degrees of bias. AI can help us remove the outliers, but it may rely on biased data.

Dwork: I believe this is the bias problem we discussed.

Q: Wouldn’t be better to train it on artificial data?

Seltzer: Where does that data come from? How do we generate realistic but unbiased data?

The post [liveblog][bkc] Algorithmic fairness appeared first on Joho the Blog.

by davidw at October 10, 2017 05:46 PM

Berkman Center front page
HUBweek 2017: Programming the Future of AI: Ethics, Governance, and Justice


featuring Harvard's Cynthia Dwork, Christopher L. Griffin, Margo I. Seltzer, and Jonathan L. Zittrain in conversation with Professor Chris Bavitz


How do we prepare court systems, judges, lawyers, and defendants to interact with autonomous systems? What are the potential societal costs to human autonomy, dignity, and due process from the use of these systems in our judicial systems?

Event Date

Oct 10 2017 12:00pm to Oct 10 2017 12:00pm
Thumbnail Image: 

Tuesday, October 10, 2017 at 12:00 pm
Berkman Klein Center for Internet & Society at Harvard University

How do we prepare court systems, judges, lawyers, and defendants to interact with autonomous systems? What are the potential societal costs to human autonomy, dignity, and due process from the use of these systems in our judicial systems?

Harvard Law School Clinical Professor and Director of the Cyberlaw Clinic Chris Bavitz, along with Harvard's Cynthia Dwork, Christopher L. Griffin, Margo I. Seltzer, and Jonathan L. Zittrain, discuss the evolution of artificial intelligence, with an emphasis on ethics, governance, and criminal and social justice. Drawing from the research, community building, and educational efforts undertaken as part of our Ethics and Governance of Artificial Intelligence initiative, leading experts in the field share and reflect on insights from ongoing activities related to the judiciary and fairness.

Download original audio and video from this event.

Subscribe to the Berkman Klein events podcast to have audio from all our events delivered straight to you!


Cynthia Dwork

John A. Paulson School of Engineering and Applied Science, Harvard University
Radcliffe Alumnae Professor, Radcliffe Institute for Advanced Study
Affiliated Faculty, Harvard Law School

Cynthia Dwork uses theoretical computer science to place societal problems on a firm mathematical foundation. She was awarded the Edsger W. Dijkstra Prize in 2007 in recognition of some of her earliest work establishing the pillars on which every fault tolerant system has been built for a generation (Dwork, Lynch, and Stockmeyer, 1984). Her contributions to cryptography include the launching of non-malleable cryptography, the subfield of modern cryptography that studies -- and remedies -- the failures of cryptographic protocols to compose securely (Dolev, Dwork, and Naor, 1991). She is a co-inventor of the first public-key cryptosystem based on lattices, the current best bet for cryptographic constructions that will remain secure even against quantum computers (Ajtai and Dwork, 1997). More recently, Dwork spearheaded a successful effort to place privacy-preserving analysis of data on a firm mathematical foundation. A cornerstone of this effort is the invention of Differential Privacy (Dwork, McSherry, Nissim, and Smith, 2006, Dwork 2006), now the subject of intense activity in across many disciplines and recipient of the Theory of Cryptography Conference 2016 Test-of-Time award. With its introduction into Apple's iOS 10 (2016) and Google's Chrome browser (2014), differential privacy is just now beginning to be deployed on a global scale. Differentially private analyses enjoy a strong form of stability. One consequence is statistical validity under adaptive (aka exploratory) data analysis, which is of great value even when privacy is not itself a concern (Dwork, Feldman, Hardt, Pitassi, Reingold, and Roth 2014, 2015a, 2015b).

Data, algorithms, and systems have biases embedded within them reflecting designers' explicit and implicit choices, historical biases, and societal priorities. They form, literally and inexorably, a codification of values. Unfairness of algorithms -- for tasks ranging from advertising to recidivism prediction -- has recently attracted considerable attention in the popular press. Anticipating these concerns, Dwork initiated a formal study of fairness in classification (Dwork, Hardt, Pitassi, Reingold, and Zemel, 2012). Dwork is currently working in all of these last three areas (differential privacy, statistical validity in adaptive data analysis, and fairness in classification). Dwork was educated at Princeton and Cornell. She received her BSE (with honors) in electrical engineering and computer science at Princeton University, where she also received the Charles Ira Young Award for Excellence in Independent Research, the first woman ever to do so. She received her MSc and PhD degrees in computer science at Cornell University. Dwork is a member of the US National Academy of Sciences and the US National Academy of Engineering, and is a fellow of the ACM, the American Academy of Arts and Sciences, and the American Philosophical Society.

Christopher L. Griffin

Research Director, Access to Justice Lab
Harvard Law School

Christopher L. Griffin. Jr. is the Research Director at the Access to Justice Lab at Harvard Law School. He earned his B.S. magna cum laude from Georgetown University, an MPhil in Economics at the University of Oxford, and his J.D. from Yale Law School, where he was an Editor for the Yale Law Journal and Editor-in-Chief of the Yale Law & Policy Review. Prior to joining the A2J Lab, Chris taught at Duke Law School (2010-2012) and William & Mary Law School (2012-2016). In addition to court administration and procedure, his research interests include employment discrimination and judicial decision-making.

Margo I. Seltzer

Herchel Smith Professor of Computer Science
John A. Paulson School of Engineering and Applied Sciences, Harvard University
Faculty Co-Director, Berkman Klein Center for Internet & Society

Margo I. Seltzer is a Herchel Smith Professor of Computer Science and the Faculty Director for the Center for Research on Computation and Society in Harvard's John A. Paulson School of Engineering and Applied Sciences. Her research interests are in systems, construed quite broadly: systems for capturing and accessing provenance, file systems, databases, transaction processing systems, storage and analysis of graph-structured data, new architectures for parallelizing execution, and systems that apply technology to problems in healthcare.

She is the author of several widely-used software packages including database and transaction libraries and the 4.4BSD log-structured file system. Dr. Seltzer was a founder and CTO of Sleepycat Software, the makers of Berkeley DB, and is now an Architect at Oracle Corporation. She is a Sloan Foundation Fellow in Computer Science, an ACM Fellow, a Bunting Fellow, and was the recipient of the 1996 Radcliffe Junior Faculty Fellowship. She is recognized as an outstanding teacher and mentor, having received the Phi Beta Kappa teaching award in 1996, the Abrahmson Teaching Award in 1999, and the Capers and Marion McDonald Award for Excellence in Mentoring and Advising in 2010.

Professor Seltzer received an A.B. degree in Applied Mathematics from Harvard/Radcliffe College in 1983 and a Ph. D. in Computer Science from the University of California, Berkeley, in 1992.

Jonathan L. Zittrain

George Bemis Professor of International Law, Harvard Law School
Vice Dean for Library and Information Resources, Harvard Law School
Faculty Chair, Berkman Klein Center for Internet and Society
Professor of Computer Science, John A. Paulson School of Engineering and Applied Sciences, Harvard University
Professor, Harvard John F. Kennedy School of Government

Jonathan L. Zittrain is the George Bemis Professor of International Law at Harvard Law School and the Harvard Kennedy School of Government, Professor of Computer Science at the Harvard School of Engineering and Applied Sciences, Director of the Harvard Law School Library, and Faculty Director of the Berkman Center for Internet & Society. His research interests include battles for control of digital property and content, cryptography, electronic privacy, the roles of intermediaries within Internet architecture, human computing, and the useful and unobtrusive deployment of technology in education. He performed the first large-scale tests of Internet filtering in China and Saudi Arabia, and as part of the OpenNet Initiative co-edited a series of studies of Internet filtering by national governments: Access Denied: The Practice and Policy of Global Internet Filtering; Access Controlled: The Shaping of Power, Rights, and Rule in Cyberspace; and Access Contested: Security, Identity, and Resistance in Asian Cyberspace.

He is a member of the Board of Directors of the Electronic Frontier Foundation and the Board of Advisors for Scientific American. He has served as a Trustee of the Internet Society, and as a Forum Fellow of the World Economic Forum, which named him a Young Global Leader, and as Distinguished Scholar-in-Residence at the Federal Communications Commission, where he previously chaired the Open Internet Advisory Committee. His book The Future of the Internet -- And How to Stop It is available from Yale University Press and Penguin UK -- and under a Creative Commons license. Papers may be found at

Moderator: Chris Bavitz

Christopher T. Bavitz is Managing Director of Harvard Law School’s Cyberlaw Clinic, based at the Berkman Klein Center for Internet & Society. He is also a Clinical Professor of Law at HLS, where he co-teaches the Counseling and Legal Strategy in the Digital Age seminar and teaches the seminar, Music & Digital Media. Chris concentrates his practice on intellectual property and media law, particularly in the areas of music, entertainment, and technology. He oversees many of the Clinic’s projects relating to copyright, speech, and advising of startups, and he serves as the HLS Dean’s Designate to Harvard’s Innovation Lab. Prior to joining the Clinic, Chris served as Senior Director of Legal Affairs for EMI Music North America. From 1998-2002, Chris was a litigation associate at Sonnenschein Nath & Rosenthal and RubinBaum LLP / Rubin Baum Levin Constant & Friedman, where he focused on copyright and trademark matters. Chris received his B.A., cum laude, from Tufts University in 1995 and his J.D. from University of Michigan Law School in 1998.


by doyolu at October 10, 2017 05:15 PM

October 09, 2017

Harry Lewis
Remarks of Professor James Engell at the October 3 FAS meeting

[These remarks were delivered from handwritten notes, not a full script. Furthermore, the Parliamentarian, literally one minute prior to the meeting being called to order, urged great brevity on the speaker personally, so some of the notes were condensed and several paragraphs crossed out, which then required ex tempore transitions. This is the best reconstruction of what was said.]
It’s good we are discussing this vexed topic today. The Clark-Khurana Committee would not exist were it not for Prof. Lewis’s motion made last December [2016], when I spoke of the statutory power of discipline resting with the Faculty.
The Clark-Khurana Committee presents a narrative both explicit and implied. Explicitly, there exist problems with some of these organizations and some new action is required. Yes. There is also an underlying narrative. It goes something like this. “Most, or many”—the Report cannot make up its mind on that score and it names no names—of these organizations are not only exclusionary in the sense of not admitting every student who wishes to join, they are places where misogyny, racism, and class prejudice are fostered and practiced. From the Report: “While the larger issue of selective membership on campus is worth further discussion, our committee’s charge was to address those groups whose members and leadership are committed to practicing discrimination against their fellow students on such bases as gender, race, and class.” Acts demeaning to women have occurred at at least a few of these clubs. The kind of racial slur mentioned in the Report is abhorrent. Some of these clubs have dues not purely nominal.
However, the Report estimates that up to one-quarter of undergraduates belong to these organizations, and more women than men. Given that first-year students don’t often belong, that pushes up the fraction to closer to one-third of students in sophomore through senior years. The Report implies that these students are undermining the educational mission of the College.
Are one-third of our students in sophomore through senior year going to private spaces to practice misogyny, racism, and class prejudice? Is that why these organizations exist and attract many students? These are the students we accept and teach and whom we graduate. The College does not require students to live in the Houses, but more than 98% of them, including more than 98% of those who join these organizations, live in the Houses and participate fully in House life. The nature of some of these organizations is evolving and for some of them quite quickly. It would be unfortunate to take a sudden and absolute action regarding these organizations.
I would ask colleagues to consider what the Report implies about these organizations, and to consider that the story of the students in these organizations is more complex and often far more benign than what the Report does imply.
James Engell, Gurney Professor of English and Professor of Comparative Literature 

by Harry Lewis ( at October 09, 2017 09:13 PM

October 08, 2017

Harry Lewis
Professor Allen’s puzzling motion

Harvard Magazine gives the text of the motion Professor Danielle Allen offered at the October 3 FAS meeting:
that the policies of the Harvard College Handbook for Students for student organizations pertain to students participating in all student organizations recognized as such by the Commonwealth of Massachusetts. The Faculty recognizes that on a college campus, as in society, basic freedoms and rights can come into conflict with each other. In such situations, the faculty and administration of Harvard College shall establish policies that protect individual freedoms while upholding the educational mission of the College.
This must have seemed benign and unobjectionable to the Faculty Council, since the Council voted in favor, 17-0. (Actually, the wording of the motion was changed after the Faculty Council voted for it – the revised version was distributed on paper at the beginning of the FAS meeting. Neither of these is the draft motion included in the report of the Clark-Khurana committee. So Professor Allen has offered three versions of her motion so far; perhaps a fourth will be offered before a vote gets taken. The Faculty Council opposed my motion, 2-16, so at least one faculty member, and perhaps two, voted for both.)

What is puzzling is that in a letterto the Crimson, Professor Allen describes the actual effect her motion would have.
Its effect would be that students who join student social groups that have become co-educational and that otherwise adhere to campus policies for student organizations will not face repercussions from the administration. Students who join social groups that have not become co-educational will be ineligible to continue enrollment at the College.
I have no idea how these conclusions follow from the text of the motion. The motion gives no protection to membership in any organization, as my motion would; it simply suggests which organizations would immediately be subject to institutional control. The statement that, if Professor Allen’s motion passes, members of noncompliant student groups would be punished harshly was also made at the FAS meeting and in an FAQ. As Harvard Magazine reports,
This approach seemingly would have the virtue of bringing regulation of USGSOs under FAS’s auspices, rather than relegating the decision to the president (to which some faculty members have objected, as described above). But it leaves what to do up to the dean. As for the risks students might face if they do not comply with regular oversight and regulation, Allen’s FAQ points, briskly, to “suspension or expulsion.”
In the Crimson, Professor Allen describes this way of proceeding as a “middle way between the two poles of the argument.” Hardly.

There is nothing new in the idea that Harvard should be trying to get all single-gender clubs to go co-ed; Dean Khurana has been doing his best at that for the past couple of years, with some success. Professor Allen’s recourse to Massachusetts Law as justifying such efforts to regulate the clubs is, as far as I can tell, without teeth. Indeed, it was exactly the fact that threats from Harvard could not be enforced that led the Clark-Khurana committee to reject the earlier draft of Professor Allen’s motion. So the argument that Harvard can legally regulate the clubs doesn’t advance the ball at all, and leaves the original question: what to do if the clubs do not cooperate. Professor Allen says their members should be suspended or expelled, and suggests that her motion implies that, but it doesn’t.

Let’s read it again:
that the policies of the Harvard College Handbook for Students for student organizations pertain to students participating in all student organizations recognized as such by the Commonwealth of Massachusetts.
Yes, of course, the policies of Harvard College apply to Harvard students—to all Harvard students. The restrictive clause that follows (students participating in this or that) adds nothing to what we already know about the applicability of Harvard rules to Harvard students. The motion does not say that students may not participate in organizations that do not comply with rules governing recognized student organizations. That is, of course, the very question my motion seeks to clarify, by guaranteeing that they may.


Professor Allen seems to have couched her motion in bland language so that it will receive broad support, on the understanding that it would authorize the administration to work out the details without ever bringing anything contentious to a vote of the full Faculty of Arts and Sciences. Bland motions are dangerous. Even the unanimously voted FAS statement in favor of student body diversity has unexpectedly been citedby the president in justifying her attack on students’ freedom to join outside organizations.

The Allen motion offers a theory to justify Harvard’s legal right to regulate outside organizations—something I haven’t questioned (though others may have). So for me, the reference to Massachusetts anti-hazing statutes seems beside the point. I am not a lawyer, but it seems to me that if Harvard wants to make a rule that no member of the Bee or the Kappa Kappa Gamma may enroll, it can. My entire argument is that it shouldn’t have rules of that kind. In the particular hypothetical Professor Allen posed, in which a Harvard Pokémon Club staged cheating as an annual club ritual, she argued that all its members should be expelled. I am all in favor of throwing out cheaters, but I don’t see why some Pokémon-loving reformer who refused to go along with the crowd should get tossed too. As I have said many times, students should be punished for what they do, not for what clubs they join.

In the course of arguing that even the state sometimes steps in to regulate private organizations, the assurances in the First Amendment notwithstanding, Professor Allen makes a curious citation to support her case. (This and other documents are available here.)
[W]hen the Supreme Court handed down its 1987 decision in Rotary International, affirming California’s decision to prohibit gender-exclusive membership policies for clubs of that kind, the Court argued that “the State’s compelling interests in eliminating discrimination against women and in assuring them equal access to public accommodations. . . extends to the acquisition of leadership skills and business contacts, as well as tangible goods and services” (481 U.S. 537 [1987]: 548-549, emphasis added).
What is odd about that citation is how the Court got to the conclusion that, in the case of the Rotary Club, it could override the usual protections of free association. It was because the Rotary Club was so … inclusive. “The evidence in this case indicates that the relationship among Rotary Club members is not the kind of intimate or private relation that warrants constitutional protection,” Justice Powell states in his opinion. As the syllabus summarizes the argument,
In determining whether a particular association is sufficiently intimate or private to warrant constitutional protection, consideration must be given to factors such as size, purpose, selectivity, and whether others are excluded from critical aspects of the relationship. Here, the relationship among Rotary Club members does not warrant protection, in light of the potentially large size of local clubs, the high turnover rate among club members, the inclusive nature of each club's membership, the public purposes behind clubs' service activities, and the fact that the clubs encourage the participation of strangers in, and welcome media coverage of, many of their central activities
To apply this to the final clubs seems to require arguing simultaneously that Harvard shouldregulate them because they are exclusive and can regulate them because they are inclusive! And to the extent that certain clubs are not particularly selective (some of the sororities, for example), that makes them more vulnerable to intervention by the authorities, not less.

I am, in any case, skeptical that Harvard’s interest in getting women jobs in investment banking has much to do with its insistence that the Porcellian Club go co-ed. It would be the easiest thing in the world to ramp up the support of women who wanted high-paying jobs in the financial sector; the industry would gladly help any Harvard effort to do that. No Harvard administrator has ever said that anything of the kind was an institutional priority, except when complaining about the final clubs.

There are other curious aspects to the argument for the motion, for example the relevance of the anti-hazing statute:
The law has been taken to apply to final clubs since it was introduced, and the final clubs adhere to its provisions.
Taken by whom? But yes. It’s a criminal statute. I am not surprised that the clubs acknowledge that they comply with it. It is a huge leap to suggest that this statute somehow justifies Harvard intervening in their membership practices. And to take another matter Professor Allen cites, I am also not persuaded that a Harvard rule against faculty having sex with students—a condition of an employment relationship—has any relevance to students’ club memberships.

Of course there can be such limits on individual freedoms in the context of our contractual relationship with the institution. The question is which ones Harvard should impose. The motion is dangerously vague on that substantive question, handing it off to “the faculty and administration.” Which seems to mean that another Clark-Khurana committee will make the rules, and the Faculty, corporately, will have no say.

I have from the beginning cited the Verba reportand its explicit rejection of the idea of punishing ROTC students for joining what was then a discriminatory organization. It seems that we can infer how Professor Allen would want such a dilemma of conflicting freedoms and rights to be handled in the future. If Harvard is unable to persuadethe government to drop its transgender ban in the military, will we expel our ROTC cadets?

So what should happen to a motion when its explanatory materials state a good deal more about its purpose and effects than does its text? It should be defeated. It is plainly intended as an alternative to my motion, and if it passes, the administration will surely use the apparatus surrounding the text to justify its harsh interpretation of the Faculty’s intentions.


One final point. The motion claims to be moderate because it limits the scope of institutional reach to student organizations as defined by Massachusetts Law. So it is meant to swat away the questions about political parties and the like.

What seems not to have been stated anywhere is that Professor Allen’s Index of Prohibited Organizations is a great deal longer than that of the Clark-Khurana Committee.For example, it would include all the ethnic fraternities and sororities. They were excluded from the Clark-Khurana list because that list was restricted to organizations that consisted mostly or entirely of Harvard students, and these organizations (the ones I know about, anyway) are joint with MIT, Tufts, BU, and/or Wellesley. (Here is a link to one of them.)

The ethnic fraternities and sororities are a piece of Harvard culture I don’t know much about. They do not turn up on police blotters or Ad Board dockets. The only time I ever hear about them is when students tell me how important their organizations are to them. They are old, some of them; W.E.B. Dubois was a member of one.

The recent funeral of Dr. Allen Counter, whose name is synonymous with inclusiveness and diversity at Harvard, ended with a moving display. About 40 of his fraternity brothers, old and young, including several senior African-American Harvard faculty and administrators, rose from the congregation, assembled in the front of the chapel, formed a ring by linking their little fingers, and gave him a solemn ritual sendoff.

I am not at all sure that Harvard understands the social structures it is seeking to destroy.

by Harry Lewis ( at October 08, 2017 11:39 PM

October 07, 2017

How should customers look to business?

The world of business has a default symbol for customers: the ones they put on restroom doors.

Outside of those, there is no universal symbol for a customer.

When business talks to itself, it mostly uses generic cartoon images such as these (from a Bing search) and these (from a Google one):

I’m sure all of us identify more with the restroom symbols (and emojis) than we do with those things.

It’s interesting how, even though we comprise 100% of the marketplace, we remain a prevailing absence in nearly every business conference, business book and business school class.

The notion that customers can be independent and fully empowered agents of themselves, with scale across all the businesses they deal with, at best gets the intellectual treatment (seeing customers, for example, as “rational actors”).

At worst, customers are seen as creatures that go moo and squit money if they’re held captive and squeezed the right ways.  Listen to the talk. Typically customers are “targets” that businesses “acquire,” “manage,” “control” or “lock in” as if we are cattle or slaves.

Often customers are simply ignored.

One example that showed up today was this press release announcing “an innovative initiative focused on the overhaul of open account trade finance infrastructure.” It’s from R3, which makes Corda, a ” distributed ledger platform designed specifically for financial services,” and is “a joint undertaking between R3, TradeIX, and twelve financial institutions.” This network, says the release, will “improve access to open account trade for the global ecosystem of banks, buyers, suppliers, technology providers, insurers, and other parties, such as logistics companies, that are critical to facilitating global open account trade flows.”

Never mind that distributed ledgers have been hailed as the second coming (or even the first) of the customer-empowering peer-to-peer world. Instead note the absence of customers: people and institutions who entrust their money and assets to all the parties listed in that long sentence.

Our goal with ProjectVRM is to equip customers (not just “consumers,” or “end users”) to say We’re not just at the same table with you guys. We are that table. And we are much bigger and far more powerful than you can ever make us on your own.

In other words, our job here is to give customers superpowers.

There are lots of people arguing that more policy is the answer. But we already have the GDPR. Huge leverage there. Let’s use it to highlight how own customer-empowering solutions put the companies that serve us in compliance.

In the last post we named one. That and many other forms of #customertech will be featured at VRM Day and IIW, later this month at the Computer History Museum in Silicon Valley. Looking forward to seeing many of you there.

Let’s make customers powerful. Then it won’t matter how they look to business, other than real.


by Doc Searls at October 07, 2017 01:19 AM

October 06, 2017

Making it impossible for business to keep customers out of the conversation

It’s time for business to stop talking to itself, and start obeying customers.

That’s right, I said obeying.

It hasn’t happened yet, but it will.

This is what we were predicted with “Markets are Conversations” in The Cluetrain Manifesto, way back at the start of the millennium.

When that didn’t happen, I launched ProjectVRM in 2006.

Encouraged by our progress, I wrote The Customer as a God, a cover story for The Wall Street Journal‘s Marketplace section. That ran not long after The Intention Economy: When Customers Take Charge came out in 2012. The image above was most of that Journal section’s front page (you can see the fold below the middle there).

That it still hasn’t happened doesn’t mean it won’t. That a glass is empty doesn’t mean it can’t be filled. Instead it’s a promise that it will be filled. There is already plenty happening in that glass—far more than there was five years ago.

One sign of the “before” state of things we’re still in is a certain absence in every business conference, every business book, every business school class.

That absence is customers.

The notion that customers can be independent and fully empowered agents of themselves, with scale across all the businesses they deal with, at best gets the intellectual treatment (seeing customers, for example, as “rational actors”). Still, mostly customers are seen as creatures that go moo and squit money if they’re held captive and squeezed the right ways.  Listen to the talk. We are “targets” businesses “acquire,” “manage,” “control” or “lock in” as if we are cattle or slaves.

A good example of customer absence at non-work is this press release announcing “an innovative initiative focused on the overhaul of open account trade finance infrastructure.” It’s from R3, which makes Corda, a ” distributed ledger platform designed specifically for financial services,” and is “a joint undertaking between R3, TradeIX, and twelve financial institutions.” This network, says the release, will “improve access to open account trade for the global ecosystem of banks, buyers, suppliers, technology providers, insurers, and other parties, such as logistics companies, that are critical to facilitating global open account trade flows.”

Never mind that distributed ledgers have been hailed as the second coming (or even the first) of the customer-empowering peer-to-peer world. Instead note the absence of the people and institutions who entrust their money and assets to all the parties listed in that long sentence.

The goal of ProjectVRM is to equip customers (not just “consumers,” or “end users”) to say We’re not just at the same table with you guys. We are that table. And we are much bigger and far more powerful than you can ever make us separately and on your own.

In other words, our job here is to give customers superpowers.

There are lots of people arguing that more policy is the answer. Or that we need a new Google or Facebook to fight the incumbents.

In fact we already have the policy we need for leverage, in the GDPR. Let’s use that leverage to point to our own customer-empowering solutions.

In our last post we named one. That and many others will be on the table and open to being worked on during VRM Day and IIW, later this month at the Computer History Museum in Silicon Valley. Looking forward to seeing many of you there.


by Doc Searls at October 06, 2017 03:01 PM

October 05, 2017

Berkman Center front page
Enabling Competition & Innovation on a City Fiber Network


The municipally owned fiber-optic network of Ammon, Idaho provides one model for U.S. public entities and policymakers seeking to increase service competition and innovation.

Publication Date

5 Oct 2017

Thumbnail Image: 

Authored by Paddy Leerssen and David Talbot

This case study of Ammon, Idaho’s city-run fiber-optic network describes an unusual strategy for encouraging retail Internet service competition and innovation. Cities, towns and counties seeking to help their citizens and businesses by building fiber-optic networks may do so under a number of models, including building unused or “dark” fiber and then finding private providers to install network electronics and provide services. By contrast, Ammon decided to take all the steps required to operate the network, then use a technology called network virtualization to make it very easy for retail providers to provide services, which are then presented to users via an online dashboard. In theory, Ammon’s residences and businesses can take multiple services simultaneously, create private networks within Ammon’s city network, and obtain city services and emergency alerts over the network even if they don’t take an Internet access subscription. As of the date of publication, two ISPs have started to provide service, and Ammon itself has developed certain public safety applications. Though at an early stage, the project represents a versatile technological and operational model for other public fiber networks.

by djones at October 05, 2017 02:42 PM

October 04, 2017

Berkman Center front page
Zero Rating & Internet Adoption

Workshop Paper & Research Agenda


The Role of Telcos, ISPs, & Technology Companies in Expanding Global Internet Access

Publication Date

5 Oct 2017


Zero rating, which allows users to access select Internet services and content without incurring mobile data charges, is not a new concept. But it has become an object of debate as mobile carriers and major app providers have used it in the developing world to attract customers, with the goal of increasing Internet access and adoption. While some feel these programs violate net neutrality and create the potential for a two-tiered Internet, others argue that zero rating programs bring the developing world online and could be modified to uphold, rather than violate, net neutrality principles. At the same time, little research evaluating zero rating programs exists, and many different program formulations are lumped under the term “zero rating,” some of which are more compatible with net neutrality than others. In March of 2016, the Berkman Klein Center for Internet & Society gathered a diverse group of stakeholders from academia, the media, the government sector, industry, and the open software community to discuss the use of zero rating as a means to improve Internet adoption in the developing world and how and when it could be an effective tool, if at all. This paper captures the resulting dialogue and recommendations. The workshop summary is followed by a collection of briefing papers representing the viewpoints of many of the workshop participants.


  • Many different models of industry initiatives currently fall into the loose definition of zero rating. Creating a better defined taxonomy of program parameters, technical mechanisms, and impacts may allow for greater nuance and understanding in the field, as well as more targeted regulatory responses.
  • Universal Internet access and adoption is a common goal but one that requires significant investment in global infrastructure. Some assert that zero rating programs may serve as a helpful stopgap measures to increase access, while others argue that these programs contribute to the creation of a tiered internet ecosystem without providing meaningful benefits to the targeted beneficiaries.
  • Zero rating initiatives may be employed in pursuit of goals other than Internet adoption, such as emergency services messages or security updates, and the goals of a particular program may make it more or less controversial.
  • More empirical research is required to fully assess the impact of specific zero rating initiatives, as well as zero rating generally, on Internet adoption in the developing world. This research will sometimes require access to usage information held by mobile carriers and zero rating service providers that should be handled with user privacy in mind.

Producer Intro

Authored by

by djones at October 04, 2017 08:36 PM

Did fake news save Kenya from an Internet shutdown? Emerging Trends in Tech and Elections in Africa
Did fake news save Kenya from an Internet shutdown? Kenya held general elections on August 8, 2017. The presidential election was nullified due to irregularities and is set for a repeat on October 26, 2017. Technology played a key role in the polls at two levels - there was use of tech in aspects such as results transmission and social media was employed massively in political campaigns with propaganda and fake news flowing freely. The talk explores emerging trends in use of technology in elections and their effect on Internet freedom and what to expect as Kenya gears up for repeat elections. About Grace Grace was a 2016/17 OTF Information Controls Fellow at the Berkman Klein Center studying freedom online during election periods in East Africa. She analysed freedom online in the Uganda elections of 2016 and is part of an election observer mission in Kenya's 2017 elections. Grace is also an advocate of the High Court of Kenya and an associate at the Kenya ICT Action Network (KICTANet) where she carries out ICT policy and legal analysis. Find out more about this event here:

by the Berkman Klein Center at October 04, 2017 08:13 PM

Berkman Center front page
Did fake news save Kenya from an Internet shutdown? Emerging Trends in Tech and Elections in Africa


featuring Grace Mutung'u, the 2016/17 OTF Information Controls Fellow at the Berkman Klein Center


Technology and elections and the politics of technology​. How use of technology in Kenyan elections is shaping Internet freedom in Africa.

Event Date

Oct 4 2017 12:00pm to Oct 4 2017 12:00pm
Thumbnail Image: 

Wednesday, October 4, 2017 at 12:00 pm
Berkman Klein Center for Internet & Society at Harvard University
23 Everett Street, Second Floor, Conference Room (campus map)

Kenya held general elections on August 8, 2017. The presidential election was nullified due to irregularities and is set for a repeat on October 26, 2017. Technology played a key role in the polls at two levels - there was use of tech in aspects such as results transmission and social media was employed massively in political campaigns with propaganda and fake news flowing freely.

In this talk Grace Mutung’u explores emerging trends in the use of technology in elections and their effects on Internet freedom. She gives a short background on the history of technology and elections in Kenya and then participates in a Q&A with BKC affiliate Ellery Biddle of Global Voices and answers questions from the audience.

Notes from the talk

People in Kenya take elections very seriously, said Mutung’u. In the seven presidential elections since independence in 1963, all but one have been disputed. The 2007 election led to reform and to a new constitution in 2010. By 2017, she said, issues related to corruption, historical injustices, and equity were all important topics for discussion and continue to be as the country gears up for the repeat elections on October 26th.

As technology and the internet play more of a role in elections, there are consequences such as an increased polarity of the country, increased policing of the internet, and a general increase in discourse. Technology was integral to the 2017 election in Kenya, including playing a role in voter registration, voter identification, and results transmission. It was the argument that voting technology had been hacked during results transmission that led to the courts to examine the paper voting records and ultimately determine that significant enough irregularities existed to warrant calling for a new election. Questions remain as to how the transmission system will function on October 26th.

Technology also played a large role in the 2017 Kenyan election in terms of the dissemination of information, both in terms of the content itself as well as how it spread. Mutung’u reported that there was a lot of fake news and propaganda, primarily spread on WhatsApp, Twitter, and Facebook and that the government attempted in various ways to regulate online information, including through regulations, surveillance, and direct online engagement with critics. Much of the fake news, however, was spread through private groups on WhatsApp, which kept it hidden from the open web and also in many cases rendered stories more powerful as they were spread through trusted networks.  At the same time, much of the fake news and negative campaigning was very professionally done, she said. “There were really nice videos and really nice memes. They really understood the landscape and made the news as easy as possible to consume,” she said. “And they shared it in closed groups.”  Later, due to people “calling them out” and sharing these stories on the open web, it was revealed that the company Cambridge Analytica may have been responsible for much of this content in their role running communications for the incumbent party. Other brigades were also producing false content that benefitted the other political parties.

Another key role technology played in the election was in campaigning. Mutung’u said that nearly all candidates used Facebook in one way or another for political engagement. Some, she said, forgoed traditional campaigning methods altogether and relied exclusively on social media. Corporations did take steps to fight back against the waves of false stories being spread (Facebook ran a full-page newspaper ad trying to educate people about how to tell what is fake news, she said), but in general, their response has not been contextualized to the situation. “They are not understanding the problem of why elections are such a high stakes game in Africa,” she said. “All this fake news is because people really want to win elections. It’s a life and death situation. It would be interesting to try another way. The people who spread harmful content - why? What do they really want to talk about? Is there a place we can sit down and find out what their issues really are and find a way to solve these issues?”

Mutung’u said that in some ways, the problem of fake news is actually helping Kenyans tackles issues of truth, justice, and reconciliation, topics that need to be addressed but that formal processes have not been successful with.  “Sometimes, fake news helps us talk about about these issues that were not on the table before.” She said that fake news also may have prevented an internet shutdown around the election. “All indications were that there would be a shutdown,” she said. But in the end the government did not shut it down. This is likely for a number of reasons, including that government was benefitting from much of the false content, that fake news was motivating more people to go to the polls, and that the political cost for shutting down the internet may have been too high, she said.

In addition to discussing the role of technology and social media on the election, Mutung’u also talked about the importance of data protection and data privacy. Kenya does not yet have a data protection framework, she said, noting that there’ a right of protection of privacy in the new constitution, but no enabling legislation. “It’s a delicate balance,” she said, citing the example of publishing of the country’s voter register so that people can see that it's clean, but noting that, too much personal information was shared. “We need to lead Africa in getting data protection laws that protect people from both companies and their own government,” she said.

About Grace

Grace was a 2016/17 OTF Information Controls Fellow at the Berkman Klein Center studying freedom online during election periods in East Africa. She analysed freedom online in the Uganda elections of 2016 and is part of an election observer mission in Kenya's 2017 elections.
Grace is also an advocate of the High Court of Kenya and an associate at the Kenya ICT Action Network (KICTANet) where she carries out ICT  policy and legal  analysis.
Follow her on Twitter a @Bomu

Links to explore in advance of event

Links to resources we want to share in advance of talk:

Download original audio and video from this event.

Subscribe to the Berkman Klein events podcast to have audio from all our events delivered straight to you!

by candersen at October 04, 2017 04:00 PM

Harry Lewis
Professor Jason Mitchell's Minority Report
Professor Mitchell has kindly allowed me to post his minority report. It is very much worth reading, a model of reasoned analysis and clear writing.


29 September 2017

In its Final Report, the Committee on Unrecognized Single-Gender Social Organizations (USGSOs) has modified its preliminary recommendation of June 2017, which would bar undergraduates from joining Final clubs, fraternities, sororities, or similar off-campus single-gender organizations (such as the Hasty Pudding).  In lieu of a single policy, the Final Report considers a variety of strategies for addressing the USGSOs, which range from the sanction policy of May 2016 to calls for engagement with students to more aggressive police enforcement of underage drinking and noise statutes.  The choice to adopt such a “multiple recommendation” approach reflects the continuing deep divisions within our community about how best to address the USGSOs.  In discussions with many colleagues and students—and over many hours of debate within the Committee itself—it has become obvious that these issues do not admit to any straightforward solution, and that colleagues who start from the same goal of making Harvard the best place it can be may nevertheless arrive at very different end points.  The purpose of this note is to offer an analysis of the main sources of these differences.  As such, it is not intended as a dissent per se, but as a formal attempt to clarify some of the principles and conceptual distinctions that seem to matter most to my colleagues and students.
As I see it, when members of our community disagree about how to address the all-male Final clubs and other USGSOs, we may be disagreeing about either of two distinct questions.  The first of these asks, “What problem are we trying to solve?”; the second, “What is the best way to solve it?”  The Committee’s Final Report makes clear that a range of answers exists to the latter—thus, its “multiple recommendations” strategy.  But, likewise, different members of our community provide very different answers to the question of what problem we are, in the first place, trying to solve.  These differences have been reflected in the seemingly different ways that the problem of Final clubs has been framed for the Faculty over time and in different documents—many colleagues feel that the rationale for sanctioning USGSO membership has morphed from an initial focus on sexual assault, to later concerns about gender-based discrimination, and most recently, to issues of inclusion, belonging, and privilege.  Indeed, the Committee spent a good deal of time discussing not only these problems, but also additional ones, such as the distorting effects of Final clubs on student social life and the health and safety concerns they pose for our students.
Some of my colleagues have decided that these shifts in rationale reflect some form of political expediency (“let’s keep making different arguments until the Faculty buy one of them”).  But we may do better to conclude instead that the problem of the all-male Final clubs is—as psychoanalysts and philosophers of science would say—overdetermined.  That is, we should not disavow the all-male Final clubs because they increase the incidence of sexual assault orbecause they discriminate against women orbecause they advance the prerogatives of a few individuals at the expense of many others or because the undermine student life or because they encourage unsafe drinking.  We should repudiate them because they do all of these things.  Perhaps any one such aspect of the clubs would be sufficient to make the case against them; together they lead, as the Final Report notes, to our community’s shared sense that we cannot afford to do nothing about them.
Note that I have restricted the point above to all-male Final clubs, and this is with intent.  Many of us believe firmly that despite its shifting rationales, the College is “really” trying to address problems specific to the all-male Final clubs.  After all, these are the only groups that own property adjacent to campus and that host the parties outside of which female undergraduates queue in the hopes of being admitted.  These are the groups that perpetuate privilege most perniciously.  And these are the groups that our colleagues have uniquely identified as important loci of sexual assault.  Indeed, it is hard not to perceive a direct line connecting the Final Report in March 2016 of President Faust’s Task Force on the Prevention of Sexual Assault to the announcement two months later of the first sanctions policy.  That Task Force repeatedly makes the case that it is the all-male Final clubs that pose serious concerns with regard to sexual assault, and that this is mainly possible because they control the space in which many undergraduates socialize (unlike other USGSOs).
My sense is that our current divide has emerged, in part, because of a continual choice to first select one or another of the specific problems caused by the all-male Final clubs and to then develop policies designed to address that problem broadly throughout undergraduate life.  This impulse is understandable—we are, after all, a community that values first articulating our principles and then developing policies that serve them.  When our goal is to achieve a particular outcome (say, the end to all-male Final clubs) we naturally want to start by defining the principles at stake, such as an opposition to gender-based discrimination, and then allow our policies to flow from that principle.  But thus far, this approach has created something of a dragnet, which threatens to sweep in student groups that many of us feel are not much of a problem (or, at least, not nearly as much of a problem as the all-male Final clubs); fraternal organizations without houses in which to host parties and womens’ Final clubs, not to mention the Hasty Pudding, do not really seem to be at the root of campus ills.  It is my own belief personally—and I think the sentiment of many faculty colleagues—that we would have done better to clearly identify what we are trying to achieve, which is an end to the noxious, distorting, and often abhorrent influence of the all-male Final clubs on undergraduate life.  This is surely the point on which the greatest number of us agree; if for no other reason, it would serve well as the starting point for discussions about what policies best achieve our goal.

Which brings us squarely to the second major source of disagreement within our community: regardless of how one answers the question of what goal we ought to be aiming for, there remains an open—and very contentious—question of how best to go about achieving it. 
To date, much of the debate around this issue has been cast a clash between competing values.  On the one hand, our community is committed to inclusion, we fight against discrimination in all its pernicious forms, and we have rightly begun to identify and dismantle the many structures that prevent members of our community from feeling that they belong at Harvard (and that it belongs to them).  On the other hand, we recognize that this set of values is one among many that progressive, well-intentioned individuals espouse.  Another set of such values includes notions of free expression, of individual autonomy, and of the right to free association.  One frame on the current faculty debate concerns how to adjudicate between these values when they conflict with one another.  The choice of some students to socialize off-campus with only certain people acts as a barrier to inclusion and belonging for other students; to whom do our responsibilities lie?  Each of us recognizes that rights (even those enumerated in the law) are not absolute but must be balanced against our responsibilities to one another—thus, the restrictions on free expression that enjoin us from shouting “fire!” in a crowded faculty meeting or the like.  One way of thinking about our current state of division is as a disagreement about whether the hazards posed by the all-male Final clubs and other USGSOs warrants a similar abrogation of (some of) our students’ rights (such as to free and lawful assembly).  [I am, of course, glossing over many nuanced aspects of this point of view, but only because I wish to make the observation below.]
This way of framing the debate tends to bottom out in the question of whether we should intervene against the all-male Final clubs and other USGSOs.  But another way we might have this discussion is by instead asking the question of how we ought to intervene.  What I mean is this: For much of the past 16 months, we have been led to think in binary terms—either we take the extraordinary step of patrolling the off-campus social lives of students, or we wave a white flag of surrender to the status quo and acquiesce as the Final clubs continue to exert an adverse effect on our community.  What is missing from this duality is any substantive discussion of how we might effect meaningful change on the Final clubs through more ordinarymeans.  The policies of sanctioning USGSO membership surely comprise extraordinarymeasures: they make extraordinary and unprecedented claims on the private, off-campus lives of our students; implementing them will require a radical reimagining (for many of us) of the relationship between the faculty and its students’ private lives; and they seem (to many of us) to contravene other values that ought to characterize a liberal institution committed to free inquiry and personal transformation.  One index of just how extraordinary these policies seem is the amount of time spent by the USGSO Committee on the question of whether the various sanctions policies are even legal.  Such policies will take us into uncharted places.
Is there nothing short of such extraordinary measures that can bring change to our campus?  The USGSO Committee’s Final Report tells us the answer to this question is no, that we have tried in vain for years to rein in the Final clubs through normal channels.  But a look at what is described suggests that the College’s ordinary attempts have been limited to various forms of “moral suasion,” mainly comprising various meetings between administrators and club leaders and alumni boards.  If the College’s efforts have indeed consisted mostly of an occasional stern talking-to, then we have little reason for surprise at their failure.  Social scientists—economists, sociologists, those in psychology departments and business schools—have learned a great deal about how to change people’s behavior, and we know that “moral suasion” is probably the least effective ways of going about it.  This is why when public health officials aim to decrease cigarette smoking, they do not simply tell people, “Cigarette smoking is bad, you shouldn’t do it.”  Instead, they have waged a sustained campaign to inform consumers of the dangers of smoking; they make it harder for young people to obtain cigarettes; they have worked relentlessly to transform cigarette smoking from something with social cachet into something that borders on shameful and “uncool”; and so on.  No, this has not proven straightforward, and yes, it has taken time and real effort.  But walk around Harvard Square on a Saturday night, and you will struggle to find an (American) student smoking a cigarette, an absence that would have leapt out for its strangeness not all that long ago.
So we might then ask ourselves: Can we use these kinds of techniques to change student behavior regarding the all-male Final clubs and other USGSOs?  Are there no such ordinary means by which to drain these clubs of their vitality (or to “shrivel” them, as a colleague has colorfully put it)?  Again, we have been led to believe not.  But many of us are skeptical of this claim.  Thus, my sense that when we look past the legislative motions and parliamentary maneuvers, the blog posts and leaks to The Crimson, a good deal of opposition to the sanction policies flows from a desire to try—seriously for the first time—to rein in the Final clubs through a full suite of methods that we ordinarily use to change social behavior.[1]  That is, we have not had—but should be having—a full-throated conversation about whether we can reach our shared goals in ways that do not require us to compromise other core institutional values.  I am not convinced we can, but many of us believe it is worth first trying. 
However, any serious attempt to use such “ordinary” measures to undermine the Final clubs’ influence on campus needs to start from an analysis of why exactly they play such an outsized role in campus social life in the first place, and thus what the College must do to drain their vitality.  During the USGSO Committee discussions, we heard, in every meeting, that the Final clubs dominate undergraduate social life precisely because few good on-campus alternatives exist.  A similar point was made manifest in the Implementation Committee’s report: that if the College wants to rob the Final clubs of their appeal, it must start by creating attractive alternative social spaces for undergraduates.  Many of our students want a place to “have fun”—which we would do well to acknowledge means drinking alcohol, acting in mildly transgressive ways, and being out from under the watchful eye of tutors and resident deans.  Wonderful as they are, the Houses do not—and perhaps cannot ever—fully serve that desire.  And although I resist the notion that Harvard College is somehow obliged to administer its students an appropriate dosage of fun (surely, something somewhere in the Boston area caters to the needs of college students?), we should acknowledge that the (real or) perceived lack of alternative spaces for “letting loose” remains a powerful draw of the Final clubs for our students.  Thus, draining energy away from the Final clubs will require that we direct it elsewhere.
Finally, it is impossible not to comment on the current campus morass without also noting the deep and abiding concerns of the Faculty regarding its role in informing College policy.  The implementation of either sanctions policy will permanently reshape the relationship between the faculty and our students (perhaps for the better, perhaps not).  At the same time, however, the specific way in which these policies have been advanced threatens at theh same time to alter the relationship between the Faculty and its Administration.  Many of us—including many of us who would otherwise not be opposed to taking extraordinary measures against the USGSOs—are deeply disturbed by what we view as unprecedented administrative overreach, including the widespread perception that our Administration is committed to avoiding a faculty vote on the proposed policies.  From my conversations with many colleagues, it is hard to overstate how divisive and demoralizing this posture towards the Faculty has been, not least because it could have been avoided in the first place.  In many ways, it is this aspect of our current situation that troubles me most.
One final note, this one of appreciation for my fellow committee members—students, staff, and faculty alike—for their unfailing civility, eloquence, and clarity of thought throughout our discussions; you have been a continual reminder of the things that make Harvard great.  Suzannah Clark deserves special recognition for her thoughtful leadership of the Committee, and for her truly herculean efforts on our behalf. 

Jason Mitchell
Professor of Psychology

[1] As a side note, I object to the Final Report’s characterization that opponents of the sanctions policy “argue that suasion is always better than sanctions” (p. 14).  That statement does not reflect my understanding of the discussion within the Committee, nor do I know any colleagues who traffic in such absolutes.  A more accurate statement might be to suggest that some opponents of the sanction policies are arguing that suasion (or other ordinary measures) are in this instance a better course of action right now than the sanction policies as formulated.

by Harry Lewis ( at October 04, 2017 03:17 PM

Professor Steven Pinker's remarks at the October 3 Faculty meeting
I was opposed to the recommendations of USGSO report, and support the motion by Professor Lewis to rule out such draconian measures in the future.  I believe they run afoul of the principles on which liberal education are based.

First, a human institution is not an omnipotent embodiment and enforcer of morality but must be grounded in a social contract with circumscribed responsibilities. A university contracts with its students to provide them with an education. It does not require them to submit to control over their lives, 24/7. Legal activities that students do on their own time and off university premises are none of the university’s business.

Second, a university is not a religion with a mandatory creed. One of the essential values in higher education is that people can differ in their values, and that these differences can be constructively discussed. Harvard has a right to value mixed-sex venues everywhere, all the time.  If some of its students find value in private, single-sex associations, some of the time, a university administration is free to argue against, discourage, or even ridicule those choices. But it is not a part of the mandate of a university to impose these values on its students over their objections.

Third, universities ought to be places where issues are analyzed, distinctions are made, evidence is evaluated, and policies crafted to attain clearly stated goals. Policies that restrict students’ freedoms should not be symbolic statements of values; they should be means to justifiable ends. But punishing students for belonging to private organizations is a sledgehammer. It doesn’t distinguish between single-sex and other private clubs. It doesn’t target illegal or objectionable behavior such as drunkenness or public disturbances. Nor by any stretch of the imagination could it be seen as an effective, rationally justified, evidence-based policy for reducing sexual assault. As my colleague Jason Mitchell argued in his minority report, there are plenty of proven ways of altering behavior between the extremes of moral suasion and authoritarian prohibition.

Finally, and perhaps most important, the policy of banning students from private organizations is widely seen outside Harvard as exemplifying some of the worst tendencies of elite universities. It can only contribute to the impression that universities are not dispassionate forums for clarifying values or analyzing problems but institutions determined to impose their ideology on a diverse population by brute force. In an era in which the credibility of reason-based institutions is vital yet endangered, this can have pernicious effects.

Let me be concrete. Those of us who engage in argument with intelligent people on the opposite end of the political spectrum often encounter the objection that the near-consensus among academic scientists (on climate change, for example), cannot be trusted. Everybody knows, they say, that university research is distorted by the political agenda of elites trying to exert control over individual choices. “No, no,” we insist; “Universities aren’t like that; we open-mindedly identify problems and try to come up with solutions.” A policy that is widely seen by the outside world as repressive virtue-signaling makes our job that much harder.

by Harry Lewis ( at October 04, 2017 02:55 AM

Professor Eric Nelson's remarks at the Faculty meeting of October 3
I would like us to reflect for just a moment on how we got here.  Two years ago, a University task force on sexual assault asserted that final clubs were responsible for a wave of sexual violence against women on our campus.  The administration responded to this report by announcing an unprecedented set of sanctions against undergraduates who join unrecognized social groups, including final clubs and sororities.  As the subsequent debate unfolded, it emerged that the task force’s claim about the link between final clubs and sexual violence was false, and that the data on which it had relied in making this claim had been misconstrued.  From that point on, we ceased to hear anything from the administration and its supporters about the problem of sexual assault—although it must be said that the task force’s assertion has not to date been retracted, nor has any apology been offered to the large numbers of Harvard students and alumni who were mistakenly (and very publicly) branded as sexual predators by the University. 

The rationale for the sanctions then shifted; the danger was no longer assault, but discrimination based on gender.  Single-gender social organizations were now said to be no less odious in principle than racially segregated ones—a remarkable finding from a College that admits hundreds of students each year from single-gender schools, maintains a host of single-gender sports teams, clubs, and performing groups, and divides students by gender in undergraduate housing.  Presumably we would not do these things if we regarded them as tantamount to Jim Crow.  Indeed, I wonder if it has been noticed that several of the fellowships from which the administration proposes to exclude members of single-sex clubs are themselves tenable at single-sex institutions. 

But before a proper debate could be had about this revised rationale, it shifted yet again—this time to the value of inclusion, full stop.  On this view, the final clubs and sororities were to be anathematized, not for admitting members of only one sex, but for choosing their members at all.  The question became whether it was so urgent to rescue some Harvard students from the discomfort of rejection that we ought to deny all of them the right to form any intimate associations of like-minded peers, even off-campus.  But this too is now old news.  The most recent justification for the proposed ban seems to be that we are worried about the integrity of the residential system at Harvard—the prospect of undergraduates fleeing the houses to live in a “Greek” world of fraternities and sororities.  This despite the fact that 99% of undergraduates voluntarily elect to live in the houses—and that, if we are truly worried about this issue, we remain perfectly free to require undergraduates to live on campus, whatever the fate of the Lewis motion. 

When the reasons offered for a given policy change as frequently as they have in this case, we should begin to wonder whether the policy in question has anything to do with reasons.  The bottom line here, as it seems to me, is that we just don’t like these clubs.  And some of them, at least, clearly merit our dislike.  But the first principle of a liberal arts education—and of the liberal society to which we are all rightly committed—is that disliking something is an insufficient reason to punish people for doing it.  Here, I think, we might profitably learn from our students, two-thirds of whom rejected the proposed sanctions in a referendum last year.

Lastly, I think we need to take seriously the broader national context in which this debate is unfolding.  It is news to no one in this room that we are currently facing a concerted effort by dangerous opponents to paint Harvard and our peer institutions as bastions of ideological groupthink, in which a frenzied and menacing political correctness has replaced common sense—and in which freedom of speech and association are routinely sacrificed upon the altar of diversity and inclusion.  I do not think this is who we are, but it strikes me that, over the last two years or so—on issue after issue—we have been doing our level best to make it easier for these opponents to caricature us.  This would perhaps be a price worth paying if we were right on the merits—but, as it happens, I believe we have not been.  We have been getting it wrong and looking foolish in the process.  My suggestion is that we ought to stop.  There are, alas, very real battles ahead of us, for which we will require the support of a united faculty, student body, and alumni community—to say nothing of our fellow citizens.  I don’t see why on earth we would further risk that support merely to destroy a bunch of tweedy Victorian relics.

That there are problems with undergraduate social life at Harvard, no one will deny.  But, as the committee report helpfully notes, these have primarily resulted from decisions that we ourselves have taken—and it is our responsibility to address them.  Let us turn to that important business.

I urge colleagues to support the motion.

Thank you.

by Harry Lewis ( at October 04, 2017 02:32 AM

October 03, 2017

Harry Lewis
My remarks introducing the motion on clubs
There was a good discussion in the FAS faculty meeting. The matter will be put to a vote on November 7. Here is what I said:

I move: HarvardCollege shall not discipline,penalize, or otherwise sanctionstudents for joining, oraffiliating with, any lawful organization,political party, or social, political,or other affinity group.

This is a simple motion. It says Harvard College can’t punish students for joining a club. It does NOT say that students who belong to clubs can’t be punished for bad things they do. It does NOT take away any tool that has been used in the past to discipline students for their behavior. It would, however, block several social club policies that have been proposed over the past year and a half.

I cannot find a single case prior to May 2016 when Harvard said it would punish a student for joining any organization -- a club or anything else. To the contrary, when Harvard barred ROTC from campus, we explicitly rejected the idea of punishing ROTC students for joining a discriminatory organization. And in the 1950s, when Senator McCarthy called on Harvard to fire one of us, Wendell Furry of the Physics Department, for being a member of the Communist Party, President Pusey refused on principle, in spite of enormous political pressure and his own anti-communist sentiments. Harvard today holds the moral high ground. We would give it up if we were to adopt any policy that would punish students for joining a club.

Some who are concerned about my motion have asked me, “but what if a student joins X”—and then name some particularly odious national organization. Well, we have survived a long time without any rules against joining hated organizations. This is not the time to institute such a rule in order to crush some off-campus sorority.

Students should not give up their rights peaceably to assemble off campus when they enroll here, any more than they give up their rights to read, write, and say what they wish. Indeed, by becoming students they do not give up their right to have private lives. All these freedoms are fundamental to our educational mission.

In a Faculty meeting last year, I teasingly referred to the possibility of an Index of Prohibited Organizations, like the Index of Prohibited Books of the medieval Church. Little did I expect that the Clark-Khurana Committee would publish exactly such an Index—in fact a list that was expanded beyond what had been proposed before the committee reviewed the policy. Let’s not go down the path of trying to maintain a list of the sort that even the Roman Church eventually realized was a bad idea.

If we can’t remember history, at least let’s look to the future. Suppose we publish a list of clubs and punish their members. What will we do when government officials again demand that we punish members of some allegedly un-American group? In the year of the Muslim ban, would anyone be surprised if the government tried to put us to the test? Would we say, “Oh no. At Harvard, we suspend civil liberties onlyfor organizations that threaten our deepest values, like the Bee and the Owl, not the ones you think are bad for the nation.”

I am grateful for the hard work of the committees that have worked on this difficult task, but I must note how little is said in their reports about the social structures they seek to destroy. The caricature of off-campus clubs as bastions of privilege, full of the stock of the Puritans learning to discriminate against other people, is not based in fact, certainly not in any facts presented in the report. Indeed, the report contains almost no facts of any kind. It does not even mention that more women than men are members of affected clubs. There is no data showing how many incidents have been reported at which clubs. That data might have shown that most of the trouble is caused by only a handful of the clubs, including only a few of the men’s and coed clubs and none of the women’s clubs. That would suggest that a narrower remedy made more sense than the broader ones that are proposed.

Data may be hard to come by, but then how will the College know who is in these private organizations? The report doesn’t say. Will we encourage students to turn each other in?

It is not true that everything else has been tried to combat bad behavior at the problematic clubs. There is no right to unpeaceable assembly; we should call in the police when students break the law. And we should tell students which clubs are dangerous places, and why. When muggings occur in Cambridge, we don’t just say, “there is crime in Cambridge, so students must stay on campus.” We tell them where they shouldn’t go, explain why, and expect them to protect themselves. To the extent that Harvard’s legal liability is driving any of this, or indeed to the extent that we are worried about student safety, education would be more effective as well as more appropriate.

I urge you to read Jason Mitchell’s superb minority report. From the beginning this has been an attempt to kill the men’s final clubs without much concern for the collateral damage from making a much broader rule. Let’s be clear what problem we are trying to solve and then go straight after it. Strengthening the Houses does not require punishing students for hanging out off campus sometimes. Opening “networks of power” to women does not require destroying the networks they have created for themselves.

And there is no silver bullet in Professor Allen’s astonishingly sweeping motion either. Toestablish policies that protect individual freedoms while upholding the educational mission of the College” is exactly what committees have been trying to do for a year; it is time for a statement of principle from the faculty, not a carte blanche handoff to the administration. As the Clark-Khurana committee notes, the Allen motion raises but does not answer the question of what to do if the clubs do not cooperate. Punishing their members is not the right answer.

I beg you, this is not a trivial matter. Students engaged in unlawful or violent behavior should pay a price for what they do. But nobody should be punished just for joining a club. Not us, and not our students. Thank you.

Added after the meeting. A medievalist points out that the Index was NOT, in fact, a medieval invention; it emerged in the sixteenth century. In other words, it was a reaction to the Enlightenment, not a piece of pre-Enlightenment church culture. I regret the error.

by Harry Lewis ( at October 03, 2017 11:05 PM

October 02, 2017

Justin Reich
To Ban or Not to Ban? Technology, Education, and the Media
Before citing research or editorials as a rationale to ban technology, we need to question the underlying assumptions that the authors make about student learning.

by Beth Holland at October 02, 2017 09:57 PM

Berkman Center front page
In AI We Trust?


Do we already have the necessary trust in AI, and if not, how do we create it?

Thumbnail Image: 

Urs Gasser shares some thoughts on how we learn to trust new technologies in a time of rapid change:

We are witnessing a wave of AI-based technologies that make their way out of the labs into industry- and user-facing applications, and we know from history that trust is an important factor that shapes the adoption of new technology. Given today’s quicksilver AI environment, it seems fair to ask: Do we already have the necessary trust in AI, and if not, how do we create it?

Read Urs Gasser's Medium post

by djones at October 02, 2017 05:47 PM

September 27, 2017

Berkman Center front page
How to Watch Them Watching You


Researching Social Media, Online Platforms, and Algorithmic Systems From the Outside


Join us at the University of Michigan for a discussion with Eric Gilbert, Cedric Langbort, Jeff Larson, Casey Pierce, and Christo Wilson on how researchers can navigate and investigate huge and complex datasets, algorithms, and online platforms to improve transparency and accountability.

Event Date

Sep 29 2017 10:00am to Sep 29 2017 10:00am
Thumbnail Image: 


Friday, September 29, 2017
10-11:30 a.m. Eastern Daylight Time (UTC/GMT -4 hours)

6050 Institute for Social Research (map/directions)
University of Michigan
Ann Arbor, MI USA

Visit the full event page for more details

A public roundtable discussion. Online streaming video available with live questions taken from Twitter (#algoaudit) and the BKC Live Question Tool.


  • Eric Gilbert, University of Michigan
  • Cedric Langbort, University of Illinois
  • Jeff Larson, ProPublica
  • Casey Pierce, University of Michigan
  • Christo Wilson, Northeastern University


The equations of big-data algorithms have permeated almost every aspect of our lives. A massive industry has grown up to comb and combine huge data sets — documenting, for example, Internet habits — to generate profiles of individuals. These often target advertising, but also inform decisions on credit, insurance and more. They help to control the news or adverts we see, and whether we get hired or fired. They can determine whether surveillance and law-enforcement agencies flag us as likely activists or dissidents — or potential security or criminal threats….Largely absent from the widespread use of such algorithms are the rules and safeguards that govern almost every other aspect of life in a democracy. There is an asymmetry in algorithmic power and accountability…Fortunately, a strong movement for greater algorithmic accountability is now under way. Researchers hope to find ways to audit for bias….Society needs to discuss in earnest how to rid software and machines of human bugs.

–Unsigned Editorial, Nature (2016)


[We] need to create a new field around the social algorithm, which examines the interplay of social and computational code.

–David Lazer, Op-Ed, Science (2015)

by djones at September 27, 2017 05:15 PM

The Computer Says No


The Bad News About Online Discrimination in Algorithmic Systems


Join us at the University of Michigan for a discussion with Solon Barocas, J. Nathan Matias, H. V. Jagadish, and Christian Sandvig on the potential for discrimination and digital redlining posed by the development of purportedly neutral algorithms.

Event Date

Sep 28 2017 4:00pm to Sep 28 2017 4:00pm
Thumbnail Image: 


Thursday, September 28, 2017
4:00-5:30 pm Eastern Daylight Time (UTC/GMT -4 hours)

1430 Institute for Social Research (map/directions)
University of Michigan
Ann Arbor, MI USA

Visit the full event page for more details

A public roundtable discussion. Online streaming video available with live questions taken from Twitter (#algoaudit) and the BKC Live Question Tool.


  • Solon Barocas, Cornell University
  • J. Nathan Matias, Princeton University
  • H. V. Jagadish, University of Michigan
  • Christian Sandvig, University of Michigan


…[Personalization] leaves room for subtle and not-so-subtle forms of discrimination in pricing, services, and opportunities….[A]lgorithmic decisions raise the specter of ‘redlining’ in the digital economy–the potential to discriminate against the most vulnerable classes of our society under the guise of neutral algorithms.

–Executive Office of the President of the United States, Report on Big Data (2014)

by djones at September 27, 2017 05:05 PM

Juan Carlos De Martin
Editoriale: "Una riforma per gli onesti"
Una riforma per gli onesti

Juan Carlos De Martin

La Repubblica, p. 1, 27 settembre 2017

Nel pieno di uno scandalo come quello di Firenze, riguardante docenti universitari di diritto tributario, non è facile scrivere nulla che non sia un attacco puro e semplice alla corruzione in ambito universitario. Da un certo punto di vista è inevitabile e forse anche doveroso. Tuttavia la riflessione pubblica non può limitarsi all'ennesima geremiade sull'ennesimo scandalo.

Occorre innanzitutto chiedersi come mai le presunte panacee di questi ultimi otto anni non abbiano funzionato, dalla riforma Gelmini (approvata nel 2010 a colpi di fiducia), riforma che avrebbe dovuto scardinare il potere dei "baroni" e che invece ha verticalizzato il potere nelle università, agli algoritmi e alle "misure oggettive" dell'Agenzia nazionale di valutazione del sistema universitario e della ricerca (ANVUR), che avrebbero - secondo le promesse - dovuto creare un paradiso di meritocrazia in terra e che, invece, a quanto pare hanno solo modificato le modalità della corruzione, non la sostanza. E' evidente che un intero approccio, basato sull'accentramento del potere e su una montagna di regole e formalismi, ha mancato il bersaglio e andrebbe ripensato da zero.

Ma bisogna anche avere il coraggio, anche in un momento come questo di forte indignazione, di ricordare che abbiamo il dovere di ragionare non solo di casi (che siano 5, 15 o 50), ma anche e soprattutto di sistema universitario.

Perché non si discute del sistema universitario nello stesso modo in cui si ragiona, per esempio, di sistema sanitario nazionale, di Forze Armate o di Forze dell'Ordine, ovvero, valutando il sistema nel suo complesso? Solo così è possibile dare un contesto a qualsiasi fenomeno, inclusi quelli di malcostume o di illeciti. Non per sminuire o sviare l'attenzione, ma per capire, a testa fredda e dati alla mano. Quante università vogliamo? Distribuite come? Di quale dimensione? Con quante risorse complessive? Con quali salari? Con quale livello di diritto allo studio e con quali tasse?

E quando volessimo ragionare di prestazioni, il sistema universitario italiano come si confronta coi sistemi francese, inglese, tedesco, ecc.? Uno dei modi più immediatamente comprensibili per valutare un sistema nazionale sanitario è guardare, per esempio, alla durata media della vita dei cittadini, nel caso dell'Italia molto alta (tra l'altro con una spesa complessiva decisamente contenuta). Perché non facciamo quasi mai lo stesso col sistema universitario?

Se lo facessimo, scopriremmo che l'Università italiana si colloca solidamente e sistematicamente tra le prime dieci al mondo per la ricerca; e se normalizzassimo questo risultato per le risorse investite (l'Italia è il penultimo paese OCSE per finanziamento pubblico all'Università), sarebbe addirittura la prima al mondo.

E i 50.000 ricercatori italiani all'estero (quasi sempre forzati ad emigrare a causa della spaventosa carenza di posti in patria), come li giustifichiamo se non con un sistema perfettamente in grado di formare persone ai massimi livelli?

Questi dati di sistema giustificano forse nepotismo o corruzione? Ovviamente no. Ma ci dicono qualcosa con cui tutte le persone intellettualmente oneste dovrebbero necessariamente fare i conti: che è materialmente impossibile che l'Università italiana sia - come a volte viene descritta - un'istituzione popolata da lazzaroni o da incapaci. Ci deve per forza essere una maggioranza di docenti e ricercatori che lavora onestamente e con onore, altrimenti le buone prestazioni del sistema sarebbero semplicemente impossibili da spiegare.

Il fatto che un sistema nazionale sia sicuramente dignitoso e forse anche qualcosa di più - come credo sia ragionevole sostenere a riguardo sia del sistema sanitario italiano, sia di quello universitario sia di quello delle Forze dell'Ordine - significa che allora va tutto bene, che possiamo tranquillamente ignorare scandali e altri problemi?

Ripeto: ovviamente no. E' doveroso contrastare con la massima energia sprechi, nepotismi, ecc., ovunque si presentino, e forse con un'energia ancora maggiore nel caso dell'Università, considerato l'alto ruolo educativo e civile che svolge l'istituzione (di cui faccio parte e che vorrei con tutte le mie forze fosse in grado di superare i suoi limiti).

Ma contemporaneamente dobbiamo tenere la testa fredda e valutare la situazione senza generalizzazioni indebite e senza scorciatoie demagogiche, avendo sempre a cura il futuro del sistema nel suo complesso. Da questo punto di vista chi oggi sostiene che gli scandali indeboliscono la richiesta di risorse per l'Università, prostrata da quasi dieci anni di tagli, non fa che assicurare una cosa: il definitivo scoraggiamento di chi va ogni giorno in aula, in laboratorio, in biblioteca.

Combattiamo senza pietà corruzione e nepotismi, ma assicuriamo contemporaneamente sia risorse adeguate, sia una ricerca di possibili soluzioni ai problemi che coinvolga, come non è mai stato fatto in passato, tutti i docenti e ricercatori, non solo i vertici accademici.

by Juan Carlos De Martin at September 27, 2017 04:54 PM

David Weinberger
[liveblog][pair] Blaise Agüera y Arcas on the source of bias

At the PAIR Symposium, Google’s Blaise Agüera y Arcas is providing some intellectual and historical perspective on AI issues.

NOTE: Live-blogging. Getting things wrong. Missing points. Omitting key information. Introducing artificial choppiness. Over-emphasizing small matters. Paraphrasing badly. Not running a spellpchecker. Mangling other people’s ideas and words. You are warned, people.

[Note: This is a talk tough to live-blog because it is carefully structured intellectually. My apologies.]

He says neural networks have been part of the computing environment from the beginning. E.g., he thinks that the loop at the end of the logic gate symbol in fact comes from a 1943 symbolization of biological neural networks. There are indications of neural networks in Turing’s early papers. So these ideas go way back. Blaise thinks that the majority of computing processes in a few years will be running on processors designed for running neural networks.

ML has raised anxiety reminiscent of Walter Benjamin’s concern — he cites The Work of Art in the Age of Mechanical Reproduction — about the mass reproduction of art that strips it of its aura. Now there’s the same kind of moral panic about art and human exceptionalism and existence. (Cf. Nick Bostrom’s SuperIntelligence). It reminds him of Jakob Mohr’s 1910 The Influencing Machine in which schizophrenics believe they’re being influenced by an external machine. (They always thought men were managing the machine.) He points to what he calls Bostrom’s ultimate colonialism, in which we are able to populate the universe with 10^58 human minds. [Sorry, but I didn’t get this. My fault.] He ties this to Bacon’s reverence for the domination of nature. Blaise prefers a feminist view, citing Kember & Zylinksa’s Life After New Media.

Many say we have a value alignment problem, he says: how do we make AI that embeds human values? But AI systems do have human values because they’re trained on human data. The problem is that our human values are off. He references a paper on judging criminality based on faces. The paper claims it’s free of human biases. But it’s based on data that is biased. Nevertheless, this sort of tech is being commercialized. E.g., Faception claims to classify people based on their faces: High IQ, Pedophile, etc.

Also, there’s the recent paper about a ML system classifies one’s gender preferences based on faces. Blaise ran a test on Mechanical Turk asking about some of the features in the composite gay and straight faces in that paper. He found that people attracted to the same sex were more likely to wear glasses. There were also significant differences in facial hair, use of makeup, and face tan, features also in the composite faces. Thus, the ML system might have been using social markers, not physiognomy, “There are a lot of tells.”

In conclusion, none of these are arguments against ML. On the contrary. The biases and prejudices, and the social signalling, are things ML lets us hold a mirror up to.

The post [liveblog][pair] Blaise Agüera y Arcas on the source of bias appeared first on Joho the Blog.

by davidw at September 27, 2017 12:48 AM

September 26, 2017

Berkman Center front page
Summer Internship Program: 2017 Wrap Up


The Berkman Klein Center summer internship program is a cohesive, integrated experience that brings together students from around the world to work closely with faculty, staff, and fellows on a wide range of projects and initiatives related to the internet and society. 

Thumbnail Image: 

The Berkman Klein Center summer internship program is a cohesive, integrated experience that brings together students from around the world to work closely with faculty, staff, and fellows on a wide range of projects and initiatives related to the internet and society. Over the last ten years, more than 400 interns have joined us in Cambridge to conduct research, exchange ideas, gain exposure to new areas for exploration and study, expand networks, and form friendships. Providing this 10-week intensive paid experience for students from a wide range of backgrounds and areas of interest, ranging from high school to PhD level, is one of the main ways the Center demonstrates its commitments to education, diversity, and network-building. The length of time and intensity of the experience allows interns to form lasting bonds with each other and with the Berkman Klein Center. Many interns return to join us as staffers or fellows, or stay in our network through ongoing projects or by joining a peer institution.

While the Center has provided opportunities for students to participate in our research projects since it’s earliest days, it was in 2008 that we began a deliberate effort to provide a structured community experience for a cohort of interns, and since then the program has grown considerably.  In 2008, we welcomed 31 interns drawn from a pool of 66 applicants. This summer, we welcomed nearly 50 interns to Cambridge selected from a pool of more than 1,000 applicants. From the beginning, the summer interns have reflected the global nature of our community, with seven countries represented in our 2008 cohort, and more than 15 countries, including Colombia, Spain, Nigeria, and India, represented in 2017.

This summer, interns worked on more than 17 projects around the Center, including the Ethics and Governance of Artificial Intelligence, Youth and Media, Harmful Speech, Privacy Tools, Lumen, Internet Monitor, the Harvard Open Access Project, Global Access in Action, and the Cyberlaw Clinic. In addition to working closely with their research teams, interns had opportunities to showcase their expertise and interests through skillshares and formal talks, and were able to get to know the Berkman Klein community through weekly programmed “intern hours,” and BKC public events, and more informally through social gatherings and connections formed from working amidst the activity here at 23 Everett Street. Each summer, interns also participate in a group project, which in the past has resulted in explainer videos, interviews with incoming community members, and, this summer, online learning experiences for students on topics such as cybersecurity, digital health, and freedom of expression. These learning experiences will be published and made freely available on the Digital Learning Resources Platform.

For an intern’s inside look at the summer 2017 experience, visit Summer Snapshot 2017, created by BKC communications intern Tym Yee. And check out the the summer internship page for more information about the program and to learn when and how to apply.

by gweber at September 26, 2017 07:20 PM

David Weinberger
[liveblog][pair] Golan Levin

At the PAIR Symposium, Golan Levin of CMU is talking about ML and art.

NOTE: Live-blogging. Getting things wrong. Missing points. Omitting key information. Introducing artificial choppiness. Over-emphasizing small matters. Paraphrasing badly. Not running a spellpchecker. Mangling other people’s ideas and words. You are warned, people.

The use of computers for serendipitous creativity has been a theme of computer science since its beginning, Golan says. The job of AI should be serendipity and creativity. He gives examples of his projects.

Put your hand up to a scanner and it shows you hand with an extra finger. Or with extra hands at the end of your fingers.

Augmented Hand Series (v.2), Live Screen Recordings from Golan Levin on Vimeo.

[He talks very very quickly. I’ll have to let the project videos talk for themselves. Sorry.]

Terrapattern provides orbital info about us. It’s an open source neural network tool which offers similar-image search for satellite imagery. It’s especially good at finding “soft” structures often not noted on maps. E.g., click on a tennis court and it will find you all of them in the area. Click on crossroads, same thing.

Terrapattern (Overview & Demo) from STUDIO for Creative Inquiry on Vimeo.

This is, he says, an absurdist tool of serendipity. But it also democratizes satellite intelligence. His favorite example: finding all the rusty boats floating in NYC harbor.

Next he talks about our obsession with “masterpieces.” Will a computer ever be able to create masterpiece, he keeps getting asked. But artworks are not in-themselves. They exist in relationship to their audience. (He recommends When the Machine Made Art by Grant D. Taylor.)

Optical illusions get us to see things that aren’t there. “Print on paper beats brain.” We see faces in faucets and life in tree trunks. “This is us deep dreaming.” The people who understand this best are animators. See The illusion of Life, a Disney book about how to make things seem alive.

The observer is not separate from the object observed. Artificial intelligence occurs in the mind as well as in the machine.

He announces a digression: “Some of the best AI-enabled art is being made by engineers,” as computer art was made by early computer engineers.

He points to the color names ML-generated by Janelle Shane. And Gabriel Goh’s synthetic porn. It uses Yahoo’s porn detector and basically runs it in reverse starting with white noise. “This is conceptual art of the highest order.”

“I’m frankly worried, y’all,” he says. People use awful things using imaging technology. E.g., face tracking can be abused by governments and others. These apps are developed to make decisions. And those are the thoughtless explicit abuses, not to mention implicit biases like HP’s face scanning software that doesn’t recognize black faces. He references Zeynep Tufecki’s warnings.

A partial, tiny, and cost-effective solution: integrate artists into your research community. [He lists sensible reasons too fast for me to type.]

The post [liveblog][pair] Golan Levin appeared first on Joho the Blog.

by davidw at September 26, 2017 07:00 PM

[liveblog][PAIR] Rebecca Fiebrink on how machines can create new things

At the PAIR symposium, Rebecca Fiebrink of Goldsmiths University of London asks how machines can create new things.

NOTE: Live-blogging. Getting things wrong. Missing points. Omitting key information. Introducing artificial choppiness. Over-emphasizing small matters. Paraphrasing badly. Not running a spellpchecker. Mangling other people’s ideas and words. You are warned, people.

She works with sensors. ML can allow us to build new interactions from examples of human action and computer response. E.g., recognize my closed fist and use it to play some notes. Add more gestures. This is a conventional suprvised training framework. But suppose you want to build a new gesture recognizer?

The first problem is the data set: there isn’t an obvious one to use. Also, would a 99% recognition rate be great or not so much? It depends on what was happening. IF it goes wrong, you modify the training examples.

She gives a live demo — the Wekinator — using a very low-res camera (10×10 pixels maybe) image of her face to control a drum machine. It learns to play stuff based on whether she is leaning to the left or right, and immediately learns to change if she holds up her hand. She then complicates it, starting from scratch again, training it to play based on her hand position. Very impressive.

Ten years ago Rebecca began with the thought that ML can help unlock the interactive potential of sensors. She plays an early piece by Anne Hege using Playstation golf controllers to make music:

Others make music with instruments that don’t look normal. E.g., Laetitia Sonami uses springs as instruments.

She gives other examples. E.g., a facial expression to meme system.

Beyond building new things, what are the consequences, she asks?

First, faster creation means more prototyping and wider exploration, she says.

Second, ML opens up new creative roles for humans. For example, Sonami says, playing an instrument now can be a bit wild, like riding a bull.

Third, ML lets more people be creators and use their own data.

Rebecca teaches a free MOC on Kadenze
: Machine learning for artists and musicians.

The post [liveblog][PAIR] Rebecca Fiebrink on how machines can create new things appeared first on Joho the Blog.

by davidw at September 26, 2017 06:33 PM

Cyberlaw Clinic - blog
Clinic Files Amicus Brief on Behalf of Members of Congress in Support of Access to Law

On September 25, 2017, the Cyberlaw Clinic and local counsel Catherine Gellis filed an amicus brief on behalf of members of Congress Zoe Lofgren (D-CA 19th District) and Darrell Issa (R-CA 49th District) in the United States Court of Appeals for the District of Columbia Circuit. The brief supports defendant-appellant (Public Resource) in the case American Society of Testing Engineers (ASTM) et. al. v., Case No. 17-7035 (D.C. Cir.). The appeal — a consolidation of two district court cases, both filed by standard developing organizations (SDOs) — addresses the copyrightability of the law and standards incorporated therein. The crux of the case is whether the text of applicable law may be shared freely by non-profit organizations like Public Resource. 

When model codes and standards become part of federal, state, or local regulations, the text is often not reproduced in the location where the law is published. Rather, citizens interested in reading the content of enacted statutes and regs must access the incorporated materials via the SDOs’ publication channels. These may come with high access fees or remain incompatible with online accessibility tools for the disabled. Public Resource acquired copies of a number of standards and codes, made them public, and was sued for copyright and trademark infringement by the SDOs.

The Clinic previously filed amicus briefs on behalf of legal scholars in support of Public Resource in both cases brought in the United States District Court for the District of Columbia.  The district court ruled in favor of both sets of plaintiffs-appellees, the “ASTM Plaintiffs”—ASTM, National Fire Protection Association, Inc. (NFPA), and American Society of Heating, Refrigerating, and Air-Conditioning Engineers (ASHRAE)—and the “AERA Plaintiffs”—American Education Resource Association (AERA), American Psychological Association (APA), and the National Council on Measurement in Education (NCME), finding copyright and trademark infringement in the publication on Public Resource’s website of model codes and standards incorporated into law.

The Brief:

In this brief, Members of Congress weighed in to discuss the public policy implications of the lower courts’ decisions. The brief discusses possible constitutional due process implications of SDOs’ using copyright to restrict on access to laws, noting that “[a] citizen’s ability to take notice of a law or regulation depends on that citizen’s ability to access the text of such law or regulation,” and that “[t]here can be no due process when people cannot remain informed of the laws by which they are bound.”

Amici also expressed concerns on the dual impact of copyright restrictions on general welfare, namely: (1) allowing SDOs to maintain their copyright interest in incorporated model codes “would open the floodgates to claims of joint authorship by all the various stakeholders who contribute to drafting laws,” tampering with the lawmaking process by undermining the fundamental principle that the law belongs to the public; and (2) restricting access to the law also poses a threat to public safety—the poor may have trouble paying the high model code access fees to ensure that their housing is safe; the disabled may not be able to access the model codes due to the incompatibility of the SDO platforms with accessibility tools.

Finally, the brief argues that, even if SDOs maintain an interest in the law, Public Resource’s open publication of the law is fair use. The model codes and standards are factual—not creative—in nature, and online reproduction of relevant material is reasonable and necessary, because “increasing public access to information constitutes a public benefit.”

Other Amicus Briefs in Support of Public Resource:

Four other amicus briefs were filed in support of Public Resource. Public Knowledge filed a  brief on behalf of a group of library associations, non-profit organizations, legal technology companies, former government officials, librarians, innovators, and law professors addresses the societal detriment of restricting access to the law—stifling innovation, fostering discrimination and bias, and violating the First Amendment right to information.

Non-profit organizations Public Citizen, Center for Science in the Public Interest, Consumers Union, National Employment Law Project, and United StatesPublic Interest Research Group, filed a brief discussing the importance of open access to regulatory texts in preserving public and environmental safety. These organizations argue that restricting access to the law would “impede the public’s ability to participate in rulemaking proceedings, engage in discussion of regulatory requirements, challenge illegal regulations, and hold those who violate regulations accountable for their actions.”

The Juelsgaard Intellectual Property and Innovation Clinic at Stanford Law School filed a brief on behalf of Sina Bahram, a digital accessibility researcher and advocate for disabled individuals. Bahram argues that Public Resource publication of model codes constitutes fair use because it provided a transformative service to the disabled in creating an accessible online format of regulatory codes. He asserts that SDO-provided online reading platforms are incompatible with accessibility tools and hence, violate the American Disabilities Act requirement that prohibits discrimination against the disabled.

Finally, a group of intellectual property filed an amicus brief addressing the trademark arguments advanced by plaintiffs-appellees, arguing that the trademark infringement claims should be barred because the SDOs objections do not create a separate trademark cause of action distinct from copyright.


Special thanks to HLS Cyberlaw Clinic students Evelyn Chang, Jillian Goodman, and Anderson Grossman who worked closely with clinical supervisors Kendra Albert and Chris Bavitz to write the brief. (Evelyn also played a key role in drafting this blog post!) Cyberlaw Clinic summer interns Steven Deolus and Teddi Josephson did a lot of the legwork that led to the filing of the brief, and Clinical Instructor Jessica Fjeld was instrumental in crafting and honing the arguments. The Clinic is also grateful to students Michael Shafer and Ben Shiroma for their assistance with filing.

Court of Appeals Image courtesy of the United States Court of Appeals for the District of Columbia Circuit website, (last accessed September 26, 2017).

by Clinic Staff at September 26, 2017 06:27 PM

David Weinberger
[liveblog][PAIR] Doug Eck on creativity

At the PAIR Symposium, Doug Eck, a research scientist at Google Magenta, begins by playing a video:

Douglas Eck – Transforming Technology into Art from Future Of StoryTelling on Vimeo.

Magenta is part of Google Brain that explores creativity.
By the way:

NOTE: Live-blogging. Getting things wrong. Missing points. Omitting key information. Introducing artificial choppiness. Over-emphasizing small matters. Paraphrasing badly. Not running a spellpchecker. Mangling other people’s ideas and words. You are warned, people.

He talks about three ideas Magenta has come to for “building a new kind of artist.”

1. Get the right type of data. It’s important to get artists to share and work with them, he says.

Magenta has been trying to get neural networks to compose music. They’ve learned that rather than trying to model musical scores, it’s better to model performances captured as MIDI. They have tens of thousands of performances. From this they were able to build a model that tries to predict the piano roll view of the music. At any moment, should the AI stay at the same time, stacking up notes into chords, or move forward? What are the next notes? Etc. They are not yet capturing much of the “geometry” of, say, Chopin: the piano-roll-ish vision of the score. (He plays music created by ML trained on scores and one trained on performances. The score-based on is clipped. The other is far more fluid and expressive.)

He talks about training ML to draw based on human drawings. He thinks running human artists’ work through ML could point out interesting facets of them.

He points to the playfulness in the drawings created by ML from simple human drawings. ML trained on pig drawings interpreted a drawing of a truck as pig-like.

2. Interfaces that work. Guitar pedals are the perfect interface: they’re indestructible, clear, etc. We should do that for AI musical interfaces, but the sw is so complex technically. He points to the NSyth sound maker and AI duet from Google Creative Lab. (He also touts deeplearn.js.)

3. Learning from users. Can we use feedback from users to improve these systems?

He ends by pointing to the blog, datasets, discussion list, and code at

The post [liveblog][PAIR] Doug Eck on creativity appeared first on Joho the Blog.

by davidw at September 26, 2017 06:11 PM

[liveblog][pair] John Zimmerman on UI for AI…and making AI the new UI

At the PAIR symposium, John Zimmerman is giving a great talk on UX for AI. But it relies on graphics that I can’t capture, and I’m about to run out of battery. Sorry :(

The post [liveblog][pair] John Zimmerman on UI for AI…and making AI the new UI appeared first on Joho the Blog.

by davidw at September 26, 2017 04:32 PM

[lliveblog][PAIR] Antonio Torralba on machine vision, human vision

At the PAIR Symposium, Antonio Torralba asks why image identification has traditionally gone so wrong.

NOTE: Live-blogging. Getting things wrong. Missing points. Omitting key information. Introducing artificial choppiness. Over-emphasizing small matters. Paraphrasing badly. Not running a spellpchecker. Mangling other people’s ideas and words. You are warned, people.

If we train our data on Google Images of bedrooms, we’re training on idealized photos, not real world. It’s a biased set. Likewise for mugs, where the handles in images are almost all on the right side, not the left.

Another issue: The CANNY edge detector (for example) detects edges and throws a black and white reduction to the next level. “All the information is gone!” he says, showing that a messy set of white lines on black is in fact an image of a palace. [Maybe the White House?] (A different example of edge detection:)


Deep neural networks work well, and can be trained to recognize places in images, e.g., beach. hotel room, street. You train your neural net and it becomes a black box. E.g., how can it recognize that a bedroom is in fact a hotel room? Maybe it’s the lamp? But you trained it to recognize places, not objects. It works but we don’t know how.

When training a system on place detection, we found some units in some layers were in fact doing object detection. It was finding the lamps. Another unit was detecting cars, another detected roads. This lets us interpret the neural networks’ work. In this case, you could put names to more than half of the units.

How to quantify this? How is the representation being built? For this: Network dissection. This shows that when training a network on places, objects emerges. “The network may be doing something more interesting than your task.”The network may be doing something more interesting than your task: object detection is harder than place detection.

We currently train systems by gathering labeled data. But small children learn without labels. Children are self-supervised systems. So, take in the rgb values of frames of a movie, and have the system predict the sounds. When you train a system this way, it kind of works. If you want to predict the ambient sounds of a scene, you have to be able to recognize the objects, e.g., the sound of a car. To solve this, the network has to do object detection. That’s what they found when they looked into the system. It was doing face detection without having been trained to do that. It also detects baby faces, which make a different type of sound. It detects waves. All through self-supervision.

Other examples: On the basis of one segment, predict the next in the sequence. Colorize images. Fill in an empty part of an image. These systems work, and do so by detecting objects without having been trained to do so.

Conclusions: 1. Neural networks build represntations that are sometimes interpretatble. 2. The rep might solve a task that’s evem ore interesting than the primary task. 3. Understanding how these reps are built might allow new approaches for unsupervised or self-supervised training.

The post [lliveblog][PAIR] Antonio Torralba on machine vision, human vision appeared first on Joho the Blog.

by davidw at September 26, 2017 04:25 PM

[liveblog][PAIR] Maya Gupta on controlling machine learning

At the PAIR symposium. Maya Gupta runs Glass Box at Google, which looks at black box issues. She is talking about how we can control machine learning to do what we want

NOTE: Live-blogging. Getting things wrong. Missing points. Omitting key information. Introducing artificial choppiness. Over-emphasizing small matters. Paraphrasing badly. Not running a spellpchecker. Mangling other people’s ideas and words. You are warned, people.

The core idea of machine learning are its role models, i.e., its training data. That’s the best way to control machine learning. She’s going to address by looking at the goals of controlling machine learning.

A simple example of monotinicity. Let’s say we’re tring to recommend nearby coffee shops. So we use data about the happiness of customers and distance from the shop. We can fit the model ot a linear model. Or we can fit it to a curve, which works better for nearby shops but goes wrong for distant shops. That’s fine for Tokyo but terrible for Montana because it’ll be sending people many miles away. A montonic example says we don’t want to do that. This controls ML to make it more useful. Conclusion: the best ML has the right examples and the right kinds of flexibility. [Hard to blog this without her graphics. Sorry.] See “Deep Lattice Networks for Learning Partial Monotonic Models,” NIPS 2017; it will soon by on the TensorFlow site.

“The best way to do things for practitioners is to work next to them”The best way to do things for practitioners is to work next to them.

A fairness goal: e.g., we want to make sure that accuracy in India is the same as accuracy in the US. So, add a constraint that says what accuracy levels we want. Math lets us do that.

Another fairness goal: the rate of positive classifications should be the same in India as in the US, e.g., rate of students being accepted to a college. In one example, there is an accuracy trade-off in order to get fairness. Her attitude: Just tell us what you want and we’ll do it

Fairness isn’t always relative. E.g., E.g., minimize classification errors differently for different regions. You can’t always get what you want, but you sometimes can or can get close. [paraphrase!] See

It can be hard to state what we want, but we can look at examples. E.g., someone hand-labels 100 examples. That’s not enough as training date, but we can train the system so that it classifies those 100 at something like 95% accuracy.

Sometimes you want to improve an existing ML system. You don’t want to retrain because you like the old results. So, you can add in a constraint such as keep the differences from the original classifications to less than 2%.

You can put all of the above together. See “Satisfying Real-World Goals with Dataset Constraints,” NIPS, 2016. Look for tools coming to TensorFlow.

Some caveats about this approach.

First, to get results that are the same for men and women, the data needs to come with labels. But sometimes there are privacy issues about that. “Can we make these fairness goals work without labels? ”Can we make these fairness goals work without labels? Research so far says the answer is messy. E.g., if we make ML more fair for gender (because you have gender labels), it may also make it fairer for race.

Second, this approach relies on categories, but individuals don’t always fit into categories. But, usually if you get things right on categories, it usually works out well in the blended examples.

Maya is an optimist about ML. “But we need more work on the steering wheel.” We’re not always sure we want to go with this technology. And we need more human-usable controls.

The post [liveblog][PAIR] Maya Gupta on controlling machine learning appeared first on Joho the Blog.

by davidw at September 26, 2017 04:04 PM

[liveblog][PAIR] Hae Won Park on living with AI

At the PAIR conference, Hae Won Park of the MIT Media Lab is talking abiut personal social robots for home.

NOTE: Live-blogging. Getting things wrong. Missing points. Omitting key information. Introducing artificial choppiness. Over-emphasizing small matters. Paraphrasing badly. Not running a spellpchecker. Mangling other people’s ideas and words. You are warned, people.

Home robots no longer look like R2D2. She shows a 2014 Jibo video.

What sold people is the social dynamic: Jibo is engaged with family members.

She wants to talk about the effect of AI on people’s lives at home.

For example, Google Home changed her morning routine, how she purchases goods, and how she controls her home environment. She shows a 2008 robot called Autom, a weight management coach.

A studied showed that the robot kept people at it longer than using paper or a computer program, and people had the strongest “working alliance” with the robot. They also had emotional engagement with it, personalizing it, giving them names, etc. These users understand it’s just a machine. Why?

She shows a video of Leonardo, a social robot that exhibits bodily cues of emotion. We seem to share a mental model.

They studied how children tell stories and listen to each other. Jin Joo Lee developed a model in which the robot appears to be very attentive as the child tells a story. It notes cues about whether the speaker is engaged. The children were indeed engaged by this reactive behavior.

Researchers have found that social robots activate social thinking, lighting up the social thinking part of the brain. Social modeling occurs between humans and robots too.

Working with children aged 4-6, they studied “growth mindset”: the belief that you can get better if you try hard. Parents and teachers have been shown to affect this. They created a growth mindset robot that plays a game with the child. The robot encourages the child at times determined by a “Boltzmann Machine””>Boltzmann Machine. [Over my head.]

Their researc showed that playing puzzles with a growth-mindset robot fosters that mindset in children. For example, the children tried harder over time.

They also studied early literacy education using personalized robot tutors. In multiple studies of about 120 children. The robot, among other things, encourages the child to tell stories. Over four weeks, they found children more effectively learn vocabulary, and when the robot provided more expressive story telling (rather than speaking in an affect-less TTY voice) the children retained more and would mimic that expressiveness.

Now they’re studying fully automonmous storytelling robots. The robot uses the child’s responses to further engage the child. The children respond more, tell longer stories, and stayed engaged over longer periods across sessions.

We are headed toward a time when robots are more human-centered rather than task focused. So we need to think about making AI not just human-like but humanistic. We hope to make AI that make us better people.

The post [liveblog][PAIR] Hae Won Park on living with AI appeared first on Joho the Blog.

by davidw at September 26, 2017 04:02 PM

[liveblog][PAIR] Karrie Karahalios

At the Google PAIR conference, Karrie Karahalios is going to talk about how people make sense of their world and lives online. (This is an information-rich talk, and Karrie talks quickly, so this post is extra special unreliable. Sorry. But she’s great. Google her work.)

NOTE: Live-blogging. Getting things wrong. Missing points. Omitting key information. Introducing artificial choppiness. Over-emphasizing small matters. Paraphrasing badly. Not running a spellpchecker. Mangling other people’s ideas and words. You are warned, people.

Today, she says, people want to understand how the information they see comes to them. Why does it vary? “Why do you get different answers depending on your wifi network? ”Why do you get different answers depending on your wifi network? These algorithms also affect our personal feeds, e.g., Instagram and Twitter; Twitter articulates it, but doesn’t tell you how it decides what you will see

In 2012, Christian Sandvig and [missed first name] Holbrook were wondering why they were getting odd personalized ads in their feeds. Most people were unaware that their feeds are curated: only 38% were aware of this in 2012. Thsoe who were aware became aware through “folk theories”: non-authoritative explanations that let them make sense of their feed. Four theories:

1. Personal engagement theory: If you like and click on someone, the more of that person you’ll see in your feed. Some people were liking their friends’ baby photos, but got tired of it.

2. Global population theory: If lots of people like, it will show up on more people’s feeds.

3. Narcissist: You’ll see more from people who are like you.

4. Format theory: Some types of things get shared more, e.g., photos or movies. But people didn’t get

Kempton studied thermostats in the 1980s. People either thought of it as a switch or feedback, or as a valve. He looked at their usage patterns. Regardless of which theory, they made it work for them.

She shows an Orbitz page that spits out flights. You see nothing under the hood. But someone found out that if you use a Mac, your prices were higher. People started using designs that shows the seams. So, Karrie’s group created a view that showed the feed and all the content from their network, which was three times bigger than what they saw. For many, this was like awakening from the Matrix. More important, they realized that their friends weren’t “liking” or commenting because the algorithm had kept their friends from seeing what they posted.

Another tool shows who you are seeing posts from and who you are not. This was upsetting for many people.

After going through this process people came up with new folk theories. E.g., they thought it must be FB’s wisdom in stripping out material that’s uninteresting one way or another. [paraphrasing].

They let them configure who they saw, which led many people to say that FB’s algorithm is actually pretty good; there was little to change.

Are these folk theories useful? Only two: personal engagement and control panel, because these let you do something. But there are poor tweaking tools.

How to embrace folk theories: 1. Algorithm probes, to poke and prod. “It would be great, Karrie says, to have open APIs so people could create tools”(It would be great to have open APIs so people could create tools. FB deprecated it.) 2. Seamful interfaces to geneate actionable folk theories. Tuning to revert of borrow?

Another control panel UI, built by Eric Gilbert, uses design to expose the algorithms.

She ends with a wuote form Richard Dyer: “All technolgoies are at once technical and also always social…”

The post [liveblog][PAIR] Karrie Karahalios appeared first on Joho the Blog.

by davidw at September 26, 2017 02:56 PM

[liveblog][PAIR] Jess Holbrook

I’m at the PAIR conference at Google. Jess Holbrook is UX lead for AI. He’s talking about human-centered machine learning.

NOTE: Live-blogging. Getting things wrong. Missing points. Omitting key information. Introducing artificial choppiness. Over-emphasizing small matters. Paraphrasing badly. Not running a spellpchecker. Mangling other people’s ideas and words. You are warned, people.

“We want to put AI into the maker toolkit, to help you solve real problems.” One of the goals of this: “How do we democratize AI and change what it means to be an expert in this space?” He refers to a blog post he did with Josh Lovejoy about human-centered ML. He emphasizes that we are right at the beginning of figuring this stuff out.

Today, someone finds a data set, and finds a problem that that set could solve. You train a model and look at its performance, and decided if it’s good enough. And then you launch “The world’s first smart X. Next step: profit.” But what if you could do this in a human-centered way?

Human-centered design means: 1. Staying proximate. Know your users. 2. Inclusive divergence: reach out and bring in the right people. 3. Shared definition of success: what does it mean to be done? 4. Make early and often: lots of prototyping. 5. Iterate, test, throw it away.

So, what would a human-centered approach to ML look like? He gives some examples.

Instead of trying to find an application for data, human-centered ML finds a problem and then finds a data set appropriate for that problem. E.g., diagnosis plant diseases. Assemble tagged photos of plants. Or, use ML to personalize a “balancing spoon” for people with Parkinsons.

Today, we find bias in data sets after a problem is discoered. E.g., ProPublica’s article exposing the bias in ML recidivism predictions. Instead, proactively inspect for bias, as per JG’s prior talk.

Today, models personalize experiences, e.g., keyboards that adapt to you. With human-centered ML, people can personalize their models. E.g., someone here created a raccoon detector that uses images he himself took and uploaded, personalized to his particular pet raccoon.

Today, we have to centralize data to get results. “With human-centered ML we’d also have decentralized, federated learning”With human-centered ML we’d also have decentralized, federated learning, getting the benefits while maintaining privacy.

Today there’s a small group of ML experts. [The photo he shows are all white men, pointedly.] With human-centered ML, you get experts who have non-ML domain expertise, which leads to more makers. You can create more diverse, inclusive data sets.

Today, we have narrow training and testing. With human-centered ML, we’ll judge instead by how systems change people’s lives. E.g., ML for the blind to help them recognize things in their environment. Or real-time translation of signs.

Today, we do ML once. E.g., PicDescBot tweets out amusing misfires of image recognition. With human-centered ML we’ll combine ML and teaching. E.g., a human draws an example, and the neural net generates alternatives. In another example, ML improved on landscapes taken by StreetView, where it learned what is an improvement from a data set of professional photos. Google auto-suggest ML also learns from human input. He also shows a video of Simone Giertz, “Queen of the Shitty Robots.”

He references Amanda Case: “Expanding people’s definion of normal” is almost always a gradual process.

[The photo of his team is awesomely diverse.]

The post [liveblog][PAIR] Jess Holbrook appeared first on Joho the Blog.

by davidw at September 26, 2017 02:26 PM

[liveblog] Google AI Conference

I am, surprisingly, at the first PAIR (People + AI Research) conference at Google, in Cambridge. There are about 100 people here, maybe half from Google. The official topic is: “How do humans and AI work together? How can AI benefit everyone?” I’ve already had three eye-opening conversations and the conference hasn’t even begun yet. (The conference seems admirably gender-balanced in audience and speakers.)

NOTE: Live-blogging. Getting things wrong. Missing points. Omitting key information. Introducing artificial choppiness. Over-emphasizing small matters. Paraphrasing badly. Not running a spellpchecker. Mangling other people’s ideas and words. You are warned, people.

The great Martin Wattenberg (half of Wattenberg – Fernanda Viéga) kicks it off, introducing John Giannandrea, a VP at Google in charge of AI, search, and more. “We’ve been putting a lot of effort into using inclusive data sets.”

John says that every vertical will affected by this. “It’s important to get the humanistic side of this right.” He says there are 1,300 languages spoken world wide, so if you want to reach everyone with tech, machine learning can help. Likewise with health care, e.g. diagnosing retinal problems caused by diabetes. Likewise with social media.

PAIR intends to use engineering and analysis to augment expert intelligence, i.e., professionals in their jobs, creative people, etc. And “how do we remain inclusive? How do we make sure this tech is available to everyone and isn’t used just by an elite?”

He’s going to talk about interpretability, controllability, and accessibility.

Interpretability. Google has replaced all of its language translation software with neural network-based AI. He shows an example of Hemingway translated into Japanese and then back into English. It’s excellent but still partially wrong. A visualization tool shows a cluster of three strings in three languages, showing that the system has clustered them together because they are translations of the same sentence. [I hope I’m getting this right.] Another example: a photo of integrated gradients hows that the system has identified a photo as a fire boat because of the streams of water coming from it. “We’re just getting started on this.” “We need to invest in tools to understand the models.”

Controllability. These systems learn from labeled data provided by humans. “We’ve been putting a lot of effort into using inclusive data sets.” He shows a tool that lets you visuallly inspect the data to see the facets present in them. He shows another example of identifying differences to build more robust models. “We had people worldwide draw sketches. E.g., draw a sketch of a chair.” In different cultures people draw different stick-figures of a chair. [See Eleanor Rosch on prototypes.] And you can build constraints into models, e.g., male and female. [I didn’t get this.]

Accessibility. Internal research from Youtube built a model for recommending videos. Initially it just looked at how many users watched it. You get better results if you look not just at the clicks but the lifetime usage by users. [Again, I didn’t get that accurately.]

Google open-sourced Tensor Flow, Google’s AI tool. “People have been using it from everything to to sort cucumbers, or to track the husbandry of cows.”People have been using it from everything to to sort cucumbers, or to track the husbandry of cows. Google would never have thought of this applications.

AutoML: learning to learn. Can we figure out how to enable ML to learn automatically. In one case, it looks at models to see if it can create more efficient ones. Google’s AIY lets DIY-ers build AI in a cardboard box, using Raspberry Pi. John also points to an Android app that composes music. Also, Google has worked with Geena Davis to create sw that can identify male and female characters in movies and track how long each speaks. It discovered that movies that have a strong female lead or co-lead do better financially.

He ends by emphasizing Google’s commitment to open sourcing its tools and research.



Fernanda and Martin talk about the importance of visualization. (If you are not familiar with their work, you are leading deprived lives.) When F&M got interested in ML, they talked with engineers. ““ML is very different. Maybe not as different as software is from hardware. But maybe. ”ML is very different. Maybe not as different as software is from hardware. But maybe. We’re just finding out.”

M&F also talked with artists at Google. He shows photos of imaginary people by Mike Tyka created by ML.

This tells us that AI is also about optimizing subjective factors. ML for everyone: Engineers, experts, lay users.

Fernanda says ML spreads across all of Google, and even across Alphabet. What does PAIR do? It publishes. It’s interdisciplinary. It does education. E.g., TensorFlow Playground: a visualization of a simple neural net used as an intro to ML. They opened sourced it, and the Net has taken it up. Also, a journal called aimed at explaining ML and visualization.

She “shamelessly” plugs deeplearn.js, tools for bringing AI to the browser. “Can we turn ML development into a fluid experience, available to everyone?”
What experiences might this unleash, she asks.

They are giving out faculty grants. And expanding the Brain residency for people interested in HCI and design…even in Cambridge (!).

The post [liveblog] Google AI Conference appeared first on Joho the Blog.

by davidw at September 26, 2017 02:25 PM

September 22, 2017

The Line Between Hate and Debate on Facebook
The Internet has been billed as the great equalizer, breaking down barriers and increasing access to information and ideas. At the same time, it has allowed for the proliferation of abuse online – whether in the form of hate, harassment or offensive content. The freedom to express oneself is an important principle, but should it persist unfettered? How and where should we draw the line, and who – or what – should play a role in moderating online debate? Monika Bickert, Facebook’s Head of Global Policy Management, and Jonathan Zittrain, Faculty Director of the Berkman Klein Center for Internet & Society and Harvard professor, discuss online abuse and the role that technology can play in addressing it. For more on this event, including video, visit:

by the Berkman Klein Center at September 22, 2017 07:05 PM

September 19, 2017

John Palfrey
“Knowledge and Goodness: The Andover Campaign” launch speech

PA Knowledge Goodness Sun Wordmark RGB

John Palfrey

Remarks at Capital Campaign Launch

Andover, MA

September 2017

Thank you, Dana Delany, for your kind introduction and for everything you’ve done for Andover over the years.

I also want to thank Peter Currie and the board of trustees for their leadership.

Most of all, I am grateful to all of you.  Thank you for joining us as we launch this ambitious campaign to secure Andover’s future.  With your help, we will make sure that Phillips Academy remains a vital source of both knowledge and goodness.

We all have our own reasons for loving Andover.  Maybe you had a teacher who unlocked your passion for science or poetry. Maybe you discovered an instrument or a sport that gave you a new sense of pride and confidence.  Maybe you fell in love for the first time. Maybe, like Catherine and me, you placed your trust in Andover, to educate and care for your child.

Whatever your story, you’re here because Andover changed your life or your child’s life.  That’s what we do.  It’s what makes this school so special.  It’s something I’ve heard our Dean of Admission Jim Ventre say a million times.  Imagine Team Shuman – Jim and his colleagues from the Admissions Office – in the parking lot outside a school or community building.  As they are getting ready to recruit our next fabulous group of students, Jim says:

“Let’s go change some lives.”

Each one of you here is proof of the results.  So is Hafsat.  Wasn’t she amazing?  And in a few minutes you’ll hear from the incredible Kevin Olusola, whose musical and vocal talents found new creative pathways at Andover.  There are thousands more stories like theirs—stories of lives changed by Andover, stories centered around knowledge and goodness.

I think about the students the admission team assisted after Hurricane Katrina when they set up a makeshift office in a Houston hotel and conducted interviews by cell phone.  One day these students are stranded, the next they’re headed to a promising new future. Alan Wesson was one of those 19 students who blew in on Katrina’s winds. He went from Andover to Yale and is now serving as director of public programs for a west coast high school’s Center for Civic Engagement.

I think about Dario Collado, of the class of 1998, who spoke at All-School Meeting this spring.  Dario grew up in a housing project in a working-class Dominican community in Lawrence.  In an All-School Meeting last year, Dario gave one of those addresses where I could tell he had gripped every pair of eyes and ears in the audience.  Dario told our students about how a teacher at the public high school saw his potential, encouraged him to apply to Andover, drove him to the interview, and even paid his application fee.  I loved watching the faces of our students as they listened to Dario tell the story of how he found self-confidence and determination at Andover, how he became the first member of his family to go to college, and how he went on to a life of service nurturing the next generation of LatinX leaders.  Dario’s story embodies our ethic of non sibi and youth from every quarter—and it’s a testament to the transformative power of the Andover experience for students from every quarter, from every socio-economic background, from all around the world.

I think also of Caroline Lind, who came here as a promising student and devoted softball player from Greensboro, NC.  When she broke her nose one season, she worked out on the erg to stay in shape.  After hitting a record time on the machine, she changed sports and joined crew.  We all know how this story turned out. Caroline went onto Princeton, starred in crew there, and has since won 2 Olympic gold medals.  Circumstances, great coaching, faculty encouragement and personal “grit” enabled her to find a career and a passion.

Your support has helped make all this possible.  I’ve seen it first-hand over the past six years.

You’ve allowed us to continue the need-blind admission policy so no student is ever turned away for financial reasons.  No other secondary school has a financial aid program as comprehensive as ours.

You’ve supported a legacy of excellence that shines most brightly in our faculty and academic program.  It’s paying off:  Last year, a record 86 percent of admitted students chose to enroll, joining us on campus just weeks ago.

You’ve also supported our efforts to provide the healthy, balanced campus life our students need and deserve. I’m enormously proud of our state-of-the-art Rebecca M. Sykes Wellness Center and the programming and care to which it is home.

You’ve helped us achieve so much.  But we can’t rest on yesterday’s success. There are many more lives to change. Knowledge and Goodness, The Andover Campaign is our catalyst.

Under the leadership of Peter and the trustees, and guided by the Academy’s strategic plan, we’ve set big goals – from ensuring that Andover remains need- blind, to building a dynamic campus that can support the needs of leading-edge 21st-century education.

Our work is more important than ever.  Andover’s mission—the charge laid down by our founders to instill both knowledge and goodness—is fiercely urgent and absolutely necessary.

We are living in a time of great change… in education for sure, but also in our society at large…  how we live, work, reason, and grow together… all of it is in flux.  It can be disorienting… for students, for parents, for all of us.

As someone whose research is focused on technological change, I see these effects on education every day on campus.  I also see the impact of our increasingly polarized politics and how hard our students are working to keep open minds and open hearts.


Here’s the good news: Andover is well positioned to thrive in this changing world—if we make the right choices and investments.

In 1959, at the start of another fundraising campaign, Headmaster John Mason Kemper said that schools like Andover needed to meet the great changes of that era “with new ideas, new attitudes, and new techniques and tools, while holding fast to the enduring values of our past.”

That’s even more true today.

Andover’s strength has always come from a special balance of continuity and change.  Our traditions have defined us.  Finis origine pendet is right there on our seal.  But our spirit of innovation is what’s made us excel.  Think of Thomas Cochran leading the way to build our modern campus in the 1920s, with our museums, library, and Chapel.  Ted Sizer bringing coeducation to Andover in the 1970s.  And Barbara Chase and Oscar Tang recommitting Andover to need blind 10 years ago, so that the Academy could live up to its promise of educating youth from every quarter.  In each case, visionary leadership and courageous thinking helped Andover set the platinum standard for secondary schools everywhere.

With Knowledge and Goodness, we’ll double down on Andover’s core values, which provide a foundation in a changing time—an enduring commitment to excellence and inclusion, an ethic of service and citizenship, and a laser-like focus on the minds and morals of our students.

At the same time, we’ll keep innovating. The Tang Institute is an incubator for emerging ideas in education. Our faculty are already adding to their teaching techniques and changing the way students learn.

Our Learning in the World program offers every student the opportunity to study off campus and experience a culture unlike their own. We are preparing global citizens like never before. I can’t think of anything more valuable in our present climate.

And with your help, our need-blind admission policy will continue allowing us to recruit the most talented, creative and diverse student body in the country.

This is what Knowledge and Goodness, The Andover Campaign is all about.  It’s how we’ll make sure Andover continues to change lives for years to come.

We like to say that the end depends upon the beginning.  Well, this is another beginning for Andover.  Right here, with all of you, tonight.  Thank you for your support of our school, our students and faculty and staff, and the values we share.  Thank you for all of it.

Now let’s go change some lives!

# # #


by jgpalfrey at September 19, 2017 10:11 PM

David Weinberger
[bkc] Hate speech on Facebook

I’m at a Very Special Harvard Berkman Klein Center for Internet & Society Tuesday luncheon featuring Monika Bickert, Facebook’s Head of Global Policy Management in conversation with Jonathan Zittrain. Monika is in charge of what types of content can be shared on FB, how advertisers and developer interact with the site, and FB’s response to terrorist content. [NOTE: I am typing quickly, getting things wrong, missing nuance, filtering through my own interests and biases, omitting what I can’t hear or parse, and not using a spelpchecker. TL;DR: Please do not assume that this is a reliable account.]

Monika: We have more than 2B users…

JZ: Including bots?

MB: Nope, verified. Billions of messages are posted every day.

[JZ posts some bullet points about MB’s career, which is awesome.]

JZ: Audience, would you want to see photos of abused dogs taken down. Assume they’re put up without context. [It sounds to me like more do not want it taken down.]

MB: The Guardian covered this. [Maybe here?] The useful part was it highlighted how much goes into the process of deciding these things. E.g., what counts as mutilation of an animal? The Guardian published what it said were FB’s standards, not all of which were.

MB: For user generated content there’s a set of standards that’s made public. When a comment is reported to FB, it goes to a FB content reviewer.

JZ: What does it take to be one of those? What does it pay?

MB: It’s not an existing field. Some have content-area expertise, e.g., terrorism. It’s not a minimum wage sort of job. It’s a difficult, serious job. People go through extensive training, and continuing training. Each reviewer is audited. They take quizzes from time to time. Our policies change constantly. We have something like a mini legislative session every two weeks to discuss proposed policy changes, considering internal suggestions, including international input, and external expert input as well, e.g., ACLU.

MB: About animal abuse: we consider context. Is it a protest against animal cruelty? After a natural disaster, you’ll see awful images. It gets very complicated. E.g., someone posts a photo of a bleeding body in Syria with no caption, or just “Wow.” What do we do?

JZ: This is worlds away from what lawyers learn about the First Amendment.

MB: Yes, we’re a private company so the Amendment doesn’t apply. Behind our rules is the idea that “You don’t have to agree with the content, but you should feel safe”FB should be a place where people feel safe connecting and expressing themselves. You don’t have to agree with the content, but you should feel safe.

JZ: Hate speech was defined as an attack against a protected category…

MB: We don’t allow hate speech, but no two people define it the same way. For us, it’s hate speech if you are attacking a person or a group of people based upon a protected characteristic — race, gender, gender identification, etc. —. Sounds easy in concept, but applying it is hard. Our rule is if I say something about a protected category and it’s an attack, we’d consider it hate speech and remove it.

JZ: The Guardian said that in training there’s a quiz. Q: Who do we protect: Women drivers, black children, or white men? A: White men.

MB: Not our policy any more. Our policy was that if there’s another characteristic beside the protected category, it’s not hate speech. So, attacking black children was ok but not white men, because of the inclusion of “children.” But we’ve changed that. Now we would consider attacks on women drivers and black children as hate speech. But when you introduce other characteristics such as profession, it’s harder. We’re evaluating and testing policies now. We try marking content and doing a blind test to see how it affects outcomes. [I don’t understand that. Sorry.]

JZ: Should the internal policy be made public?

MB: I’d be in favor of it. Making the training decks transparent would also be useful. It’s easier if you make clear where the line is.

JZ: Do protected categories shift?

MB: Yes, generally. I’ve been at FB for 5.5 yrs, in this are for 4 yrs. Overall, we’ve gotten more restrictive. Sometimes something becomes a topic of news and we want to make sure people can discuss it.

JZ: Didi Delgado’s post “all white people are racist” was deleted. But it would have been deleted if had said that all black people are racist, right?

MB: Yes. “If it’s a protected characteristic, we’ll protect it”If it’s a protected characteristic, we’ll protect it. [Ah, if only life were that symmetrical.]

JZL How about calls to violence, e.g., “Someone shoot Trump/Hillary”? If you think it should be taken down. [Sounds like most would let it stand.]

JZ: How about “Kick a person with red hair.” [most let it stand]

JZ: “How about: To snap a bitch’s neck, make sure to apply all your pressure to the middle of her throat.” [most let it stand][fuck, that’s hard to see up on the screen.]

JZ: “Let’s beat up the fat kids.” [most let it stand]

JZ: “#stab and become the fear of the Zionist” [most take it down]

MB: We don’t allow credible calls for violence.

JZ: Suppose I, a non-public figure, posted “Post one more insult and I’ll kill you.”

MB: We’d take that down. We also look at the degree of violence. Beating up and kicking might not rise to the standard. Snapping someone’s neck would be taken down, although if it were purely instructions on how to do something, we’d leave it up. “Zionist” is often associated with hate speech, and stabbing is serious, so we’d take them down. We leave room for aspirational statements wishing some bad thing would happen. “Someone should shoot them all” we’d count as a call to violence. We also look for specifity, as in “Let’s kill JZ. He leaves work at 3.” We also look at the vulnerability of people; if it’s a dangerous situation,
we’ll tend to treat all such things as calls to violence, [These are tough questions, but I’m not aligned with FB’s decisions on this.]

JZ: How long does someone spend reviewing this stuff?

MB: Some is easy. Nudity is nudity, although we let breast cancer photos through. But a beheading video is prohibited no matter what the context. Profiles can be very hard to evaluate. E.g., is this person a terrorist?

JZ: Given the importance of FB, does it seem right that these decisions reside with FB as a commercial entity. Or is there some other source that would actually be a relief?

MB: “We’re not making these decisions in a silo”We’re not making these decisions in a silo. We reach out for opinions outside of the company. We have Safety Advisory Board, a Global Safety Network [got that wrong, I think], etc.

JZ: These decisions are global? If I insult the Thai King…

MB: That doesn’t violate our global community standard. We have a group of academics around the world, and people on our team, who are counter-terrorism experts. It’s very much a conversation with the community.

JZ: FB requires real names, which can be a form of self-doxxing. Is the Real Name policy going to evolve?

MB: It’s evolved a little about what counts as their real name, i.e., the name people call you as opposed to what’s on your drivers license. Using your real name has always been a cornerstone of FB. A quinessential element of FB.

JZ: You don’t force disambiguation among all the Robert Smiths…

MB: When you communicate with people you know, you know you know them. “We don’t want people to be communicating with people who are not who you think they are”We don’t want people to be communicating with people who are not who you think they are. When you share something on FB, it’s not public or private. You can choose which groups you want to share it with, so you know who will see it. That’s part of the real name policy as well.

MB: We have our community standards. Sometimes we get requests from countries to remove violations of their law, e.g., insults to the King of Thailand. If we get such a request, if it doesn’t violate the standards, we look if the request is actually about real law in that country. Then we ask if it is political speech; if it is, to the extent possible, we’ll push back on those requests. E.g., Germans have a little more subjectivity in their hate speech laws. They may notify us about something that violates those laws, and if it does not violate our global standards, we’ll remove it in Germany only. (It’s done by IP addresses, the language you’re using, etc.) When we do that, we include it in our 6 month reports. If it’s removed, you see a notice that the content is restricted in your jurisdiction.


Q: Have you spoken to users about people from different cultures and backgrounds reviewing their content?

A: It’s a legitimate question. E.g., when it comes to nudity, even a room of people as homogenous as this one will disagree. So, “our rules are written to be very objective”our rules are written to be very objective. And we’re increasingly using tech to make these decisions. E.g., it’s easy to automate the finding of links to porn or spam, and much harder for evaluating speech.

Q: What drives change in these policies and algorithms?

A: It’s constantly happening. And public conversation is helpful. And our reviewers raise issues.

Q: a) When there are very contentious political issues, how do you prevent bias? b) Are there checks on FB promoting some agenda?

A: a) We don’t have a rule saying that people from one or another country can review contentious posts. But we review the reviewers’ decisions every week. b) The transparency report we put out every six months is one such check. If we don’t listen to feedback, we tend to see news stories calling us out on it.

[Monika now quickly addresses some of the questions from the open question tool.]

Q: Would you send reports to Lumen? MB: We don’t currently record why decisions were made.

Q: How to prevent removal policies from being weaponized but trolls or censorious regimes? MB: We treat all reports the same — there’s an argument that we shouldn’t — but we don’t continuously re-review posts.

JZ: For all of the major platforms struggling with these issues, is it your instinct that it’s just a matter of incrementally getting this right, bringing in more people, continue to use AI, etc. OR do you think sometimes that this is just nuts; there’s got to be a better way.

There’s a tension between letting anyone see what they want, or have global standards. People say US hates hate speech and the Germans not so much, but there’s actually a spectrum in each. The catch is that there’s content that you’re going to be ok seeing but we think is not ok to be shared.

[Monika was refreshingly direct, and these are, I believe, literally impossible problems. But I came away thinking that FB’s position has a lot to do with covering their butt at the expense of protecting the vulnerable. E.g., they treat all protected classes equally, even though some of us — er, me — are in top o’ the heap, privileged classes. The result is that FB applies a rule equally to all, which can bring inequitable results. That’s easier and safer, but it’s not like I have a solution to these intractable problems.]

The post [bkc] Hate speech on Facebook appeared first on Joho the Blog.

by davidw at September 19, 2017 06:24 PM

Berkman Center front page
The Line Between Hate and Debate


featuring Monika Bickert, Facebook’s Head of Global Policy Management in conversation with Professor Jonathan Zittrain


As society figures out what is acceptable and what is harmful, can technology play a role in improving online debate?

Parent Event

Berkman Klein Luncheon Series

Event Date

Sep 19 2017 12:00pm to Sep 19 2017 12:00pm
Thumbnail Image: 

Tuesday, September 19, 2017 at 12:00 pm
Berkman Klein Center for Internet & Society at Harvard University

The Internet has been billed as the great equalizer, breaking down barriers and increasing access to information and ideas. At the same time, it has allowed for the proliferation of abuse online – whether in the form of hate, harassment or offensive content. The freedom to express oneself is an important principle, but should it persist unfettered? How and where should we draw the line, and who – or what – should play a role in moderating online debate? Monika Bickert, Facebook’s Head of Global Policy Management, and Jonathan Zittrain, Faculty Director of the Berkman Klein Center for Internet & Society and Harvard professor, discuss online abuse and the role that technology can play in addressing it.

About Monika

Monika Bickert is Facebook’s Head of Global Policy Management. Her global team manages the policies for what types of content can be shared on Facebook and how advertisers and developers can interact with the site. She also manages the company’s response to terrorist content online. Monika originally joined Facebook in 2012 as lead security counsel, advising the company on matters including child safety and data security.

Prior to joining Facebook, Monika served as Resident Legal Advisor at the U.S. Embassy in Bangkok, Thailand, where she specialized in Southeast Asian rule of law development, including Southeast Asian governments’ response to terrorist insurgencies, child exploitation, and human trafficking. She also served as an Assistant United States Attorney for 11 years in Washington, DC, and Chicago, prosecuting federal crimes ranging from public corruption to gang-related violence. During her time in Chicago, Monika was the Northern District of Illinois’ coordinator for the prosecution of crimes against children, and she also received the Investigation of the Year award from Drug Enforcement Administration for U.S. v. James Austin, et al., a prosecution of 50 gang members and associates for coordinated narcotics trafficking that caused over a dozen deaths.

Monika received a B.A. from Rice University and a J.D. from Harvard Law School.

About Jonathan

Jonathan Zittrain is the George Bemis Professor of International Law at Harvard Law School and the Harvard Kennedy School of Government, Professor of Computer Science at the Harvard School of Engineering and Applied Sciences, Vice Dean for Library and Information Resources at the Harvard Law School Library, and co-founder of the Berkman Klein Center for Internet & Society.  His research interests include battles for control of digital property and content, cryptography, electronic privacy, the roles of intermediaries within Internet architecture, human computing, and the useful and unobtrusive deployment of technology in education.

He performed the first large-scale tests of Internet filtering in China and Saudi Arabia, and as part of the OpenNet Initiative co-edited a series of studies of Internet filtering by national governments: Access Denied: The Practice and Policy of Global Internet FilteringAccess Controlled: The Shaping of Power, Rights, and Rule in Cyberspace; and Access Contested: Security, Identity, and Resistance in Asian Cyberspace.

He is a member of the Board of Directors of the Electronic Frontier Foundation and the Board of Advisors for Scientific American.  He has served as a Trustee of the Internet Society and as a Forum Fellow of the World Economic Forum, which named him a Young Global Leader. He was a Distinguished Scholar-in-Residence at the Federal Communications Commission, and previously chaired the FCC’s Open Internet Advisory Committee. His book The Future of the Internet -- And How to Stop It predicted the end of general purpose client computing and the corresponding rise of new gatekeepers.  That and other works may be found at <>.

Links to resources we want to share in advance of talk:

Download original audio and video from this event.

Subscribe to the Berkman Klein events podcast to have audio from all our events delivered straight to you!

by candersen at September 19, 2017 04:00 PM

September 18, 2017

Justin Reich
Who, Me? Distributed Leadership in Schools
Teachers play an essential role in leading changes in teaching and learning in schools.

by Justin Reich at September 18, 2017 11:26 PM

Jonathan Zittrain on Technology for the Social Good
Berkman Klein Center Faculty Chair Jonathan Zittrain discusses the development of the Internet — from its earliest stages to its present manifestations — as a technology for good or harm, depending on the human forces that wield it. Find out more about this event, and the Berkman Klein Center, here:

by the Berkman Klein Center at September 18, 2017 02:57 PM

September 15, 2017

Harry Lewis
Motion update
For those who are just coming up to speed, Harvard Magazine published a good summary of the situation at the time I filed the new version of the motion about club memberships.
The Crimson reports, on authority of a member of the Faculty Council, that the motion will be discussed at the October meeting of the Faculty, but not voted until the November meeting. This may or may not be true; I can't confirm it, since these matters are decided by the Docket Committee, which has not communicated any such decision to me. (The December 2016 FAS meeting was adjourned without a vote in a rather odd way, so the signatories to the motion are watching these procedural decisions rather closely this time.)
The Crimson has another story of interest, about a Title IX complaint against the University in which the Final Clubs are featured prominently. This news tends to support the theory that what got the ball rolling toward the mess we are in was fear that Harvard might be legally liable for a tort that happened at a final club. That would explain, for example, the insistence that the final decision about the clubs is for the president to make rather than the Faculty, in spite of the unambiguous language of the Statutes (she would be acting as a fiduciary, would go the logic, notwithstanding the assignment to the Faculty of responsibility for the discipline of students). It would explain the early public involvement of the Senior Fellow, who traditionally has not weighed in on questions about student parties, and the "regular" discussions taking place about final clubs between the College administration and the Corporation.  It would also explain the otherwise peculiar decision, even in the recent harsh version of the proposals, that the ethnic fraternities and sororities would be left alone, in spite of being doubly "exclusionary," on the basis of both gender and ethnicity. As they draw from several local universities, Harvard may have calculated that the risk of liability was small for events that might happen at them.
Connecting these dots suggests how the ball may have gotten rolling, but it is not in meant to question the sincerity of those who support the sanctions for other reasons. Whether or not this speculation is correct, I would fully support Harvard taking strong action to limit its risks, and protect students' safety to boot. (Though if risk mitigation really got this started, I wish the Corporation had been equally risk-conscious back in 2008, when it lost billions from the endowment overnight.)
On the other hand, the Title IX origin would make a great deal of what has been said over the past year rather incomplete and beside the point, if not disingenuous, and the actual solution proposed both overbroad and not even sure to include the original target in its kill zone. (Is the champagne being chilled in anticipation of Harvard's glorious victory over the KKG sorority?) It would also be worrisome in suggesting that the Corporation's interpretation of University governance is that the president has limitless authority to make any kind of decision over matters in which the University can be said to have any risk.
One more news item. The President has some words about the motion in her opening of term greetings. I will leave it to readers to make up their own minds about her characterization of the matter under discussion. But I would note that it is fairly unusual, and perhaps unprecedented, for the president to speak so strongly in public against a motion to be debated and voted by the Faculty at a meeting over which she will preside.

The first Crimson story linked above describes a one-pager I shared with the Faculty Council; I include it below for your information.


Harvard College shall not discipline, penalize, or otherwise sanction students for joining, or affiliating with, any lawful organization, political party, or social, political, or other affinity group.

Explanatory note. This motion is intended to give students who join or form legal clubs or similar organizations the same protections that existing policies afford to all other students.  It also secures their right of free association.  If the policy is adopted, students could not, simply because of membership in a legal club, social or political organization, be sanctioned by the Administrative Board or by the Honor Council, or deprived of any academic or extracurricular opportunity or honor for which they would otherwise be eligible.

This motion is a version of the motion submitted last year in response to the USGSO policy announced in May of 2016. When that policy was reconsidered, the motion was withdrawn. Now that the recommendations of the Clark-Khurana committee have proven to be even more expansive than the original proposal, it is time for the full Faculty to debate and decide the question of principle: should students ever be punished for joining private organizations?

To recap very briefly the main points raised in support of the motion last year:
  • Students should be punished for their acts, not their memberships.
  • The right of free association, like the right of free speech, should apply to students as it applies to all US citizens.
  • Harvard has never in modern times blacklisted any organization by prohibiting membership.
  • Harvard specifically prohibits even asking faculty candidates about club memberships.
  • In the 1950s, Harvard famously stood behind the right of faculty to join publicly unpopular but legally constituted organizations.
  • The Verba report on ROTC (written while ROTC banned gay students) stated the principle explicitly: To punish students for joining a discriminatory organization would be “a paternalistic policy inconsistent with Harvard’s general approach.” The Verba principle is in diametrical contrast to the view expressed last year that “of course we can discriminate against people who discriminate.”
The motion has been reworded to drop the term “discrimination,” which some found confusing or objectionable, and to use direct language instead. Further relevant discussion appears on the FAS Wiki.

In response to last year’s motion, some protested that bringing the McCarthy era into the conversation was alarmist, as the policy then proposed was narrowly targeted and could not possibly be a step down any slippery slope. The fact that the new proposal does indeed take several steps down that slope confirms that a broad statement of principle is needed.

It is urgent that the Faculty’s voice be heard corporately, not via hand-picked representatives. When the USGSO policy was announced in 2016, the president accepted it simultaneously. When the Implementation Committee report was issued last year, Dean Khurana accepted most of its recommendations simultaneously. There has been no indication that the Faculty will be asked to vote on the final policy. The president, the deans, and the Faculty itself need to know the Faculty’s view on the fundamental question asserted by this motion.

This motion has twenty-one signatories, including all twelve who signed last year’s version.

Harry Lewis, 12 September 2017

by Harry Lewis ( at September 15, 2017 02:40 AM

September 14, 2017

Berkman Center front page
Cross-Border Data Access Reform

A Primer on the Proposed U.S.-U.K. Agreement


A brief primer on how cross-border data access requests currently work, options for reform, and major challenges to reform ahead

Publication Date

13 Sep 2017


Thumbnail Image: 


Cross-border data access reform may be on the legislative agenda in late 2017, with recent House and Senate judiciary committee hearings revisiting the topic. In light of this increasing interest, we thought it would be helpful to provide a brief primer on how cross-border data access requests currently work, options for reform, and major challenges to reform ahead. This document presents a short, high-level background review of the debate as it currently stands, particularly focusing on the DOJ’s 2016 proposal for reform.*

Governments need evidence to investigate and prosecute crimes, but increasingly that evidence takes the form of data stored on the servers of U.S. tech companies. In July 2016, the U.S. Department of Justice (DOJ) released draft legislation that would address some of the challenges foreign governments face when seeking data related to criminal investigations from U.S. companies. Interest in making such changes continues to grow, with relevant laws, including the Electronic Communications Privacy Act (ECPA), maybe seeing Congressional attention in late 2017, especially as the Foreign Intelligence Surveillance Act (FISA) comes up for renewal.

To access electronic content – including email, social media messages, and more – held by U.S. companies, a foreign country currently relies primarily on the processes set out in agreements called Mutual Legal Assistance Treaties (MLATs), if that country has negotiated one with the U.S. MLATs with the U.S. require countries to meet U.S. legal standards when making requests for electronic content data, with less strict standards for metadata. Countries have grown frustrated with both the normative implications of the MLAT process and its typical lengthiness.

After substantial debate, and with many proposed ideas from civil society, industry, and academia, the Department of Justice (DOJ) in July 2016 released draft legislation intended to address these concerns. The proposal moves away from the treaty-based system currently underpinning the mutual legal assistance process. Instead, the new legislation would require “lighter touch” bilateral agreements on this issue between the United States and participating countries. Once countries are approved for these bilateral agreements, the legislation would allow them to submit requests for data, made pursuant to the requesting countries’ laws and stipulations in the legislation, directly to U.S. electronic service providers, instead of first going through U.S. courts. The U.K. would likely be the first country approved to make requests under this new legislation, but the legislation would also pave the way for agreements with other qualifying countries. This legislation advances a legal solution for cross-border data access that proponents hope is sufficiently appealing to foreign governments to forestall more damaging alternative responses to data access concerns, including country-wide service bans, mandated data localization, or forcing companies to make decisions in the face of a conflict of laws.

* This publication is an adaptation of a briefing document originally created by the authors to inform discussions in the Berklett Cybersecurity project meetings about the proposed U.S.-U.K. agreement on cross-border data sharing and related issues.

About the Berklett Cybersecurity Project

Launched in 2015, the Berklett Cybersecurity project is a unique forum for discussing true and important, and often novel, facts, and perspectives, and achieving surprising consensus on enduring questions of cybersecurity.  The project is led by Prof. Jonathan Zittrain and former National Security Agency (NSA) Director of Compliance John DeLong, in close collaboration with security technologist Bruce Schneier, and Matthew Olsen, the former Director of the U.S. National Counterterrorism Center (NCTC). More information about the project can be found on the Berkman Klein Center’s website:

The Berklett Cybersecurity project is generously supported by the William and Flora Hewlett Foundation. Research efforts that contributed to this publication were also supported by the John D. and Catherine T. MacArthur Foundation and the Ford Foundation. 

Producer Intro

Authored by

by djones at September 14, 2017 07:00 PM

September 13, 2017

John Palfrey
Opening All School Meeting, Phillips Academy, 2017-2018

All-School Meeting

John Palfrey, Head of School, Phillips Academy

September 13, 2017

Good morning, Andover!  I am psyched to see you all here.

The main point of this All School Meeting is simply for us to gather, in one space, to celebrate the start of the year.  It’s a great chance to acknowledge the special role that the faculty and staff play in our lives here.  And it’s a moment to celebrate the start to the senior year of the great Andover Class of 2018.

Phillips Academy is not a great place because it’s old.  It’s a great place because generation after generation of faculty and students, staff and alumni, have refused to rest on the laurels of past greatness.  Phillips Academy has always been a place where tradition and values matter a great deal – and you’ll hear much about non sibi, knowledge with goodness, and youth from every quarter during your time here – but also a place where innovation happens, where reform has happened in ways that are consistent with the school’s founding principles.

At this time of year, I always think of footsteps – those left by those who came before us and those that we will leave during OUR time at Andover.

First, let’s think about the effect of our footsteps on our natural environment.  I hope and trust that we are entering a new era of stewardship, in which we are all thinking carefully about how might protect the environment around us and do our part to combat the dangers of climate change.

My thoughts about footsteps this morning relate to treading lightly and carefully during our time here.  Those who are returning students to Andover know the rules when it comes to walking around campus.  One big one is to be sure to press the button before you cross the street, whether the sun is shining or not.  Take out your earbuds.  Look out for cars, make eye contact with drivers, and smile and wave if you are crossing in front of a car.  Please do this 100% of the time.

When it comes to walking on the grass, the rule goes something like this: one may walk on the grass if one is going to a spot on the grass, say, to have a picnic; but one should use the path, if one is merely walking from point to point on campus, across the grass.

And if you must cross the grass to get from point A to point B, returning students, what do you need to do?  Yes, zig-zag.

This rule seems quite sensible; I like it.  Please do play Frisbee and soccer and have picnics on the lawns at Andover.  Shame on us if we don’t take the time to enjoy the natural beauty of this campus, to enjoy the hard work of our friends in OPP, to share the gifts of the landscape architecture of Frederick Law Olmsted, Charles Platt, and others.  This rule means that we are both enjoying and respecting the land we have been given, as stewards for the future.

I am reminded, too, of my own first all-school meeting, when I was a student.  It turns out my school called it “assembly.”  And it turns out, in case you haven’t heard, that I was at Exeter at the time.  The only thing I remember about that first assembly, other than the sense of excitement and electricity in the room, was that the head of school, Mr. Kurtz, built his remarks around a single line.  At Exeter at the time, one was not permitted to walk on the grass at all.  The main line of his speech, to his student body, was “keep off the grass.”  It was, of course, a double entendre – for those not taking French, he had a double meaning.  I didn’t forget either of those meanings for the four years I was boarding there.

At Andover, we have a different rule.  You are encouraged to use the grass in one of those two senses.

I underscore the second meaning: do not ignore those rules of community conduct.  Students may not in any instances use drugs and alcohol on this campus.  For that matter, we expect you to uphold all of our community expectations with respect to how we treat one another – in everyday encounters and in intimate moments.  We expect you to know what we mean by consent and to act accordingly – and yes, I am now talking about sex.  If anything about our community expectations is unclear, come see me or a dean or your house counselor or advisor.  It’s essential that we are all on the same page at the start of the year about the rules.

This metaphor is useful in thinking about the balance we seek to strike at Andover.  I encourage you to zig-zag on purpose; not all who wander, as the saying goes, are lost.  Do have fun; do take routes that are not-exactly-linear as you make your way through the school; and do follow the rules, with fidelity, along the way.

A second context for footsteps, meant as a metaphor for the effect of our footsteps on Phillips Academy as an institution.

One small suggestion I have for all of you is that, during your time here, you find for yourselves a favorite spot, somewhere on campus.  We all need a part of the school that gives us a sense of serenity, or happiness, or hope, for those days when we need something to help us to re-center ourselves, to reflect, to recharge our batteries.

Now in my sixth year as Head of School, I have come to love many parts of campus:  the inside of SamPhil, because I teach US History there: this chapel, because I cherish being with all of you (I mean that); the entryway to the Addison; the reading room of the OWH Library; a small library area of Phelps House, where I live with my family.

My very favorite place on campus happens to be a staircase – actually, two staircases.  These stairs are the stairs leading from the first floor to the second floor of Paresky Commons.  There is something about progress upwards, toward the divine, or towards the future, that I like about them.  Perhaps it has to do with the food, which is very good.  But mostly it has to do with the steps themselves.

The steps have indentations in the marble – indentations made by generations of students, faculty and staff who have gone before us.  I love these indentations because they remind me that we are not alone in this journey, not alone today and not alone over time.

As I walk up those steps, I realize that I am making those indentations deeper than they were before.  If I put a foot in the deepest part, I am making that indentation just a bit deeper.  If I step where others have not stepped so often, perhaps closer to the middle of the stair, then I make a tiny mark where others have not so frequently walked.

I know that my steps do matter, as your head of school.  But I also know that my steps do not really matter any more than any of your steps.  Perhaps I weigh a bit more than some of you, so my indentation is a bit deeper, or my footfall heavier than yours is, as you sprint more quickly from the first to the second floor.  But none of us can change this place very quickly with our footsteps.  None of us can change those steps, all that much, on our own.  And we will be followed – there will be a sixteenth head of school.  There will be a class of 2048, perhaps with some of your children in it, or my grandchildren.

These steps bring to mind one of the most memorable conversations I’ve had with an alumnus of Andover.  One morning, in my first summer on the job, I was invited to visit with Mr. and Mrs. Paresky, in their home to have a glass of lemonade and to hear about Andover.  I asked them why they loved the school so much and why they had given us the generous gift to renovate the “Commons” into “Paresky.”  I loved what the Pareskys said that day: it had to do with how much the school had given to David Paresky as a student, and to their own daughter Pamela, in particular, when she followed him to the school.

But it also was about the way that Mr. Paresky thinks about obligation: the notion that he had been given much by the school, at an early age; that he had gone out and done well – and many good works, in the true non sibi spirit – in his life; and that he believed that he needed to be a steward of Andover, that he had an obligation to give back.  We all get more from Andover than we give, he told me, and he wanted to be sure that the students at Andover today know about both the wonderful opportunity that you have while you are here – seize it! – and also about the extent to which great institutions like Andover don’t just happen.  They become great because generation after generation, students have been mindful of their own footsteps here and then have given back, when they’ve moved on from life on campus, out of a sense of love for the place and also obligation.

And that’s the key point about the footsteps.  Our words and our deeds while we are at Andover matter, just as they matter after we are gone from here.

As I wrote to you this summer, our theme for the year is citizenship.  As you think about the mark you want to make at Andover, I urge you to do so in the context of the larger world – not just what is going on inside the Andover bubble.  I expect every Andover student to engage in the issues of our time.  This summer gave plenty of examples: senseless violence in Charlottesville and Barcelona; lives disrupted by Hurricane Harvey and Hurricane Irma; proposals to end and reform DACA; and on and on.  Andover students come from a long and proud tradition of making a mark in the world through their footsteps.  I expect us to continue that tradition and in doing so, to be informed, engaged, productive citizens of our communities, nations, and the world.

As we do so, we should have fun – good, wholesome fun, of course.  We should have picnics and games on the grass.  We will work hard and we should play a lot too, and enjoy this community that we are so lucky to be a part of.

Before we go, I’d like to do a few quick things.  I’ve been so happy to hear the joyful voices of all of you students lighting up this campus since the Blue Keys started to cheer on the corner as new students arrived.

First: Juniors, Lowers, Uppers: the seniors came in with a lot of spirit this morning.  I want all the juniors, lowers, and uppers, to make some noise in appreciation of those students who go before you.  Let’s hear it for the seniors!

Seniors, you get another shot.  Let’s hear it for the juniors, lowers, and uppers, who are following in your enormous footsteps!  Make some noise!

And last, after this last cheer, the All School Meeting is adjourned.  I want you to do one last cheer – hold on! – and then walk out of this chapel into brilliant sunshine, ideally with a big smile on your faces, and perhaps a little attention, in the back of your minds, to your footsteps as you go.

All students, you are going to do this last cheer.  You are surrounded, in this community, by some of the finest adults I have ever had the privilege of meeting.  This is a mindful, inspired, caring community of teachers – and citizens – who have CHOSEN to devote their professional lives – and in many respects, their personal lives, too, as they live in the dorms with you and eat together and play together – to your education.  For our last cheer of the day, Andover students, and as our last act as we leave the chapel: Let’s hear it for ALL the teachers on this campus!

Thank you – All School Meeting is dismissed.

by jgpalfrey at September 13, 2017 05:53 PM

September 12, 2017

Berkman Center front page
Berkman Klein: Technology for the Social Good featuring Jonathan Zittrain


Learn more about us in an interactive discussion with Prof. Jonathan Zittrain, Berkman Klein Center Faculty Chair


If you’re interested in connecting with the Berkman Klein Center's research, community, and events, or are just generally interested in digital technologies and their impact on society, please join us for this talk!

Parent Event

Berkman Klein Luncheon Series

Event Date

Sep 12 2017 12:00pm to Sep 12 2017 12:00pm
Thumbnail Image: 

Tuesday, September 12, 2017 at 12:00 pm
Berkman Klein Center for Internet & Society at Harvard University

Join us for our first Tuesday Luncheon Series of the academic year to learn more about the Berkman Klein Center for Internet & Society at Harvard University -- and its network of researchers, activists, faculty, students, technologists, entrepreneurs, artists, policy makers, lawyers, and more -- in an interactive conversation lead by Berkman Klein Center Faculty Chair Jonathan Zittrain. If you’re curious about connecting with our research, our community, or our events, or are just generally interested in digital technologies and their impact on society and the social good, please join us!

About Berkman Klein

We  bring together the sharpest, most thoughtful people from around the globe to tackle the biggest challenges presented by the Internet. As an interdisciplinary, University-wide center with a global scope, we have an unparalleled track record of leveraging exceptional academic rigor to produce real- world impact. We pride ourselves on pushing the edges of scholarly research, building tools and platforms that break new ground, and fostering active networks across diverse communities. United by our commitment to the public interest, our vibrant, collaborative community of independent thinkers represents a wide range of philosophies and disciplines, making us a unique home for open-minded inquiry, debate, and experimentation.

About Jonathan Zittrain

Jonathan Zittrain is the George Bemis Professor of International Law at Harvard Law School and the Harvard Kennedy School of Government, Professor of Computer Science at the Harvard School of Engineering and Applied Sciences, Vice Dean for Library and Information Resources at the Harvard Law School Library, and co-founder of the Berkman Klein Center for Internet & Society.  His research interests include battles for control of digital property and content, cryptography, electronic privacy, the roles of intermediaries within Internet architecture, human computing, and the useful and unobtrusive deployment of technology in education.

He performed the first large-scale tests of Internet filtering in China and Saudi Arabia, and as part of the OpenNet Initiative co-edited a series of studies of Internet filtering by national governments: Access Denied: The Practice and Policy of Global Internet FilteringAccess Controlled: The Shaping of Power, Rights, and Rule in Cyberspace; and Access Contested: Security, Identity, and Resistance in Asian Cyberspace.

He is a member of the Board of Directors of the Electronic Frontier Foundation and the Board of Advisors for Scientific American.  He has served as a Trustee of the Internet Society and as a Forum Fellow of the World Economic Forum, which named him a Young Global Leader. He was a Distinguished Scholar-in-Residence at the Federal Communications Commission, and previously chaired the FCC’s Open Internet Advisory Committee. His book The Future of the Internet -- And How to Stop Itpredicted the end of general purpose client computing and the corresponding rise of new gatekeepers.  That and other works may be found at <>.

Download original audio and video from this event.

Subscribe to the Berkman Klein events podcast to have audio from all our events delivered straight to you!

by candersen at September 12, 2017 04:00 PM

danah boyd
Data & Society’s Next Stage

In March 2013, in a flurry of days, I decided to start a research institute. I’d always dreamed of doing so, but it was really my amazing mentor and boss – Jennifer Chayes – who put the fire under my toosh. I’d been driving her crazy about the need to have more people deeply interrogating how data-driven technologies were intersecting with society. Microsoft Research didn’t have the structure to allow me to move fast (and break things). University infrastructure was even slower. There were a few amazing research centers and think tanks, but I wanted to see the efforts scale faster. And I wanted to build the structures to connect research and practices, convene conversations across sectors, and bring together a band of what I loved to call “misfit toys.”  So, with the support of Jennifer and Microsoft, I put pen to paper. And to my surprise, I got the green light to help start a wholly independent research institute.

I knew nothing about building an organization. I had never managed anyone, didn’t know squat about how to put together a budget, and couldn’t even create a check list of to-dos. So I called up people smarter than I to help learn how other organizations worked and figure out what I should learn to turn a crazy idea into reality. At first, I thought that I should just go and find someone to run the organization, but I was consistently told that I needed to do it myself, to prove that it could work. So I did. It was a crazy adventure. Not only did I learn a lot about fundraising, management, and budgeting, but I also learned all sorts of things about topics I didn’t even know I would learn to understand – architecture, human resources, audits, non-profit law. I screwed up plenty of things along the way, but most people were patient with me and helped me learn from my mistakes. I am forever grateful to all of the funders, organizations, practitioners, and researchers who took a chance on me.

Still, over the next four years, I never lost that nagging feeling that someone smarter and more capable than me should be running Data & Society. I felt like I was doing the organization a disservice by not focusing on research strategy and public engagement. So when I turned to the board and said, it’s time for an executive director to take over, everyone agreed. We sat down and mapped out what we needed – a strategic and capable leader who’s passionate about building a healthy and sustainable research organization to be impactful in the world. Luckily, we had hired exactly that person to drive program and strategy a year before when I was concerned that I was flailing at managing the fieldbuilding and outreach part of the organization.

I am overwhelmingly OMG ecstatically bouncing for joy to announce that Janet Haven has agreed to become Data & Society’s first executive director. You can read more about Janet through the formal organizational announcement here.  But since this is my blog and I’m telling my story, what I want to say is more personal. I was truly breaking when we hired Janet. I had taken off more than I could chew. I was hitting rock bottom and trying desperately to put on a strong face to support everyone else. As I see it, Janet came in, took one look at the duct tape upon which I’d built the organization and got to work with steel, concrete, and wood in her hands. She helped me see what could happen if we fixed this and that. And then she started helping me see new pathways for moving forward. Over the last 18 months, I’ve grown increasingly confident that what we’re doing makes sense and that we can build an organization that can last. I’ve also been in awe watching her enable others to shine.

I’m not leaving Data & Society. To the contrary, I’m actually taking on the role that my title – founder and president – signals. And I’m ecstatic. Over the last 4.5 years, I’ve learned what I’m good at and what I’m not, what excites me and what makes me want to stay in bed. I built Data & Society because I believe that it needs to exist in this world. But I also realize that I’m the classic founder – the crazy visionary that can kickstart insanity but who isn’t necessarily the right person to take an organization to the next stage. Lucky for me, Janet is. And together, I can’t wait to take Data & Society to the next level!

by zephoria at September 12, 2017 02:34 PM

September 11, 2017

Justin Reich
Research "Proves" - Very Little
When reading articles that reference empirical research, we need to examine how the author makes assumptions before considering the study as "proof."

by Beth Holland at September 11, 2017 01:27 PM

September 06, 2017

Berkman Center front page
Massachusetts Supreme Judicial Court Ruling on Bail Instructive Re: Algorithms & Criminal Justice


Prof. Chris Bavitz notes how a recent Massachusetts Supreme Judicial Court ruling might impact the use of algorithms in criminal justice.

Thumbnail Image: 

Prof. Chris Bavitz notes the Massachusetts Supreme Judicial Court’s recent ruling in Brangan v. Commonwealth  concerning the process of making bail determinations in Massachusetts, and how it might impact the use of algorithms in criminal justice.

If we are going to allow algorithms to play a role in certain aspects of the bail determination process, it is vitally important that we understand and make clear: (a) which questions those algorithms are helping to answer; (b) on what data sets those algorithms draw in reaching their conclusions; and (c) at what stages of a given legal analysis the outcomes prescribed by those algorithms may be relevant.

Read Chris Bavitz's Medium post

by djones at September 06, 2017 03:44 PM

September 05, 2017

Cyberlaw Clinic - blog
MA SJC Ruling on Bail Instructive Re: Algorithms and Criminal Justice

One track of the Berkman Klein Center’s work on artificial intelligence ethics and governance concerns the use of algorithms, machine learning, and related technologies in ways that impact social and criminal justice. Among other things, this research examines technologies employed by courts in their disposition of criminal cases. Increasingly, judicial determinations are informed by software that helps judges perform “risk assessments” of defendants or otherwise process and weigh factors relevant to decisions about sentencing, parole, and the like. The Center (along with collaborators at the MIT Media Lab) is undertaking a number of efforts to evaluate ways in which these kinds of technologies might mitigate or exacerbate bias. The initiative has both technical and legal components, and a significant amount of our work to date has involved technologists and lawyers working together to correlate technical concepts with legal standards (and vice-versa). In the context of these efforts, the Massachusetts Supreme Judicial Court’s recent ruling in Brangan v. Commonwealth — which has nothing to do with algorithms but concerns, broadly, the process of making bail determinations in Massachusetts — is of significant interest.

In assessing the use of technical tools in the context of bail, it is important to go back to first principles and understand the purposes behind bail and the forces that animate bail decisions in individual cases. In making a bail determination, a court is faced with a decision whether and on what terms (monetary and otherwise) to release a criminal defendant pending full adjudication of that defendant’s rights. Such decisions are complex and depend upon on analysis of factors that are often in direct tension:

  • To state the obvious, incarceration has significant adverse impacts on those who are incarcerated. Thus, any decision to deprive someone of her liberty (either prior to trial or following a finding of guilt) must take into consideration the defendant’s right to due process and the extraordinary damage that a jail term can have on one’s livelihood, family relations, and the like. At the bail stage, that decision must recognize the presumption that criminal defendants are innocent unless and until the state proves otherwise.
  • At the same time, lenient standards for pre-trial release might increase the likelihood that a criminal defendant will fail to return to court for a full adjudication of underlying charges. In some cases, releasing a criminal defendant might put victims or others members of the community at risk.

Against this backdrop, the Brangan case deals with a situation in which a defendant is “unable to give the necessary security for his appearance at trial because of his indigence” — a situation in which “the purpose of bail is frustrated.” The opinion (issued August 25, 2017 and authored by Justice Geraldine S. Hines) concerns a criminal defendant, Jahmal Brangan, held in Hampden County, Massachusetts jail since January 17, 2014 following his arrest for armed robbery. During the intervening three-and-a-half years, Mr. Brangan has been held subject to bail set at amounts ranging from $20,000 to $50,000 in cash (or $200,000 to $500,000 in surety). Mr. Brangan was unable to pay any of these amounts, and he thus remains incarcerated despite never having received a full and final adjudication of the criminal charges against him.1 On appeal to the Massachusetts Supreme Judicial Court, Mr. Brangan argued that, in setting bail, the lower court had “violated his right to due process because the judge failed to give adequate consideration to his financial resources, and set bail in an amount so far beyond his financial means that it resulted in his long-term detention pending resolution of his case.”

In its opinion, the SJC notes that the relevant statute — Massachusetts General Laws c. 276, § 57 — does not expressly identify ability to pay as among the factors a judge must consider when setting bail. The statute indicates that one imposing bail in the Commonwealth “shall take into consideration” the following factors (links added):

the nature and circumstances of the offense charged, the potential penalty the person faces, the person’s family ties, employment record and history of mental illness, the person’s reputation, the risk that the person will obstruct or attempt to obstruct justice or threaten, injure or intimidate or attempt to threaten, injure or intimidate a prospective witness or juror, the person’s record of convictions, if any, any illegal drug distribution or present drug dependency, whether the person is on bail pending adjudication of a prior charge, whether the acts alleged involve abuse, as defined in said section 1 of said chapter 209A, a violation of a temporary or permanent order issued pursuant to said sections 18 or 34B of said chapter 208, said section 32 of said chapter 209, said sections 34 or 5 of said chapter 209A or said sections 15 or 20 of said chapter 209C, whether the person has any history of issuance of such orders pursuant to the aforesaid sections, whether the person is on probation, parole or other release pending completion of sentence for any conviction and whether the person is on release pending sentence or appeal for any conviction.

Nevertheless, the Court identifies common law support for the proposition that “[a] Superior Court judge . . . must still consider a defendant’s financial resources when setting bail.” The Court specifically points to the SJC’s prior decisions in Commonwealth v. Torres, 441 Mass. 499, 504 (2004) and Querubin v. Commonwealth, 440 Mass. 108, 115 n. 6 (2003), which included “financial resources” as among the factors a judge should consider when conducting a bail hearing:

the factors that a judge is to consider when conducting a bail hearing are “(1) the nature and circumstances of the offense charged, (2) the accused’s family ties, (3) his financial resources, (4) his length of residence in the community, (5) his character and mental condition, (6) his record of convictions and appearances at court proceedings or of any previous flight to avoid prosecution or (7) any failure to appear at any court proceedings.”

Beyond statutory language and common law, the SJC looks to the federal and state constitutions and notes that both the Eighth Amendment to the United States Constitution and Article 26 of Massachusetts Declaration of Rights (i.e., the state constitution in the Commonwealth) expressly prohibit excessive bail. Such prohibition also finds support in the broader constitutional principles of due process and equal protection. Avoiding imposition of excessive bail and complying with constitutional mandates regarding due process and equal protection necessarily requires individualized, case-by-case determinations that include consideration of defendants’ ability to pay

The Court does not go so far as to hold that imposition of unaffordable bail constitutes a due process violation per se. On the contrary, the Brangan opinion serves to establish a framework for bail-setting without pre-determining outcomes in any particular case:

Although the judge must take a defendant’s financial resources into account in setting bail, that is only one of the factors to be considered, and it should not override all the others. Bail that is beyond a defendant’s reach is not prohibited. Where, based on the judge’s consideration of all the relevant circumstances, neither alternative nonfinancial conditions nor an amount the defendant can afford will adequately assure his appearance for trial, it is permissible to set bail at a higher amount, but no higher than necessary to ensure the defendant’s appearance.

Indeed, the Court notes that — in some cases — it may be permissible to detain a defendant prior to trial without bail as long as fundamental requirements of both substantive and procedural process have been met in reaching that result.

Applying this legal framework to the facts at hand, the Court expresses three sets of concerns in support of its decision that a new bail determination for Mr. Brangan is required. First, the Court notes that a defendant’s “dangerousness” is separate from whether or not he poses a “flight risk.” Under the applicable statute, the former may not be taken into consideration in assessing the amount of bail and may only be considered in the context of other conditions of pretrial release. The SJC finds that arguments presented in the Commonwealth’s briefs below conflated dangerousness and risk of flight and holds that the lower court’s bail determination did not adequately disentangle the two.

Second, and perhaps most importantly, the Court holds (footnotes and citations omitted):

[W]here, based on a defendant’s credible representations and any other evidence before the judge, it appears that the defendant lacks the financial resources to post the amount of bail set by the judge, such that it will likely result in the defendant’s long-term pretrial detention, the judge must provide findings of fact and a statement of reasons for the bail decision, either in writing or orally on the record. The statement must confirm the judge’s consideration of the defendant’s financial resources, explain how the bail amount was calculated, and state why, notwithstanding the fact that the bail amount will likely result in the defendant’s detention, the defendant’s risk of flight is so great that no alternative, less restrictive financial or nonfinancial conditions will suffice to assure his or her presence at future court proceedings.

Applying this standard, the Court finds that the bail order in question was deficient insofar as it did not evince consideration of Mr. Brangan’s financial resources and offered no explanation of “how the judge calculated the bail amount, how Brangan’s criminal history demonstrated that he posed a serious flight risk, or why that risk was so great that it necessitated a bail amount beyond his means.” (The Court also notes that the order failed to explain why the judge “rejected Brangan’s alternative proposal that he be released on $5,000 cash with the condition that he wear a GPS tracking bracelet.”)

Third, the Court holds that the long delay in bringing Mr. Brangan to trial must be considered in determining whether continuing pre-trial detention is justified.

The Brangan case is important on its merits and serves as a roadmap for judges, prosecutors, and defendants. For those approaching bail from a technical or design perspective, the case may be of interest for a slightly different set of reasons:

  • Bail may appear to present a paradigm for the type of straightforward judicial decisionmaking framework in which algorithms can play a role in processing information and balancing competing interests. But, the SJC’s opinion in Brangan underscores the complexities of bail and multivariate nature of a bail determination in any given case.
  • For those interested in “interpretability” in the context of algorithms,2 the opinion highlights the need for decisionmaking models that are transparent, trustworthy, replicable, and explainable. It also might serve to inform development of audit mechanisms for risk assessment tools in the context of bail, offering guidance regarding how judges must explain their decisionmaking processes as we seek to ensure the ability to review algorithmic decisions.
  • The decision underscores the need to draw from multiple data sources in order to arrive at reasonable and supportable bail determinations. Decisions may be informed by general principles (e.g., about the circumstances in which risk of flight is likely). But, individualized determinations based on specific facts about specific defendants’ financial circumstances are necessary to satisfy constitutional requirements.
  • On that note, if we are going to allow algorithms to play a role in certain aspects of the bail determination process, it is vitally important that we understand and make clear: (a) which questions those algorithms are helping to answer; (b) on what data sets those algorithms draw in reaching their conclusions; and (c) at what stages of a given legal analysis the outcomes prescribed by those algorithms may be relevant. Brangan‘s distinction between dangerousness and flight risk in the context of bail determinations made under 276 M.G.L. § 57 is instructive in this regard. The case makes clear that a bail determination might involve separate evaluations of two distinct factors: (i) whether a defendant, if released on bail, will pose a danger to a community, generally (or to particular past or potential future victims); and (ii) whether a defendant, if released on bail, is likely to leave the jurisdiction and/or otherwise fail to return for future court appearances. The case also makes clear that an analysis of each factor may be relevant at some stages of a monetary bail assessment and may be forbidden at other stages. If an algorithmic tool is used in making a legal determination (such as a decision about the conditions for pre-trial release), it is vital that a judge understand what that tool is assessing and in what ways that assessment may be relevant (or, conversely, in what circumstances consideration of that assessment may be prohibited) during the various stages of that legal determination.
  • In holding that the lower court erred by failing to consider Mr. Brangan’s alternative proposal – regarding a lower bail amount plus GPS tracking – the SJC demands not just an absolute assessment of whether to permit bail and at what amount but a relative assessment of the merits of alternative proposals and the ways in which those alternative proposals might, in different ways, help achieve the underlying goals of bail. This need to consider multiple options may inform the ways in which algorithmic risk assessment tools might be used in making bail or other similar determinations.
  • Finally, the Brangan decision highlights the importance of constitutional considerations in guiding bail determinations and defining both the universe of acceptable outcomes and the universe acceptable processes one might employ to achieve those outcomes in any given case. This is true regardless of the statutory architecture in place in a particular state.  Those designing tools to support judges must do so in ways that go beyond merely comporting with state statutes and must ensure such tools operate cognizant of and consistent with overarching considerations about due process and equal protection under law.

1 Mr. Brangan was tried, but the trial judge declared a mistrial due to improper conduct by the prosecution. Various appeals and other procedural consequences followed, but a court-ordered retrial had yet to commence by the time the SJC reached its decision.

See, e.g.Finale Doshi-Velez, Been Kim, “Towards A Rigorous Science of Interpretable Machine Learning, v. 2” arXiv:1702.08608v2 (last revised March 2, 2017)Zachary C. Lipton, “The Mythos of Model Interpretability, v. 3” arXiv:1606.03490v3 (last revised March 6, 2017)Jesse Johnson, “Goals of Interpretability,” The Shape of Data (November 17, 2016).

Image, “John Adams Courthouse – Suffolk County Courthouse – Boston, MA – DSC04718.JPG,” courtesy of Wikimedia Commons, user Daderot, made available under Creative Commons CC0 1.0 Universal Public Domain Dedication.

by Christopher Bavitz at September 05, 2017 02:42 PM

September 03, 2017

Good news for publishers and advertisers fearing the GDPR

The GDPR (General Data Protection Regulation) is the world’s most heavily weaponized law protecting personal privacy. It is aimed at companies that track people without asking, and its ordnance includes fines of up to 4% of worldwide revenues over the prior year.

The law’s purpose is to blow away the (mostly US-based) surveillance economy, especially tracking-based “adtech,” which supports most commercial publishing online.

The deadline for compliance is 25 May 2018, just a couple hundred days from now.

There is no shortage of compliance advice online, much of it coming from the same suppliers that talked companies into harvesting lots of the “big data” that security guru Bruce Schneier calls a toxic asset. (Go to and see whose ads come up.)

There is, however, an easy and 100% GDPR-compliant way for publishers to continue running ads and for companies to continue advertising. All the publisher needs to do is agree with this request from readers:

That request, along with its legal and machine-readable expressions, will live here:

The agreements themselves can be recorded anywhere.

There is not an easier way for publishers and advertisers to avoid getting fined by the EU for violating the GDPR. Agreeing to exactly what readers request puts both in full compliance.

Some added PR for advertisers is running what I suggest they call #Safeds. If markets are conversations (as marketers have been yakking about since  The Cluetrain Manifesto), #SafeAds will be a great GDPR conversation for everyone to have:

Here are some #SafeAds benefits that will make great talking points, especially for publishers and advertisers:

  1. Unlike adtech, which tracks eyeballs off a publisher’s site and then shoot ads at those eyeballs anywhere they can be found (including the Web’s cheapest and shittiest sites), #SafeAds actually sponsor the publisher. They say “we value this publication and the readers it brings to us.”
  2. Unlike adtech, #SafeAds carry no operational overhead for the publisher and no cognitive overhead for readers—because there are no worries for either party about where an ad comes from or what it’s doing behind the scenes. There’s nothing tricky about it.
  3. Unlike adtech, #SafeAds carry no fraud or malware, because they can’t. They go straight from the publisher or its agency to the publication, avoiding the corrupt four-dimensional shell game adtech has become.
  4. #SafeAds carry full-power creative and economic signals, which adtech can’t do at all, for the reasons just listed. It’s no coincidence that nearly every major brand you can name was made by #SafeAds, while adtech has not produced a single one. In fact adtech has an ugly history of hurting brands by annoying people with advertising that is unwelcome, icky, or both.
  5. Perhaps best of all for publishers, advertisers will pay more for #SafeAds, because those ads are worth more.

#NoStalking and #SafeAds can also benefit social media platforms now in a world of wonder and hurt (example: this Zuckerberg hostage video). The easiest thing for them to do is go freemium, with little or no ads (and only safe ones on the paid side, and nothing but #SafeAds on the free side, in obedience to #NoStalking requests, whether expressed or not.

If you’re a publisher, an advertiser, a developer, an exile from the adtech world, or anybody else who wants to help out, talk to us. That deadline is a hard one, and it’s coming fast.

by Doc Searls at September 03, 2017 07:07 PM

David Weinberger
Free e-book from Los Angeles Review of Books

I’m proud that my essay about online knowledge has been included in a free e-book collecting essays about the effect of the digital revolution, published by the Los Angeles Review of Books.

It’s actually the first essay in the book, which obviously is not arranged in order of preference, but probably means at least the editors didn’t hate it.


The next day: Thanks to a tweet by Siva Vaidhyanathan, I and a lot of people on Twitter have realized that all but one of the authors in this volume are male. I’d simply said yes to the editors’ request to re-publish my article. It didn’t occur to me to ask to see the rest of the roster even though this is an issue I care about deeply. LARB seems to feature diverse writers overall, but apparently not so much in tech.

On the positive, this has produced a crowd-sourced list of non-male writers and thinkers about tech with a rapidity that is evidence of the pain and importance of this issue.

The post Free e-book from Los Angeles Review of Books appeared first on Joho the Blog.

by davidw at September 03, 2017 03:37 PM

September 01, 2017

Justin Reich
Why Change? Launching Innovation in Schools
Preparing students for the challenges of the decades ahead requires understanding how computers are changing societies and labor markets.

by Justin Reich at September 01, 2017 02:01 PM

August 31, 2017

Center for Research on Computation and Society (Harvard SEAS)
Neil Thompson: Science is Shaped by Wikipedia: Evidence from a Randomized Control Trial

Title: Science is Shaped by Wikipedia: Evidence from a Randomized Control Trial


“I sometimes think that general and popular treatises are almost as important for the progress of science as original work.” - Charles Darwin, 1865

As the largest encyclopedia in the world, it is not surprising that Wikipedia reflects the state of scientific knowledge. However, Wikipedia is also one of the most accessed websites in the world, including by scientists, which suggests that it also has the potential to shape

Read more about Neil Thompson: Science is Shaped by Wikipedia: Evidence from a Randomized Control Trial

by kmavon at August 31, 2017 06:40 PM

Neil Thompson
Assistant Professor of Innovation and Strategy at the MIT Sloan School of Management
Co-director of the Experimental Innovation Lab (X-Lab)
Associate member of the MIT Computer Science and Artificial Intelligence Lab (CSAIL) and the Broa

by kmavon at August 31, 2017 06:12 PM

August 29, 2017

Justin Reich
Research "Says" - Or Does It?
As educators, we need to be critical consumers of educational research before assuming that findings present the "truth."

by Beth Holland at August 29, 2017 09:56 PM

Miriam Meckel
Jenseits der Verantwortung

Donald Trump bezieht jede Position und keine. Das ist ein Mangel an moralischer Führung, der auch der Wirtschaft schadet.

Wer auf vielen Seiten steht, steht auf keiner. Das mag man als opportunistisch geschickten Schachzug betrachten. Wer auf keiner Seite steht, tut ja niemandem weh. Ganz so einfach ist es aber nicht. Seit Tagen verläuft ein Riss durch die US-Wirtschaft, seit bei einem Aufmarsch von Ku-Klux-Klan und Neonazis in Charlottesville ein mutmaßlicher Rechtsextremist mit dem Auto in eine Gruppe von Gegendemonstranten gerast war, eine Frau getötet und mehrere Menschen verletzt hatte. US-Präsident Donald Trump weigerte sich, Position zu beziehen. Er sieht die Verantwortung „on many sides“.

Für CEOs gilt das Primat der Wirtschaft. US-Nobelpreisträger Milton Friedman hat das 1970 auf den Punkt gebracht: „The business of business is business.“ Gibt es in der Wirtschaft eine Verantwortung jenseits von Ergebnisverantwortung? Aus moralischer Perspektive muss das jede Chefin und jeder Chef für sich entscheiden. Aus systemischer Sicht lässt sich Friedmans Allsatz infrage stellen. Wer sich nicht gegen Gewalt und politischen Missbrauch positioniert, hält vielleicht Trump bei Laune, nicht aber die eigenen Kunden. Im Internet tobt ein Kampf um die Reaktionen auf Charlottesville. „Ich liebe Under Armour“, schreibt eine Kundin der Sportbekleidungsfirma. „Aber wenn der Chef im Beirat des Präsidenten bleibt, werde ich zum Nike-Girl.“ Kevin Plank, Gründer und CEO von Under Armour, verließ das Gremium dann auch. Ebenso wie die CEOs von Intel und Merck USA. Elon Musk und Disney-Chef Bob Iger sind schon draußen, seit Trump das Pariser Klimaabkommen gekündigt hat. So langsam beginnen die Topmanagerinnen und -manager zu verstehen, dass Appeasement, die wiederholte Beschwichtigung, bei Trump nicht weiterhilft.

Loyalität durch Zugehörigkeit, das funktioniert nicht bei diesem Präsidenten, der immer alles auf sich bezieht und wie ein politischer Egoshooter auf alles feuert, was ihm in die Quere kommt – völlig losgelöst von bislang aufgebauten Beziehungen. Das ist das Kernproblem mit Trump und seinen Ausfällen. Er stellt die „Licence to operate“, die Betriebslizenz von Unternehmen infrage, sobald es eine Meinungsverschiedenheit gibt. Beziehungsmanagement zwischen Unternehmen und Regierungschef wird zum Pokerspiel.

Kenneth Frazier, CEO von Merck USA, hat das deutlich gespürt. Er war der Erste, der auf Charlottesville mit Rückzug aus dem Trump-Beirat reagierte. Der Präsident bedachte ihn daraufhin mit zwei Tweets, in denen er das Geschäftsgebaren des Unternehmens und dessen „Beschiss“ mit zu hohen Medikamentenpreisen angriff.

Auf allen Seiten gleichzeitig sein, das klappt nicht. Die Wirtschaft steht aufseiten ihrer Kundinnen und Kunden. Und auf der Seite unternehmerischer Freiheit, die in den USA immer besonders ausgeprägt und für viele andere Länder Vorbild war. Trump ruiniert diese herausragende Tradition. Das ist moralisches und ökonomisches Versagen.

by Miriam Meckel at August 29, 2017 03:39 PM

August 28, 2017

Cyberlaw Clinic - blog
Welcoming Kendra Albert and Kicking Off the 2017-18 Academic Year!

With September just around the corner, we here in the Cyberlaw Clinic are eager to get the fall semester underway. And, we are especially excited to announce that the start of the new term comes with a new addition to our practice and teaching team in the form of the one and only Kendra Albert! Kendra is a familiar face around Harvard Law School and the Berkman Klein Center, having worked at Berkman before attending law school at HLS. Kendra was a student in the Cyberlaw Clinic during the spring term of their third year, back in 2016. Kendra spent a year in private practice at Zeitgeist Law in San Francisco from 2016-17 before rejoining us as a Clinical Instructional Fellow this week.  We are delighted to have Kendra on board and anticipate that they will contribute to a wide variety of our projects involving privacy, copyright, and related issues.

Kendra’s arrival comes in the midst of some additional staff changes at the Clinic. We are delighted to report that Jessica Fjeld has assumed the role of Acting Assistant Director of the Clinic and has been appointed a Lecturer on Law at Harvard Law School. In that capacity, Jess will co-teach the Cyberlaw Clinic Seminar along with Clinical Professor Chris Bavitz this fall. Vivek Krishnamurthy will take on the role of Clinic Attorney, splitting his time between Clinic projects concerning technology and human rights and Berkman Klein Center research initiatives (primarily from his new homebase on the west coast).

Susan Crawford will continue to oversee projects relating to government use of technology and civic innovation, with support from Maria Smith on the Clinic staff and Clinic Advisor Waide Warner. And, Mason Kortz will round out the teaching team, continuing his first year as a Fellow with us and spearheading a lot of our initiatives involving civil liberties and big data.

Our incoming students will be pleased to know that the Clinic’s docket for the coming year is as diverse as ever, with the usual array of projects that touch on copyright, speech, privacy, security, and civic tech.  We anticipate playing a role in the United States Copyright Office’s triennial review of exemption requests pursuant to 17 USC § 1201; engaging in some amicus advocacy; and counseling cutting-edge organizations in fields ranging from arts, to ed tech, to legal services.

We also anticipate that the Cyberlaw Clinic staff and students will play a significant role in the Berkman Klein Center’s Ethics and Governance of Artificial Intelligence initiative, launched earlier this year. Kira Hessekiel on the Clinic staff will help to support a number of research initiatives that focus on the broad theme of “algorithms and justice,” considering the ways in which technology can help to inform government decisions about allocation of scarce resources and assessments of risk in the criminal justice system.

All in all, it promises to be an extraordinary year!

by Clinic Staff at August 28, 2017 04:15 PM

Feeds In This Planet