Meta: Innovating By Breaking The Rules

At Meta, the prevailing culture is one of risk – borne by everyone else.

The past few weeks have brought three new examples – of varying severity – of the Company’s tendency to behave ethically only when forced to do so:

• Following a BBC investigation, the Indian government ordered Meta to immediately remove Instagram ads promoting content depicting the sexual abuse of children. To put it bluntly, protecting children does not appear to be one of Meta’s highest priorities. Indeed, at the same time, the Company has been lobbying California lawmakers to shield it from proposed legislation that would increase legal penalties in cases involving harm to minors.
• The Company shut down a tool designed to monitor how employees interact with their computers in order to help train its artificial intelligence models. The decision came after a major internal security breach made sensitive data collected by the tool accessible to the entire workforce, vindicating concerns that many employees had already raised.
• Meta suspended a default feature of its new Muse Image AI system that allowed it to generate visuals from Instagram users’ public accounts. Users had to opt out if they wanted to protect their content. The backlash from countless users, and perhaps more importantly, from Hollywood’s largest actors’ union, eventually persuaded Meta to reconsider its approach to its members’ privacy.

This after-the-fact approach to governance creates the conditions for growth through transgression. That strategy is hardly new. It is embedded in the Company’s culture, which was originally guided by the motto “Move fast and break things.” The problem is that while this agility enabled Mark Zuckerberg’s teams, for example, to transform Facebook into a highly effective mobile platform, it also left an extraordinary amount of damage in its wake.

American sociologist Diane Vaughan described the mechanisms through which organizations gradually normalize deviant behavior. Similarly, management professor David Welsh and his coauthors have shown that people are more likely to commit serious violations when these are preceded by a series of smaller breaches than when they are suddenly asked to commit a major offense.

This gradual escalation makes moral disengagement easier. People progressively find ways to justify behavior they would never have accepted at the outset. Organizational misconduct therefore rarely begins with one dramatic leap into illegality. It advances incrementally, with each violation made more acceptable by the one that came before it.

This research leads to a far more demanding conclusion than it may initially appear to: Culture is never an accident. It is the product of what leaders accept, encourage, or refuse to correct. As legendary college football coach Mike Leach famously put it, “You’re either coaching it or allowing it to happen.”

In Mark Zuckerberg’s case, he has not merely tolerated misconduct because he lacked information or involvement. Time and again, he was warned and allowed it to continue.

The examples are too numerous for me to claim anything close to an exhaustive list, so I will focus on the most emblematic:

• Facebook did not meaningfully strengthen its oversight of users’ personal data until the global Cambridge Analytica scandal erupted.
• Meta did not significantly step up its efforts to control misinformation on its platforms until the dangers had been exposed to the entire world through a succession of crises: The 2016 U.S. presidential election, Brexit, the Covid pandemic, and the January 6, 2021 attempt to overturn the U.S. election in Washington, D.C. Incidentally, after Donald Trump was elected to the White House for a second time, Meta largely reversed course in an apparent effort to curry favor with him.
• Meta introduced new safety features intended to limit harmful content shown to teenagers on Facebook, Instagram, and Messenger only after losing two legal cases on the issue.
• Most disturbingly, Meta is, to my knowledge, the only company in history to have been implicated in two successive genocides or mass-atrocity campaigns. In 2018, the United Nations Independent International Fact-Finding Mission on Myanmar emphasized Facebook’s decisive role in spreading hatred against the Rohingya. An Amnesty International investigation likewise concluded that Facebook’s recommendation system had “proactively amplified” anti-Rohingya content and that the company had ignored warnings it began receiving as early as 2012. Yet, the same deadly pattern was repeated only a few years later in Ethiopia. Amnesty International reported that Facebook’s algorithms dramatically increased the circulation of hateful content targeting Tigrayans during the 2020–2022 civil war, even though the Company had once again been warned before the conflict began.

As this incomplete list suggests, ethics at Meta too often appears to serve as the adjustment variable for the Company’s expansion. The risks are internalized when they generate profits and externalized when they produce consequences.

What we are seeing at Meta is not a succession of isolated mistakes. It is a culture that keeps pushing boundaries until external, or occasionally internal, pressure forces the Company to retreat.

You do not have to take my word for it. One of the people who described this strategy most clearly was Andrew Bosworth, one of Mark Zuckerberg’s closest lieutenants. In a 2016 internal memo, he argued, quite literally, that the deaths of innocent people were an acceptable consequence as long as Facebook continued to grow.

Meta deserves at least this much credit: Over the years, its words and its actions have remained remarkably consistent.