Working Backwards: Separable, Single Threaded Teams and Herbert Simon

Working Backwards by Colin Bryar and Bill Carr describes in detail the inner workings of Amazon, including the development of its best practices over a nearly 3-decade history.

The chapter Organizing: Separable, Single-Threaded Leadership quickly jumped out at me.

Separable teams? Where had I heard that idea before?

It didn’t take me long to remember: Herbert Simon’s concept of Nearly Decomposable systems.

In this piece, I discuss Amazon’s concept of Separable, Single Threaded Leadership through Herbert Simon’s writings on organizations and complexity.

Note: I do not / have not worked at Amazon myself; my description of the company’s organizational structure here refers to the content in the book.

In Working Backwards authors Bryar and Carr explore the evolution of Amazon’s organizational structure. With company growth came an increasing array of dependencies… not just dependencies of product, but dependencies of team. Contrary to the default assumption that a healthy dose of coordination and communication helps teams mitigate dependencies, Amazon’s leadership team realized that “all this cross-team communication didn’t really need refinement at all– it needed elimination.”

Seeing that our best short-term solutions would not be enough, Jeff proposed that instead of finding new and better ways of managing dependencies, we figure out how to remove them. We could do this, he said, by reorganizing software engineers into smaller teams that would be essentially autonomous, connected to other teams only loosely, and only when unavoidable. These largely independent teams could do their work in parallel. Instead of coordinating better, they could coordinate less and build more.

I recall coming to the same realization several years ago; I was speaking with a friend and budding CEO about team coordination. The startup I was working at had just crested into the false summit of 50+ people (miniscule compared to Amazon, but the feeling was still real) and our small team vibe was starting to break down. At the time, my friend– whose own org was already 10x that size– remarked that what we needed was less collaboration, not more. I remember adamantly disagreeing… didn’t better collaboration solve everything?

It was Herbert Simon’s description of nearly decomposable systems that changed my mind.

In nearly decomposable systems, interactions are tighter *within* teams rather than *among* them. Said a different way: autonomous subsystems, loosely coupled, allow for greater stability. By partitioning activities based on rate of interaction, “near-decomposability is a means of securing the benefits of coordination while holding down its costs,”  Simon remarked in one of his last public lectures. It is only at identified collecting points that information should flow between subsystems, enforcing the need for greater localized decision making.

You can read more about nearly decomposable systems here.

Amazon explored this concept of nearly decomposable systems (and the speed and stability they deliver) with their own term: separable “two-pizza teams.” For Amazon, defining characteristics of a two-pizza team included:

• Small in size
• Autonomous in operation
• Business owner capable
• Self funded

Now let’s describe that using Herbert Simon’s terms. That gives us:

Once in practice, Amazon’s concept of separable two-pizza teams only solved part of their growing coordination problem. While this structure reduced dependencies and enabled each team to be faster, the concept of size (i.e. “two” units of pizza) didn’t scale well; it didn’t match with reality. “We later came to realize that the biggest predictor of a team’s success was not whether it was small but whether it had a leader with the appropriate skills, authority, and experience to staff and manage a team whose sole focus was to get the job done.”

Through trial and error, Amazon realized there was more they could do. Although the concept of separable (i.e. near decomposability) enabled their goal of speed while reducing unnecessary transaction costs, something was missing. That missing piece was single-threaded leadership: accountability to a singular goal and decision making framework.

(Single-threaded, i.e. a single sequence, one command at a time. Simon– with his interest in attention and condensing information into simple outlines– would be proud.)

Single-threaded teams have a clear leader with the “right” amount of  attention and empowerment . (I want to say “maximum,” but I suspect Simon would scold me for not saying “bounded!”) These small teams are organized with clear accountability and empowerment, “by focusing their scope, providing dedicated resources, and establishing clear objectives.”

Herbert Simon approached this need for focus and accountability with his use of the term hierarchy. For Simon, “hierarchy” didn’t refer to power but to form: “By a hierarchic system, or hierarchy, I mean a system that is composed of interrelated subsystems, each of the latter being, in turn, hierarchic in structure until we reach some lowest level of elementary subsystem.”

Through hierarchies, it is clear where the accountability lies. Subsystems nest into subsystems which nest into subsystems; the hierarchic span for each acts as leader with sole focus for those within their bounds.

And it scales with size.

Amazon’s concept of separable single-threaded teams are the real-world example of Herbert Simon’s hierarchic and nearly decomposable systems.

For more reading, see

• Org Design, Part 1: Nearly decomposable systems, with Serendipity & Slack
• By Herbert Simon
• The Architecture of Complexity (1962)
• Shape of Automation (1965)
• Last Public Lecture (2000)

Post Script

I delight in finding examples of Herbert Simon’s theories in the real world.

I hope to learn that Jeff Bezos is actually a Herb Simon fan, and that he modeled Amazon’s organizational structure after Simon’s teachings all along.

I also find it fun that the Amazon team uncovered the truths of hierarchic, nearly decomposable systems in the reverse order from how Simon articulated them. For example, in his essay The Architecture of Complexity, Simon starts by describing the importance of hierarchic systems (single threaded) before digging into near decomposability (separable).

We all learn through trial and error 🙂

Notes on my daily writing practice

I started this one with 20 minutes on Day 3, but then got stuck. A key point I wanted to make eluded me. Luckily, on a run several days later the idea came to me! Of course, that doesn’t mean this was any easier to write; I spent MUCH more time on this than intended. Still, I think it was worth it.

Day: 6/12 [Wondering what happened to Day 5/12? That became a personal email to friends!]

Total time: 5 hours :/

Update: June 30, 2021 – Added more “oomph” to a point, 25 mins