Text notes: Architecture of Complexity

A collection of notes from Herbert Simon’s 1962 essay, “The Architecture of Complexity


Useful concepts from Cybernetics: Feedback, Homeostasis, Analyzing “adaptiveness” in terms of “theory of selective information”

Point: Describe the usefulness of complex systems across fields. By being abstract, may have relevance (careful to not say application) to other systems like those in social, biological, and physical systems. Discuss points that are  “applicable to the complexity of systems without specifying the exact content of that complexity.”

Definition: Complex systems can be defined, by Warren Weaver, as the whole is greater than the sum of its parts

Central theme:

  • Complexity frequently takes the form of hierarchy
  • Hierarchic systems have common properties independent of their specific content

The paper has 4 parts:

  1. Complex systems frequently take the form of hierarchy
  2. Relation between structure and time required for it to emerge through evolutionary process
  3. Nearly decomposability
  4. Relation between systems and description


Hierarchy systems: systems composed of interrelated subsystems, each in turn also hierarchic in structure; parts within parts (Not narrowed to structures of levels of subordination, but inclusive of)


  • Social: organizations, families
  • Biological: cells
  • Physical: particles, atoms, molecules
  • Symbolic: book

How to describe hierarchic structure?

  • Physical / biological systems typically described in spatial terms, while Social systems typically described as interactions (not necessarily spatial)
  • To reconcile this: define hierarchy in terms of intensity of interaction (where for phys / bio, intense interaction implies spatial propinquity”
  • Note: specialized communications and transportation types allow spatial propinquity to be less determinative of structure (e.g. think local supply global demand)

Definition of span: number of subsystems into which subsystems are further partitioned


Problem solving as natural selection

Parable: Hora & Tempus

Human problem solving can be described as searching through a maze

Starting with the axioms and previously proved theorems, various transformations allowed by the rules of the mathematical systems are attempted, to obtain new expressions. These are modified in turn until, with persistence and good fortune, a sequence or path of transformations is discovered that leads to the goal.

The process usually involves a great deal of trial and error. Various paths are tried; some are abandoned, others are pushed further. Before a solution is found, a great many paths of the maze may be explored. The more difficult and novel the problem, the greater is likely to be the amount of trial and error required to find a solution. At the same time, the trial and error is not completely random or blind ; it is, in fact, rather highly selective. The new expressions that are obtained by transforming given ones are examined to see whether they represent progress toward the goal. Indications of progress spur further search in the same direction; lack of progress signals the abandonment of a line of search. Problem solving requires selective trial and error.

A little reflection reveals that cues signaling progress play the same role in the problem-solving process that stable intermediate forms play in the biological evolutionary process. In fact, we can take over the watchmaker parable and apply it also to problem solving. In problem solving, a partial result that represents recognizable progress toward the goal plays the role of a stable subassembly.

Selectivity ~ Feedback of information from the environment

2 sources of selectivity in problem solving:

  1. Iterative trial and error
    • Those transformations that are stable provide building blocks for further construction
    • It is THIS information about stable configurations (and not free energy / negentropy from the sun) that guides the process of evolution and provides the selectivity essential
  1. Previous experience
    • In trial and error, search is greatly reduced
    • Reduce it to a problem previously solved

To conclude: Complex systems will evolve from simple systems much more rapidly if there are stable intermediate forms than if there are not


Interaction among vs. within subsystems:

  • Interactions among subsystems are weak but not negligible
  • Higher frequency dynamics associated with subsystems and lower frequency dynamics with larger systems
    • Note to self: Similar to pace layers!
  • Short run behavior of each component subsystem is approx. independent of short run behavior of the others
  • Long run behavior of any one component depends only in an aggregate way on the behavior of the others
    • Note to self: Assuming stablity?

See “Aggregation of Variables in Dynamic Systems” Econometrica 1961


When information is in outline form, it’s easier to include information about the relations among major parts and info about internal relations of parts of each of the suboutlines

For ND systems: little info is lost by representing hierarchies in outline form

  • Sub parts belonging to different parts only interact in an aggregated fashion– the detail of their interaction can be ignored

There is no conservation law that requires the description of a complex system to be as cumbersome as the object described

Simplicity, Redundancy:

  1. Hierarchic systems usually composed of only a few different kinds of subsystems, in various combinations and arrangements
  2. Hierarchic systems are often nearly decomposable
    • Only aggregative properties of their parts enter into the description of interactions of those parts
    • By adopting a descriptive language that allows the absence of something to go unmentioned, a nearly empty world can be described concisely
  1. By “recoding” the redundancy that is present but unobvious in the structure of a complex system can often be made patent
    • Example of recoding: Replacing a description of the time path with a description of a differential law that generates that path

Two types of descriptions, the warp and weft of our experience:

  • State: World as sensed
  • Process: World as acted upon

The distinction between the world as sensed and the world as acted upon defines the basic condition for survival of adaptive organisms:

  • The organism must develop correlations between goals in the sensed world and actions in the world of process. {‘hen they are made conscious and verbalized, these correlations correspond to what we usually call means-end analysis. Given a desired state of affairs and an existing state of affairs, the task of an adaptive organism is to find the difference between these two states, and then to find the correlating process that will erase the difference

Problem solving is a continual translation between state and process descriptions

Given a blueprint, find the corresponding recipe

To conclude:

  • How complex or simple a structure is depends critically upon the way in which we describe it.
  • Use redundancy to simplify their description but must make sure to have the right representation
  • Art by Wu Guanzhong