The book is organized into four parts, based on descriptions of objects at four different levels of abstraction.
1. Generalized images-images and image-like entities.
2. Segmented images-images organized into subimages that are likely to correspond to "interesting objects."
3. Geometric structures-quantitative models of image and world structures.
4. Relational structures-complex symbolic descriptions of image and world structures.

The parts follow a progression of increasing abstractness. Although the four parts are most naturally studied in succession, they are not tightly interdependent. Part I is a prerequisite for Part II, but Parts III and IV can be read independently.