The Brain Is a Lossy Compressor

September 8, 2009

You probably have an MP3 player; perhaps an iPod.  Chances are, the songs on it are compressed – their file size is smaller than the CD versions.  The more that song files can be compressed, the more songs can be fit on a player.  There are two kinds of compression: lossless and lossy.

Lossless compression reduces the size of the file while retaining all of the information that was in the original.  Even though a song file compressed losslessly is smaller than the CD version, every bit of sound information that was in the original can be reproduced from the compressed file.

Sometimes there’s a need for files to be compressed to even smaller sizes than lossless compression typically yields.  Lossy compression achieves this result by actually throwing away a small amount of information that was in the original file.  It’s very careful in how it does this – you don’t hear any gaps in the song; in fact, most people can’t hear any difference between a song that was compressed lossily and one compressed losslessly.

Other things can be compressed besides songs.  The common JPEG image format, for example, applies lossy compression to pictures.  The more an image is compressed, the smaller the resulting file is… but the more information is thrown away.  This photo of a flower was compressed to progressively higher degrees from left to right.  The right side is the most compressed – and the grainiest.

Photo by André Karwath

Photo by André Karwath

Compression can be applied to any set of information. If you wanted to squeeze a book into a smaller number of pages losslessly, you could lower the font size and decrease the spacing between lines. The compressed book would still have 100% of the information – every word – of the original; it would just be smaller. Alternatively, you could reduce it even more with lossy compression by removing all the articles (a, and & the). Some information would be missing, but articles are relatively expendable because they are redundant of other information in the text – we can figure out what’s missing from the context. If we remove “the” from “The quick brown fox jumped over the lazy dog,” we still know what it’s trying to say. If we were to remove nouns or verbs instead, we’d lose much of the message.

The goal of lossy compression is to maximize efficiency – to reduce the size of a data set as much as possible while retaining as much of its gist as possible. It draws on information theory, a mathematical discipline whose principles apply to a wide variety of domains including communications, finance, physiology and statistics. Using information theory, we can quantify the amount of entropy (or uncertainty, or noise) a particular lossy compression scheme introduces, the goal of course being to keep the signal-to-noise ratio as high as possible.

Human cognition relies heavily on lossy compression in constructing and working with mental representations of the objective world – i.e., beliefs. Our cognitive capacity, like your MP3 player’s memory, is zero-sum – our ability to process and store information is limited, and any data given to our brains to crunch takes up resources that become unavailable for processing anything else. We can’t possibly hold in our heads every bit of information that describes an object in the physical world (if it were possible to collect it all in the first place).

Our ability to know and successfully navigate objective reality, then, relies on the effectiveness of our cognitive shorthand as a stand-in for the objective facts it represents. Our information processing mechanism works not with rich images of the external world but with coarse representations that toss away a lot of the detail, of necessity.

Does lossy compression in and of itself compromise objectivity? Not if the compression process is attribute-neutral and doesn’t introduce any kind of bias. Faced with a decision about which pixels in an image of a flower to discard, if we take them from all over the flower in a way that makes the resulting image consistently fuzzy – as opposed to, say, removing a specific petal – the correspondence of the resulting compressed image to the original changes in granularity, but not in direction. It introduces uncertainty, but diffuses it evenly.

As it turns out, though, we cannot count on cognition’s compression process to be unbiased. Worse, the data our senses collect and pass on for processing is prone to corruption by a host of biases before it even gets to the information-reduction stage, as well as in processing steps after that stage. These challenges to living in the objective world will be the subject of my next post.


Leave a Comment

Previous post:

Next post: