The Guessing Game
One idea ran through every room. It is easiest to see in a game that Claude Shannon invented in 1951.
Shannon showed people a sentence one letter at a time and asked them to guess the next. The sentence he printed in his paper is hidden here. It is in English, without capitals or punctuation.
Type the letter you think comes next, and keep trying until you have it. Play for as long as you like and then go on.
From games like this Shannon worked out that English carries about one bit a letter to somebody who can read it, somewhere between 0.6 and 1.3.
gzip knows nothing about English, and needs 3.0 bits a letter for the novels of Dickens. xz needs 2.2. A reader does better than either, by knowing what is likely to come next.
A good guesser is a good compressor. Every room in this museum held a way of guessing: the next letter, the next sound, the next square of a picture, the next frame, the next point on a rabbit, the next reading, the next person's DNA.
Whatever is guessed right costs nothing to send.
For most of this story the guessing was done by rules that engineers wrote out by hand. Lately more and more of it has been handed to neural networks, which learn to guess from examples.
In 2023 researchers at DeepMind took a language model that had only ever been trained on text and used it to compress photographs and speech. It beat PNG and FLAC, the usual lossless formats for each, so long as nobody counted the size of the model, which is enormous.
To compress something well you have to be able to predict it, and to predict it you have to understand it a little.
That is as far as the story has got.