General data compression limits

我们先回看一下这张图：

作者解释：

First, we’ve chosen the probabilities to be all of the form 1/2^k where k is a positive integer.

为什么呢？为了避免取对数之后有浪费、冗余：

This technicality is why the entropy is in general less than or equal to the expected length of the shortest binary encoding, rather than always exactly equal. 

而案例也是特殊构建的：

We’ve specififically picked the case where it is equal for illustrative purposes.

现实中，为达到极限（等号）：

It is possible to get closer to this bound in the more general case by encoding sequences of multiple events together into a single binary string, otherwise known as block encoding.

这两个案例：

These sequences in Figure 2 are called typical sequences.