An unlikely event can be more informative than an event that was already expected. Self-information quantifies this idea.

For an event with probability $p>0$, its information content in bits is

$$I(p)=-\log_2 p.$$

Certain events carry no surprise

If $p=1$, then

$$I(1)=0.$$

Learning that a certain event occurred provides no new information.

Rarer events carry more information

As $p$ decreases, $-\log_2p$ increases. An event with probability $1/2$ carries one bit, while one with probability $1/8$ carries three bits.

Why a logarithm

If two independent events have probabilities $p$ and $q$, their joint probability is $pq$. The logarithm makes their information additive:

$$I(pq)=I(p)+I(q).$$

This matches the idea that independent pieces of evidence contribute separate amounts of information.

The choice of logarithm base determines the unit: base 2 gives bits, while the natural logarithm gives nats.