麻省理工计算机系统结构课件

Examples:  information in one coin flip: log2(2/1) = 1 bit  roll of 2 dice: log2(36/1) = 5.2 bits  learning card drawn from fresh deck is a club = log2(52/13) = 2 bits I (message) = log2 1 pmessage ! " # # $ % & & Common case: Suppose you’re faced with N equally probable choices, and you receive a message that narrows it down to M choices. The probability that message would be sent is M*(1/N) so the amount of information you have received is I (message) = log2 1 M 1 N ! " # # # $ % & & & = log2 N M ! " # $ % & bits Even when a message doesn’t resolve all the uncertainty & = log2 N M ! " # $ % & bits Even when a message doesn’t resolve all the uncertainty