A Characterization of Entropy in Terms of Information Loss

作者：Baez John C; Fritz Tobias^*; Leinster Tom

来源：Entropy, 2011, 13(11): 1945-1957.

DOI：10.3390/e13111945

• Shannon entropy
• Tsallis entropy
• information theory
• measure-preserving function

There are numerous characterizations of Shannon entropy and Tsallis entropy as measures of information obeying certain properties. Using work by Faddeev and Furuichi, we derive a very simple characterization. Instead of focusing on the entropy of a probability measure on a finite set, this characterization focuses on the "information loss", or change in entropy, associated with a measure-preserving function. Information loss is a special case of conditional entropy: namely, it is the entropy of a random variable conditioned on some function of that variable. We show that Shannon entropy gives the only concept of information loss that is functorial, convex-linear and continuous. This characterization naturally generalizes to Tsallis entropy as well.

• 出版日期2011-11