A Bregman Matrix and the Gradient of Mutual Information for Vector Poisson and Gaussian Channels

作者：Wang Liming^*; Carlson David Edwin; Rodrigues Miguel R D; Calderbank Robert; Carin Lawrence

来源：IEEE Transactions on Information Theory, 2014, 60(5): 2611-2629.

DOI：10.1109/TIT.2014.2307068

• Vector Poisson channels
• vector Gaussian channels
• gradient of mutual information
• Bregman divergence
• Bregman matrix
• minimum mean squared error (MMSE)

A generalization of Bregman divergence is developed and utilized to unify vector Poisson and Gaussian channel models, from the perspective of the gradient of mutual information. The gradient is with respect to the measurement matrix in a compressive-sensing setting, and mutual information is considered for signal recovery and classification. Existing gradient-of-mutual-information results for scalar Poisson models are recovered as special cases, as are known results for the vector Gaussian model. The Bregman-divergence generalization yields a Bregman matrix, and this matrix induces numerous matrix-valued metrics. The metrics associated with the Bregman matrix are detailed, as are its other properties. The Bregman matrix is also utilized to connect the relative entropy and mismatched minimum mean squared error. Two applications are considered: 1) compressive sensing with a Poisson measurement model and 2) compressive topic modeling for analysis of a document corpora (word-count data). In both of these settings, we use the developed theory to optimize the compressive measurement matrix, for signal recovery and classification.

• 出版日期2014-5