Display options
Cite
Tangkaratt V, Sasaki H, Sugiyama M. Direct Estimation of the Derivative of Quadratic Mutual Information with Application in Supervised Dimension Reduction. Neural Comput. 2017;29(8):2076-2122doi: 10.1162/NECO_a_00986.
Tangkaratt, V., Sasaki, H., & Sugiyama, M. (2017). Direct Estimation of the Derivative of Quadratic Mutual Information with Application in Supervised Dimension Reduction. Neural computation, 29(8), 2076-2122. https://doi.org/10.1162/NECO_a_00986
Tangkaratt, Voot, et al. "Direct Estimation of the Derivative of Quadratic Mutual Information with Application in Supervised Dimension Reduction." Neural computation vol. 29,8 (2017): 2076-2122. doi: https://doi.org/10.1162/NECO_a_00986
Tangkaratt V, Sasaki H, Sugiyama M. Direct Estimation of the Derivative of Quadratic Mutual Information with Application in Supervised Dimension Reduction. Neural Comput. 2017 Aug;29(8):2076-2122. doi: 10.1162/NECO_a_00986. Epub 2017 Jun 09. PMID: 28599116.
Copy Download .nbib Format: NLM
Share it on

Neural Comput. 2017 Aug;29(8):2076-2122. doi: 10.1162/NECO_a_00986. Epub 2017 Jun 09.

Direct Estimation of the Derivative of Quadratic Mutual Information with Application in Supervised Dimension Reduction.

Neural computation

Voot Tangkaratt, Hiroaki Sasaki, Masashi Sugiyama

Affiliations

  1. University of Tokyo, Bunkyo-ku, Tokyo, 113-033, Japan [email protected].
  2. Nara Institute of Science and Technology, Ikoma, Nara 630-0192, Japan, and RIKEN Center for Advanced Intelligence Project, Chuo-ku, Tokyo 103-0027, Japan [email protected].
  3. RIKEN Center for Advanced Intelligence Project, Chuo-ku, Tokyo 103-0027, Japan, and University of Tokyo, Bunkyo-ku, 113-033, Japan [email protected].

PMID: 28599116 DOI: 10.1162/NECO_a_00986

Abstract

A typical goal of linear-supervised dimension reduction is to find a low-dimensional subspace of the input space such that the projected input variables preserve maximal information about the output variables. The dependence-maximization approach solves the supervised dimension-reduction problem through maximizing a statistical dependence between projected input variables and output variables. A well-known statistical dependence measure is mutual information (MI), which is based on the Kullback-Leibler (KL) divergence. However, it is known that the KL divergence is sensitive to outliers. Quadratic MI (QMI) is a variant of MI based on the [Formula: see text] distance, which is more robust against outliers than the KL divergence, and a computationally efficient method to estimate QMI from data, least squares QMI (LSQMI), has been proposed recently. For these reasons, developing a supervised dimension-reduction method based on LSQMI seems promising. However, not QMI itself but the derivative of QMI is needed for subspace search in linear-supervised dimension reduction, and the derivative of an accurate QMI estimator is not necessarily a good estimator of the derivative of QMI. In this letter, we propose to directly estimate the derivative of QMI without estimating QMI itself. We show that the direct estimation of the derivative of QMI is more accurate than the derivative of the estimated QMI. Finally, we develop a linear-supervised dimension-reduction algorithm that efficiently uses the proposed derivative estimator and demonstrate through experiments that the proposed method is more robust against outliers than existing methods.

Publication Types