Display options
Cite
Goodfellow IJ, Courville A, Bengio Y. Scaling up spike-and-slab models for unsupervised feature learning. IEEE Trans Pattern Anal Mach Intell. 2013;35(8):1902-14doi: 10.1109/TPAMI.2012.273.
Goodfellow, I. J., Courville, A., & Bengio, Y. (2013). Scaling up spike-and-slab models for unsupervised feature learning. IEEE transactions on pattern analysis and machine intelligence, 35(8), 1902-14. https://doi.org/10.1109/TPAMI.2012.273
Goodfellow, Ian J, et al. "Scaling up spike-and-slab models for unsupervised feature learning." IEEE transactions on pattern analysis and machine intelligence vol. 35,8 (2013): 1902-14. doi: https://doi.org/10.1109/TPAMI.2012.273
Goodfellow IJ, Courville A, Bengio Y. Scaling up spike-and-slab models for unsupervised feature learning. IEEE Trans Pattern Anal Mach Intell. 2013 Aug;35(8):1902-14. doi: 10.1109/TPAMI.2012.273. PMID: 23787343.
Copy Download .nbib Format: NLM
Share it on

IEEE Trans Pattern Anal Mach Intell. 2013 Aug;35(8):1902-14. doi: 10.1109/TPAMI.2012.273.

Scaling up spike-and-slab models for unsupervised feature learning.

IEEE transactions on pattern analysis and machine intelligence

Ian J Goodfellow, Aaron Courville, Yoshua Bengio

Affiliations

  1. Departement d’Informatique et de Recherche Operationelle, Université de Montréal, Montréal, QC H3C 3J7, Canada. [email protected]

PMID: 23787343 DOI: 10.1109/TPAMI.2012.273

Abstract

We describe the use of two spike-and-slab models for modeling real-valued data, with an emphasis on their applications to object recognition. The first model, which we call spike-and-slab sparse coding (S3C), is a preexisting model for which we introduce a faster approximate inference algorithm. We introduce a deep variant of S3C, which we call the partially directed deep Boltzmann machine (PD-DBM) and extend our S3C inference algorithm for use on this model. We describe learning procedures for each. We demonstrate that our inference procedure for S3C enables scaling the model to unprecedented large problem sizes, and demonstrate that using S3C as a feature extractor results in very good object recognition performance, particularly when the number of labeled examples is low. We show that the PD-DBM generates better samples than its shallow counterpart, and that unlike DBMs or DBNs, the PD-DBM may be trained successfully without greedy layerwise training.

Publication Types