Abstract
The performance of unsupervised learning models for natural images is evaluated quantitatively by means of information theory. We estimate the gain in statistical independence (the multi-information reduction) achieved with independent component analysis (ICA), principal component analysis (PCA), zero-phase whitening, and predictive coding. Predictive coding is translated into the transform coding framework, where it can be characterized by the constraint of a triangular filter matrix. A randomly sampled whitening basis and the Haar wavelet are included in the comparison as well. The comparison of all these methods is carried out for different patch sizes, ranging from . In spite of large differences in the shape of the basis functions, we find only small differences in the multi-information between all decorrelation transforms (5% or less) for all patch sizes. Among the second-order methods, PCA is optimal for small patch sizes and predictive coding performs best for large patch sizes. The extra gain achieved with ICA is always less than 2%. In conclusion, the edge filters found with ICA lead to only a surprisingly small improvement in terms of its actual objective.
© 2006 Optical Society of America
Full Article | PDF ArticleMore Like This
Michael S. Lewicki and Bruno A. Olshausen
J. Opt. Soc. Am. A 16(7) 1587-1601 (1999)
Thomas Wachtler, Te-Won Lee, and Terrence J. Sejnowski
J. Opt. Soc. Am. A 18(1) 65-77 (2001)
Mitchell G. A. Thomson
J. Opt. Soc. Am. A 16(7) 1549-1553 (1999)