Abstract

Maximum Entropy (ME) estimation has been applied in various forms with various names to a wide variety of problems ranging from the depths of seismic spectral analysis, sonar and radar beam forming and filter formation, to astronomical imaging and beyond to economics. The particular techniques and theoretical points of view differ greatly among the several disciplines. Our very general interpretation, which includes these others as special cases, is based on two considerations. Any image, measured as signal, pattern or spectrum, whatever it represents, is necessarily a degraded version of the true object because real measurement systems have limited spatial and temporal bandwidth. The samples are finite and perhaps undersampled. Furthermore noise cannot be ignored. Therefore, many different possible object patterns can produce the same measured image pattern. One way to resolve this ambiguity is to apply the ME method. In our interpretation, a probability is assigned to every possible object pattern and the most probable pattern is chosen as the estimated or restored object. Patterns are assigned probabilities based on the physics and statistics of the immediate problem. The entropy is understood to mean the logarithm of the probability, following Boltzmann. So, to find a maximum of the entropy is to find a maximum of the probability, subject to the measured image data constraints and any a priori bias. No new "principle of ME" or appeal to information theory is needed to justify the method, though they may enrich our understanding. Sometimes misunderstandings have arisen in the use of the information theoretic entropy of Shannon, –f log f, and it has been used inappropriately. These considerations have been developed at length,¹ so only a brief summary will be given here. We develop the idea of ME in an analogy to the well known statistical mechanical principle of the minimization of free energy, and derive some useful benefits in the consideration of fluctuations or noise. The degree of confidence in the ME estimate is derived and some examples of the ME method are given showing super-resolution.

© 1986 Optical Society of America