Abstract

There has been increasing interest in quantitative performance evaluation of computer vision algorithms. The usual method is to vary parameters of the input images or parameters of the algorithms and then construct operating curves that relate the probability of misdetection and false alarm for each parameter setting. Such an analysis does not integrate the performance of the numerous operating curves. In this paper we outline a methodology for summarizing many operating curves into a few performance curves. This methodology is adapted from the human psychophysics literature and is general to any detection algorithm. We demonstrated the methodology by comparing the performance of two line detection algorithms. The task was to detect the presence or absence of a vertical edge in the middle of an image containing a grating mask and additive Gaussian noise. We compared the Burns line finder and an algorithm using the facet edge detector and the Hough transform. To determine each algorithm's performance curve, we estimated the contrast necessary for an unbiased 75% correct detection as a function of the orientation of the grating mask. These functions were further characterized in terms of the algorithm's orientation selectivity and overall performance. An algorithm with the best overall performance need not have the best orientation selectivity. These performance curves can be used to optimize the design of algorithms.

© 1992 Optical Society of America