Open Access
Add to BibSonomy
Abstract
The accuracy and precision of the modulation transfer function (MTF) of a sampled imaging system are affected by the shift-variant nature of subpixel binning of the pixel values in edge-based methods. This study demonstrates that a binning phase selected from a small number of binning phases can achieve a practical precision criterion for the MTF measurement. Furthermore, the new method proposed in this paper approximates the non-aliased, fundamental MTF without edge angle estimation and the following subpixel binning. The algorithm simply averages the aliased MTFs calculated from the row-by-row edge gradients in the region of a bitonal edge image and removes an assumed aliasing component. This method is also applicable to an oblique and non-straight edge.
© 2019 Optical Society of America under the terms of the OSA Open Access Publishing Agreement
1. Introduction
Spatial resolution is the ability of an imaging system to capture or render fine details. The modulation transfer function (MTF) describes the magnitude of the response of an optical system to sinusoids of different spatial frequencies. The imaging system for MTF analysis must be approximately linear and shift invariant. To preserve the convenience of a transfer-function approach for a sampled imaging system, the object being imaged is assumed to have spatial frequency components with random phases that are uniformly distributed with respect to the sampling sites [1].
In a recently published work describing a new edge-based method [2], the MTF of a sampled imaging system is measured from the edge response of a bitonal edge target with an arbitrary edge angle rather than only a near-vertical or near-horizontal direction, as in the ISO 12233 edge-based method [3]. The edge angle in a region of interest (ROI) is estimated by fitting a parameterized two-dimensional normal cumulative distribution function to the ROI pixel values with significantly high accuracy, precision, and robustness against camera noise. The ROI pixels are then projected along the direction of the estimated edge onto the horizontal axis, which is divided into subpixel-wide bins to reduce the influence of signal aliasing on the estimated MTF. The values of the pixels collected in each bin are averaged to generate an oversampled one-dimensional (1D) edge spread function (ESF). A finite difference filter is used to approximate the derivative of the ESF to obtain a line spread function (LSF). After applying a Tukey window to the LSF, a discrete Fourier transform is performed. The modulus of the corresponding optical transfer function (OTF) is then normalized to unity at zero frequency, and the MTF is estimated over a range of horizontal spatial frequencies beyond the Nyquist frequency. The edge-based method is implemented in a real-time MTF measurement system. The bin locations corresponding to the ROI pixel positions computed in the projection process are recorded once in a lookup table (LUT); then, the ROI pixels are mapped to the bins, based on the LUT in the subsequent input video frames without estimating the edge angle, assuming the edge is static during the measurement.
Edge-based methods, however, have some practical issues. The accuracy and precision depend on the binning phase and oversampling ratio [1]. Although the average of the MTFs calculated with several binning phase shifts over the bin width improves the accuracy and precision, it also increases the computational cost. Furthermore, inaccurate edge angle estimation causes an error in the resultant MTF [4]. A sophisticated algorithm is usually needed to increase the accuracy of the edge angle estimation. The real-time MTF measurement system uses a quadratic function to fit a curved edge [5]. Although a higher-order function may fit more complex curves, the computation will become costly and less stable. Unfortunately, the accuracies of the edge angle and resultant MTF estimations are not usually addressed in edge-based methods.
In this paper, the dependency of MTF measurement precision on the binning phase shift is simulated, and a method is demonstrated to find an optimal binning from a small number of binning phase shifts under a practical criterion. Next, a simplified, hardware-friendly edge-based MTF measurement method is introduced that is applicable to an oblique and non-straight edge without the edge angle estimation and the following subpixel binning.
2. Binning phase optimization in edge-based method
The edge-based MTF measurement is valid only when the edge positions imaged are assumed to be uniformly distributed relative to the sampling sites. However, validating this assumption is complicated because of the shift-variant nature of subpixel binning used in the edge-based method, in addition to the sampling process employed by the image sensor. Masaoka [1] computed the accuracy and precision of MTFs estimated from synthesized bitonal edge images with edge angles ranging from 0° to 45° with various binning phase shifts, where ξ is the spatial frequency in cycles/pixel. Figure 1 shows a schematic of the binning phase shifts. The number of binning phase shifts is denoted by m. The oversampling ratios of the binning nbin were set to 4× , 8×, 16×, and 32×, with the binning phase shifts occurring at intervals of 1/1024th of the bin width w, yielding 1024 MTF results (i.e., m = 1024) for each nbin; here, w is calculated by dividing the pixel width W by nbin. The results show that the precision, which depends on the edge angle, is counterintuitively higher at a higher oversampling ratio of the binning, regardless of the smaller pixel counts in the bins. It is also shown that significantly high accuracy is achievable by taking an ensemble average of the MTFs even at low oversampling ratios, which is more appropriate for smaller ROIs to prevent zero-count bins. However, calculating as many as 1024 MTFs for each ROI is impractical. Therefore, it is of interest to determine how many binning phase shifts are required to meet a practical precision criterion for MTF estimations.
Fig. 1 Schematic of binning phase shifts (nbin = 4 and m = 8). Note that horizontally aligned pixels are rotated such that the edge angle orients upright and perpendicular to the bin array in the new edge-based method.
Typically, the spatial resolution characteristics of a broadcast camera are measured to determine if the camera satisfy the minimum threshold criterion, which is typically an MTF of 35% at 0.37 cycles/pixels [2]. A sinc4(ξ) curve can be used to approximate a typical MTF curve of high-performance broadcast cameras that meet the MTF criterion of 35% at 0.37 cycles/pixel. Figure 3 in Ref. [1] shows MTF curves estimated from a synthesized edge image with a critical edge angle having a sinc4(ξ) MTF for nbin of 4 and 8 and m of 32. The 32 discrete MTFs curves are diverse for each nbin; they are more diverse for the 4× oversampling ratio than for the 8× oversampling ratio, whereas both the mean curves are close to the sinc4(ξ) curve. This suggests a practical solution to improve precision by averaging the multiple MTFs obtained with several binning phase shifts rather than simply estimating the MTF with a single binning phase (m = 1), as is done in conventional edge-based methods.
Figure 2 shows the standard deviations (SDs) of the averaged MTF values at 0.37 cycles/pixel calculated from synthesized 200 (W) × 200 (H)-pixel edge images with edge angles ranging from 0° to 45° having a sinc4(ξ) MTF for nbin of 4 and 8 and m of 1 (conventional), 4, 8, and 16 with binning phase offsets ranging from 0 to 1023w/1024/m at intervals of w/1024/m. It should be noted that the binning phase offset is different from the binning phase shift (see Fig. 1). For example, when m = 1, a total of 1024 MTF curves were estimated for each edge angle and nbin with binning phase offsets ranging from 0 to 1023w/1024 at w/1024 intervals. When m = 16, 16 MTF curves were averaged at each binning phase offset ranging from 0 to 1023w/1024/16 at the w/1024 intervals to yield a total of 64 averaged MTF curves for each edge angle and nbin. The results show that the precision increases with not only higher nbin but also higher m. When an SD of less than 1% is required for any edge angle, 8× oversampling (nbin = 8) with four binning phase shifts (m = 4) would be a desirable choice. When measuring the MTFs of a video camera in real time, however, even averaging only four MTFs for each frame will raise the computational cost. To solve this problem, it is proposed to calculate MTFs with a small number of different binning phase shifts, average them, and then select a binning phase shift that provides an MTF value closest to the averaged MTF value at 0.37 cycles/pixel. Figure 3 shows the SDs of the MTF values calculated from the synthesized edge images using the selected binning phase shifts with nbin of 4 and 8 and m of 4, 8, and 16 with binning phase offsets. When an SD of less than 1% is required for any edge angle, 8× oversampling (nbin = 8) with eight binning phase shifts (m = 8) is suitable. Note that these simulations results are valid when the edge angle estimation is accurate. The next section proposes a simplified edge-based method that does not require edge angle estimation and the following binning process.
Fig. 2 SDs of averaged MTF values at 0.37 cycles/pixel from synthesized edge images having a sinc4(ξ) MTF with nbin of 4 (blue) and 8 (orange) and m of 1 (conventional), 4, 8, and 16.
Fig. 3 SDs of the MTF values at 0.37 cycles/pixel calculated from synthesized edge images having a sinc4(ξ) MTF using the selected binning phase shifts with nbin of 4 (blue) and 8 (orange) and m of 4, 8, and 16.
3. Simplified edge-based method
The novel edge-based method proposed in this section approximates the MTF of a sampled imaging system without edge angle estimation and the following subpixel binning. Although the imaged edge positions are assumed to be distributed uniformly relative to the sampling sites, as in the ISO and other edge-based methods, this method does not use a supersampled ESF. In the algorithm, the aliased MTFs calculated from the row-by-row edge gradients in an ROI are averaged and corrected by removing an assumed aliasing component to approximate the fundamental MTF.
The ensemble average of the aliased MTFs ⟨|Fs(ξ)|⟩ of the sampled imaging system is obtained as follows:
where step(x – s) is a step function to represent an ideal edge target with a transition from 0 to 1 at the position of s, f(x) is the LSF of the imaging system whose sampling procedure is expressed by comb(x), δ(x) – δ(x – 1) is a difference filter to approximate the 1D derivative of the ESF, F(ξ) represents the OTF calculated by the Fourier transform ℱ{ } of f(x), and sinc(ξ) represents the attenuation on MTF in ⟨|Fs(ξ)|⟩ due to the difference filter. Here, the aliased MTF yielded by averaging the MTFs calculated from row-by-row edge gradients in an ROI with correction of the sinc–1(ξ) factor is denoted as ⟨|Fs(ξ)|⟩CORR (≈ ⟨|Fs (ξ)|⟩/sinc(ξ)). ⟨|Fs(ξ)|⟩CORR suffers from aliasing by the convolution of F(ξ) and comb(ξ) with a phase shift of e–j2πsξe–jπξ. Figure 4 shows diverse 200 MTF curves calculated from the rows of a synthesized 100 (W) × 200 (H)-pixel slanted-edge image with a fundamental MTF |F(ξ)| of sinc4(ξ) [1] and ⟨|Fs(ξ)|⟩CORR. The averaged MTF curve is closer to the sinc4(ξ) curve at lower spatial frequencies but deviates upward at higher frequencies due to the aliasing components. Here, the fundamental MTF and aliasing MTF in the range of 0 ≤ ξ ≤ 0.5 are denoted as |F(ξ)| and |F(1 – ξ)|, respectively. Figure 5 illustrates the relationships among a fundamental MTF |F(ξ)|, aliasing MTF |F(1 – ξ)|, and ensemble average of the aliased MTFs with correction ⟨|Fs(ξ)|⟩CORR. Note that ⟨|Fs(ξ)|⟩CORR is not equivalent to their simple sum because of the phase between F(ξ) and F(1 – ξ). For example, at the Nyquist frequency ξN ( = 0.5), ⟨|Fs(ξN)|⟩CORR is smaller than the double of |F(ξN)|.Fig. 4 MTF curves (gray) calculated from each row of a synthesized slanted-edge image (MTF: sinc4(ξ); ROI size: 100 (W) × 200 (H) pixels; slant angle: 3°) with the averaged MTF curve (black solid line) and |F(ξ)| (red dashed line).
Fig. 5 Fundamental MTF |F(ξ)|, aliasing MTF |F(1 – ξ)|, and ensemble average of aliased MTFs with correction ⟨|Fs(ξ)|⟩CORR.
Let us approximate the fundamental MTF from ⟨|Fs(ξ)|⟩CORR. First, assume that |F(ξ)| becomes zero with ξ ≥ 1 and ξ ≤ –1. According to Eq. (1), ⟨|Fs(ξ)|⟩ CORR is equivalent to the average of the length of the vector sum F(ξ) + F(1 – ξ) obtained by rotating the two vectors in opposite directions. The average length does not depend on the phase difference of e–jπξ between F(ξ) and F(1 – ξ). When r(ξ) is the ratio of |F(1 – ξ)| to |F(ξ)|, the length of the vector sum with a phase difference of 2θ is expressed as (1 + 2r(ξ)cos(2θ) + r(ξ)2)1/2. This formula can be transformed to (1 – r(ξ)sin2(θ))1/2, and its integral with θ from 0 to π/2 yields the ratio of ⟨|Fs(ξ)|⟩/sinc(ξ) to |F(ξ)| using the elliptic integral of the second kind E as follows:
Figure 6 shows the relation between ⟨|Fs(ξ)|⟩CORR/|F(ξ)| and r(ξ) ( = |F(1 – ξ)|/|F(ξ)|).To determine the unknown values of r(ξ), we approximate |F(ξ)| as roughly sincp(ξ), which is simple in the formula as it has a null point at the sampling frequency. At the Nyquist frequency, r(ξN) is 1, and the fundamental MTF |F(ξN)| is obtained by dividing ⟨|Fs(ξN)|⟩CORR by 4/π ( = 1.2732) based on Eq. (2). Then, p is determined such that sincp(ξN) passes through the modulation by using the following equation:
Then, |F(ξ)| is calculated from ⟨|Fs(ξ)|⟩CORR by substituting sincp(1 – ξ)/sincp(ξ) ( = (ξ/(1 – ξ))p) for r(ξ) in Eq. (2).The algorithm of the proposed method is simple and hardware-friendly. Although E in Eq. (2) is computationally expensive, the scaling factors shown in Fig. 6 can be approximated using a 1D LUT. Moreover, the proposed method is applicable to not only near-vertical or near-horizontal edges, but also to oblique and/or non-straight edges by using parallelogram ROIs. Figure 7 compares the MTF calculated from an oblique, non-straight edge with a fundamental MTF of sinc4(ξ) with the one calculated with a conventional edge-based method. Note that the scanning direction is still in the vertical or horizontal direction, although the spatial frequency is scaled to correct for the rotational difference between the horizontal direction of the spatial frequency in the analysis and the edge angle [6], which can be simply set to the slant angle of the user-selected parallelogram ROI.
Fig. 7 MTF calculated by the simplified method (black solid line) and a conventional edge-based method (blue solid line) from a synthesized image with an oblique and non-straight edge having a sinc4(ξ) MTF (red dashed line).
Note that this method will provide a good approximation for a camera only if the fundamental MTF is a sinc-like monotonously decreasing curve. A deviation from the sinc-like shapes (especially near the Nyquist frequency) due to digital image processing can give rise to erroneous corrections on the aliased MTF. Therefore, it is recommended that the accuracy of the MTF approximation be verified with conventional edge-based methods. Another problem is that this method is not robust against camera noise because the algorithm analyzes each line of an ROI image instead of yielding a single ESF through a binning that reduces camera noise. Figure 8 shows the overestimated MTFs calculated from 100 edge images with camera noise and an accurate MTF calculated from the averaged edge image. A Tukey window was applied to apodize the LSFs. In practice, several edge images must be averaged to make camera noise negligible.
Fig. 8 MTFs (gray) calculated from each of 100 edge images (MTF: sinc4(ξ), red dashed line; ROI size: 100 (W) × 200 (H) pixels; edge angle: 3°; signal-to-noise ratio: 40 dB) and MTF calculated from the averaged edge image (black solid line).
4. Summary
This study demonstrates that unlike conventional edge-based methods using a fixed binning phase, averaging MTFs with a small number of binning phase shifts significantly improves the precision. For video image measurement, it is suggested that using an optimized binning phase selected from a small number of different binning phases rather than several different binning phases for each frame reduces the computational cost. Furthermore, the simplified, hardware-friendly MTF measurement method approximates the fundamental MTF by averaging aliased MTFs calculated from row-by-row edge-gradients in an averaged ROI image and by removing an assumed aliasing component without edge angle estimation and the following subpixel binning. This method is also applicable to an oblique and non-straight edge when the MTF to be measured is a sinc-like monotonously decreasing function. Thus, practical MTF measurement will be facilitated by using this new method as complementary to conventional edge-based methods with binning phase optimization.
5. Acknowledgements
I thank the anonymous reviewers for their insightful comments and suggestions.
References
1. K. Masaoka, “Accuracy and precision of edge-based modulation transfer function measurement for sampled imaging systems,” IEEE Access 6(1), 41079–41086 (2018). [CrossRef]
2. K. Masaoka, K. Arai, and Y. Takiguchi, “Real-time measurement of ultra-high definition camera modulation transfer function,” SMPTE Motion Imag. J. 127(10), 1 (2018).
3. Photography—electronic still picture imaging—resolution and spatial frequency responses, document ISO 12233:2017, 2017.
4. P. D. Burns, “Slanted-edge MTF for digital camera and scanner analysis,” in Proceedings of IS&T PICS Conference 2000, Portland, OR, USA, 2000, pp. 135–138.
5. K. Masaoka, T. Yamashita, Y. Nishida, and M. Sugawara, “Modified slanted-edge method and multidirectional modulation transfer function estimation,” Opt. Express 22(5), 6040–6046 (2014). [CrossRef] [PubMed]
6. J. K. M. Roland, “A study of slanted-edge MTF stability and repeatability,” Proc. SPIE 9396, 93960L (2015). [CrossRef]
