Color filter array patterns for small-pixel image sensors with substantial cross talk

Leo Anzagira; Eric R. Fossum

doi:10.1364/JOSAA.32.000028

1. INTRODUCTION

In the world of image sensors, increasing pixel count while decreasing pixel size has been an important trend that has seen pixel pitch reduced to 1 μm and below and pixel count increase to over 40 million pixels. This drive to decrease pixel size poses a number of significant challenges. A fundamental challenge is the reduction in light collection that has been mitigated by a variety of approaches including the use of microlenses and backside illumination (BSI) [1].

Another problem that remains persistent in small pixels is the increased occurrence of cross talk. Cross talk occurs in two different ways. First, light incident above one pixel may penetrate into a neighboring pixel and generate photocharge. This is known as optical cross talk and tends to be very important in frontside illuminated (FSI) pixels. As pixel sizes decrease to levels comparable to the wavelength of the visible light, increased diffraction increases this form of cross talk in both FSI and BSI pixels. In the second cross-talk mechanism, the charge generated in a pixel diffuses into neighboring pixels and contributes to the wrong signal. This is known as electrical or diffusion cross talk. Decreasing pixel sizes shortens the length over which the charge has to diffuse to reach neighboring pixels, aggravating the impact.

Cross talk in color image sensor pixels diminishes the color signal of affected color channels and increases the overlap in the spectral responses of the different color channels. For instance, in the Bayer pattern, the cross talk in the red pixel extends its spectral response into the green wavelength region and decreases the response in the red spectral region. The diminished color signal as a result of cross talk reduces the color gamut that can be reproduced from the raw color signal without color correction.

Typically, the standard sRGB color gamut can be reproduced by means of color correction. However, if the cross talk substantially diminishes the color gamut of the device, more intensive color correction is required. The color correction must perform an amplification operation to transform the reduced gamut, and increased signal subtraction is required to compensate for the increased overlap in spectral responses. Increased cross talk therefore increases the noise amplification of the color correction process and leads to reduced SNR performance. Color correction matrices for sensors with increased cross talk will therefore sacrifice either color reproduction accuracy or SNR or both.

Several approaches have been suggested both for reducing the effects of cross talk and for mitigating its impact on SNR and color reproduction accuracy. Most approaches have been centered on the idea of modifying the pixel structure to minimize cross-pixel light absorption. A double light shield is used in [2] to suppress cross talk caused by obliquely incident light, and deep trench isolation is used in [3] to reduce electrical cross talk between pixels. More recently, Samsung’s ISOCELL pixel technology has targeted cross-talk reduction by introducing an insulating layer between pixels to prevent electrical cross talk and lower cross talk occurrence by a reported 30% [4].

Other approaches such as [5,6] modify the color filter array design specifically to mitigate the occurrence and effect of cross talk. In the patented approach in [5], the cross-talk behavior of the pixel is first characterized, and then a transformation is determined to map the spectral response in the presence of cross talk to the desired spectral response. This transformation is implemented as an adjustment in the composition of pigments in the color filters.

In this paper, we present an alternative approach to mitigate the effect of cross talk on the color reproduction accuracy and SNR performance in image sensors where high cross talk cannot be avoided. We propose color filter array patterns that show a markedly better color reproduction than the CFAs widely used in industry today.

2. DESCRIPTION OF PROPOSED COLOR FILTER ARRAY PATTERNS

Many color filter array patterns have been researched and implemented in image sensors to date with their various benefits and drawbacks, as highlighted in [7]. The design of CFA patterns is often discussed with regards to specifications such as spatial resolution, aliasing, and immunity to color artifacts. However, as pixels sizes decrease to the submicrometer range, most of the restrictions on CFA design such as the spatial resolution become inconsequential. For such small pixels, also known as subdiffraction-limit (SDL) pixels, where the pixel pitch may be less than the Airy disk diameter of the diffraction-limited point response of the optical system, a sampling rate less than the Nyquist rate can be used since the pixel size is much smaller than the smallest resolvable point. The Airy disk diameter for an optical system is dependent on its $F$ -number $F$ and the wavelength of the illumination:

D = 2.44 λ F .

For optical systems in most mobile applications with F-number of about 2.8, the smallest resolvable spot is 3.8 μm (assuming green light, 550 nm). In image sensor concepts such as the quanta image sensor (QIS) [8], pixels/jots are expected to be only a fraction of a micrometer. Conventional restrictions on CFA design regarding spatial resolution and color moiré therefore become trivial.

The Bayer pattern [9], which is inarguably the most widely used CFA pattern for image sensors in digital cameras, has its red and blue pixels surrounded vertically and horizontally by green pixels. Cross-talk signal into red and blue pixels is therefore predominantly from green pixels. This extends the red and blue pixel responses into the green region of the spectrum. Likewise green pixels receive cross-talk signal from two red and two blue pixels. This has the effect of reducing the actual signal for each of the red blue and green pixels while increasing the overlap in their spectral responses.

In our proposed color filter array, we introduce a secondary color pixel between every two primary color pixels in the regular Bayer pattern. The secondary color introduced is the color obtained by summing the two Bayer primary colors. A yellow pixel is placed between red and green pixels and a cyan pixel between blue and green pixels, as shown in Fig. 1. A green pixel is situated in the middle in one CFA pattern. This is because the middle position has the same neighbors as the primary green in the expanded Bayer pattern. This is depicted as RGBCY. An alternative design aimed at increasing light sensitivity uses a white/panchromatic filter in place of the middle green. This pattern is depicted as RGBCWY.

Fig. 1. CFAs of the Bayer pattern and two proposed patterns with black square demarcating the kernel for each CFA.

Download Full Size | PDF

In the new patterns, spectral overlap caused by cross talk is minimized since most of the cross talk is now in the same spectral region as the signal. Each primary color pixel is surrounded vertically and horizontally by secondary color pixels. The individual primary color pixels have negligible cross-talk contributions to each other. As a result of this spectral overlap reduction, the color correction process causes less noise amplification and SNR reduction when RGBCY and RGBCWY patterns are used.

The new CFA patterns have five or six colors in their kernels compared to the three in the Bayer pattern. Assuming traditional interpolation methods are used, these new patterns result in $6 \times N$ or $5 \times N$ outputs requiring $3 \times 6$ and $3 \times 5$ color correction matrices, respectively. It is understood that these color correction matrices will increase computational costs. Also the added color filters will increase fabrication costs since two additional masks will have to be used. However, it is worth noting that the five and six channel outputs can effectively be combined to produce robust R, G, B channels that can then utilize the conventional tristimulus color correction requiring $3 \times 3$ matrices. For instance, for the five-channel RGBCY, it is possible to combine channels to produce $R^{'}$ , $G^{'}$ , and $B^{'}$ where

R^{'} = \frac{R + Y - G}{2},

An alternative implementation of the RGBCY pattern uses three primary color filters, like the Bayer pattern, thereby eliminating the additional mask costs. In this alternate form, referred to as sRGBCY, each secondary color filter is replaced with two half-primary filters whose colors sum up to give the secondary color. Thus the active region of each yellow pixel is half-covered by a red filter and half-covered by a green filter, as shown in Fig. 2. The red half of the yellow pixel is the half-closest to the red pixel, and the green half is closest to the neighboring green pixel. In a similar fashion, the cyan pixels are half-covered by blue filters and half-covered by green. The kernel of the new CFA is transformed as shown in Fig. 2.

Fig. 2. Transformation of the RGBCY kernel to sRGBCY showing yellow pixels composed of half-red and half-green filters and cyan pixels half-covered by blue and green filters.

Download Full Size | PDF

It is important to note that using two half-primary color filters in place of the secondary color filters will reduce their light transmission by half. The resulting sRGBCY now has the same sensitivity as the Bayer pattern. The CFA pattern obtained using this alternative implementation is shown in Fig. 3.

Fig. 3. Full CFA pattern showing RGBCY pattern and its modified form sRGBCY, which uses two half-primary filters for each secondary color filter.

Download Full Size | PDF

3. EXPERIMENTAL PROCEDURE AND MODEL DESCRIPTION

In the evaluation of the new color filter array patterns, only computer simulations have been performed at this time. Test images were created for the different CFA patterns. The color filters used in this simulation were Gaussian curves centered at the wavelengths stated in Table 1 above. It is assumed that the secondary color filters are a combination of the two primary filter responses, and primary filters are scaled to have a maximum transmittance of 0.33 at the center wavelength. We also assume an ideal imager such that the only source of variability is the shot noise. The analysis here therefore describes the best performance scenario for all the CFAs investigated.

Table 1. Sensor Simulation Parameter Values

View Table | View all tables in this article

In our simulations, the pixel response was determined using the incident photon flux, $Φ (λ)$ in $photons / {μm}^{2} s$ , the target spectral reflectance $M (λ)$ , and the spectral transmittance $C T (λ)$ of the color filter above each pixel. The signal collected at each pixel is given by

S (λ) = k \cdot Φ (λ) \cdot M (λ) \cdot C T (λ) .

The target used is the Macbeth chart. The proportionality constant

k

accounts for pixel parameters such as pixel size, lens F-number, etc. A step-by-step explanation of the derivation of pixel response from incident illumination to electron generation is provided in [10].

Five CFA patterns were simulated and compared. These include the Bayer RGB and CMY patterns and the new RGBCWY, RGBCY, and sRGBCY patterns. For each CFA, a $128 \times 192$ test image of Macbeth chart was created using Eq. (3) to generate pixel responses. Shot noise is simulated by means of the Poisson random function generator in MATLAB.

In our simulations pixel cross talk is modeled through a cross-talk kernel similar to the approach in [11]. The cross-talk kernel for each pixel location is a $3 \times 3$ matrix that depicts the loss of signal from the central pixel into adjacent pixels:

X_{R} = [\begin{matrix} a_{11} & a_{12} & a_{13} \\ a_{21} & a_{22} & a_{23} \\ a_{31} & a_{32} & a_{33} \end{matrix}] .

The middle element

a_{22}

represents the fraction of signal that remains in the pixels after cross-talk signal has been subtracted. The surrounding terms represent the fraction of signal the middle pixel loses to its neighbors. This cross-talk model is illustrated in Fig. 4. The figure shows the cross talk for red, green, and blue pixels.

Fig. 4. Cross-talk illustration. Only the central pixel is illuminated, but the surrounding pixels receive some signal due to cross talk from the illuminated pixel.

Download Full Size | PDF

In this illustration, it is assumed that only the central pixel is illuminated. Therefore, the off-center pixels ideally should have no signal if cross talk is absent. As illustrated in Fig. 4, the red pixel has the highest level of cross talk into surrounding pixels and the blue pixel has the least. Cross-talk values used in our simulations are linearly scaled versions of data obtained by means of technology computer-aided design (TCAD) simulation. These cross-talk values are listed in Table 1.

In our analysis, we assume that cross talk only occurs between horizontal and vertical neighbor pixels. Cross talk to diagonally neighboring pixels is negligible. For ease of analysis, we also assume that the cross talk is independent of the wavelength in the different spectral regions. Thus all wavelengths in the red region have the same cross talk, which is higher than cross talk for wavelengths in the green and blue owing to the deeper penetration. This is less so in BSI pixels where the photodiode is located away from the light-incident surface.

For each pixel, the net signal is calculated by subtracting the percentage of signal lost due to cross talk and also adding the signal gained as a result of cross talk from its four neighboring pixels. For instance for a red pixel in the Bayer pattern that is surrounded by four green pixels, the net signal following cross-talk addition is given by

S_{R}^{'} = (1 - x r r) \cdot S_{R} + \sum_{i = 1}^{4} (\frac{x g g}{4}) \cdot S_{G, i} .

The

S_{R}

term is the signal in the red pixel before cross talk addition whereas

S_{G 1}

to

S_{G 4}

are the signals of the surrounding green pixels before cross-talk addition. The parameters

x r r

and

x g g

are fraction of signal lost from red and green pixels due to cross talk. Note that only a fourth of the total cross talk from each surrounding pixel is added to the red pixel’s net signal.

Simple bilinear interpolation was used to create full test images for each CFA pattern. For this method of interpolation, the new filter array patterns with kernel size of $4 \times 4$ require an interpolation kernel of size at least $5 \times 5$ . The Bayer and CMY test images can be interpolated with $3 \times 3$ kernels; however, the interpolation step is an averaging operation that reduces noise and increases SNR. It is therefore important that the same kernel size be used for all test images so that a fair comparison can be made. For this reason, the Bayer RGB and CMY test images are interpolated with $5 \times 5$ kernels as well.

A simple white balance operation was performed on all images. White balance weights were determined to equalize mean values of R, G, and B channels of the white patch of the test image. The weights calculated for our simulation are presented in Table 2.

Table 2. White Balance Weights

View Table | View all tables in this article

Following the white balancing operation, a color correction step similar to the white point preserving least squares color correction algorithm in [12] was applied. This color correction method uses the fmincon function in MATLAB to determine the color correction matrix that minimizes the color difference between the raw image and a reference image of the Macbeth color checker. To preserve the white point, we enforce a linear constraint that requires that the white point be mapped exactly. Besides preserving the white point, this constraint also ensures that the column sum of the color correction matrices are the same for all CFA patterns, which makes comparison easier.

4. EVALUATING CFA PERFORMANCE IN THE PRESENCE OF CROSS TALK

To evaluate and compare the performance of the different CFA patterns in the presence of cross talk, we consider two metrics. The deterioration in color reproduction accuracy caused by cross talk is quantified by means of the CIEDE2000 metric and the sensitivity metamerism index. The SNR deterioration as a result of cross-talk addition is also quantified using the color SNR as defined in (ISO) 12232 [13]. It is understood that color error and SNR performances are greatly dependent on the CFA as well as the source of illumination. Therefore two illumination sources have been used in these simulations. Results are reported for D65 and CIE A illuminants.

A. Color Reproduction Accuracy

As explained earlier, cross talk in color image sensor pixels both diminishes the color signal and increases the spectral overlap of the color channels, thus reducing the reproducible gamut and decreasing the color reproduction accuracy. Ideally, the sensitivity of a color imager should be a linear function of the sensitivity of cones in the human eye. This condition, known as the Luther–Ives condition, is rarely ever achieved. Deviation from this condition results in objects with different reflectance spectra producing the same color output, a phenomenon known as metamerism.

As the reproducible color gamut decreases, metameric error increases since fewer colors can be reproduced accurately by the imager. This error is quantified by the sensitivity metamerism index (SMI) proposed in [14]. The SMI is given by

R_{i} = 100 - 4.6 Δ E_{i},

where

Δ E_{i}

is the color difference between the test image and a reference image. An imager with SMI of 100 is an ideal imager with no metameric error. Typical high performance imagers have SMI values around 85.

To calculate SMI, the CIEDE2000 metric was used to determine the color difference between the test and reference images. This metric provides a more perceptual measure of the color difference between two images and is given by

Δ E_{00} = {[{(\frac{Δ L^{'}}{k_{L} S_{L}})}^{2} + {(\frac{Δ C^{'}}{k_{C} S_{C}})}^{2} + {(\frac{Δ H^{'}}{k_{H} S_{H}})}^{2} + R_{T} (\frac{Δ C^{'}}{k_{C} S_{C}}) (\frac{Δ H'}{k_{H} S_{H}})]}^{\frac{1}{2}} .

[

Δ L^{'}

,

Δ C^{'}

and

Δ H^{'}

] and [

S_{L}

,

S_{C}

and

S_{H}

] are the lightness, chrominance, and hue differences and weighting factors, respectively. In our experiments the parametric factors (

k_{L}

,

k_{H,}

and

k_{C}

) are set to unity. The color difference is calculated as the difference between reference values of the Macbeth chart and the test images generated for the different CFAs. A detailed discussion of the derivation of the CIEDE2000 metric can be found in [15].

For each test image, the color difference value is calculated for each patch of the Macbeth chart, and the mean color error over all patches can be determined using Eq. (8),

Δ E = \frac{1}{N} \sum_{i = 1}^{N} Δ E_{i},

where

N = 24

is the number of patches of the Macbeth color checker.

Color correction matrices were determined by using the fmincon function in MATLAB to minimize the color difference between the reference chart and the test image. The color error for each CFA was determined by performing 100 runs using the $128 \times 192$ test images—the Poisson random function generator in MATLAB was used to introduce shot noise so that images varied from run to run. The average color difference of the 100 runs is computed for each CFA pattern. The results obtained for the color difference and SMI can be found in Tables 3 and 4.

Table 3. Color Error and SMI using D65 Illuminant

View Table | View all tables in this article

Table 4. Color Error and SMI Using CIE A Illuminant

View Table | View all tables in this article

From the simulations, better color reproduction is obtained when the CIE A illuminant is used. The Bayer pattern has the best color reproduction for both D65 and CIE A illuminants when there is no cross talk between pixels. However, when cross talk is added, the Bayer RGB and CMY patterns record the worst color performance for both illuminants. On the other hand, the new CFA patterns have the best color reproduction when cross talk is substantial. In fact, for the CIE A illuminant, cross-talk addition doesn’t cause any significant deterioration in the color performance of the new CFA patterns whereas the color difference for the Bayer RGB pattern increases from 2.7 to 4.3.

B. SNR Performance

The luminance signal to noise ratio (YSNR) discussed in [10] is the most widely used metric for comparing different color images because it provides a single overall SNR measure that combines the SNRs of the different color channels using their luminance coefficients. However, this YSNR metric ignores the correlation between color channels as a result of different color processing steps. As a result, this metric tends to underestimate the contributions of the blue channel to the visible noise and overemphasize the green channel contribution [16].

For the purpose of this investigation, the visual noise calculation is performed using the noise metric specified in ISO 12232 [13]. The SNR is calculated as the ratio of the luminance to the visual noise. The luminance evaluated using linearized RGB values is given by

Y = 0.2125 R + 0.7154 G + 0.0721 B .

The visual noise in [13] is calculated from the noise in the luminance channel and two chrominance channels (

R - Y

) and (

B - Y

) and is given by

σ (D) = [σ^{2} (Y) + C_{1} σ^{2} (R - Y) + C_{2} σ^{2} (B - Y)]^{\frac{1}{2}},

where

D

is the illumination.

The SNR for each CFA’s test image was determined both with and without cross talk. The color correction matrices used in Section 4.A. for optimal color reproduction were used here so that a fair comparison of the SNRs for the different CFAs can be done. It should be mentioned that the SNRs calculated here are only meant to highlight SNR deterioration due to cross talk assuming optimal color reproduction is desired. A more detailed comparison of the SNR performance of the different CFAs at fixed levels of color reproduction accuracy will be done later.

For the SNR comparison, we consider the fourth gray patch of the test images. This is the patch on the Macbeth chart that has a reflectance value closest to 18%. Again, 100 runs are performed and the luminance ( $Y$ ) and chrominance ( $R - Y$ and $B - Y$ ) values are stored for each run. The standard deviations of luminance and chrominance channels at each pixel location are calculated and used in Eq. (10) to determine the visual noise. The results of this evaluation are presented in Table 5.

Table 5. SNR Results for Different CFA Patterns

View Table | View all tables in this article

It can be noted that the Bayer RGB pattern has the highest SNR in the absence of cross talk regardless of the illuminant used. However, upon addition of cross talk, the SNR of the Bayer RGB decreases by 4 dB (for D65) whereas the new patterns show significantly less deterioration in SNR. This trend holds true for both D65 and CIE A illuminants. The SNR advantage of the new CFA patterns is higher in simulations using the CIE A illuminant. For this illuminant, SNRs of the Bayer RGB and CMY patterns decrease by 3.9 dB and 1.8 dB, respectively, whereas the new patterns have no discernible change in SNR. Therefore, when the color correction matrices are optimized for the best color reproduction, the new CFA patterns we propose have both better color reproduction and SNR performance than the conventional CFAs under conditions of high cross talk.

C. Trade-Off between YSNR and Color Reproduction

Color correction matrices optimized for minimizing color error tend to produce less than optimal SNR performance. Generally SNR performance can be improved at the expense of the color accuracy. In this section, we investigate the SNR–color accuracy trade-off behavior for the different CFAs. This relationship gives us an idea of the degradation in color accuracy that must be conceded for a given increment in SNR. For the selection of color correction matrices, the SNR–color accuracy relationship is very important.

For this investigation, a multi-objective optimization function was used to determine color correction matrices (CCMs) that produce optimal SNR at different color errors levels. The SNR–color error curve obtained using these CCMs is pareto optimal. The objective function used in this optimization is shown in Eq. (11):

\min_{T} F (T) = [(Δ E (T); - SNR (T)] .

The objective functions

Δ E (T)

and

SNR (T)

are the color difference and SNR of the test image after it has been color corrected using the matrix

T

. Note that the SNR is negated here because a minimization operation is used. As stated earlier, a linear constraint was used to ensure white point preservation.

A sample of the color correction matrices determined for various color error levels is provided in Table 6. As the color error increases, the components of the CCMs decrease–this decreases the noise amplification of the matrices and increases the SNR. The new patterns have smaller terms in their correction matrices than the Bayer RGB and CMY patterns.

Table 6. Color Correction Matrices for Different Color Error Levels for High Cross Talk Conditions (D65 Illuminant)

View Table | View all tables in this article

Figure 5 shows the SNR–color error trade-off curves for the different CFAs tested. As expected, the SNR increases as we relax the color error requirement. For the same color error level, simulations with CIE A illuminant give a much higher SNR than simulations with the D65 illuminant.

Fig. 5. Color error—SNR trade-off curves for simulations using D65 illuminant (left) and CIE illuminant A (right) in high cross-talk conditions.

Download Full Size | PDF

The new patterns have better color performance compared to the Bayer RGB and CMY regardless of the illuminant used. This advantage is higher when CIE A illuminant is used. The CMY pattern shows the worst performance across the range of color error levels surveyed. As a higher level of color error is allowed, the SNR of the Bayer pattern becomes comparable to the new CFA patterns. For instance, at $Δ E_{00} = 5$ , the sRGBCY pattern has an SNR value about 5 dB higher than the Bayer pattern. However, at $Δ E_{00} = 10$ , this gap reduces to about 1 dB.

Therefore, for the range of color errors considered, it is evident that the new CFA patterns have a significant SNR advantage over the Bayer patterns in conditions of high cross talk.

5. CONCLUSION

We have presented a comparative study of the effect of cross talk on the color reproduction accuracy and SNR of images produced using various CFA patterns. In this work, color filter array patterns are proposed for mitigating the effects of cross talk. The CIEDE2000 metric was used to quantify the color error. The SNR metric was also used to compare different color filter array patterns. Evaluation of the filter array patterns was done for two different illuminants D65 and CIE A illuminants.

The analysis performed demonstrates higher sensitivity metamerism indices for the new CFA patterns in high cross talk conditions. These patterns show significantly less deterioration in their color performance when cross talk is added. This suggests that cross talk does not greatly alter the response of these CFA patterns compared to the Bayer patterns.

It should be reiterated that ideal Gaussian curves were used to model the spectral transmittance of the color filters used in this work. It is expected that the center wavelength of these filters can be optimized to attain the better performance. It is also expected that increasing the width of the spectral transmittances of these filters will increase the overlap in the spectral response of the pixels and worsen the SNR performance. In conditions of low cross talk, there isn’t a significant advantage in the performance of the new color filter array patterns.

Finally, from the analysis of the SNR–color accuracy relationship, it is evident that when the CCMs are optimized for the same color accuracy, the Bayer RGB and CMY have inferior SNR performance compared to the new patterns. The Bayer RGB attains SNR levels comparable to the new patterns only at high color error levels.

ACKNOWLEDGMENTS

The authors gratefully acknowledge the support of Rambus Inc. for this work.

REFERENCES

1. J. C. Ahn, C.-R. Moon, B. Kim, K. Lee, Y. Kim, M. Lim, W. Lee, H. Park, K. Moon, J. Yoo, Y. J. Lee, B. Park, S. Jung, J. Lee, T.-H. Lee, Y. K. Lee, J. Jung, J.-H. Kim, T.-C. Kim, H. Cho, D. Lee, and Y. Lee, “Advanced image sensor technology for pixel scaling down toward 1.0 μm (Invited),” in IEDM Digest of Technical Papers (IEEE, 2008), pp. 1–4.

2. M. Furumiya, H. Ohkubo, Y. Muramatsu, S. Kurosawa, and Y. Nakashiba, “High sensitivity and no-cross-talk pixel technology for embedded CMOS image sensor,” IEEE Trans. Electron Devices 48, 2221–2227 (2001). [CrossRef]

3. S.-Y. Chen, C.-C. Chuang, J.-C. Liu, and D.-N. Yaung, “Image sensor with deep trench isolation structure,” U.S. patent 20120025199 (February 2, 2012).

4. Samsung Tomorrow, “Samsung launches ISOCELL: Innovative image sensor technology for premium mobile devices,” http://global.samsungtomorrow.com/?p=28442

5. J. Kim and H. Tanaka, “Color filter array with reduced crosstalk effect and image sensor and image pickup apparatus having the same,” U.S. patent 8,054,352 (November 8, 2011).

6. Y. Qian, H.-C. Tai, D. Mao, V. Venezia, and H. E. Rhodes, “Image sensors having dark sidewalls between color filters to reduce optical crosstalk,” U.S. patent 20120019695 (January 26, 2012).

7. R. Lukac and K. N. Plataniotis, “Color filter arrays: design and performance analysis,” IEEE Trans. Broadcast Telev. Receivers 51, 1260–1267 (2005).

8. E. R. Fossum, “The quanta image sensor (QIS): concepts and challenges,” in Imaging and Applied Optics, OSA Technical Digest (CD) (Optical Society of America, 2011), paper JTuE1.

9. B. E. Bayer, “Color imaging array,” U.S. patent 3971065, Issued July 20, 1976.

10. J. Alakarhu, “Image sensors and image quality in mobile phones,” in Proceedings of International Image Sensor Workshop (2007).

11. F. Li, H. Eliasson, and A. Dokoutchaev, “Comparison of objective metrics for image sensor crosstalk characterization,” Proc. SPIE 7876, 78760L (2011).

12. G. D. Finlayson and M. S. Drew, “White-point preserving color correction,” in Color and Imaging Conference (Society for Imaging Science and Technology, 1997), pp. 258–261.

13. ISO, “Photography—Digital still cameras—Determination of exposure index, ISO speed ratings, standard output sensitivity, and recommended exposure index,” ISO 12232:2006 (International Organization for Standardization, Geneva, Switzerland, 2006).

14. P.-C. Hung, “Sensitivity metamerism index for digital still camera,” Proc. SPIE 4922, 1–14 (2002).

15. M. R. Luo, G. Cui, and B. Rigg, “The development of the CIE 2000 colour-difference formula: CIEDE2000,” Color Res. Appl. 26, 340–350 (2001). [CrossRef]

16. S. Koskinen, E. Tuulos, and J. Alakarhu, “Color channel weights in a noise evaluation,” in International Image Sensor Workshop (2011), pp. 114–117.

Parameter	Value
Illuminants	D65, CIE A
Pixel parameter constant, $k$	$0.27 {μm}^{2} s$
Red filter center/half-width	600/50 nm
Green filter center/half-width	555/66 nm
Blue filter center/half-width	450/33 nm
Red pixel cross talk	45%
Green pixel cross talk	30%
Blue pixel cross talk	20%

	D65		CIE A
CFA Pattern	No Cross Talk	With Cross Talk	No Cross Talk	With Cross Talk
RGB	[1.8 1.0 1.3]	[1.5 1.0 1.0]	[1.2 1.0 3.5]	[1.2 1.0 1.7]
CMY	[1.8 1.0 1.3]	[2.0 1.1 1.0]	[1.2 1.0 3.5]	[1.3 1.0 1.1]
RGBCWY	[1.8 1.0 1.3]	[2.1 1.0 1.2]	[1.2 1.0 3.5]	[1.2 1.0 4.5]
RGBCY	[1.8 1.0 1.3]	[2.5 1.0 1.1]	[1.2 1.0 3.5]	[1.6 1.0 2.1]
sRGBCY	[1.8 1.0 1.3]	[1.5 1.0 1.0]	[1.2 1.0 3.5]	[1.0 1.0 2.7]

	No Cross Talk		With Cross Talk
CFA Pattern	Mean $Δ E_{00}$	Mean SMI	Mean $Δ E_{00}$	Mean SMI
Bayer RGB	3.0	86	4.9	77
CMY	3.7	83	4.4	80
RGBCWY	4.1	81	4.2	81
RGBCY	3.6	83	3.8	83
sRGBCY	3.6	83	3.8	82

	No Cross Talk		With Cross Talk
CFA Pattern	Mean $Δ E_{00}$	Mean SMI	Mean $Δ E_{00}$	Mean SMI
Bayer RGB	2.7	87	4.3	80
CMY	3.6	83	5.4	75
RGBCWY	3.3	85	3.3	85
RGBCY	3.1	86	3.0	86
sRGBCY	3.3	85	3.2	85

	SNR (dB)–D65		SNR (dB)–CIE A
CFA Pattern	No Cross Talk	With Cross Talk	No Cross Talk	With Cross Talk
RGB	24.4	20.2	25.3	21.4
CMY	21.9	20.7	24.0	22.2
RGBCWY	23.8	22.2	24.7	24.3
RGBCY	24.1	23.1	24.9	25.0
sRGBCY	23.4	22.6	24.4	24.5

Color filter array patterns for small-pixel image sensors with substantial cross talk

Abstract

1. INTRODUCTION

2. DESCRIPTION OF PROPOSED COLOR FILTER ARRAY PATTERNS

3. EXPERIMENTAL PROCEDURE AND MODEL DESCRIPTION

4. EVALUATING CFA PERFORMANCE IN THE PRESENCE OF CROSS TALK

A. Color Reproduction Accuracy

B. SNR Performance

C. Trade-Off between YSNR and Color Reproduction

5. CONCLUSION

ACKNOWLEDGMENTS

REFERENCES

Cited By

Figures (5)

Tables (6)

Equations (28)

Journal of the Optical Society of America A

	Color Error
CFA Pattern	$E_{00} = 5$	$E_{00} = 7$	$E_{00} = 9$
Bayer RGB	$[\begin{matrix} 2.92 & 0.44 & 0.54 \\ - 2.70 & 0.82 & - 1.22 \\ 0.74 & - 0.27 & 1.78 \end{matrix}]$	$[\begin{matrix} 1.95 & 0.61 & 0.75 \\ - 1.43 & 0.48 & - 1.21 \\ 0.43 & - 0.09 & 1.55 \end{matrix}]$	$[\begin{matrix} 1.48 & 0.68 & 0.90 \\ - 0.81 & 0.34 & - 1.17 \\ 0.28 & - 0.01 & 1.36 \end{matrix}]$
CMY	$[\begin{matrix} 1.11 & 0.37 & - 0.63 \\ - 0.90 & 1.14 & - 1.74 \\ 0.74 & - 0.51 & 3.47 \end{matrix}]$	$[\begin{matrix} 0.82 & 0.41 & - 0.36 \\ - 0.15 & 0.94 & - 1.22 \\ 0.28 & - 0.35 & 2.62 \end{matrix}]$	$[\begin{matrix} 0.74 & 0.44 & - 0.14 \\ 0.05 & 0.79 & - 0.69 \\ 0.17 & - 0.22 & 1.92 \end{matrix}]$
RGBCWY	$[\begin{matrix} 0.79 & 0.30 & 0.32 \\ - 0.21 & 0.70 & - 0.22 \\ 0.37 & 0.00 & 0.99 \end{matrix}]$	$[\begin{matrix} 0.61 & 0.31 & 0.42 \\ - 0.02 & 0.58 & - 0.40 \\ 0.37 & 0.11 & 1.08 \end{matrix}]$	$[\begin{matrix} 0.49 & 0.31 & 0.45 \\ 0.10 & 0.50 & - 0.46 \\ 0.36 & 0.19 & 1.10 \end{matrix}]$
RGBCY	$[\begin{matrix} 0.97 & 0.24 & 0.16 \\ - 0.10 & 1.02 & 0.00 \\ 0.09 & - 0.27 & 0.92 \end{matrix}]$	$[\begin{matrix} 0.90 & 0.44 & 0.20 \\ 0.01 & 0.76 & 0.00 \\ 0.04 & - 0.19 & 0.82 \end{matrix}]$	$[\begin{matrix} 0.84 & 0.56 & 0.42 \\ 0.12 & 0.59 & - 0.03 \\ 0.00 & - 0.15 & 0.70 \end{matrix}]$
sRGBCY	$[\begin{matrix} 1.25 & 0.34 & 0.36 \\ - 0.72 & 0.67 & - 0.67 \\ 0.43 & - 0.01 & 1.39 \end{matrix}]$	$[\begin{matrix} 0.96 & 0.32 & 0.42 \\ - 0.45 & 0.57 & - 0.71 \\ 0.44 & 0.11 & 1.38 \end{matrix}]$	$[\begin{matrix} 0.72 & 0.30 & 0.46 \\ - 0.22 & 0.49 & - 0.75 \\ 0.45 & 0.21 & 1.38 \end{matrix}]$