Open Access
Add to BibSonomyAbstract
We describe a color image reconstruction method that enables both direct visualization and direct digital image acquisition from one oral tissue by using various light sources and color compensating filters. In this method, the image of the oral tissue with white light emitting diodes (LEDs) with blue color compensating filter has a larger color difference between the normal and inflamed tissues. The enhanced visualization comes from the white light color mixing between the red normal tissue and bluish white light from the LEDs. With our method, we evaluate the perceived tissue reflectance in each pixel of the image and color reproduction with different illuminated spectra. Our approach to enhancement of visually perceived color difference between normal and inflamed oral tissue involves optimization of illumination and observation conditions by allowing a significant optical contrast of illuminated spectrum to reach the observer’s eyes. In comparison with a conventional daylight LED flashlight, a LED with blue filter as the illuminant for oral cavity detection enhances the color difference between normal and inflamed tissues by 32%.
©2010 Optical Society of America
1. Introduction
Inspection of the oral cavity, which contains mucous membrane, the tongue, and parts of larynx, is a fundamental diagnostic tool for doctors to investigate diseases such as viral infection, enterovirus, herpetic gingivostomatitis, and oral cancer. The early detection of oral cancer has been a very popular subject of study for around two decades. However, the oral cavity diagnosis of viral infection or enterovirus is also common and essential for everyone, especially children. Enterovirus is the most common cause of aseptic meningitis, which can cause serious disease in infants and the immunocompromised. Human enteroviruses (family Picornaviridae) infect millions of people worldwide each year, resulting in a wide range of clinical outcomes, from apparently symptom-free infection to mild respiratory illness (common cold), hand-foot-and-mouth disease, acute hemorrhagic conjunctivitis, aseptic meningitis, myocarditis, severe neonatal sepsis-like disease, and acute flaccid paralysis [1–3]. Mouth disease caused by the enterovirus herpangina is a common early symptom of enterovirus, and is a simple diagnostic tool for doctors [4,5]. The illumination design for oral cavity detection is a key issue in the visualization of the oral cavity. Early identification of the enterovirus could greatly reduce both serious disease and mortality due to mistaking the enterovirus for a less serious viral infection or cold.
Light emitting diodes (LEDs) are new illumination sources for the detection of disease in oral cavity. They can provide higher color purity, lighting efficiency and more colors than traditional tungsten filament light. Rahman et al. developed a low-cost, multimodal, and portable screening system for early detection of oral cancer by using blue, white, and polarized LEDs to form the fluorescence image, standard reflectance image, and low scattering image, respectively. It is an attractive method to screen for oral cancer in low-resource environments where clinical expertise is often unavailable [6]. Pierre et al. demonstrated a simple handheld device to excite green-red fluorescence from fluorophores in the oral tissues by using blue excitation light. Fluorescence detected at the tissue surface is a function of tissue morphology and biochemistry [7]. Rhonda et al. reported a multi-spectral imaging approache for significant contrast enhancement between normal and pre-cancerous tissues in the oral cavity. The contrast of reflectance is best perceived when filtered near 557 nm with a 60 nm bandpass filter [8]. The neoplastic tissue in the oral cavity has shown enhanced absorption in the green light range and more reflection in the red light range by using multispectral and hyperspectral imaging methods [9]. Imaging tissue pathology with polarized light emphasizes image contrast on the basis of light scattering in the surface layers of the tissue [10]. As indicated above, multi-image analysis methods show more information from oral tissue for early identification of oral cancer particularly. Those techniques have some problems such as, complicated image acquirement, long analysis time, and non-real-time diagnosis, especially for general diagnosis or in the emergency room. Another problem is that more image acquirement means more pictures focused at the same oral cavity. Constant conditions such as angle of picture, location of camera or light source, brightness of environment cannot be maintained from image to image. This will cause artificial differences in the images, making comparison between normal and inflamed tissues more difficult and less precise.
This paper reports on the production of a high color difference between normal and inflamed oral tissue based on the implementation of multi-band image reproduction [11,12]. Multi-band image reproduction is taken as the main foundation to achieve a system of high-accuracy color gauging and color distinction of oral tissues. A readily available digital camera (Canon, 860IS) is used to record the color images and reconstruct the reflection spectrum of the target by using principal component analysis, multi-dimensional linear regression algorithms, and a high-accuracy spectrophotometer (Konica Minolta, CS-1000). This system is used also to replace the light source spectra, with the color images calculated to achieve the same result as the actual images. By applying this technology, external conditions can be held constant, producing the color images based on different light sources. Hence this paper mainly discusses the images of oral inflammation in the mouth, and makes the range of color difference of inflamed tissue more apparent, so doctors can determine the cause quickly and precisely.
This paper is organized as follows: In Section 2, the estimation processes of spectral reflectance curves from multispectral image data and the implementation of the color image recording and reflection spectra of color checker experiment are presented. The reflection spectra reproduction of the color-checker and the color image reconstructions based on different light sources are reported in Section 3. Here, discussions about the significant color differences between normal and inflamed tissues are also given. Finally, the conclusions are drawn in Section 4.
2. Estimation processes of spectral reflectance curves from multispectral image data
In color science, the CIE (the Commission Internationale de l’Éclairage) XYZ color space defines all the colors in terms of three imaginary primaries based on the human visual system. X, Y and Z denote tristimulus values of a color stimulus (S(λ)) and are expressed as
where, , , and are the color matching functions, S(λ) represents the spectral radiance at a certain wavelength λ which is the product of the spectral power distribution of the illuminant I(λ) and the spectral reflectance function of the object R(λ), and k is a constant [13]. The unit of the spectral power distribution is measured in power units, watts. The normalization constant k in equation is defined differently for relative and absolute colorimetry. In absolute colorimetry, k is set equal to 683 lumen/W, and making the system of colorimetry compatible with the system of photometry. For relative colorimetry, k is defined by equationThe normalization for relative colorimetry in Eq. (2) results in tristimulus values that are scaled from zero to approximately 100 for various materials. In this study, we use relative colorimetry during estimation processes. The spectral radiometric quantity for each pixel of one color image is simulated, then the new light source spectrum in S(λ) is replaced to reproduce new color images with different light sources. Color images of the same oral tissue based on different color light sources in the same conditions are demonstrated.Figure 1 shows the estimation processes of the color image reproduction with multispectral image data. This flowchart contains two parts: one is the multispectral image acquisition of the color-checker by the digital camera, and the other is the reflection spectra measurement by the spectrophotometer. The multispectral image is captured by the digital camera incorporating a rotating wheel comprising six Kodak compensating color filters in front of the camera lens. Each color filter has the higher correlation value with the same color filter group. The new digital counts of the color-checker’s 24 patches are given after correcting the camera with the spectrophotometer. The spectral reflectance functions of the 24 patches on the color-checker are measured by spectrophotometer. In the set up for measurement, two 4Bank system artificial solar illuminant lamps (Kino Flo, Mega 4Bank) are used as the illuminant [14]. The 4Bank system consists of ballast and a lamp that has four tubes. The ballast keeps the lamp flicker-free at any speed or angle of the camera shutter by controlling the high output power of the tubes. The correlated color temperature of this illuminant is about 6500K. For uniform measurement of the spectral reflectance function of each color patch, an x-y axis precise-translation-stage table is used to move the color-checker. The power distribution of the illuminant on each color patch of the color-checker is the same. In the measurement system, the illuminant is warmed up about 30 minutes before starting to measure the spectrum; the layout of the setup is based on the 45°/0° geometry [15]. The illumination is projected at an incident angle of 45°. The spectrophotometer is placed at the 0° angle and is facing the center of the object.
Fig. 1 Schematic diagram of this method which is used for estimating the spectral reflectance of each pixel of an image by using a digital camera and a set of color compensating filters.
These spectral reflectance functions can be represented by the linear combination of several major principal components after the principal component analysis method. The coefficients of the linear combination will be called the PC (principle component) coefficients in this study. The relation between the PC coefficients and the digital counts of the digital camera is linear, and the measured digital counts and the corresponding PC coefficients are analyzed to obtain the transformation matrix between the digital camera and spectrophotometer. After obtaining the transformation matrix, the spectral reflectance functions of any given object surface can be reconstructed with a multi-spectral image acquisition system. First, six images of the object are captured by a 6-filter digital camera. According to the digital counts and the obtained transformation matrix, the PC coefficients of the object’s spectral reflectance function can be calculated. With the obtained PC coefficients, the spectral reflectance can be reconstructed by calculating the linear combination of the principal components. One can see that the reconstructed spectral reflectance functions almost completely match the reflection spectrum of the color-checker for red, green, blue, and yellow, which is represented by the blue curves in Fig. 2 (a)-(d) . For color reproduction, the color difference between the original reflectance function and the approximated reflectance function of the color-checker under artificial solar illuminant are estimated. In color vision, the CIE defines a uniform color space as the LAB color space. The conversion between LAB and XYZ can be expressed as
In these equations, the tristimulus values Xn, Yn, and Zn are the XYZ coordinates of the reference white. The definition of reference white is when the X, Y, and Z values are on condition that the CIE standard illuminant projects on the perfect diffusion objects. In this study, we use CIE D65 as standard illuminant. The color difference in the CIELAB space is used to qualify the performance of the spectrum reconstructed results. The color difference, is defined asFig. 2 Original spectra (pink curves) and reconstructed spectra (blue curves) of color checkers for (a) red, (b) green, (c) blue, and (d) yellow using the estimation processes of spectral reflectance curves from multispectral image data.
Figure 3 shows the color difference between the original reflectance functions and the reconstructed reflectance functions of the 24 color patches.
Fig. 3 Color difference between the original reflectance functions and reconstructed reflectance functions of the 24 color patches under an artificial solar illuminant lamp.
3. Color image reproduction based on different light sources
This paper demonstrates the color image reproduction with multispectral image data. This paper’s hypothesis is that high-recognition color images comparing normal and inflamed oral cavity tissue should exist at some particular-colored light sources. In this work, color images of the same oral cavity with different-colored light sources, but all other external conditions constant, are simulated. Figure 4 (a), and (b) , show the color images from the digital camera under two light sources, fluorescent lamp and tungsten lamp, respectively. The chromaticity coordinates and the correlated color temperature of the fluorescent lamp is (0.324, 0.330) and 5892, respectively. The calculation equation of correlated color temperature is referred to McCamy’s proposition [16]. Color images under tungsten lamps are more reddish than under fluorescent lamps due to the correlated color temperature of a tungsten lamp being about 3000 K lower than that of a fluorescent lamp. Figure 4(c), the simulated image of the spectrophotometer, shows the simulated color image of Fig. 4(a), but with a tungsten lamp replacing the fluorescent one. Because the image in Fig. 4(c) is simulated using this method with computer, the spectrum of each pixel in Fig. 4 (c) can be determined. Figures 4(b) and 4(c) have the same color performance because they are both under tungsten lamps. This result represents a successful color image reproduction by replacing the colored light source.
Fig. 4 Color images from camera under fluorescent lamps (a) and tungsten lamps (b). The color image reproduction (c) is under replacing the light source of (a) with tungsten lamps.
In the production of the color image of the oral cavity, an LED flashlight with four color compensating filters is used as the illuminant, as well as one tungsten flashlight as the control illuminant. The output spectra of light sources are shown in Fig. 5 . The LED flashlight is a conventional white-light-output model based on blue-light-excited yellow phosphor. In Fig. 6(b) , which was a cropped photograph from white frame of the chromaticity diagram of the CIE 1931 (Fig.6(a)), the chromaticity coordinates of five light sources are (0.197, 0.166), (0.211, 0.177), (0.204, 0.220), (0.180, 0.123), and (0.219, 0.237) for the LED flashlight alone and with R, G, B, and Y color compensating filters, respectively. For demonstrating the color reproduction of oral cavity based on various color lighting, using LEDs with color compensating filters as illuminant is simpler though LED-based lighting technology allows tailoring illumination spectra by mixing emission from colored LEDs with any ratio of fluxes controlled by driving currents.
Fig. 5 Output spectra of colored light sources: an LED flashlight alone and combined successively with four different colored filters, and a tungsten flashlight.
Fig. 6 (a) CIE 1931 chromaticity diagram; (b) Chromaticities of five light sources, in which color points move away in the direction determined by which color compensating filter is on the LED flashlight. NTSC is color range from National Television System Committee.
Figure 7 shows the color images of an oral cavity with one dysplasia (left) with LED illuminant. The enlargements of the dysplasia under different illuminants are shown in (A) LED flashlight alone, (B) LED with red color compensating filter, (C) LED with green color compensating filter, (D) LED with blue color compensating filter, (E) LED with yellow color compensating filter, and (F) tungsten flashlight. In the left-hand image, the pathological changes in the oral cavity are clear. Each enlargement, (A) to (F), can be divided into three parts: normal tissue, inflamed tissue surrounding the dysplasia, and dysplasia tissue. This study focuses on how to increase the color contrast between normal tissue and inflamed tissue under a special illuminant. Increased color contrast will increase the recognizability of the pathological changes in the oral cavity, aiding clinical judgment. In Fig. 7, the image (B) shows the lowest color contrast between normal and inflamed tissue due to the low contrast between the reddish light source and the oral cavity. Image (F) shows a little color saturation under the tungsten flashlight due to the tungsten lamp’s reddish color output and lower correlated color temperature. From images (B) and (F), it can be surmised that in the oral cavity or any blood-filled tissue, an illuminant with a reddish light source or lower correlated color temperature cannot be used. In the quantification of the color differences in the images in Fig. 7, the average chromaticity of normal tissue, inflamed tissue, and dysplasia is calculated. The calculation area of normal tissue is the upper part of the image that has a lighter color with an inverted U shape. The calculation area of inflamed tissue is around the dysplasia, with the red ring.
Fig. 7 Color images of an oral cavity with one dysplasia (left) with LED illuminant. The enlargements of the dysplasia under different illuminants: (A) LED flashlight, (B) LED with red color compensating filter, (C) LED with green color compensating filter, (D) LED with blue color compensating filter, (E) LED with yellow color compensating filter, and (F) tungsten flashlight.
The calculation area of dysplasia tissue is the center part of the image with whiter color. The results are shown in Table 1 . In the CIELAB color space, a value of color difference (ΔE*Lab) smaller than three represented the tolerance for color difference, i.e., cannot be distinguished by eye, a value between three and six represented the acceptable color difference of vision, i.e., could be distinguished by eye by some people [17]. In this study, the values of color difference are larger than 25, meaning the color differences can be easily to identify. In Table 1, we find the LED with blue color compensating filter as illuminant for oral tissue has the largest recognizability between normal and inflamed tissue. Compared with an unfiltered LED light source, the enhancement of the color difference is about 32% for the LED with blue color compensating filter as illuminant. When the illuminant is the LED with green or yellow filter, the color difference between normal and inflamed tissue increases only slightly due to most of the green/yellow light can be absorbed by hemoglobin [9]. For improving color discrimination by saturating colors is known to be attainable by using narrow-band RGB sources or incandescent lamps with neodymium filter, which removes yellow light, rather than by using yellow-blue sources such as the phosphor-conversion daylight LED or other sources containing a yellow component [18]. In this study, compared with the phosphor-conversion daylight LED, the LED with green or yellow filter as illuminant shows that a little color saturation change will cause the slightly color difference in oral cavity. When the illuminant is the LED with blue filter, the light source provides the blue emission color that can mix with reddish tissues to be fairly close to a white signal to the observer. In such a condition, the detector can collect more light signals that come from the reflection from oral tissue, and the increased number of light signals provides more information about the tissue to enhance the recognizability of the oral cavity. The average output spectra of normal and inflamed oral tissue under different illuminants are shown in Fig. 8 . In color science, the physical mechanism of the bluish illumination condition can enhance recognizability of oral tissue based on two effects: one is white light generation by complementary wavelengths, i.e., complementary colors [19,20]; the other is the Helson-Judd effect of the color appearance model [21]. The main peak of the reflection spectrum in oral tissue is about 630 nm corresponding to 486 nm complementary wavelength for white light generation by dichromatic sources. In this case, the LED with blue filter as illuminant is close to the complementary color for white light generation in oral cavity illumination. The Helson-Judd effect suggests that nonselective samples, viewed under highly chromatic illumination, take on the hue of the light source if they are lighter than the background and take on the complementary hue if they are darker than the background. In an oral cavity, the inflamed tissue is darker than the normal tissue, meaning the background (normal tissue) will appear close to white in the image. Combining these two effects, the color contrast between inflamed and normal tissue will be enhanced. In addition, the bluish light source avoided color saturation and absorption in the oral cavity illumination, in contrast to the saturation of red illuminant and the absorption of the green and yellow green color light sources, respectively.
Table 1. Calculation results of color difference between normal and inflamed tissue under various color light sources.
Fig. 8 Estimated spectral radiometric quantity of (a) normal and (b) inflamed oral tissue are under various illuminants using the multispectral image reproduction method.
4. Conclusions
In conclusion, a high color difference between normal and inflamed oral tissue based on the implementation of multi-band image reproduction has been achieved. Multi-band image reproduction was taken as the main foundation to achieve a system of high accuracy color gauging and the color distinction of oral tissues. The color image reproduction of the oral cavity tissue by replacing the various color illuminants under the same conditions as taking a picture is demonstrated. According to this study, the reddish illuminant produces the color saturation in the oral cavity image; the use of the green and yellow illuminants results in absorption of the light by hemoglobin, in turn resulting in less light reflection from oral tissue, and the bluish illuminant produces the largest color difference between normal and inflamed tissue, thus increasing the recognizability in the oral cavity for early inflamed tissue detection. This method can help the doctor to quickly detect pathological changes in the oral cavity and precisely predict viral infection, especially for enterovirus and herpetic gingivostomatitis. In the future, this method using particular special-colored illuminants to enhance color difference/contrast, might also be applied to semiconductors and to food and other agricultural products to check for defects while they are still on the production lines.
Acknowledgement
This research was supported by the National Science Council, The Republic of China, under grant NSC 97-2221-E-194-008.
References and links
1. M. S. Oberste, K. Maher, D. R. Kilpatrick, and M. A. Pallansch, “Molecular evolution of the human enteroviruses: correlation of serotype with VP1 sequence and application to picornavirus classification,” J. Virol. 73(3), 1941–1948 (1999). [PubMed]
2. J. R. Wang, Y. C. Tuan, H. P. Tsai, J. J. Yan, C. C. Liu, and I. J. Su, “Change of major genotype of enterovirus 71 in outbreaks of hand-foot-and-mouth disease in Taiwan between 1998 and 2000,” J. Clin. Microbiol. 40(1), 10–15 (2002). [CrossRef] [PubMed]
3. Z. H. Li, C. M. Li, P. Ling, F. H. Shen, S. H. Chen, C. C. Liu, C. K. Yu, and S. H. Chen, “Ribavirin reduces mortality in enterovirus 71-infected mice by decreasing viral replication,” J. Infect. Dis. 197(6), 854–857 (2008). [CrossRef] [PubMed]
4. M. Ho, E. R. Chen, K. H. Hsu, S. J. Twu, K. T. Chen, S. F. Tsai, J. R. Wang, S. R. Shih, and Taiwan Enterovirus Epidemic Working Group, “An epidemic of enterovirus 71 infection in Taiwan,” N. Engl. J. Med. 341(13), 929–935 (1999). [CrossRef] [PubMed]
5. K. T. Chen, H. L. Chang, S. T. Wang, Y. T. Cheng, and J. Y. Yang, “Epidemiologic features of hand-foot-mouth disease and herpangina caused by enterovirus 71 in Taiwan, 1998-2005,” Pediatrics 120(2), e244–e252 (2007). [CrossRef] [PubMed]
6. M. Rahman, P. Chaturvedi, A. M. Gillenwater, and R. Richards-Kortum, “Low-cost, multimodal, portable screening system for early detection of oral cancer,” J. Biomed. Opt. 13(3), 030502 (2008). [CrossRef] [PubMed]
7. P. M. Lane, T. Gilhuly, P. Whitehead, H. Zeng, C. F. Poh, S. Ng, P. M. Williams, L. Zhang, M. P. Rosin, and C. E. MacAulay, “Simple device for the direct visualization of oral-cavity tissue fluorescence,” J. Biomed. Opt. 11(2), 024006 (2006). [CrossRef] [PubMed]
8. E. Svistun, U. Utzinger, R. Jacob, R. K. Rebecca, A. El-Naggar, and A. Gillenwater, “Optimal visual perception and detection of oral cavity neoplasia reflectance and fluorescence,” Biomedical Topical Meeting, TuA3 (2002).
9. D. Roblyer, C. Kurachi, A. El-Naggar, M. D. Williams, A. M. Gillenwater and R. Richards-Kortum, “Multispectral and hyperspectral in vivo imaging of the oral cavity for neoplastic tissue detection,” Biomedical Optics, BTuD1 (2008).
10. S. L. Jacques, J. C. Ramella-Roman, and K. Lee, “Imaging skin pathology with polarized light,” J. Biomed. Opt. 7(3), 329–340 (2002). [CrossRef] [PubMed]
11. Y. Murakami, T. Obi, M. Yamaguchi, N. Ohyama, and Y. Komiya, “Spectral reflectance estimation from multi-band image using color chart,” Opt. Commun. 188(1-4), 47–54 (2001). [CrossRef]
12. T. N. Kanagawa, “Method and apparatus for outputting multi-band image,” Patent No. US 6885394 B1 (2005).
13. G. Wyszecki, and W. Stiles, Color Science -- Concepts and Methods, Quantitative Data and Formulate (2nd Edition, Wiley, New York, 1982).
15. R. S. Berns, Principles of color technology (Wiley, 83-92, 2000).
16. C. S. McCamy, “Correlated color temperature as an explicit function of chromaticity coordinates,” Color Res. Appl. 17(2), 142–144 (1992). [CrossRef]
17. P. Green, and L. MacDonald, Colour Engineering: Achieving Device Independent Colour (Wiley, New York, 2002).
18. A. Žukauskas, R. Vaicekauskas, and M. S. Shur, “Solid-state lamps with optimized color saturation ability,” Opt. Express 18(3), 2287–2295 (2010). [CrossRef] [PubMed]
19. E. F. Schubert, Light-Emitting Diodes (2nd Edition, Cambridge, New York, 2006), p. 333-338.
20. A. Žukauskas, M. S. Shur, and R. G aska, Introduction to Solid-State Lighting, (Wiley, New York, 2002), p. 119-121
21. M. D. Fairchild, Color appearance models, page 122-125 (John Wiley & Sons, 2006).
