Simple and affordable imaging of multiple physiological parameters with RGB camera-based diffuse reflectance spectroscopy

Izumi Nishidate; Masashi Minakawa; Daniel McDuff; MD. Abdul Wares; MD. Abdul Wares; Kazuya Nakano; Hideaki Haneishi; Yoshihisa Aizu; Kyuichi Niizeki

doi:10.1364/BOE.382270

1. Introduction

Quantitative evaluation of biological chromophores is useful for detecting various skin diseases including cancers, monitoring health status and tissue metabolism, and assessing clinical and physiological vascular functions. The major chromophores in the superficial skin layer are melanin, oxygenated hemoglobin, and deoxygenated hemoglobin, which show distinctive optical absorption properties in the visible wavelength range [1]. When the concentration of each chromophore varies, the corresponding change may be observed on diffusely reflected light from the skin tissue in this wavelength range. Peripheral skin tissues and cells demand a continuous supply of oxygen, which are delivered via the blood circulation. The absorption spectra of oxygenated hemoglobin and deoxygenated hemoglobin differ since the binding of oxygen to hemoglobin changes the light absorption spectrum of hemoglobin [2]. Delivery of oxygen to peripheral skin tissues can be evaluated from the diffuse reflectance spectrum based on the absorption spectra of oxygenated hemoglobin and deoxygenated hemoglobin. The percentage of oxygenated hemoglobin in a volume of tissue is called tissue oxygen saturation (StO₂) or hemoglobin oxygen saturation [3]. It is a useful indicator for monitoring peripheral tissue oxygen consumption, hypoperfusion, cyanosis, and tissue viability. Diffuse reflectance and transmittance spectra of peripheral tissue are fluctuated by the periodic temporal variation in blood volume due to the cardiac pulse traveling through the body. Photo-plethysmograph (PPG) has been widely used to assess systemic physiological parameters such as heart rate, blood pressure, cardiac output, vascular compliance, and percutaneous oxygen saturation (SpO₂) [4,5].

In addition to hemoglobin, bilirubin is an important chromophore for evaluating health conditions such as neonatal jaundice, liver cirrhosis, and hepatitis. Bilirubin is a hemoglobin breakdown product and has a broad spectrum of light absorption in the wavelength range from 400 nm to 500 nm [6]. A condition in which the serum bilirubin concentration exceeds the holding capacity of the serum albumin is known as hyperbilirubinemia. This condition is responsible for the yellowish skin color in jaundice. Hyperbilirubinemia reflects a normal transitional phenomenon called physiological jaundice in most infants. In some infants, however, significant hyperbilirubinemia may cause bilirubin to accumulate in the brain tissue, potentially causing irreversible brain damage called kernicterus [7]. It is therefore recommended to carefully measure serum bilirubin concentration or transcutaneous bilirubin concentration (C_bil) [8,9] levels in neonate jaundice, especially in the first 24 hours.

Diffuse reflectance spectra with a continuous-wave light can be easily acquired using a white light source, affordable optical components, and a spectrometer. Analysis of diffuse reflectance spectra provide useful information on tissue activities and functions that are related to biological chromophores. Thus, diffuse reflectance spectroscopy (DRS) has been widely used for the evaluating skin chromophores [10–20]. Multi-spectral imaging based on DRS has been employed for visualizing the spatial distribution of chromophore contents in living tissue using a series of discrete optical interference filters, a liquid crystal wavelength tunable filter [21–23], and a hyperspectral imaging system [24–27]. A simple method for quantitative imaging of melanin and hemoglobin concentrations in in vivo skin tissue based on diffuse reflectance images at six wavelengths (500, 520, 540, 560, 580, and 600 nm) using multiple regression analysis aided by Monte Carlo simulations has been previously proposed [28].

For practical uses, especially in low-resource settings, more simple, cost-effective, and portable equipment is needed. Imaging with a digital red-green-blue (RGB) camera is one of the promising tools to satisfy those demands. Several techniques with an RGB camera-based imaging have been used for the non-invasive characterization of biological tissues where contrast is obtained from the absorption of light by biological chromophores [29–31]. This can be achieved using a general purpose digital color camera and a white light source such as a white light emitting diode (LED), which enable cost-effective, easy-to-use, battery-powered, portable, remotely administrable, and/or point-of-care solutions. Therefore, an RGB camera-based diffuse reflectance imaging technique meets healthcare needs in resource-constrained environments. We have already developed an approach with a digital RGB camera to visualize the concentrations of oxygenated hemoglobin (C_HbO), deoxygenated hemoglobin (C_HbR), total hemoglobin (C_HbT), and melanin (C_m) for in vivo human skin [32]. In this article, this approach is modified and extended to simultaneous imaging of hemodynamics and jaundice. Therefore, we propose a method to visualize the spatial maps of C_HbO, C_HbR, C_HbT, StO₂, SpO₂, C_bil, and C_Mel using a digital RGB image. In this approach, the RGB-values were converted into tristimulus values in the CIE (Commission Internationale de l’Eclairage) XYZ color space, which is compatible with the common RGB working spaces. Monte Carlo simulation for light transport in biological tissue is then used to specify the relationship among the tristimulus XYZ-values and C_HbO, C_HbR, C_bil and C_m. The values of C_HbT and StO₂ are also calculated from the resultant values of C_HbO and C_HbR. An experimental model of obstructive jaundice caused by bile duct ligation in rats is introduced to evaluate hyperbilirubinemia quantitatively. Experiments with rat dorsal skin while changing the fraction of inspired oxygen (FiO₂) are performed to demonstrate the ability of the method to estimate C_HbT and StO₂ under the conditions of normoxia, hypoxia, and anoxia. We further extend the method to a non-contact PPG imaging and visualization of SpO₂. An empirical formula to estimate SpO₂ was derived from the pulse wave amplitudes (PWAs) of C_HbO and C_HbR extracted from the PPG signal.

2. Principle

2.1 Estimation of chromophore concentrations

The responses of RGB channels (R, G, and B) in each pixel of the skin tissue color image acquired by a digital RGB camera can be expressed as

(1)$$\left[ \begin{array}{l} R\\ G\\ B \end{array} \right] = {{\textbf L}_{\textbf 1}}\left[ {\begin{array}{c} X\\ Y\\ Z \end{array}} \right],$$

where L₁ is a transposition matrix to convert the tristimulus values in the Commission Internationale de l’ Éclairage XYZ (CIEXYZ) color system X, Y, and Z to corresponding responses of R, G, and B, and ${[{\cdot}{\cdot} \cdot ]^T}$ represents transposition of a vector. The tristimulus values X, Y, and Z in the above equation are defined as

(2)$$X = \kappa \sum {{\rm E}(\lambda )\bar{x}(\lambda )\Theta (\lambda )} ,$$

(3)$$Y = \kappa \sum {{\rm E}(\lambda )\bar{y}(\lambda )\Theta (\lambda )} ,$$

(4)$$Z = \kappa \sum {{\rm E}(\lambda )\bar{z}(\lambda )\Theta (\lambda )} ,$$

where λ, E(λ), and Θ(λ) are the wavelengths, the spectral distribution of the illuminant, and the diffuse reflectance spectrum of liver tissue, respectively, whereas $\bar{x}(\lambda )$, $\bar{y}(\lambda )$, and $\bar{z}(\lambda )$ are color matching functions in the CIEXYZ color system. The values of constant κ that result in Y being equal to 100 for the perfect diffuser is given by

(5)$$\kappa = \frac{{\textrm{100}}}{{\sum {{\rm E}(\lambda )\bar{y}(\lambda )} }}.$$

In Eqs. (2)–(5), the summation can be carried out using data at 10 nm intervals, from 400 to 700 nm. Assuming that the skin tissue mainly consists of epidermis containing the melanin and dermis containing oxygenated hemoglobin, deoxygenated hemoglobin, and bilirubin, the diffuse reflectance of skin tissueΘ can be expressed as

(6)$$\begin{aligned}\Theta &= \frac{I}{{{I_\textrm{0}}}} = \left[ {\int_0^\infty {{P_e}({\mu_{s,e}}^\prime ,{l_e})\exp ({ - {\mu_{a,m}}{l_e}} )d{l_e}} } \right] \\ & \quad \times \left[ {\int_0^\infty {{P_d}({\mu_{s.d}}^\prime ,{l_d})\exp [ - ({\mu_{a,\textrm{HbO}}}} + {\mu_{a,\textrm{HbR}}} + {\mu_{a,bil}}){l_d}]d{l_d}} \right],\end{aligned}$$

where I and I₀ are the detected and incident light intensities respectively, P(μ_s’, l) is the path length probability function that depends on the scattering properties as well as on the geometry of the measurements, μ_s’, μ_a, and l are the reduced scattering coefficient, absorption coefficient, and the photon path length, respectively. In addition, the subscripts m, HbO, HbR, bil, e, and d indicate melanin, oxygenated hemoglobin, deoxygenated hemoglobin, bilirubin, epidermis, and dermis, respectively. The absorption coefficient μ_a of each chromophore can be expressed as the product of its concentration C and the extinction coefficient ε, and can be defined as

(7)$${\mu _a} = C \times \varepsilon .$$

Therefore, the responses of I_R, I_G, and I_B can be expressed as a function of concentrations of melanin (C_m), oxygenated hemoglobin (C_HbO), deoxyhemoglobin (C_HbR), and bilirubin (C_bil).

Figure 1 shows a flowchart describing the estimation process using the proposed method. First, the responses of the RGB channels I_R, I_G, and I_B in each pixel of the image are transformed into XYZ-values by a matrix N₁ as

(8)$$\left[ {\begin{array}{c} X\\ Y\\ Z \end{array}} \right] = {{\textbf N}_{\textbf 1}}\left[ \begin{array}{l} R\\ G\\ B \end{array} \right].$$

We determined the matrix N₁ based on measurements of a standard color chart (Color Checker, X-Rite Incorporated, Michigan, USA) that has 24 color chips and is supplied with data giving the CIEXYZ values for each chip under specific illuminations and corresponding reflectance spectra. The values of X, Y, and Z are then transformed into C_m, C_HbO, C_HbR, and C_bil by matrix N₂. It is difficult to determine the matrix N₂ based on L₁ and Eqs. (2)–(6) because P(μ_s’, l) and l for each layer are usually unknown. We calculated 1,800 diffuse reflectance spectra Θ(λ) in a wavelength range from 400 to 700 nm at intervals of 10 nm by Monte Carlo simulation (MCS) for light transport [33] in skin tissue. The simulation model consisted of two layers representing the epidermis and dermis. In a single simulation of diffuse reflectance at each wavelength, 5,000,000 photons were randomly launched. The absorption coefficients of melanin μ_a_,m(λ), oxygenated hemoglobin μ_a_,HbO(λ), deoxygenated hemoglobin μ_a_{, HbR}(λ), and bilirubin μ_a_,bil(λ) were obtained from the values of ε_m(λ), ε_HbO(λ), ε_HbR(λ), and ε_bil(λ) in the literature [2,6,34]. The absorption coefficient of the epidermis depends on the volume concentration of melanin in the epidermis C_m. We used the absorption coefficient of a melanosome given in the literature [35] as the absorption coefficient of melanin μ_a_,m(λ). This corresponds to the absorption coefficient of the epidermis for the case in which C_m = 100%. We subsequently derived the absorption coefficients of the epidermis for 10 lower concentrations of C_m= 1 to 10% at intervals of 1%, by simply proportioning them to that of C_m= 100%, and the absorption coefficients were input for the epidermis. The hemoglobin concentration of blood, having a 44% hematocrit with 150 g∕L of hemoglobin, was converted to 100% volume concentration (100 vol.%) of hemoglobin. The sum of the absorption coefficients of oxygenated hemoglobin and deoxygenated hemoglobin, μ_a_{, HbO}(λ) and deoxyhemoglobin μ_a_{, HbR}(λ), represents the absorption coefficient of total hemoglobin μ_a_{, HbT}(λ); the values for C_HbT= 0.2, 0.4, 0.6, 0.8, and 1.0 vol.% were used as input for the dermis layer in the MCS. Tissue oxygen saturation StO₂ was determined by μ_a_{, HbO}(λ)/μ_a_{, HbT}(λ), and values of 0%, 20%, 40%, 60%, 80%, and 100% were used for the simulation. The absorption coefficient of bilirubin μ_a_,bil(λ) was derived as μ_a_,bil(λ) = 2.303(C_HbT/P_Mbil)ε_bil(λ), where C_bil g/L, ε_bil(λ) cm^-1M^-1, and P_Mbil g/mol are the bilirubin concentration in the whole blood, the extinction coefficient of bilirubin, and the gram molecular weight of bilirubin, respectively. The absorption coefficient of bilirubin was then scaled by a coefficient f_bil. The value of f_bil was kept at 0.2 [36]. The values of μ_a_,bil(λ) for C_bil= 0, 1, 5, 10, 15, and 20 mg/dl were used as input for the dermis layer in the MCS. The refractive index of the epidermis and dermis layers was assumed to be the same and fixed at 1.4 for all simulations. The thicknesses of the epidermis and dermis layer were set to 0.06 and 4.94 mm, respectively. The reduced scattering coefficient μ_s’(λ) calculated from the typical values for the scattering coefficient μ_s(λ) [17] and anisotropy factor g(λ) [17] were used for both the epidermis and dermis. The XYZ-values were then calculated based on the simulated Θ(λ). The above calculations were performed for the various combinations of C_m, C_HbO, C_HbR, and C_bil in order to obtain the data sets of chromophore concentrations and XYZ-values. Multiple regression analysis with 1,800 data sets established four regression equations for C_m, C_HbO, C_HbR, and C_bil:

(9)$${C_\textrm{m}} = {\alpha _0} + {\alpha _1}X + {\alpha _2}Y + {\alpha _3}Z,$$

(10)$${C_{\textrm{HbO}}} = {\beta _0} + {\beta _1}X + {\beta _2}Y + {\beta _3}Z,$$

(11)$${C_{\textrm{HbO}}} = {\gamma _0} + {\gamma _1}X + {\gamma _2}Y + {\gamma _3}Z,$$

(12)$${C_{\textrm{bil}}} = {\omega _0} + {\omega _1}X + {\omega _2}Y + {\omega _3}Z,$$

Fig. 1. Flowchart of the process for estimating the concentration of melanin C_m, bilirubin C_bil, oxygenated hemoglobin C_HbO, deoxygenated hemoglobin C_HbR, total hemoglobin C_HbT, and tissue oxygen saturation, StO₂ from the measured RGB-values.

Download Full Size | PDF

The regression coefficients α_i, β_i, γ_i, and ω_i (i = 0, 1, 2, 3) reflect the contributions of the XYZ-values to C_m, C_HbO, C_HbR, and C_bil, respectively, and were used as the elements of a 4×4 matrix N₂ as

(13)$${{\textbf N}_2} = \left[ {\begin{array}{cccc} {{\alpha_0}}&{{\alpha_1}}&{{\alpha_2}}&{{\alpha_3}}\\ {{\beta_0}}&{{\beta_1}}&{{\beta_2}}&{{\beta_3}}\\ {{\gamma_0}}&{{\gamma_1}}&{{\gamma_2}}&{{\gamma_3}}\\ {{\omega_0}}&{{\omega_1}}&{{\omega_2}}&{{\omega_3}} \end{array}} \right].$$

Transformation with N₂ from the tristimulus values to chromophore concentrations is thus expressed as

(14)$$\left[ {\begin{array}{c} {{C_\textrm{m}}}\\ {{C_{\textrm{HbO}}}}\\ {{C_{\textrm{HbR}}}}\\ {{C_{\textrm{bil}}}} \end{array}} \right] = {{\textbf N}_2}\left[ {\begin{array}{c} 1\\ X\\ Y\\ Z \end{array}} \right].$$

Once we determine the matrices N₁ and N₂, images of C_m, C_HbO, C_HbR, and C_bil are reconstructed without the MCS. The total hemoglobin concentration image is simply calculated as C_HbT= C_HbO + C_HbR and tissue oxygen saturation of hemoglobin as StO₂% = (C_HbO/C_HbT) × 100.

3. Materials and method

3.1 Imaging system

The instrument developed for the RGB imaging system is illustrated in Fig. 2. A white-light emitting diode (LED) (LA-HDF5010, Hayashi Watch Works Co., Ltd., Tokyo, Japan) illuminated the surface of the rat skin via a light guide and a ring-shaped illuminator with a polarizer. The light source covered a range from 400 nm to 740 nm. Diffusely reflected light is received by a 24-bit RGB charge-coupled device camera (DFK23U618, Imaging source LLC, Charlotte, NC, USA) via an analyzer and a camera lens to acquire RGB images of 640×480 pixels. An IR-cut filter in the camera rejects unnecessary longer wavelength light (>700 nm). The ring shaped polarizer and analyzer were placed in a crossed Nicols alignment in order to reduce specular reflection of light from the liver surface. A standard white diffuser with 99% reflectance (SRS-99-020, Labsphere Incorporated, New Hampshire, USA) was used to regulate the white balance of the camera, to correct instrument-to-instrument differences in output of the camera, and to correct the spatial nonuniformity of illumination. The RGB images were then used to estimate the images of C_m, C_bil, C_HbO, C_HbR, and C_HbT, and StO₂ according to the process described above.

Fig. 2. Schematic illustration of setup for RGB imaging of skin tissue. A white LED is used to illuminate the skin surface, which is imaged using an RGB CCD camera. The computer acquires raw RGB images.

Download Full Size | PDF

3.2 Animal experimental protocols

Male Wister rats (n = 21) weighing from 300 to 630 g were used for the animal experiments. All experimental procedures were conducted according to the protocols approved by the Animal Care Committee of Tokyo University of Agriculture and Technology (Approval numbers 22-28 and 31-25). Anesthesia of rats was performed with isoflurane and maintained at a depth such that the rat had no response to toe pinching. After induction of anesthesia, the dorsal region was shaved and a depilatory agent including thioglycolic acid was applied on the rat dorsal skin.

First, we performed measurements of RGB color images of in vivo rat dorsal skin with obstructive jaundice. Obstructive jaundice was induced in 6 rats after common bile duct ligation. In the laparotomy, the ligamentous attachments from the liver to the diaphragm and abdominal wall were dissected in order to mobilize the liver lobes. The bile duct was exposed and carefully separated from the portal vein and hepatic artery. A suture was placed around the bile duct and secured with two surgical knots. The abdominal wall was closed with separate running sutures. Concurrently with the diffuse reflectance measurements, the transcutaneous bilirubin level was measured by a transcutaneous bilirubinometer (JM-105 Konica Minolta, Inc, Tokyo, Japan). This bilirubinometer is a hand held device with a contact measuring probe. The measuring probe was placed in the same region as the field of view vertically against the rat dorsal skin immediately after the imaging at each time point. The first imaging was performed immediately after completing the bile duct occlusion surgery and the imaging time points were different with each rat. We put several marks on each rat’s dorsal skin out of the region of interest (ROI) to image the same ROI across multiple days.

Second, we carried out measurements of RGB color images of in vivo dorsal skin with 15 rats while varying the fraction of inspired oxygen (FiO₂). The value of FiO₂ was regulated by mixing 95%O₂–5%CO₂ gas and 95%N₂–5%CO₂ gas in an arbitrary ratio and monitored by an oxygen gas sensor (OM-25MS10; TAIEI DENKI, Tokyo, Japan). Hyperoxia (FiO₂= 95%) was induced by 95%O₂–5%CO₂ gas inhalation, for which a breath mask was used under spontaneous respiration, whereas anoxia (FiO₂ = 0%) was induced by 95%N₂–5%CO₂ gas inhalation. In order to identify respiration arrest (RA), the respiration of the rat was confirmed by observing the periodical movement of the lateral region of the abdomen during the experiments. The RGB images are acquired in a personal computer and analyzed according to the visualizing process described above. Simultaneously with optical imaging for skin tissue, the value of SpO₂ and heart rate (HR) were measured for 4 rats by a pulse oximeter (MOUSEOX Pulse Oximeter; Star Life Science, Oakmont, PA) as systemic physiological parameters.

4. Results and discussion

As the preliminary experiment, we measured the diffuse reflectance spectra of rat dorsal skin after bile duct occlusion using a spectrometer in order to compare with the simulated diffuse reflectance spectra by the Monte Carlo model used in this study. Figures 3 (a) and 3 (b) show the diffuse reflectance spectra of a rat dorsal skin after bile duct occlusion and the simulated diffuse reflectance spectra by the Monte Carlo model under the several bilirubin concentrations. It was observed that the measured diffuse reflectance spectra change their shape at the shorter wavelength range from 400 to 520 nm with the increase in the bilirubin concentration as shown in Fig. 3 (a). The same tendency was also observed in the simulated results of the Monte Carlo model as shown in Fig. 3 (b). Therefore, it is reasonable to expect that the current Monte Carlo simulation model can work well for representing the dependence of the diffuse reflectance spectrum on the bilirubin levels.

Fig. 3. (a) Diffuse reflectance spectra of a rat dorsal skin after bile duct occlusion and (b) simulated diffuse reflectance spectra by the Monte Carlo model under the several bilirubin concentrations.

Download Full Size | PDF

Figure 4 shows typical sequential images obtained from rat dorsal skin for RGB color and bilirubin concentration C_bil after bile duct ligation. Figure 5 shows typical time courses of estimated bilirubin concentration C_bil averaged over the ROIs (white squares) in Fig. 4 and that measured by a commercially available bilirubinometer (JM-105) after bile duct ligation. The estimated value of C_bil increases monotonically and reaches 22.5 mg/dl at 116 hours after the onset of bile duct ligation, which shows the same tendency as the transcutaneous bilirubin concentration measured by the bilirubinometer. In this study, inflammatory response such as erythema was not observed in a rat dorsal skin during and after the bile duct surgery. Therefore, we believed that the surgery did not impact the measurements of hemoglobin. Changes in the spatial distribution of estimated melanin concentration over time could be due to the non-uniform illumination resulting from misalignment of the curved surface of rat dorsal skin. The cause of the increased tissue oxygen saturation during the bile duct ligation experiment is unclear. One possible explanation for this behavior is the overestimation of tissue oxygen saturation due to cross talk between bilirubin, oxygenated hemoglobin, and deoxygenated hemoglobin.

Fig. 4. Typical sequential images obtained from rat dorsal skin including RGB color images, C_bil, C_HbO, C_HbR, C_HbT, StO₂, and C_m after bile duct ligation. The scale bar in each color image is 10 mm.

Download Full Size | PDF

Fig. 5. Typical time courses of transcutaneous bilirubin concentration C_bil estimated by the proposed method and those measured by a commercially available bilirubinometer (JM-105) after bile duct ligation.

Download Full Size | PDF

Figure 6 shows a comparison between the bilirubin concentration C_bil estimated by the proposed method and that measured by the commercially available bilirubinometer, obtained from 6 samples after bile duct ligation. The value of C_bil estimated by the proposed method agrees well with that measured by the transcutaneous bilirubinometer. The value of Pearson’s correlation coefficient between the estimated value of C_bil obtained from the proposed method and the ground truth value measured by the commercially available bilirubinometer is R = 0.93 (p < 0.0001). The root mean square percentage error in bilirubin concentration C_bil between the estimated and measured values was 6.45%.

Fig. 6. Comparison between the transcutaneous bilirubin concentration C_bil estimated by the proposed method and those measured by a commercially available bilirubinometer (JM-105) after bile duct ligation (n = 6).

Download Full Size | PDF

Figure 7 shows the histograms of (a) tissue oxygen saturation StO₂, (b) total hemoglobin concentration C_HbT, and (c) melanin concentration C_m estimated by the proposed method, obtained from 6 rats before bile duct ligation. The estimated values of StO₂ were distributed within the range from 80 to 100%. The average value of 90.6 ± 5.2% for StO₂ is lower than the typical arterial oxygen saturation (SaO₂), which ranges from 90% to 98%. The typical value of venous oxygen saturation SvO₂ is around 75%.

Fig. 7. Histograms of estimated values for (a) the concentration of melanin C_m, (b) the concentration of total hemoglobin C_HbT, and (c) the tissue oxygen saturation StO₂, obtained from six samples during bile duct ligation (n = 6).

Download Full Size | PDF

The value of StO₂ estimated by the proposed method represents the oxygen saturation for the mixture of arterio-venous blood. Almost 75% of the total blood volume in the whole body is contained within the veins and venules, whereas 25% of it is contained within the arteries and arterioles. Assuming that the blood volume ratio of venules and arterioles in the skin tissue is similar to that of the whole body, and the value of SaO₂ under the normal condition is 96%, the tissue oxygen saturation of skin is calculated to be 80.6%. This value is close to the average value of 90.6 ± 5.2% for StO₂ obtained by our method. The estimated values of C_HbT were distributed within the range from 0.3 to 0.9 vol.%, corresponding to the typical cutaneous hemoglobin content reported in the literature [17,26,37]. The results of C_m exhibited low values distributed within the range from 0 to 1.6 vol.%, which is consistent with the fact that albino rats were used in this study.

Figure 8 shows typical in vivo resultant images obtained from rat dorsal skin including RGB color images, C_HbO, C_HbR, and StO₂ during changes in FiO₂. Figure 9 shows the time courses of (a) C_HbO, (b) C_HbR, (c) C_HbT and (d) StO₂ averaged over the entire region of each image for 11 rats during changes in FiO₂. Values of C_HbO and C_HbR decreased and increased, gradually, as FiO₂ decreased, which caused decreases in StO₂. On the other hand, the value of StO₂ dropped dramatically when FiO₂ was below 19%, indicating the onset of hypoxemia due to hypoxia. The value of C_HbT gradually increased after the onset of severe hypoxia (FiO₂=9%) and anoxia (FiO₂=0%), implying an increase in blood flow compensating for hypoxia. Time courses of C_HbO, C_HbR, C_HbT, and StO₂ while changing FiO₂ were consistent with well-known physiological responses to changes in FiO₂.

Fig. 8. Typical sequential images obtained from rat dorsal skin including RGB color images, C_HbO, C_HbR, C_HbT, StO₂, C_m, and C_bil during changes in FiO₂. The scale bar in each color image is 5 mm.

Download Full Size | PDF

Fig. 9. Time courses of average chromophore concentration and tissue oxygen saturation averaged over the ROI for (a) oxygenated hemoglobin C_HbO, (b) deoxygenated hemoglobin C_HbR, (c) total hemoglobin C_HbT, (d) tissue oxygen saturation StO₂, (e) melanin C_m, and (f) bilirubin C_bil. Data are expressed as mean ± SD (n = 11).

Download Full Size | PDF

Figure 10 shows an example of temporal change in C_HbT averaged over the ROI with 100×100 pixels on in vivo rat scalp under the normoxic condition. The pulse wave forms with approximately 6 beats per second or 360 beats per minute can be observed periodically, which is closed to the value of HR under control condition (380 ± 8 beats per minute) [38]. High frequency fluctuation due to noise in CCD sensor and low frequency component originated from vasomotor are also observed in temporal signal of C_HbT. Applying the fast Fourier transform band pass filters to each pixel of the sequential images for C_HbT along the time line, two-dimensional PPG can be reconstructed as well as C_HbO and C_HbR. The frequency bands of the FFT filters were selected as 5.0-8.5 Hz taking the variability of HR in rat into account. Figure 11 shows a comparison between the estimated HR obtained by the proposed method and the reference HR measured by the commercially available pulse oximeter. The value of Pearson’s correlation coefficient between the estimated values by the proposed method and the measurements by the pulse oximeter was calculated to be 0.92 (p < 0.0001) whereas the root mean square percentage error in HR between the estimated and measured values was 0.4%. This result revealed a significant relationship between the proposed method and measurements using the conventional heart rate monitor.

Fig. 10. Typical time courses of C_HbT and the extracted PPG signal averaged over the region of interest on in vivo rat scalp under the normoxic condition (FiO₂=21%).

Download Full Size | PDF

Fig. 11. Comparison between the estimated HR obtained by the proposed method and the ground truth HR measured by the commercially available pulse oximeter. (n = 4)

Download Full Size | PDF

Figure 12 shows the typical PPG signals of C_HbO and C_HbR obtained from the scalp of rat for (a) FiO₂=95%, (b) FiO₂=21%, (c) FiO₂=14%, and (d) FiO₂=6%. The number of pulse waves for both C_HbO and C_HbR increased as FiO₂ decreased, indicating the increase in HR for compensating hypoxia. The pulse wave amplitude (PWA) for C_HbO and that for C_HbR are decreased and increased, respectively, as FiO₂ decreased. It is known that the partial pressure of oxygen in arterial blood (PaO₂) decreases due to the reduction of FiO₂. Moreover, the value of SpO₂ is decreased as PaO₂ decreases according to the oxygen dissociation curve. We assumed that the ratio of PWA for C_HbO (PWA_HbO) and that for C_HbR (PWA_HbR) are related to the value of SpO₂ from the observation of changes in PWAs for both C_HbO and C_HbR according to the value of FiO₂ shown in Fig. 12, In this study, the ratio of PWA_HbO and PWA_HbR was calculated as ϕ = PWA_HbO/ PWA_HbR and used to evaluate SpO₂.

Fig. 12. Typical PPG signals of C_HbO and C_HbR obtained from the scalp of rat under the condition of (a) FiO₂=95%, (b) FiO₂=21%, (c) FiO₂=14%, and (d) FiO₂=6%.

Download Full Size | PDF

Figure 13 shows the dependence of SpO₂ measured by the pulse oximeter on the value of ϕ obtained from the proposed method. Once the value of ϕ reaches 3.0 the curve is almost flat, indicating there is little change in SpO₂ above this point. But, at less than 3.0 of ϕ the curve is very steep, and small changes in ϕ greatly reduce the SpO₂.

Fig. 13. Dependence of SpO₂ measured by the pulse oximeter on the value of ϕ obtained from the proposed method.

Download Full Size | PDF

The function SpO₂ (ϕ) is approximated by an exponential function as,

(15)$$Sp{O_2}(\phi )= 100 - \frac{A}{{1 + \exp \left( {\frac{{\phi - B}}{C}} \right)}}.$$

Applying the function of SpO₂ (ϕ) to each pixel of a two-dimensional PPG image, the estimated mage of SpO₂ was reconstructed. Figure 14 shows the typical resultant images for (a) raw RGB color, (b) SpO₂, (c) StO₂, (d) C_m, and (e) C_bil obtained from the scalp of rat while changing FiO₂. The value of SpO₂ dropped remarkably when FiO₂ was below 18%, indicating the onset of hypoxemia due to hypoxia. The value of StO₂ was lower than that of SpO₂ for each condition of FiO₂. On the other hand, the value of StO₂ decreased gradually according to the reduction of FiO₂. The results show the possibility of the remote PPG to evaluate both SpO₂ and StO₂ simultaneously.

Fig. 14. Typical resultant images for (a) raw RGB color, (b) SpO₂, (c) StO₂, (d) C_m, and (e) C_bil obtained from the scalp of rat while changing FiO₂. The scale bar in each color image is 3 mm.

Download Full Size | PDF

In this study, we did not confirm the feasibility of the proposed method to measure the change in melanin level experimentally since the skin of albino rats used in this study contained no melanin granule. Instead of animal experiments, we evaluated the accuracy of the melanin estimation based on the results of the Monte Carlo simulations as the preliminary numerical investigations. Figure 15 shows the estimated and ground truth values of melanin concentration obtained from the preliminary numerical investigations. The estimated values are well correlated with the ground truth values for given ranges of melanin concentration. The value of Pearson’s correlation coefficient between the estimated and ground truth values of melanin concentration was calculated to be 0.93 (p < 0.0001). The root mean square percentage error was 3.46%.

Fig. 15. Comparison between the estimated melanin concentration obtained by the proposed method and the ground truth melanin concentration.

Download Full Size | PDF

Although oxygenated hemoglobin, deoxygenated hemoglobin, bilirubin, and melanin are the major chromophores in the skin tissue, light absorption by the minor chromophores such as methemoglobin, carotenoids, and collagen also contributes to the diffuse reflectance in the visible wavelength region. This method, which is based on diffusing reflection, integrates all information along the depth direction, and therefore, it does not have depth resolution. On the other hand, the diffuse reflectance have a sensing depth that estimate how deep a photon travels into the tissue before diffusely reflected to the surface and re-collected by the imaging system. We estimate a sensing depth in a wavelength range from 400 to 700 nm at intervals of 10 nm by the MCS under the typical condition for C_bil=1.0 mg/dl, C_m=1.0 vol.%, C_HbT=0.2 vol.%, StO₂=60%. The sensing depth averaged over the visible wavelength region was estimated to be 4.2 ± 2.0 mm.

It may be possible to estimate directly C_HbO, C_HbR, C_bil, and C_m from the RGB-values without the CIEXYZ color space. In that case, it is necessary to know the spectral sensitivities of red, green, and blue channels of camera used. Generally, the spectral sensitivities of color camera are device-dependent values and some of them have not been published by manufacturers. On the other hand, the color matching functions of CIEXYZ color system has been published and available for calculating the XYZ-values from the diffuse reflectance spectrum. Moreover, the XYZ-values can be estimated from the measured RGB-values based on the measurements of Color Checker. For this reason, we used the conversion from the measured RGB-values to XYZ-values.

It was considered that the illumination and detection were performed in a non-contact fashion in the Monte Carlo simulations. Although polarization of the incident light was not taken into account for the Monte Carlo model, the diffuse reflectance was calculated separately from the specular reflection component in the Monte Carlo simulation. The geometry of the ring-shaped illuminator was also not taken into consideration in the Monte Carlo simulation model.

We observed a slight movement of rat dorsal skin surface due to respiration. Although we didn’t perform any post-processing corrections, this respiratory-induced motion could cause artifacts in the estimated images of chromophore concentrations. A post-processing correction technique such as an image registration will be useful to reduce respiratory-induced motion artifacts. The current algorithm can be applied not only to rats but also to human subjects. In this study, we used a single thickness for the epidermis and the dermis and only one typical wavelength-dependent scattering value in the Monte Carlo model to derive the empirical formulae for C_HbO, C_HbR, C_bil, and C_m. However, thickness and light scattering properties of human skin could vary with subjects. Therefore, the Monte Carlo simulation model with the variations in the scattering properties and the thickness of each layer could be reconstructed to apply this algorithm to human skin imaging. Although we assumed that the thicknesses for the epidermis and dermis are 0.06 and 4.94 mm, respectively, in the MCS model, the thicknesses of epidermis and dermis could vary with species and regions of body. Changes in the thickness of epidermis and dermis will impact the regression equations for C_HbO, C_HbR, C_bil, and C_m. Therefore, the Monte Carlo model should be adjusted to account for human skin thickness versus rat skin thickness. We took the variation in melanin concentration into account for the Monte Carlo model to establish the empirical formulae for C_HbO, C_HbR, and C_bil. However, if the melanin concentration of sample is out of a setting range in the Monte Carlo model, the estimation errors in C_HbO, C_HbR, and C_bil could be increased.

We assumed a single thickness for the epidermis and the dermis and only one typical wavelength-dependent scattering coefficient and anisotropy factor for skin in the Monte Carlo simulation to establish the empirical formulae for C_HbO, C_HbR, C_bil, and C_m. Changes in these parameters could affect the empirical formulae. Diffusely reflected light from the skin tissue will increase as the absorption of light in skin becomes small and will increase as the scattering coefficient μ_s(λ) becomes small. Therefore, if the scattering coefficients μ_s(λ) of skin tissue measured are larger than the typical values used in the Monte Carlo simulation, the regression model will underestimate the chromophore concentrations. Diffusely reflected light from the skin tissue could also be increased as the anisotropy factors g(λ) decrease. Therefore, if the value of g(λ) are larger than the typical one, the empirical formulae will underestimate C_HbO, C_HbR, C_bil, and C_m. The probability that light is absorbed by melanin in the epidermis will be higher than the probability that light is absorbed by hemoglobin and bilirubin in the dermis as the epidermis becomes thicker. Thus, if the thickness of the epidermis is larger than the typical values, then the concentration C_HbO, C_HbR, and C_bil in the dermis may be overestimated. Hence, the estimated values of C_HbO, C_HbR, C_bil, and C_m could be varied with the changes in the scattering properties and the thickness of each layer. Above discussion is summarized in Table 1. Studies on the Monte Carlo simulation model with the variations in the scattering coefficients, the anisotropy factors, and the thickness of each layer should be investigated in the future.

Table 1. Summary of expected impacts of changes in light scattering properties and epidermal thickness on the estimated chromophore]e concentration.

View Table

5. Conclusions

In summary, a method for measuring and imaging bilirubin concentration, oxygenated hemoglobin concentration, deoxygenated hemoglobin concentration, and melanin concentration of skin tissues using RGB camera-based diffuse reflectance spectroscopy was demonstrated in the present study. In vivo experiments with rat dorsal skin after bile duct ligation showed good correlation in transcutaneous bilirubin concentrations between the proposed method and a commercially available transcutaneous bilirubinometer, which indicates the ability of the proposed method to evaluate transcutaneous bilirubin concentration. Time courses of total hemoglobin concentration and tissue oxygen saturation while changing the fraction of inspired oxygen coincide with well-known physiological responses to hyperoxia, normoxia, and anoxia, which demonstrate the feasibility of the method for monitoring skin hemodynamics due to loss of tissue viability and vitality. We also found that the PWA of the systolic peak for C_HbO and that for C_HbR are decreased and increased, respectively, as FiO₂ decreased. The ratio of the PWAs for C_HbO and C_HbR was associated to the reference value of SpO₂ measured by a commercially available pulse oximeter to obtain an empirical formula for evaluating SpO₂ from the sequential RGB images. Spatiotemporal changes in SpO₂ according to the reduction of FiO₂ were successfully visualized by the proposed approach. The results show the possibility of the remote PPG to evaluate both SpO₂ and StO₂ simultaneously. To the best of our knowledge, this is the first example of simultaneous imaging of bilirubin, melanin, oxygenated hemoglobin, deoxygenated hemoglobin, tissue oxygen saturation, and arterial oxygen saturation of in vivo skin tissue using a digital RGB camera. The results of animal experiments have novelty and significance in the field of biomedical optics and indicate the potential of the method for imaging of multiple physiological parameters in skin tissue and realizing an optical biomedical imaging technique that enable cost-effective, easy-to-use, portable, remotely administrable, and/or point-of-care solutions.

Funding

Japan Society for the Promotion of Science (JP24560513); The Precise Measurement Technology Promotion Foundation.

Disclosures

The authors declare that there are no conflicts of interest related to this article.

References

1. V. Tuchin, Tissue Optics: Light Scattering Methods and Instruments for Medical Diagnosis, 2nd ed., (SPIE, 2007).

2. S. A. Prahl, “Tabulated molar extinction coefficient for hemoglobin in water,” http://omlc.org/spectra/hemoglobin/summery.html (1991). Accessed September 30, 2019.

3. D. A. Boas and M. A. Franceschini, “Haemoglobin oxygen saturation as a biomarker: the problem and a solution,” Philos. Trans. R. Soc., A 369(1955), 4407–4424 (2011). [CrossRef]

4. J. Allen, “Photoplethysmography and its application in clinical physiological measurement,” Physiol. Meas. 28(3), R1–R39 (2007). [CrossRef]

5. A. A. Kamal, J. B. Harness, G. Irving, and A. J. Mearns, “Skin photoplethysmography–a review,” Comput. Meth. Programs Biomed. 28(4), 257–269 (1989). [CrossRef]

6. G. Agati and F. Fusi, “New trends in photobiology (invited review). Recent advances in bilirubin photophysics,” J. Photochem. Photobiol., B 7(1), 1–14 (1990). [CrossRef]

7. F. Groenendaal, J. van der Grond, and L. S. de Vries, “Cerebral metabolism in severe neonatal hyperbilirubinemia,” Pediatrics 114(1), 291–294 (2004). [CrossRef]

8. F. F. Rubaltelli, G. R. Gourley, N. Loskamp, N. Modi, M. Roth-Kleiner, A. Sender, and P. Vert, “Transcutaneous bilirubin measurement: a multicenter evaluation of a new device,” Pediatrics 107(6), 1264–1271 (2001). [CrossRef]

9. M. J. Maisels, E. M. Ostrea Jr, S. Touch, S. E. Clune, E. Cepeda, E. Kring, K. Gracey, C. Jackson, D. Talbot, and R. Huang, “Evaluation of a new transcutaneous bilirubinometer,” Pediatrics 113(6), 1628–1635 (2004). [CrossRef]

10. G. Zonios, J. Bykowski, and N. Kollias, “Skin melanin, hemoglobin, and light scattering properties can be quantitatively assessed in vivo using diffuse reflectance spectroscopy,” J. Invest. Dermatol. 117(6), 1452–1457 (2001). [CrossRef]

11. J. W. Feather, M. Hajizadeh-Saffar, G. Leslie, and J. B. Dawson, “A portable scanning reflectance pectrophotometer using visible wavelengths for the rapid measurement of skin pigments,” Phys. Med. Biol. 34(7), 807–820 (1989). [CrossRef]

12. D. K. Harrison, S. D. Evans, N. C. Abbot, J. S. Beck, and P. T. McCollum, “Spectrophotometric measurements of haemoglobin saturation and concentration in skin during the tuberculin reaction in normal human subjects,” Clin. Phys. Physiol. Meas. 13(4), 349–363 (1992). [CrossRef]

13. D. J. Newton, D. K. Harrison, C. J. Delaney, J. S. Beck, and P. T. McCollum, “Comparison of macro- and maicro-lightguide spectrophotometric measurements of microvascular haemoglobin oxygenation in the tuberculin reaction in normal human skin,” Physiol. Meas. 15(2), 115–128 (1994). [CrossRef]

14. A. A. Stratonnikov and V. B. Loschenov, “Evaluation of blood oxygen saturation in vivo from diffuse reflectance spectra,” J. Biomed. Opt. 6(4), 457–467 (2001). [CrossRef]

15. G. Zonios, J. Bykowski, and N. Kollias, “Skin melanin, hemoglobin, and light scattering properties can be quantitatively assessed in vivo using diffuse reflectance spectroscopy,” J. Invest. Dermatol. 117(6), 1452–1457 (2001). [CrossRef]

16. G. N. Stamatas and N. Kollias, “Blood stasis contributions to the perception of skin pigmentation,” J. Biomed. Opt. 9(2), 315–322 (2004). [CrossRef]

17. I. Nishidate, Y. Aizu, and H. Mishina, “Estimation of melanin and hemoglobin in skin tissue using multiple regression analysis aided by Monte Carlo simulation,” J. Biomed. Opt. 9(4), 700–710 (2004). [CrossRef]

18. P. R. Bargo, S. A. Prahl, T. T. Goodell, R. A. Sleven, G. Koval, G. Blair, and S. L. Jacques, “In vivo determination of optical properties of normal and tumor tissue with white light reflectance and empirical light transport model during endoscopy,” J. Biomed. Opt. 10(3), 034018 (2005). [CrossRef]

19. S.-H. Tseng, P. Bargo, A. Durkin, and N. Kollias, “Chromophore concentrations, absorption and scattering properties of human skin in vivo,” Opt. Express 17(17), 14599–14617 (2009). [CrossRef]

20. K. Yoshida and I. Nishidate, “Rapid calculation of diffuse reflectance from a multilayered model by combination of the white Monte Carlo and adding-doubling methods,” Biomed. Opt. Express 5(11), 3901–3920 (2014). [CrossRef]

21. M. G. Sowa, J. R. Payette, M. D. Hewko, and H. H. Mantsch, “Visible-near infrared multispectral imaging of the rat dorsal skin flap,” J. Biomed. Opt. 4(4), 474–481 (1999). [CrossRef]

22. A. K. Dunn, A. Devor, H. Bolay, M. L. Andermann, M. A. Moskowitz, A. M. Dale, and D. A. Boas, “Simultaneous imaging of total cerebral hemoglobin concentration, oxygenation, and blood flow during functional activation,” Opt. Lett. 28(1), 28–30 (2003). [CrossRef]

23. I. Kuzmina, I. Diebele, D. Jakovels, J. Spigulis, L. Valeine, J. Kapostinsh, and A. Berzina, “Towards noncontact skin melanoma selection by multispectral imaging analysis,” J. Biomed. Opt. 16(6), 060502 (2011). [CrossRef]

24. B. S. Sorg, B. J. Moeller, O. Donovan, Y. Cao, and M. W. Dewhirst, “Hyperspectral imaging of hemoglobin saturation in tumor microvasculature and tumor hypoxia development,” J. Biomed. Opt. 10(4), 044004 (2005). [CrossRef]

25. S. F. Bish, M. Sharma, Y. Wang, N. J. Triesault, J. S. Reichenberg, J. X. J. Zhang, and J. W. Tunnell, “Handheld diffuse reflectance spectral imaging (DRSi) for in-vivo characterization of skin,” Biomed. Opt. Express 5(2), 573–586 (2014). [CrossRef]

26. A. Nkengne, J. Robic, P. Seroul, S. Gueheunneux, M. Jomier, and K. Vie, “SpectraCam®: a new polarized hyperspectral imaging system for repeatable and reproducible in vivo skin quantification of melanin, total hemoglobin, and oxygen saturation,” Skin Res. Technol. 24(1), 99–107 (2018). [CrossRef]

27. F. Vasefi, N. MacKinnon, R. B. Saager, A. J. Durkin, R. Chave, E. H. Lindsley, and D. L. Farkas, “Polarization-sensitive hyperspectral imaging in vivo: a multimode dermoscope for skin analysis,” Sci. Rep. 4(1), 4924 (2015). [CrossRef]

28. I. Nishidate, A. Wiswadarma, Y. Hase, N. Tanaka, T. Maeda, K. Niizeki, and Y. Aizu, “Non-invasive spectral imaging of skin chromophores based on multiple regression analysis aided by Monte Carlo simulation,” Opt. Lett. 36(16), 3239–3241 (2011). [CrossRef]

29. J. O’Doherty, P. McNamara, N. T. Clancy, J. G. Enfield, and M. J. Leahy, “Comparison of instruments for investigation ofmicrocirculatory blood flow and red blood cell concentration,” J. Biomed. Opt. 14(3), 034025 (2009). [CrossRef]

30. N. Tsumura, H. Haneishi, and Y. Miyake, “Independent-component analysis of skin color image,” J. Opt. Soc. Am. A 16(9), 2169–2176 (1999). [CrossRef]

31. D. Nakao, N. Tsumura, and Y. Miyake, “Real-timemulti-spectral image processing for mapping pigmentation in human skin,” IS&T/SID’s 9th Color Imaging Conference, Color Science, Systems and Applications, Scottsdale, AZ, 80–84 (2001).

32. I. Nishidate, N. Tanaka, T. Kawase, T. Maeda, T. Yuasa, Y. Aizu, T. Yuasa, and K. Niizeki, “Noninvasive imaging of human skin hemodynamics using a digital red-green-blue camera,” J. Biomed. Opt. 16(8), 086012 (2011). [CrossRef]

33. L. Wang, S. L. Jacques, and L. Zheng, “MCML: Monte Carlo modeling of light transport in multi-layered tissues,” Comput. Methods Programs Biomed. 47(2), 131–146 (1995). [CrossRef]

34. T. Sarna and R. C. Sealy, “Photoinduced oxygen consumption in melanin systems. Action spectra and quantum yields for eumelanin and synthetic melanin,” Photochem. Photobiol. 39(1), 69–74 (1984). [CrossRef]

35. S. L. Jacques, R. D. Glickman, and J. A. Schwartz, “Internal absorption coefficient and threshold for pulsed laser disruption of melanosomes isolated from retinal pigment epithelium,” Proc. SPIE 2681, 468–477 (1996). [CrossRef]

36. J. A. Delgado Atencio, S. L. Jacques, and S. Vázquezy Montiel, “Monte Carlo Modeling of Light Propagation in Neonatal Skin,”in Applications of Monte Carlo Methods in Biology, Medicine and Other Fields of Science, C. J. Mode, ed. (InTechOpen, 2011).

37. S. L. Jacques, I. S. Saidi, A. Ladner, and D. Oelberg, “Developing an optical fiber reflectance spectrometer to monitor bilirubinemia in neonates,” Proc. SPIE 2975, 115–124 (1997). [CrossRef]

38. A. U. Ferrari, A. Daffonchio, F. Albergati, and G. Mancia, “Inverse relationship between heart rate and blood pressure variabilities in rats,” Hypertension 10(5), 533–537 (1987). [CrossRef]

	Larger or smaller than the typical value	Expected impact on the estimated chromophore concentration
	Larger or smaller than the typical value	C_HbO	C_HbR	C_bil	C_m
μ_s (λ)	Larger	Underestimation	Underestimation	Underestimation	Underestimation
μ_s (λ)	Smaller	Overestimation	Overestimation	Overestimation	Overestimation
g (λ)	Larger	Underestimation	Underestimation	Underestimation	Underestimation
g (λ)	Smaller	Overestimation	Overestimation	Overestimation	Overestimation
Epidermis thickness	Larger	Overestimation	Overestimation	Overestimation	Overestimation
Epidermis thickness	Smaller	Underestimation	Underestimation	Underestimation	Underestimation

Simple and affordable imaging of multiple physiological parameters with RGB camera-based diffuse reflectance spectroscopy

Abstract

1. Introduction

2. Principle

2.1 Estimation of chromophore concentrations

3. Materials and method

3.1 Imaging system

3.2 Animal experimental protocols

4. Results and discussion

5. Conclusions

Funding

Disclosures

References

Cited By

Figures (15)

Tables (1)

Equations (15)

Biomedical Optics Express