1. Introduction

An excitation–emission matrix (EEM, also called a fluorescence fingerprint) contains two-dimensional spectral data consisting of fluorescence spectra measured by thoroughly scanning the excitation wavelength. The EEM has been widely applied for nondestructive measurement of the physical and chemical properties of objects [1–5] because it shows an object-specific fluorescence characteristic map just like a fingerprint. Specifically, this technique has greater potential than single fluorescence spectroscopy if the object contains materials with different fluorescence characteristics. Therefore, EEM spectrophotometry is effective for the identification and quantification of an object’s composition. Furthermore, in recent years the EEM measurement technique has been expanded to EEM image measurement, similar to the evolution of spectrophotometry to hyperspectral imaging. The EEM image contains an EEM for each pixel because it consists of multiple excitation and emission wavelength fluorescence images. Tsuta et al. used EEM imaging to visualize the internal structure of soybean seeds [6], and Kokawa et al. showed that the gluten and starch distribution in dough can be visualized by EEM imaging [7].

Dimension reduction methods are typically applied to EEM fluorescence spectra as the first step of the analysis; multivariate analyses are then used to identify and quantify an object’s composition [8]. However, this procedure requires the measurement of the full EEM, which is quite time consuming. Thus, nondestructive measurement based on EEM fluorescence spectra analysis is unreasonable for real-time monitoring.

In this study, to offer a solution to this problem, we propose a method of optically reducing the dimensions of the EEM by designing the optimal combination of excitation–emission band-pass filters (BPFs) and apply the proposed technique to the quantification and visualization of viable bacteria on the surface of pork meat. A bacteria test can cover only a narrow region and requires a culture period. Therefore, it cannot be used to check all processed meat products. If viable bacteria counts can be estimated from a fluorescence image, the process will provide a useful hygiene monitoring system for a meat-processing plant.

To determine the performance of hyperspectral imaging without hyperspectral image measurement, Nishino et al. proposed a method of designing an optical filter that can change the spectral sensitivity of an RGB camera to maximize the ability to discriminate predefined targets [9,10]. Nakauchi et al. proposed a more generalized method in which spectral BPFs are designed to distinguish the targets’ spectra [11]. He applied the technique to the detection and visualization of road freezing. The result showed that the two designed filters allowed road freezing to be detected and visualized with the same performance as hyperspectral imaging using the entire spectra. Tsuta et al. extended this technique to EEM analysis and proposed the excitation and emission wavelength band selection method [12]. The proposed method was applied to the quantification of the riboflavin content of yogurt and yielded a higher estimation accuracy than the typical EEM analysis method [parallel factor analysis plus partial least square regression (PLSR)]. In addition, this method does not require a full EEM measurement, and it can be applied to image measurement using optically realized interference filters. However, the earlier method did not consider the differences in signal-to-noise ratio (S/N) between a fluorescence spectrophotometer and the imaging device. The fluorescence spectrophotometer used to measure the training data set has a quite good S/N, and the interference-filter-based EEM imaging device has many noise factors such as uneven illumination, the optical density of the elimination wavelength band of the interference filter, and the camera’s photon shot noise. For this reason, the optimal filter for the training data set might not be the best filter for optical implementation. In this study, we improved the filter selection method so as to obtain the best filters for optical implementation. Specifically, the proposed method enables the design of the best combination of filters under an arbitrary S/N. If the S/N of the implemented imaging device is predicted, the best filter for visualization can be designed.

As mentioned above, estimation of viable bacteria on the surface of pork meat was chosen as the target for filter design. It is well known that fluorescence spectra are effective for identification of bacteria [13], and many studies on the detection, quantification, and imaging of bacteria using fluorescence light have been reported in food and biomedical engineering research. Konig et al. proposed a method of in-vivo detection and imaging of the fluorescence of porphyrin-producing bacteria for application to acne vulgaris, caries, and squamous cell carcinoma diagnosis [14]. Several studies reported measurement methods for imaging of dental plaque and enamel caries using quantitative light-induced fluorescence. Pretty et al. summarized these techniques [15], focusing on the autofluorescence characteristics of teeth and plaque. On the other hand, measurement of autofluorescence was applied to the early detection of bacterial spoilage in meat. Sahar et al. reported that N-PLS regression for synchronous front-face fluorescence spectroscopy allowed the determination of the total viable count (TVC), Pseudomonas, Enterobacteriaceae, and Brochothrix thermosphacta on chicken breast with a high correlation (R² = 0.99) [16]. Aït-Kaddour et al. developed a portable fluorescence spectrometer for measuring the TVC, Pseudomonas, lactic acid bacteria, and yeast/molds on minced beef [17]. Oto et al. used PLS regression to obtain EEMs of pork loins to estimate the plate count and adenosine triphosphate content [18]. However, these methods provided point measurements, and a high-speed imaging system has not been developed yet.

In this paper, we first measured the EEM fluorescence spectra of pork meat samples and the counts of viable bacteria on their surfaces to construct the EEM data set. Next, the excitation–emission filters were designed using the proposed method. The performance of filters designed for several S/N conditions was evaluated computationally by comparison with typical methods (PLSR and single-peak observation). Finally, the performance of the designed filters in an imaging device was evaluated by measuring the EEM images.

In the second section, the theory of the proposed excitation–emission filter optimization method is described. The details of the measurement and filter design are given in the third section. The results of the measurements, theoretical filter design, and evaluation of the filters and viable bacteria distribution visualization are presented in the fourth section. Finally, this study is concluded in the fifth section.

2. Theory

The filter design method proposed by Nakauchi et al. [11] for the discrimination of spectra is described as follows. The output signals O_i of a camera equipped with the i-th filter can be computed by Eq. (1). Here λ is the wavelength, T_i(λ) is the spectral transmittance of the i-th filter $(i=1,\cdots ,N)$, I(λ) is the spectral radiance of the incoming light, and S(λ) is the spectral sensitivity of the camera. The transmittance function of a filter, T_i(λ), has the band-pass characteristics described in Eq. (2).

This method was expanded to excitation and emission wavelength band selection by Tsuta et al. [12] (see Fig. 1). The computation of the camera’s output signals O_i described in Eq. (1) was modified to consider the fluorescence characteristics described in Eq. (3).

 

Fig. 1 Excitation–emission filters for EEM-derived fluorescence imaging system. Left: EEM-derived fluorescence imaging device setup. Right: Proposed excitation–emission filter.

Download Full Size | PDF

The filter design method described above was improved to include the effect of noise in the fluorescence image measurement conditions. In this study, multivariate regression was used to establish the standard curve, and the estimation accuracy was evaluated using the standard error of prediction (SEP). The multivariate regression using the camera output signal vector can be described as

3. Materials and methods

3.1 Measurement samples

In this study, pork loin pieces of different freshness were used for the filter design and evaluation. The pieces were cut to 45 mm by 45 mm, and the thickness was 1 cm. A total of three individual pieces were prepared for this research. Two of them were used to construct two EEM data sets, the calibration and validation data sets (see 3.2), and the other was used for EEM image measurement (see 3.3). These individual samples were measured 5–7 days after slaughter treatment. In each measurement, the samples were placed in a lidded petri dish and kept at 15 °C, and the EEM or EEM images of the samples and their viable bacteria counts were measured after different retention periods. The first measurement was performed within at most 4 h after sample trimming, which is denoted as 0 h. To obtain the viable bacteria counts, the colony-forming units per square centimeter (CFU/cm²) on the surfaces of the pork loin samples were measured using an aerobic count plate (Petrifilm^TM, 3M^TM). The counted region covered 40 mm by 40 mm of the inner area on the surface.

3.2 Excitation–emission matrix measurement

EEM data sets were measured by a fluorescence spectrophotometer (F-7000, Hitachi). In a sample measurement, a fused quartz plate was placed on the surface of a pork loin, and the EEM data at four different positions were measured using a contact probe through the fused quartz plate. Both the excitation and emission wavelengths ranged from 260 nm to 450 nm in 10 nm steps. The wavelength range was determined according to the possible measurement wavelengths of the EEM imaging system.

In the EEM measurement, one of two individual samples was measured to construct the calibration data set, and the other was measured for the validation data set. The training data set measurements were scheduled from 0 h to 72 h at 24 h intervals, and five samples were measured at each time. Thus, a total of 20 samples were prepared for this measurement, and 80 EEM data were obtained. The measurements for the validation data set were scheduled at 0 h, 24 h, 36 h, 48 h, and 72 h after slaughter, and three samples were measured at each time. However, only six EEM data (two EEMs for each sample) were available at the 0 h condition because of a measurement mistake. Thus, a total of 54 EEM data were used for validation.

3.3 EEM image measurement

The EEM images were measured to evaluate the performance of the designed filters when they were installed on the imaging device. To evaluate the filters, two EEM image data sets were measured. One was the training data set for establishing the standard curve, and the other was for the visualization of time-dependent changes in the viable bacteria distribution. The training data set measurements were scheduled from 0 h to 72 h at 12 h intervals, and three samples were measured at each time. Thus, a total of 21 samples were prepared for the training data set measurement. For the visualization of the viable bacteria distributions, EEM images of a single sample were captured from 0 h to 72 h at 24 h intervals, and the viable bacteria count was not measured.

EEM images were captured using the interference-filter-type EEM imaging system shown in Fig. 2. There are two filter wheels for the excitation and emission filters, respectively, in front of both the camera and the illumination fiber. Eight interference filters can be mounted on each wheel. The selectable interference filters ranged from 260 nm to 450 nm in 10 nm steps, and all of them had half-bandwidths of 10 nm. In this research, the interference filters for measurement were selected according to the theoretically designed filters. The detector was an ORCA-ER (Hamamatsu Photonics), and the illumination light source was a high-power xenon illuminant (MAX-303, 300 W, Asahi Spectra) equipped with a UV-vis mirror module that can exclude the spectral energy outside of 300–600 nm. Furthermore, cold filters that transmit only at 300 nm to 450 nm were installed on the excitation filters. The lens was an InfiniProbe UV microscope lens (STD S-80, Photo-Optical). The imaging area size ( = field of view) was about 6 cm × 6 cm, and the working distance was about 30 cm. In this geometry, the filter transmission was not observed to shift with the viewing angle. An image size of 168 × 128 pixels was obtained by using 8 × 8 binning.

 

Fig. 2 Interference-filter-type EEM imaging system. Left: Photo of EEM imaging device. Right: Measurement geometry.

Download Full Size | PDF

To remove unevenness and time-dependent variations in the illumination and dark noise of the sensor, a dark image and white reference image were also measured before the pork sample measurement for each time condition. The dark image and pork sample measurement images were both captured at a 5 s exposure time by closing the shutter of the illuminant. In the white reference measurement, a Spectralon white target (Labsphere) was measured across the entire excitation wavelength range by removing the emission filters. The exposure times for the excitation wavelength images were automatically adjusted so that their average pixel intensities were 70% of saturation. The multispectral images of the white reference and their exposure times allow calibration of the unevenness of the illumination and changes in the spectral distribution. Before image processing, the EEM images of pork, dark images, and white images were divided by their exposure times, and then the dark image was subtracted from the other images. Finally, each EEM image was divided by the white image measured with the same excitation filter

3.4 Optimization conditions

The filters were optimized according to the theory described in section 2 using the training data set described above. The objective of this optimization was to minimize SEP_n in Eq. (8) computed from the measured and estimated viable bacteria counts (CFU/cm²). Here SEP_n was computed from the calibration data set with cross-validation to avoid overfitting. The EEM data set was reconstructed so that EEM spectra of five samples measured at each time were categorized into different groups, and five-fold cross-validation was applied to them for the SEP_n computation. The EEM data set measured by the fluorescence spectrophotometer was used for $F({\lambda }_{ex},{\lambda }_{em})$ in Eq. (3). Here P(λ_ex) and S(λ_em) were assumed to be P(λ_ex) = 1 and S(λ_em) = 1 over an entire wavelength range. If the spectral radiance and sensitivity of the viable bacteria monitoring system had been given, they could have been used as P(λ_ex) and S(λ_em) for rigorous optimization. However, we expect that this assumption and calibration after optical implementation might work well, as demonstrated in an earlier study [11]. The maximum bandwidth for both excitation (${\lambda }_2^i-{\lambda }_1^i$) and emission (${\lambda }_4^i-{\lambda }_3^i$) in $T_i({\lambda }_{ex},{\lambda }_{em})$ in Eq. (4) was limited to 80 nm because the maximum number of interference filters was eight for each filter wheel. In addition, several combinations of filters were unavailable for optimization; the emission wavelengths were less than the excitation wavelengths, ± 30 nm for the excitation wavelength and ± 40 nm for its overtone wavelength. The number of filter combinations N was defined as two, and an all-possible-combinations search method was used to find the optimal filters. More filters can also be designed if the multiple selection method proposed by Nakauchi et al. [11] is used. To compare the filter sets designed with different S/N assumptions, the noise parameter of the imaging device, σ₀, was determined so that the S/N was 100, 316 (50 dB), or 1000. Here σ₀ is a comprehensive noise parameter including camera noise, variation in the illuminant, and the spectral transmittance of the interference filters. The S/N ratios were chosen on the basis of the S/N of the EEM imaging device. The dynamic range of the Orca-ER camera was 1:2250. However, light from the elimination band was observed in the captured fluorescence image, and the S/N ratio was 182. Therefore, the optimal filters for σ₀ = 100 or σ₀ = 316 might exhibit the best accuracy

4. Results and discussion

4.1 Results of EEM measurement

Figure 3 shows an example of EEM fluorescence spectra at 0 h (left) and 72 h (right). The horizontal axis, vertical axis, and color bar represent the emission wavelength, excitation wavelength, and fluorescence intensity, respectively. A high fluorescence energy appeared with a peak at (λ_ex, λ_em) = (300 nm, 340 nm), and the intensity clearly decreased with time. The fluorescence energy is attributed to the autofluorescence of tryptophan [13]. The change in the fluorescence characteristics with time is explained by degradation of the tryptophan in pork meat by the growing bacteria [18]. The number of CFUs is illustrated in Fig. 4 as a function of time. The error bars show the standard deviations. The number of CFUs increased with the retention time.

 

Fig. 3 Measured EEM fluorescence spectra at 0 h (left) and 72 h (right). Horizontal axis shows the emission wavelength; vertical axis shows the excitation wavelength; color bar indicates the fluorescence intensity.

Download Full Size | PDF

 

Fig. 4 Measured viable bacteria count (CFU/cm²) on the surface of pork loins used for training data set measurement versus time after the first measurement. Error bars show the standard deviations.

Download Full Size | PDF

4.2 Results of filter design and computational evaluation

Table 1 shows the designed transmittance functions, their SEPs for each S/N [not SEP_n in Eq. (8)] computed with five-fold cross-validation, and the SEPs of the calibration data set. The optimal filters designed for S/N = 1000 and 316 had the same transmittance functions. The SEPs of the validation data set were mostly the same (the filters for S/N = 100 performed slightly better), even though the filters designed for S/N = 1000 and 316 showed better performance for the calibration data set.

Table 1. Transmittance Functions of Designed Filters and Their Estimation Errors

View Table

The transmittance functions $T_i({\lambda }_{ex},{\lambda }_{em})$ for S/N = 1000 and 100 are shown in Fig. 5. The rectangles in the EEM indicate the transmittance. The selected bands reflect not the peak of tryptophan’s fluorescence but its slope. The same result was also obtained in an earlier study [12]. We suspect that the reason the optimal filters detect the slope is that the other autofluorescence properties that contribute to the estimation of the viable bacteria count also appear in the slope. As shown in the figure, the detected wavelength bands approach the peak along decreasing S/N values. This indicates that the high fluorescence intensity allows low S/N device to work, whereas the slope has more information than the peak.

 

Fig. 5 Theoretically designed filters. Rectangles on the EEM indicate the transmittance function.

Download Full Size | PDF

Figure 6 shows SEP_n, the SEP computed assuming noise, of the designed filters. As described above, the filters designed for S/N = 100 performed better for the validation data set, even though the filters for S/N = 1000 performed better for the calibration data set. As Fig. 6 shows, these trends were reversed when the S/N was about 200. The advantage of the filter for S/N = 100, its high tolerance for noise, might provide robustness to variations between samples.

 

Fig. 6 SEP computed assuming noise, SEP_n, for two filters designed for S/N = 1000 (or 316) and 100, as described in Fig. 5, under different S/N conditions. The calibration data set was used for this computation.

Download Full Size | PDF

Figure 7 shows the results of the computational evaluation. The SEPs of the calibration data set with five-fold cross-validation and the SEPs of the validation data set described in Table 1 were compared with the SEPs of two conventional methods, single regression using the peak intensity of the fluorescence of tryptophan and PLSR using the entire EEM. As illustrated in the figure, the designed filters were more accurate than these conventional methods. The proposed method is better than the analysis using the entire EEM because unwanted wavelength regions are also used in PLSR. A similar result was obtained in an earlier study [11], in which Nakauchi et al. showed that conventional spectral analysis with a wavelength band limitation outperformed the proposed filter design method.

 

Fig. 7 Computational evaluation results. The SEP for each filter is compared with those of conventional methods (single regression using the peak intensity of the fluorescence of tryptophan and PLSR using the entire EEM).

Download Full Size | PDF

4.3 Evaluation of designed filters using EEM images

EEM images of cut pork loins were measured in the wavelength range of the designed filters (excitation: 270–340 nm, emission: 370–440 nm) mentioned above. Measured fluorescence images of (λ_ex, λ_em) = (300 nm, 370 nm) at 0 h and 72 h are shown in Fig. 8. This excitation and emission combination has the highest fluorescence intensity of tryptophan among all of the measured combinations. The left panel shows RGB color images of the same sample at 0 h and 72 h, and the right panel shows their fluorescence images. The intensity of the fluorescence image clearly decreased, as shown for the training data set (4.1, Fig. 4), although it is nearly impossible to distinguish the two RGB color images. The time-dependent change in the viable bacteria counts is illustrated in Fig. 9. The error bars show the standard deviations. The CFU range is similar to that of the training data set measurement (from 10⁻² to 10⁻⁸).

 

Fig. 8 Comparison of (a) RGB and (b) fluorescence images measured at 0 h and 72 h.

Download Full Size | PDF

 

Fig. 9 Measured viable bacteria count (CFU/cm²) on the surface of pork loins used for EEM image measurement versus time after first measurement. Error bars show the standard deviations.

Download Full Size | PDF

The outputs of the filter combinations shown in Table 1 were computed using the measured EEM images, and the standard curves for the estimation of the CFU counts in Fig. 9 were established using the average fluorescence intensity of the pork region. Figure 10 shows the estimation accuracies of the filters designed for S/N = 1000 (left) and 100 (right) computed with three-fold cross-validation. The filters designed for S/N = 100 showed better estimation accuracy. These filters can perform better when the S/N of the imaging device is less than 200 (see Fig. 6). These are reasonable results because the S/N of the imaging device was less than 182, as mentioned in 3.4.

 

Fig. 10 CFU estimation accuracies of filters designed for S/N = 1000 (left) and 100 (right).

Download Full Size | PDF

Finally, the viable bacteria distribution was visualized, and the time-dependent spatial growth tendency was evaluated. The EEM images of the sample for visualization (the same sample that was measured at different times) were used for the evaluation. Thus, the sample was not used to establish the standard curve. The outputs of the filters designed for S/N = 100 were computed from the EEM image, and the standard curve established above was applied to all of the pixels. The visualization results are shown in Fig. 11. From left to right, the figure shows viable bacteria images at 0 h, 24 h, 48 h, and 72 h after the pork piece was cut. As shown in the figure, the viable bacteria increased with time and appeared to increase from the outer edge of the surface. One possible explanation for this spatial pattern of bacteria growth is that most of the bacteria from the knife used to cut the samples remained at the edge of the piece, initiating bacteria growth. However, further investigations are required to clarify this point. Our proposed method, excitation–emission filters for the visualization of viable bacteria, allows an approach to this topic.

 

Fig. 11 Viable bacteria distributions. Left to right: viable bacteria images at 0 h, 24 h, 48 h, and 72 h after the pork piece was cut. These results were obtained using the outputs of the filters for S/N = 100 and the standard curve established above.

Download Full Size | PDF

The wideband BPFs that pass the selected wavelength band described in Table 1 can replace the interference filters on the EEM imaging device. With this replacement, only two measurements of fluorescence images will be required to visualize the viable bacteria distribution, and the integration time of each measurement may be improved significantly. To consider the replacement, the spectral error between the ideal rectangular pass band used in this study and the spectral transmittance of the real optical filter should be calibrated. We expect that a redesign of the calibration curve will solve this problem, as in the earlier study [11].

We also checked the optimal set of filters in a wider wavelength range, 200–900 nm. In the EEMs in this wider wavelength range, the fluorescence of nicotinamide adenine dinucleotide phosphate (NADPH, Ex = 335, Em = 450) was obtained in addition to the fluorescence of tryptophan, as described in an earlier work [18]. The NADPH content increased with microorganism growth, and fluorescence from NADPH contributed to the CFU estimation, according to the loading map of PLS regression for the entire EEM. However, the designed optimal filter set still detected the fluorescence of tryptophan despite the contribution from the fluorescence of NADPH. We suspect that the fluorescence of NADPH was too weak for CFU estimation by fluorescence imaging, and the fluorescence of tryptophan yielded a better S/N ratio. If the number of filter sets is increased, some of the filters may detect the fluorescence of NADPH, and both fluorescence intensities may be used for CFU estimation. A dual-band filter design for the excitation of two fluorophores is the other way to use both tryptophan and NADPH fluorescence. To design such an optimal filter, the optimization method should be made more reasonable. For example, simulated annealing, which was used in an earlier work to design a multiband filter, is one possibility [9].

5. Conclusions

This paper proposed a method of optically reducing the dimensionality of an EEM fluorescence spectra data set by using optimal excitation and emission BPFs. The method was applied to the visualization of viable bacteria on the surface of a pork loin piece. The concept of optimizing the combination of excitation and emission wavelength bands was proposed in an earlier study, and the previous method was improved by providing a method of designing the optimal filters for optical implementation of the imaging device. The theory of this filter design method was described in section 2.

EEM data for pork loin pieces with different storage lifetimes measured by fluorescence spectrophotometry were used as the training data set for filter design (Fig. 3). The fluorescence properties of tryptophan appeared in the EEM and became weaker with time. This is due to degradation of the tryptophan in the pork meat by the growing bacteria.

Filters for the visualization of viable bacteria were designed for S/N values of 1000, 316, and 100. The objective variable in this optimization was the CFUs per square centimeter measured by an aerobic count plate, and the objective function was the noise-corrected SEP [SEP_n in Eq. (8)]. All of the designed filters showed a higher estimation accuracy than PLSR using the entire EEM and single regression using the peak intensity of the fluorescence properties of tryptophan (see Fig. 7).

Finally, EEM images were measured using an interference-filter-type EEM imaging device and used to evaluate the designed filters and visualize the viable bacteria distribution. Although the filters designed for S/N = 1000 showed better estimation accuracy in the theoretical evaluation, the filters for S/N = 100 performed better on the EEM images. The reason was that the S/N of the EEM imaging device may be 100 or less. Figure 11 shows the visualized viable bacteria distributions at 0 h, 24 h, 48 h, and 72 h. Viable bacteria grew inward with time from the edge of the piece.

As described above, our proposed method, reducing the dimensionality of the EEM by optimizing the excitation and emission BPFs, allowed the visualization of an object’s properties while maintaining the EEM data analysis performance without full EEM measurement. We expect our method to be a revolutionary nondestructive visualization technique for the application of EEM imaging. At least, time-dependent changes in the viable bacteria distribution were visualized in this study. It seems that our proposed method will be a useful hygiene monitoring system for meat-processing plants. However, only three individual samples were used in this study, and the mechanism of fluorescence quenching has not been clarified yet. In our future work, we will measure more samples and investigate which mechanism is actually detected in order to develop the hygiene monitoring system. As the goal of this study, optimal wideband BPFs that pass the selected wavelength bands will be developed, and a high-speed monitoring system will be implemented.