1. Introduction

Compressive Sensing (CS) [1,2], also known as sparse sampling, paved the way for new data acquisition schemes in various fields such as photonics, communications and networks or atomic force microscopy [3–6]. CS basically enables the reconstruction of sparse signals from measurements with less samples than postulated by the Shannon-Nyquist theorem. A particularly elegant utilization of CS is its combination with the matured technology of digital micro-mirror devices (DMD). This combination led to a method for imaging with a single element detector, known as single pixel imaging (SPI) [7,8]. In recent years several applications and improvements of this technology in the visible spectral range were shown, e.g. for SNR enhancement [9], real-time imaging (frame rates up to 10 Hz) [10,11] or higher compression rates [12,13].

However, the main benefit of SPI, namely replacing a costly focal plane array (FPA) becomes only effective in the infrared spectral region. While FPAs are low-cost in the visible spectral range, those devices become expensive in the upper near-infrared (NIR) and even more expensive in the mid-infrared (MIR) spectral region. Furthermore, DMDs offer much higher pixel numbers than infrared FPAs and single pixel detectors offer superior detectivity. Recently, these advantages of SPI were exploited for low-cost near-infrared (800-1700 nm) thermographic microscopy [14]. Nevertheless, up to now, hyperspectral SPI was only executed from the visible range [15] up to approximately 1100 nm [16]. Shibuya et al. showed hyperspectral SPI at 1550 nm based on frequency combs with the small spectral bandwidth of ca. 16 nm [17]. However, chemical identification in a hyperspectral imaging approach would require broadband spectral coverage, ideally covering higher wavelength regions in the upper NIR and even MIR spectral range [18,19].

In this study, we present a significant extension of the spectral coverage for hyperspectral SPI up to a wavelength of 2200 nm thereby gaining access to molecular-specific vibrational absorption bands. At the same time the size of used components was decreased further. This was achieved by employing a DMD for spatial encoding in combination with a fully-integrated spectrally selective MEMS based Fabry-Pérot tunable filter (FPTF) with built-in single element InGaAs-detector. Such FPTFs are the enabling technology for low-cost and robust miniature spectrometers (also known as “microspectrometers”), which are continuously replacing the standard FTIR spectrometers in field applications. In this contribution we are introducing MEMS based microspectrometer technology for SPI. We demonstrate for the first time that FPTFs are ideally suited for hyperspectral SPI as they provide high throughput, a large aperture size and small overall size. The performance of this approach was characterized. In particular, the employed CS reconstruction method was validated, and hyperspectral images of different samples, including adhesives on a textile surface were recorded. Chemical classification of the obtained hyperspectral images was executed by means of principle component analysis (PCA).

2. Materials and methods

CS allows to sample data in a compressed way under the premise that the respective data is sparse or at least sparse in some domain, which is typically valid for natural images [8]. Spatial encoding of the scene is achieved by applying patterns of a sampling basis via a DMD according to

The developed imaging system (Fig. 1) operates in diffuse reflection mode. A broadband halogen lamp with parabolic reflector illuminates the sample and the objective lens creates the image of the object in the DMD plane, resulting in a field of view (FOV) of 25 mm by 25 mm. An optical system, consisting of a telescope (10 × demagnification) and an additional lens, was used to focus light on the detector in order to maximize the throughput of the signal at the aperture of the detector (Ø 0.3 mm). The spatial resolution of this system can be calculated from the FOV (25 mm) and the amount of effective pixels, being the pixels displayed on the DMD, resulting in approximately 0.4 mm for 64 pixels.

 

Fig. 1 Schematic drawing of the single pixel camera. Broadband light from a halogen lamp gets diffusely scattered by the sample. An objective lens projects the sample image onto the DMD which spatially modulates the scene. An optical magnification system is attached to focus the NIR radiation onto the MEMS-based spectrometer consisting of a tunable Fabry-Pérot cavity and an InGaAs detector.

Download Full Size | PDF

Spectral detection was executed by commercially available MEMS-based FPTF spectrometers operating in the NIR spectral range (1450 nm – 1850 nm, Hamamatsu C13272-02 and 1750 nm – 2200 nm, C14273; both with InGaAs detector). The spectral resolution was approximately 22 nm and the time constant 1 ms. Spectral cross-talk between the pixels of the final image can be excluded, as a fixed set of Hadamard patterns was applied at each filter position of the tunable filter spectrometer. The DMD (Texas Instruments DLP4500, 912 × 1140 pixels) was controlled by using it as second monitor (via HDMI connection). For each Hadamard pattern displayed 3 frames were acquired. This resulted in an acquisition time of less than 4 minutes at a compression rate of 50% (for a single image at a specific wavelength with 2048 patterns plus inverse patterns applied, indicating a resolution of 64 by 64 pixels).

Hyperspectral images of three samples were recorded. First, images of a graphic printed with black toner on white paper (standard office paper) were collected. The second sample was created by drawing two different markers (edding 360, edding 3000; labelled as marker 1 and marker 2) on white paper. The third sample consisted of four different adhesives deposited on a green colored textile (65% polyester, 35% cotton). The adhesives were labelled from 1 to 4 corresponding to following brand names: ‘Loctite Power Flex’ (ethyl-2-cyanoacrylate), ‘UHU Alleskleber’ (ethanol, acetone), ‘Pattex Kraft Mix’ (epoxy) and ‘RS Superglue Gel’ (ethyl-2-cyanoacrylate).

The pixel values of the hyperspectral images shown in this work are calculated as absorbance normalized to a background recording. Preprocessing and statistical treatment of the absorption spectra, that were obtained pixel-wise from the data cube, was carried out in Orange 3.16 [22] by executing smoothing (Savitzky-Golay filter, window size: 15, polynomial order: 2), baseline correction (rubber band method) and standard normal variate (SNV) normalization. PCA was applied to mean centered data from representative pixels of each adhesive type and background. Logistic regression using a L₂-norm (i.e. ridge) penalty was used to fit a classification model on the previously selected pixels. Optimization of the ridge parameter was achieved through 10-fold cross-validation. The final model was subsequently employed to classify the remaining pixels in the hyperspectral image. The dynamic range (DR) of the setup was calculated by

3. Results

The calculation of the dynamic range yields 43 dB at zero compression and 40 dB at 90% compression at a wavelength of 2190 nm. In Fig. 2 the reconstructed images with different compression rates are shown. Using only ten percent of the most contributing patterns results in an already well recognizable image with a PSNR of 23 dB. For specific applications, the exclusive selection of the most contributing patterns of the Hadamard-set will strongly reduce acquisition times. Examples of such applications would be repeated measurements of similar samples (e.g. process monitoring in production lines) or measurements of samples changing their chemical composition over time, while preserving spatial distribution.

 

Fig. 2 Contribution of the most significant Hadamard patterns to the performance of the reconstruction. Images of a black toner logo on white paper were recorded at 2190 nm wavelength and reconstructed using different amounts of patterns. It can be seen that the patterns with large Norm(S) values have significantly higher influence on the reconstruction quality. The FOV is 25 mm by 25 mm.

Download Full Size | PDF

NIR spectroscopy has the potential to resolve structures not perceivable in the visible range (VIS). This characteristic is illustrated (Fig. 3) by imaging a sample consisting of the letter ‘R’ written with marker 1 and subsequently overpainted by marker 2. While the photograph in the visible range, Fig. 3(b), cannot resolve the two types of black markers, the single pixel image in Fig. 3(c) (λ = 1820 nm, 64 by 64 pixels, 50% compression) reveals the different absorption characteristics of the two respective markers. In contrast to thermal imaging cameras, which are integrating over a broad spectral range, hyperspectral SPI allows selective analysis of certain spectral positions at which the desired chemical information is expected.

 

Fig. 3 Two different markers applied on standard white office paper; The letter ‘R’ was overpainted with a chemically different marker illustrated in (a); Photograph of the sample (b); NIR image recorded at 1820 nm wavelength (c); The written letter ‘R’ can be clearly recognized.

Download Full Size | PDF

In order to show the benefit of exploiting the full spectral information by hyperspectral SPI in the NIR spectral range, adhesive samples on a textile (Fig. 4(c)) were measured at 60 different wavelengths between 1560 and 1835 nm. The single-pixel images with 64 by 64 pixels were recorded with 50% compression. Representative absorption spectra of each adhesive were extracted pixel-wise and plotted in Fig. 4(a), showing high quality of spectra recorded with the MEMS-FPTF (spectral resolution = 22 nm). The observed absorption bands correspond to the overtone stretching vibrations of C-H. These bands slightly differ for the used adhesives because of their particular chemical compositions. We performed PCA analysis, shown in Fig. 4(b), and fitted a logistic regression model on the previously selected pixels in order to generate a false color image. This classification results in a clear differentiation between the four adhesives, presented in Fig. 4(d). At the margins of the adhesive stains, some pixels were misclassified indicating the influence of the surface texture at the edges. In Figs. 4(e) and 4(f) the absorption images at 1765 nm and 1605 nm respectively, are shown. The pixel values of the different adhesives correspond well to the shown absorption spectra, as at 1605 nm adhesive 3 exhibits the highest absorption and at 1765 nm adhesive 2 exhibits the lowest absorption.

 

Fig. 4 Multivariate analysis and classification of reconstructed absorption images of different adhesives on a textile; (a) Representative absorbance spectra (stacked) of the four adhesive types; (b) Principle component analysis of selected pixels from each adhesive (ellipses indicate class-wise 95% confidence intervals); (c) Photograph of the sample; (d) Pixel-wise classification from a logistic regression model fitted to data shown in (b); (e) Absorption image at 1765 nm; (f) Absorption image at 1605 nm; SPI parameters: 64 by 64 pixels and 50% compression; Blue: Adhesive 1, Orange: Adhesive 2, Green: Adhesive 3, Red: Adhesive 4, Grey: Background.

Download Full Size | PDF

4. Conclusions and outlook

In summary, a broadband NIR hyperspectral single pixel imaging setup able to perform measurements up to 2200 nm using fully-integrated MEMS-based FPTF spectrometers has been developed and characterized. By means of compressive sensing, images with compression rates of up to 90 percent have been obtained. The developed setup enabled spectrally and thereby chemically resolved imaging. Experimental verification was performed by resolving an image consisting of two chemically different markers. Furthermore, PCA-based classification of recorded hyperspectral images lead to a differentiation of four different adhesives deposited on a textile. The achieved acquisition time per image was less than 4 minutes. However, as shown in [24,25] there is a significant potential for a further reduction of the acquisition time by applying an elaborated synchronization approach, which is taking advantage of the full 60 Hz frame rate and the 24 bit-planes per frame. With this approach pattern rates of up to 650 Hz could be achieved with the employed DMD [24]. Furthermore, newly available DMDs offer frame rates higher than 40 kHz enabling the execution of our approach at video rate.

In spectral regions above 1100 nm highly sensitive focal plane arrays, such as InGaAs, InSb and MCT become rather expensive. The obtained results suggest a potential low-cost replacement for such arrays using the developed SPI approach. Thereby, larger array sizes and higher pixel numbers at significantly lower costs become available. Furthermore, single element detectors typically offer a higher detectivity than array detectors of the same detector material increasing the sensitivity. MEMS-based FPTF spectrometers have been proven to be an ideal implementation for obtaining spectral resolution in a single pixel imaging system. These filter devices provide relatively large aperture sizes not reducing the FOV. Thus, they also bear great potential for miniaturized low-cost hyperspectral imaging systems operating in the MIR range [26]. This spectral range is even preferable over the NIR range as it offers more sensitive chemical response. The presented results are a significant step towards sensitive low-cost hyperspectral imaging technologies using fully integrated single-pixel spectrometers in the respective domain. Furthermore, the miniaturized and highly robust MEMS filter technology used to gain spectroscopic information can facilitate hyperspectral imaging for high volume applications.