Expand this Topic clickable element to expand a topic
Skip to content
Optica Publishing Group

Spatial scanning hyperspectral imaging combining a rotating slit with a Dove prism

Open Access Open Access

Abstract

We present a new concept for spatial scanning hyperspectral imaging. Spatial scanning is one of the main methods used for hyperspectral data acquisition and can provide high spectral resolution over a wide spectral range. However, conventional techniques, such as the whiskbroom and the pushbroom techniques, suffer from the need for relative motion between the target and the imaging system, which increases the complexity on the hardware side and limits the application possibilities. Our new approach combines a rotating slit and a co-rotating Dove prism. The rotating slit scans the target image by selecting one line from the image at each angular position of the slit. The rotating Dove prism is used to synchronously re-align the transmitted light from the selected image line with respect to the transmission grating to allow the projection of the diffracted light over the same range of pixel columns of the image sensor to facilitate data acquisition and extraction of spectral information. The new approach enables the spatial scanning of the target image without the need for relative linear motion or the use of additional external equipment and therefore opens the door for more application scenarios.

© 2019 Optical Society of America under the terms of the OSA Open Access Publishing Agreement

1. Introduction

Hyperspectral imaging (HSI) is a technology that allows acquiring a spectrum for every pixel in an image. This is achieved by combining the techniques of spectroscopy and spatial imaging into a single instrument. The acquired data is represented using two spatial dimensions and one spectral dimension, and is often referred to as the “hyperspectral cube”. The spatially-resolved spectral information can be used to reveal the chemical composition of the target in a contact-free non-destructive fashion. For this reason, hyperspectral imaging has found many applications in different fields in recent years. The medical field is one of the main areas where hyperspectral imaging has shown great potential [1], with several studies demonstrating using hyperspectral imaging in detecting cancer [2, 3], monitoring ischemia [4], detecting dental caries [5], and guided surgery [6]. Other areas of application include industrial sorting [7], agriculture [8], and security [9].

Different techniques were developed for hyperspectral data acquisition, which can be divided under the following categories:

  • Spatial scanning: this method is based on the sequential scanning of the target image to acquire the spectral information for one part of the image at a time. The spectrum is acquired by passing the light through a dispersive optical element, such as a grating or a prism, and then recording the signal using an image sensor. Two main approaches for spatial scanning have been developed. One is known as the “whiskbroom” approach, and is based on using a pinhole to scan the image in a point-by-point fashion, which requires scanning across the two spatial dimensions of the image [10]. The other is known as the “pushbroom” approach, and is based on using a slit aperture to scan the image in a line-by-line fashion, which is faster and more efficient since it requires scanning across only one spatial dimension of the image [11].
  • Spectral scanning: a tunable filter, such as a liquid crystal tunable filter, or an acousto-optic tunable filter, is placed before the illumination source or before the image sensor to acquire a set of 2D images of the target, each at a different wavelength. By stacking the images together, the hyperspectral cube can be constructed [12].
  • Spatio-spectral scanning: while spatial scanning and spectral scanning techniques are based on slicing the hyperspectral datacube along orthogonal dimensions, the spatio-spectral scanning technique is based on acquiring diagonal slices of the hyperspectral cube, where each slice contains a 2D spatial image of the target that is wavelength-encoded. This is achieved by moving a camera transverse to the slit of a basic slit-spectroscope [13].
  • Time scanning: a superposition of the spatial and the spectral data is recorded by acquiring a set of images over time. The spectral information can then be extracted by using mathematical transformations. For example, an interferometer can be used to record an interferogram of the target. Using Fourier transformation, the spectral information can be extracted from the recorded data [14].
  • Snapshot: enables acquiring the entire datacube, with all spatial and spectral information at once. One example is based on image duplication and tiled bandpass filters [15]. The target image is duplicated using a lens array. Each duplicate image is projected at the image sensor through one of the filter tiles, which are arranged in a two-dimensional matrix. By combining the resulting set of images, where each image corresponds to a different spectral band, the datacube can be constructed.

In contrast to the other methods, spatial scanning offers the possibility to acquire data with high spectral resolution over a wide spectral range. This is more challenging with spectral scanning and time scanning methods due to the demanding hardware requirements, such as having a tunable filter with high tuning precision over a wide tuning range, or acquiring interferograms over a large range of optical path differences. In snapshot methods, both spectral and spatial resolution are sacrificed for the sake of high-speed acquisition without scanning. The spatio-spectral technique requires a more complex hardware setup to enable transverse motion of the camera at a non-zero distance behind the slit-spectroscope.

However, one major limitation of the spatial scanning techniques is the need for relative motion between the imaging system and the target. The dependency on this relative motion for data acquisition goes back to the origins of the spectral imaging technology, which was developed by NASA in the 1980’s for remote sensing applications [16]. Spaceborne and airborne spectral imaging systems were developed to scan the surface of the earth. The dependency on the relative motion continued when the technology was introduced in ground-based systems. Conveyor belts and motorised stages were used to introduce the relative motion between the target and the imaging system [17]. The dependency on relative motion adds more complexity to the equipment requirements. In many cases, the relative motion is not feasible or not practical, such as endoscopy and guided surgery in the medical field. In an attempt to mitigate the need for motorized scanning stages, Yoon et al. proposed a hyperspectral endoscopy system, where the spatial scanning can be done by the freehand movement [18]. However, this does not remove the need for the relative motion and requires additional equipment for the co-registration of the wide-field image and also requires additional data processing time to remove the artefacts resulting from the freehand motion. Other examples for applications where the relative motion is problematic include environmental monitoring to detect the spread of toxic and flammable gases and liquids in air and water, and geological exploration to detect raw materials and mineral resources on site.

A number of schemes to overcome the need for relative motion were reported over the past years. Different techniques were developed, based on the pushbroom approach, to enable spatial scanning without moving the system nor the target. The common key concept behind these techniques is to scan a projected image of the target, rather than scanning the target itself directly. One technique uses a planar mirror, fixed on a high-resolution stepper motor, to reflect the target image and scan it across an imaging spectrograph [19]. Other techniques scan the image of the target at the focal plane of the front optics across the entrance slit of the spectrograph. This is done by using linear actuators to scan the front optics and the spectrograph across each other [20,21]. While the previous techniques overcome the need for relative motion between the target and the imaging system, they still achieve this by using additional external hardware to achieve the spatial scanning.

In recent years, a new paradigm is emerging, where systems are designed and developed to perform the spatial scanning of the target image internally. One technique uses a digital micromirror device to scan the target image by actuating one row of micromirrors at a time to reflect the light from one line of the image to the image sensor through a transmission grating [22]. However, switching from one row of micromirrors to the next results in a change in the incidence angle of the reflected light with respect to the grating. This leads to a change in the position of the resulting spectrum on the image sensor, which can affect the spectral calibration of the system.

In another technique, known as the “Snapscan”, an internal piezoelectric stage is used to displace the CMOS image sensor in order to scan the image circle projected by the lens. Thin-film Fabry-Perot optical bandpass filters are monolithically integrated on top of the CMOS image sensor, where each line of pixels is assigned to a different wavelength. During the line-scanning motion of the CMOS sensor, the spectral data is sequentially recorded for each spatial position of the image [23]. However, the transmittance peak of the thin-film Fabry-Perot filters is angle-dependent, which causes shifts in the acquired spectra, and therefore limits the spectral resolution of this technique.

In our previous work [24, 25], we demonstrated a pushbroom system based on an internal scanning slit and a rotating camera mechanism. This combination enabled spatial scanning without relative motion and without compromising the spectral calibration nor the imaging resolution of the system. However, this was achieved with an increased level of system complexity, which required the development of a feed-forward compensation function to synchronize the scanning motion of the slit and the rotation motion of the camera.

In this paper, we present a novel concept for a spatial scanning system that overcomes many of the prior disadvantages. We demonstrate, for the first time, a hyperspectral imaging system based on a rotating slit together with a rotating Dove prism. The rotating slit enables the spatial scanning of the target image internally without the need for relative motion and without additional external hardware. The rotating Dove prism, also called a “derotator” ensures that the light is transmitted to the diffraction grating without change in incidence angle, and therefore simplifies the data acquisition procedure. The aim of this work is to provide a simpler hardware configuration for a spatial scanning hyperspectral imaging system to open the door for more applications in the future. The system concept is explained in more detail in section 2. A comprehensive description of the experimental setup is provided in section 3. In section 4, experiments and results demonstrating the slit and the Dove prism rotation are presented, and a validation experiment is conducted where the system is used to acquire and differentiate between spectra of different plant leaves.

2. System concept

We propose a new concept for a spatial scanning hyperspectral imaging system, illustrated in Fig. 1, which combines a rotating slit with a rotating Dove prism to enable the spatial scanning of the target image internally without the need for additional external hardware. The front optics produces an intermediate image of the target at the focal plane. The rotating slit is positioned to select one line of the intermediate image. The light from the image line is collimated and transmitted through the Dove prism to the diffraction grating. The grating produces a spectrum for every point on the line, which is recorded by the image sensor. The recorded spectra represent one plane of the 3D hyperspectral datacube. In order to acquire the entire datacube, the slit is rotated to scan the target image one line at a time.

 figure: Fig. 1

Fig. 1 Schematic illustration of the system concept featuring a rotating slit and a co-rotating (at half the slit rotation angle) Dove prism.

Download Full Size | PDF

When the rotation angle of the slit and the Dove prism is zero, with respect to the vertical y-axis, the slit aperture and the selected image line are aligned with the grating. Accordingly the diffracted light for each point on the line is projected over the same range of pixel columns of the image sensor, which allows the extraction of a spectrum for every point. However, when the slit is rotated to scan the target image, the selected image line will have an angle θ with respect to the grating. This means that for every point on the image line, there will be a different transmission point on the grating along the x-axis. Accordingly, the diffracted light for every point on the image line will be projected over a different range of pixel columns of the image sensor. This does not allow the extraction of the spectral information from the recorded signal due to the overlap between the different spectral bands within the pixel columns of the image sensor. This is demonstrated experimentally in Fig. 5. Geometrically, an oriented Dove prism DP(α)is a linear optical element that contains a reflection symmetry plane positioned at angle α, and optically inverts a traversing image I about this plane, through total internal reflection. For a rotating image I(θ) incident on the Dove prism, if the reflection plane is rotated at half the rate of the image, the image is always reflected back to the same reference position θref, so that I(θref) = DP((θθref)/2) · I(θθref). Thus, when the Dove prism is rotated with half the rotation angle of the slit, the image line is rotated back and transmitted from the Dove prism with a vertical upright orientation aligned with the diffraction grating, as illustrated in Fig. 1. Therefore, for each slit position, the Dove prism can be used to rotate the selected image line back to a vertical upright position aligned with the grating to avoid the overlap of the spectral bands when the diffracted light is projected over the image sensor. This means that by using the combination of the rotating slit and the Dove prism, the target image can be spatially scanned through the slit rotation, while the Dove prism is used to cancel the rotation effect to facilitate the data acquisition and extraction of spectral information.

3. Experimental setup

In order to demonstrate the system concept, the experimental setup, shown in Fig. 2(a), is constructed. The front optics consists of a wide angle machine vision lens (3.5 mm EFL, product no. MVL4WA, Thorlabs, USA) combined with an f60 biconvex lens. Together they form an intermediate image of the target. The slit has a width of 20 μm (Thorlabs No. S20R) and is fixed into a rotating mount (Thorlabs No. CRM1P/M). The slit is positioned at the intermediate image plane to select one line of the target image. The rotating mount allows scanning the slit over 360°. However a rotation of 180° is sufficient to cover all the spatial positions of the target due to the symmetrical arrangement of the slit and the system components with respect to the optical axis of the system.

 figure: Fig. 2

Fig. 2 (a) Construction of the experimental setup showing the system components. (b) Imaging target for spatial mode with red, green, and blue lasers. (c) Imaging target for spectral mode where different number of stripes is used to highlight each angular position.

Download Full Size | PDF

The image line from the slit is collimated and magnified using a microscope objective (GF Planachromat 12.5×/0.25, Zeiss Jena, Germany). The collimated light beams then enter the Dove prism (Thorlabs No. PS992M-A) [26], which is also fixed to a 360° rotating mount. This allows rotating the Dove prism by half the angle of the slit rotation during data acquisition in order to bring the image line back to a vertical upright orientation before the light is transmitted to the diffraction grating. The grating (300 lines/mm - Thorlabs No. GT13-03) diffracts the incoming light and produces a spectrum for every point on the image line. A monochrome CCD camera (DMK 23U274, The Imaging Source, Germany) with an imaging lens (Xenoplan 2.8/50-0902, Schneider Kreuznach, Germany) is positioned at an angle corresponding to the first diffraction order to record the spectra for the selected image line.

In order to test the system concept, the imaging targets shown in Figs. 2(b) and 2(c) are used. The two images were acquired by adjusting the setup for spatial 2D imaging by removing the slit and the grating and aligning the CCD camera along the optical axis of the system, where an identical color camera (DFK 23U274, The Imaging Source, Germany) is used in place of the monochrome camera for better visibility of the targets. In Fig. 2(b), three lasers are mounted above each other and positioned with an offset to the optical axis of the setup. This arrangement allows selecting only one laser at a time to be viewed when using the rotating slit to scan through the field of view. Therefore, this target is used to simultaneously demonstrate the effect of the slit and the Dove prism rotation in the spatial imaging mode. Figure 2(c) shows the imaging target used to demonstrate the effect of the slit and Dove prism rotation in the spectral imaging mode. Five angular positions are highlighted, which are 0°, 30°, 45°, 60°, and 90°. Each angular position is marked with a number of black stripes to facilitate its recognition when selected by the rotating slit in the spectral imaging mode.

4. Results & discussion

The experimental work was divided into three parts to show how the system works in the spatial and spectral imaging modes, and to provide validation of the system. The obtained results are explained and discussed in the following sub-sections.

4.1. Spatial imaging mode

The aim of these experiments is to show how the rotating slit can be used to scan the field of view and select different points in the target image, and how the Dove prism can be used to bring the selected image line to a vertical upright orientation.

First, the slit was rotated with different angles to select each laser of the imaging target in Fig. 2(b), while keeping the Dove prism fixed without rotation. This is shown in Figs. 3(a)–3(c), where the red laser was selected with a slit rotation angle of 30°, the green laser was selected when the slit was in a horizontal orientation corresponding to a rotation angle of 90°, and the blue laser was selected with a slit rotation angle of 150°. After that, the Dove prism was rotated with half the slit angle for each slit position to bring the selected image line to a vertical upright orientation, as shown in Figs. 3(d)–3(f). Such orientation is specifically important to facilitate spectral imaging as explained in the following sub-section.

 figure: Fig. 3

Fig. 3 Experimental demonstration of the the slit and the Dove prism rotation in spatial imaging mode. (a–c) The slit is rotated to access the red, green, and blue lasers, which are located at different spatial positions as shown in Fig. 2(b), while keeping the Dove prism fixed. (d–f) By rotating the Dove prism, with half the rotation angle of the slit, the slit image can be re-oriented to the vertical upright position.

Download Full Size | PDF

Using the rotating slit to scan the field of view means that the spatial sampling will be a function of the radius along the slit from the center of the image. For each spatial point located at (x, y) from the center, the number of sampling times N(x,y) during a complete scan of the slit (rotation of 180°) will be inversely proportional to the radius r(x,y) and will range between N(x,y) = 1 for points at the perimeter of the slit rotation (r(x,y)=12×slitlength) and N(x,y) = M for the center point, where M is the number of angular slit positions required for a complete scan, assuming no overlap between slit positions. The spatial resolution, therefore, will depend on the reconstruction of the scanned image from the individual images of all slit positions. This is demonstrated in Fig. 4, where the 1951 USAF resolution test chart is used to demonstrate the spatial resolution of the system and to show the effect of the slit rotation and image reconstruction on the spatial sampling and the spatial resolution.

 figure: Fig. 4

Fig. 4 Spatial resolution of the system demonstrated in spatial imaging mode (a–c) and in a constructed image of 474 slit positions corresponding to a slit rotation of 360° (d–f). (a) Spatial mode image of the 1951 USAF resolution test chart. (b) Magnification of group 1 (red rectangle) indicates a spatial resolution of 140.31 μm, which corresponds to element 6. (d) Constructed image of the test chart by combining the images from 474 angular slit positions corresponding to a slit rotation of 360°. (e) Deterioration of spatial resolution to 250 μm, corresponding to element 1, due to image construction. Comparing the magnifications of element 2 in group −2 (green rectangle) in the upper left corner shows the difference between the spatial mode image (c) and the constructed image where the effect of the slit rotation and image construction can be observed (f). The red arrow in (d) points to a black circular spot in the center of the constructed image which is due to the slit aperture being offset from the rotation axis of the slit component.

Download Full Size | PDF

The slit used in these experiments has a width of 20 μm and a length of 3 mm (Thorlabs No. S20R). This means that the number of angular slit positions required to scan a complete image of the target is approximately 236, with a step angle of approximately 0.76°, taking into account that a rotation of 180° is sufficient due to the symmetrical arrangement of the slit and the system components with respect to the optical axis of the system.

Figures 4(a)–4(c) show the USAF test chart in spatial imaging mode while the slit was removed from the optical path. These images serve as a reference for comparison with the reconstructed image in Figs. 4(d)–4(f). The red rectangle in Fig. 4(a) is magnified in Fig. 4(b) and shows group 1 of the test chart where the lines of element 6 can still be distinguished. This indicates a spatial resolution of 140.31 μm. Figure 4(c) is a magnification of the green rectangle in the upper left corner of Fig. 4(a) and shows element 2 in group −2 of the test chart without any signs of slit scanning nor image reconstruction. In order to acquire the images in Figs. 4(d)–4(f), the slit was inserted into the system. The manual re-arrangement of the setup to insert the slit caused the slight translation in position of the test chart image in Fig. 4(d) compared to Fig. 4(a). However, this slight translation does not affect the information obtained from the figures. Due to the large number of slit positions (≈236) and the small step angle (≈0.76°) required for a full scan of the image, which are difficult to control by manual adjustment, the rotational slit mount (Thorlabs No. CRM1P/M)) was replaced with a rotational stepper motor (Thorlabs No. K10CR1) with a step angle resolution of 0.0073°. Due to a slight offset of the slit aperture from the rotation axis of the slit component with approximately 80 μm, the scanning is done by rotating the slit over 360° rather that 180° in order to minimize the area of the image which cannot be accessed by the slit rotation to the circular spot indicated by the red arrow in the reconstructed image in Fig. 4(d). This, however, demonstrates the importance of the rotational symmetry and the high precision alignment of the system components for this technique. With 360° slit rotation and with a step angle of 0.76°, 474 slit images were acquired for the reconstruction of the test chart image in Fig. 4(d). The image reconstruction was done using MATLAB by combining the acquired images in a 3D matrix and then selecting the highest intensity from the scanned images for each pixel in the lateral image. The red rectangle in the reconstructed image is magnified in Fig. 4(e) and shows group 1 of the test chart. Comparison between Figs. 4(b) and 4(e) shows a deterioration of spatial resolution after image reconstruction from 140.31 μm (element 6) to 250 μm (element 1). Figure 4(f) is a magnification of the green rectangle in the reconstructed image and shows element 2 in group −2 of the test chart, where the signs of slit scanning and image reconstruction can be observed. This is due to the reduced number of sampling times and reduced overlap between slit positions in the upper left corner of the image, compared to other parts of the image closer to the center.

4.2. Spectral imaging mode

For these experiments, the setup was re-arranged by inserting the diffraction grating into the optical path, and positioning the CCD camera with an angle corresponding to the first diffraction order. The color camera was used for better demonstration of the results. The imaging target in Fig. 2(c) was used and was illuminated by the three lasers in Fig. 2(b), in addition to a broadband light source (Thorlabs No. QTH10/M) projected through a horizontal rectangular aperture. A diffuser glass plate was used to help spread the light over the target. The resulting image, shown in Fig. 5, displayed blue, green, and red lines, in addition to a broad spectrum corresponding to the diffraction of the light transmitted from the three lasers and the broadband source.

 figure: Fig. 5

Fig. 5 Experimental demonstration of the joint slit and Dove prism rotation in spectral imaging mode using three lasers and a broadband source, where the stripes crossing the spectral lines indicate the slit angular position according to Fig. 2(c). (a–d) Rotation of the slit while the Dove prism is fixed leads to a rotation of the spectral lines of the three lasers and to loss of spectral information due to overlap of the spectral bands within the pixel columns of the sensor. (e–h) By rotating the Dove prism with half the rotation angle of the slit, the orientation of the spectral lines can be maintained in the vertical upright position. (h) At 90° rotation, the broadband light passes through the entire slit which leads to a larger vertical distribution of its spectral response.

Download Full Size | PDF

First, the slit was rotated to different angles corresponding to those highlighted on the imaging target, while fixing the Dove prism without any rotation. The result is shown in the left column of Fig. 5, where the spectral lines of the three lasers are seen to rotate as a result of the slit rotation. The rotation represents a problem for spectral imaging and for the extraction of the spectral information. This is due to the fact that, with the rotation, each pixel column of the CCD detector is recording data from different spectral bands. For example, a pixel column around the center in Figs. 5(a)–5(c) shows signals recorded from the blue, the green, and the red spectral bands, with their geometrical separation a function of the angle of rotation. In Fig. 5(d), all spectral bands overlap with each other as the band separation reduces to zero. This does not allow establishing the spectral calibration for the system where each pixel column is assigned to a unique spectral band, allowing the extraction of spectral information for each point on the image line.

In order to solve this problem, the Dove prism is used to bring the image line back to a vertical upright orientation before the light is transmitted to the diffraction grating. By rotating the Dove prism with half the slit angle for each slit position, the vertical orientation of the spectral lines of the three lasers can be maintained, as can be seen in the right column of Fig. 5. The black stripes crossing the spectral lines are used to recognize the rotation angle of the slit based on the imaging target in Fig. 2(c). In Fig. 5(h), with a slit angle of 90° and a Dove prism angle of 45°, the rectangular aperture of the broadband source is now projected to the grating in a vertical upright orientation, resulting in the broadband spectrum covering a larger area of the sensor. Maintaining the vertical orientation of the spectral bands facilitates establishing the spectral calibration of the system by assigning each pixel column to a unique spectral band within the spectral range of the system, which is limited between 400 nm and 750 nm in our setup. This can be done by using the wavelengths of the three lasers as a calibration reference, and by using the distances between the three spectral lines to interpolate the wavelength to be assigned for each pixel column. After establishing the spectral calibration, the system can be used for spectral imaging and for extracting the spectral information of the target.

Figure 6 demonstrates the spectral resolution of the system using the spectral lines of an Hg(Ar) spectral calibration lamp (product no. 6035, Newport, US). The spectrum of the Hg(Ar) lamp shows two contiguous lines in the yellow range with wavelengths of 576.96 nm and 579.06 nm. The two lines are magnified in the inset, which shows that the two lines can be distinguished, although being partially overlapped. This demonstrates that a spectral resolution close to 2 nm can be achieved with the presented system.

 figure: Fig. 6

Fig. 6 Demonstration of the spectral resolution of the system using an Hg(Ar) spectral calibration lamp. The magnification in the inset shows two spectral lines with 2.1 nm separation partially overlapping.

Download Full Size | PDF

4.3. System validation

The aim of this experiment is to provide validation for the system concept combining the rotating slit and the rotating Dove prism. This is achieved by using the system to acquire and extract spectral information from an imaging target, which is shown in Fig. 7. Eight disk samples taken from real and artificial plant leaves are fixed to the target plane and arranged in a circular array. Pairs of samples are fixed along the same angular line, so that two samples can be viewed at a time when the slit is rotated to the corresponding angle. The target is illuminated with two Quartz Tungsten-Halogen lamps (Thorlabs No. QTH10/M) with broadband emission, and using two diffuser plates to distribute the light over the samples.

 figure: Fig. 7

Fig. 7 Imaging target for the validation experiment, combining disk samples from real and artificial plant leaves.

Download Full Size | PDF

Starting with leaf samples 1 & 2, the slit and the Dove prism are kept in the vertical upright position without any rotation to allow the light to transmit directly to the diffraction grating. The diffraction image carrying the spectral information for the two leaves is recorded by the CCD camera, where the monochrome camera is used to record the light intensity in each spectral band without any filters. For the rest of the leaves, the slit is rotated with the designated angle for each pair of leaves, the Dove prism is rotated with half the slit angle, and the diffraction image for the two leaves is recorded by the camera. The rotation of the slit and the Dove prism was done manually using two identical rotational mounts (Thorlabs No. CRM1P/M).

In the next step, the spectral information for each leaf is extracted using MATLAB. This is done by plotting the light intensity recorded in each spectral band over the spectral range of the system, where the intensity was normalized with respect to a spectrum obtained from imaging a white reference. The acquired spectra for the eight leaf samples are shown in Fig. 8. By comparing the spectra in the figure, the real and artificial leaves can be readily distinguished. The spectra of leaves 2, 3, 5, and 6 are similar to each other, and also similar to the spectrum of Chlorophyll, which is a dye found in green plant leaves [27, 28]. This experiment provides validation for the system concept, and demonstrates how the system can be used in applications such as material sorting or target detection.

 figure: Fig. 8

Fig. 8 Extracted spectra for the plant leaves samples in the validation experiment.

Download Full Size | PDF

5. Conclusions

In this paper, we presented a new concept for spatial scanning hyperspectral imaging, which combines a rotating slit with a rotating Dove prism. We demonstrated how the rotating slit can be used to select one line of the target image at each angular position, and how the Dove prism, when rotated with half the rotation angle of the slit, can be used to cancel the rotation effect and project the selected image line in a vertical upright orientation, which is aligned with the grating. This prevents the pixel columns of the image sensor from recording signals from different spectral bands, which allows maintaining the spectral calibration of the system and facilitates the extraction of spectral information. We demonstrated a spatial resolution of 140.31 μm in the spatial mode and a spectral resolution close to 2 nm in the spectral mode, which is sufficient for a broad range of applications. A proof-of-concept was provided, which showed how the system can be used for material sorting.

By introducing this new concept, we add to the set of techniques developed for spatial scanning hyperspectral imaging. The new technique offers the further advantage of performing the spatial scanning internally without the need for relative motion between the entire imaging system and the target, which would otherwise require additional external hardware in conventional systems. This advantage opens the door for more applications for spatial scanning hyperspectral imaging, where the relative motion is either not feasible or not practical. Such applications include, for example, endoscopy and guided surgery in the medical field, monitoring the spread of air and water pollutants in the environmental field, and identifying raw materials and mineral resources on site in the geological field.

Using the slit rotation for scanning leads to spatial sampling being a function of the radius from the center of the image, with the number of sampling times for each point being inversely proportional to its distance from the center, due to the overlap between adjacent slit positions. In addition, scanning a full image requires rotating the slit over a large number of angular positions with a small step angle for each position (≈236 positions with ≈0.76° for 180° rotation of a slit with 20 μm width and 3 mm length). This demands high precision in rotational symmetry and alignment of the systems components, using high precision actuators to synchronize the angular positions of the slit and the Dove prism, and careful selection of the sampling routine and image reconstruction algorithm.

In our experiments, a slight offset of the slit aperture from the rotational axis of the slit component led to loss of information from a circular part around the center which could not be accessed by the rotation of the slit. Using manual actuators to synchronize the rotation of the slit and the Dove prism led to a long measurement time for a small number of data acquisition points, in the order of 4 – 5 minutes for the 4 slit positions in the validation experiment. This can be significantly improved using automated rotational motors, such as (Thorlabs K10CR1) with an angular speed of 10°/s, where it would be feasible to scan 236 slit positions in less than 30 seconds, with 30 ms of exposure time for each position. For image reconstruction, we used a simple algorithm based on selecting the maximum intensity for each pixel from the scanned slit images. The reconstructed image showed reduced spatial resolution of 250 μm in addition to slight artefacts from slit rotation and image reconstruction towards the peripheries of the image. This can be improved using a more sophisticated sampling and reconstruction algorithm that allows more overlap between slit positions and uses averaging rather than maximum intensities to improve the signal-to-noise ratio. The spectral resolution can also be improved by using a narrower slit width or a grating with a higher diffractive power.

In our future work, we plan to further develop this system. This includes using higher precision components to ensure rotational symmetry and perfect alignment of the system components. We also plan to replace the manual rotational mounts with rotational motors and to develop a software to automate and synchronize the rotation of the slit and the Dove prism to enable the acquisition of full hyperspectral 3D data sets. We will test different sampling and image reconstruction algorithms to improve the quality of the reconstructed image. Finally, we will investigate the potential of miniaturizing the system components, and will explore potential applications for which this concept would be most suited.

Funding

Ministry of Science, Research and the Arts (MWK) Baden-Württemberg (Forschungsallianz Oberrhein AZ 7533-30-20/1); German Research Foundation (DFG) (SPP 1337).

Acknowledgments

The authors acknowledge early discussions on hyperspectral imaging during the SPP project with our research partners Prof Robert Brunner from the Ernst-Abbe-Hochschule Jena, and Prof Ulrike Wallrabe from the University of Freiburg. The authors also would like to thank the KIT for providing the infrastructure which made the realization of this work possible.

References

1. G. Lu and B. Fei, “Medical hyperspectral imaging: a review,” Biomed. Opt. 19(1), 010901 (2014). [CrossRef]  

2. D. T. Dicker, J. Lerner, P. Van Belle, D. Guerry 4th, M. Herlyn, D. E. Elder, and W. S. El-Deiry, “Differentiation of normal skin and melanoma using high resolution hyperspectral imaging,” Cancer Biol. Ther. 5(8), 1033–1038 (2006). [CrossRef]   [PubMed]  

3. Z. Liu, H. Wang, and Q. Li, “Tongue tumor detection in medical hyperspectral images,” Sensors 12(1), 162–174 (2012). [CrossRef]   [PubMed]  

4. W. R. Johnson, D. W. Wilson, W. Fink, M. S. Humayun, and G. H. Bearman, “Snapshot hyperspectral imaging in ophthalmology,” Biomed. Opt. 12(1), 014036 (2007). [CrossRef]  

5. P. Usenik, M. Buermen, A. Fidler, F. Pernus, and B. Likar, “Evaluation of cross-polarized near infrared hyperspectral imaging for early detection of dental caries,” Proc. SPIE 8208, 82080G (2012). [CrossRef]  

6. S. V. Panasyuk, S. Yang, D. V. Faller, D. Ngo, R. A. Lew, J. E. Freeman, and A. E. Rogers, “Medical hyperspectral imaging to facilitate residual tumor identification during surgery,” Cancer Biol. Ther. 6(3), 439–446 (2007). [CrossRef]   [PubMed]  

7. P. Tatzer, T. Panner, M. Wolf, and G. Traxler, “Inline sorting with hyperspectral imaging in an industrial environment,” Proc. SPIE 5671, 162–174 (2005). [CrossRef]  

8. E. Bauriegel and W. Herppich, “Hyperspectral and chlorophyll fluorescence imaging for early detection of plant diseases, with special reference to Fusarium spec. infections on wheat,” Agriculture 4(1), 32–57 (2014). [CrossRef]  

9. M. P. Nelson, A. Basta, R. Patil, O. Klueva, and P. J. Treado, “Development of a handheld widefield hyperspectral imaging (HSI) sensor for standoff detection of explosive, chemical, and narcotic residues,” Proc. SPIE 8726, 872605 (2013). [CrossRef]  

10. I. Wilson and T. Cocks, “Development of the airborne reflective emissive spectrometer (ARES)-a progress report,” Presented at the 3rd EARSeL Workshop on Imaging Spectroscopy, Herrsching, Germany, 13–16 May 2003.

11. T. Weser, F. Rottensteiner, J. Willneff, and C. Fraser, “An improved pushbroom scanner model for precise georeferencing of ALOS prism imagery,” in Proceedings of ISPRS Congress Beijing 2008 (2008), pp. 723–730.

12. N. Gat, “Imaging spectroscopy using tunable filters: a review,” Proc. SPIE 4056, 50–65 (2000). [CrossRef]  

13. S. Grusche, “Basic slit spectroscope reveals three-dimensional scenes through diagonal slices of hyperspectral cubes,” Appl. Opt. 53(20), 4594–4603 (2014). [CrossRef]   [PubMed]  

14. T.-Y. Tseng, P.-J. Lai, and K.-B. Sung, “High-throughput detection of immobilized plasmonic nanoparticles by a hyperspectral imaging system based on Fourier transform spectrometry,” Opt. Express 19(2), 1291–1300 (2011). [CrossRef]   [PubMed]  

15. B. Geelen, N. Tack, and A. Lambrechts, “A snapshot multispectral imager with integrated tiled filters and optical duplication,” Proc. SPIE 8613, 861314 (2013). [CrossRef]  

16. A. F. Goetz, G. Vane, J. E. Solomon, and B. N. Rock, “Imaging spectrometry for earth remote sensing,” Science 228(4704), 1147–1153 (1985). [CrossRef]   [PubMed]  

17. A. Gowen, C. O’Donnell, P. Cullen, G. Downey, and J. Frias, “Hyperspectral imaging–an emerging process analytical tool for food quality and safety control,” Trends Food Sci. Technol. 18(12), 590–598 (2007). [CrossRef]  

18. J. Yoon, J. Joseph, D. J. Waterhouse, A. S. Luthman, G. S. Gordon, M. Di Pietro, W. Januszewicz, R. C. Fitzgerald, and S. E. Bohndiek, “A clinically translatable hyperspectral endoscopy (HySE) system for imaging the gastrointestinal tract,” Nat. Commun. 10(1), 1902 (2019). [CrossRef]   [PubMed]  

19. F. Sigernes, D. A. Lorentzen, K. Heia, and T. Svenøe, “Multipurpose spectral imager,” Appl. Opt. 39(18), 3143–3153 (2000). [CrossRef]  

20. C. Yang, J. H. Everitt, M. R. Davis, and C. Mao, “A CCD camera-based hyperspectral imaging system for stationary and airborne applications,” Geocarto International 18(2), 71–80 (2003). [CrossRef]  

21. M. A. Lanoue, “Hyperspectral/multispectral dispersive system with scanning entry slit moving across lens focus plane,” U.S. Patent Application No. 12/104,631 (2008).

22. R. Arablouei, E. Goan, S. Gensemer, and B. Kusy, “Fast and robust pushbroom hyperspectral imaging via DMD-based scanning,” Proc. SPIE 9948, 99480A (2016).

23. J. Pichette, W. Charle, and A. Lambrechts, “Fast and compact internal scanning CMOS-based hyperspectral camera: the Snapscan,” Proc. SPIE 10110, 1011014 (2017). [CrossRef]  

24. M. Abdo, E. Förster, P. Bohnert, M. Stürmer, V. Badilita, R. Brunner, U. Wallrabe, and J. G. Korvink, “Automatic correction of diffraction pattern shift in a pushbroom hyperspectral imager with a piezoelectric internal line-scanning unit,” Proc. SPIE 10110, 1011004 (2017). [CrossRef]  

25. M. Abdo, E. Förster, P. Bohnert, V. Badilita, R. Brunner, U. Wallrabe, and J. G. Korvink, “Dual-mode pushbroom hyperspectral imaging using active system components and feed-forward compensation,” Rev. Sci. Instrum. 89(8), 083113 (2018). [CrossRef]   [PubMed]  

26. Thorlabs Inc., “Mounted Dove Prisms,” https://www.thorlabs.com/newgrouppage9.cfm?objectgroup_id=6810&pn=PS992M-A. Accessed 7 June 2019.

27. C. Daughtry, C. Walthall, M. Kim, E. B. De Colstoun, and J. McMurtrey Iii, “Estimating corn leaf chlorophyll concentration from leaf and canopy reflectance,” Remote. Sens. Environ. 74(2), 229–239 (2000). [CrossRef]  

28. A. A. Gitelson, Y. Gritz, and M. N. Merzlyak, “Relationships between leaf chlorophyll content and spectral reflectance and algorithms for non-destructive chlorophyll assessment in higher plant leaves,” Plant Physiol. 160(3), 271–282 (2003). [CrossRef]  

Cited By

Optica participates in Crossref's Cited-By Linking service. Citing articles from Optica Publishing Group journals and other participating publishers are listed here.

Alert me when this article is cited.


Figures (8)

Fig. 1
Fig. 1 Schematic illustration of the system concept featuring a rotating slit and a co-rotating (at half the slit rotation angle) Dove prism.
Fig. 2
Fig. 2 (a) Construction of the experimental setup showing the system components. (b) Imaging target for spatial mode with red, green, and blue lasers. (c) Imaging target for spectral mode where different number of stripes is used to highlight each angular position.
Fig. 3
Fig. 3 Experimental demonstration of the the slit and the Dove prism rotation in spatial imaging mode. (a–c) The slit is rotated to access the red, green, and blue lasers, which are located at different spatial positions as shown in Fig. 2(b), while keeping the Dove prism fixed. (d–f) By rotating the Dove prism, with half the rotation angle of the slit, the slit image can be re-oriented to the vertical upright position.
Fig. 4
Fig. 4 Spatial resolution of the system demonstrated in spatial imaging mode (a–c) and in a constructed image of 474 slit positions corresponding to a slit rotation of 360° (d–f). (a) Spatial mode image of the 1951 USAF resolution test chart. (b) Magnification of group 1 (red rectangle) indicates a spatial resolution of 140.31 μm, which corresponds to element 6. (d) Constructed image of the test chart by combining the images from 474 angular slit positions corresponding to a slit rotation of 360°. (e) Deterioration of spatial resolution to 250 μm, corresponding to element 1, due to image construction. Comparing the magnifications of element 2 in group −2 (green rectangle) in the upper left corner shows the difference between the spatial mode image (c) and the constructed image where the effect of the slit rotation and image construction can be observed (f). The red arrow in (d) points to a black circular spot in the center of the constructed image which is due to the slit aperture being offset from the rotation axis of the slit component.
Fig. 5
Fig. 5 Experimental demonstration of the joint slit and Dove prism rotation in spectral imaging mode using three lasers and a broadband source, where the stripes crossing the spectral lines indicate the slit angular position according to Fig. 2(c). (a–d) Rotation of the slit while the Dove prism is fixed leads to a rotation of the spectral lines of the three lasers and to loss of spectral information due to overlap of the spectral bands within the pixel columns of the sensor. (e–h) By rotating the Dove prism with half the rotation angle of the slit, the orientation of the spectral lines can be maintained in the vertical upright position. (h) At 90° rotation, the broadband light passes through the entire slit which leads to a larger vertical distribution of its spectral response.
Fig. 6
Fig. 6 Demonstration of the spectral resolution of the system using an Hg(Ar) spectral calibration lamp. The magnification in the inset shows two spectral lines with 2.1 nm separation partially overlapping.
Fig. 7
Fig. 7 Imaging target for the validation experiment, combining disk samples from real and artificial plant leaves.
Fig. 8
Fig. 8 Extracted spectra for the plant leaves samples in the validation experiment.
Select as filters


Select Topics Cancel
© Copyright 2024 | Optica Publishing Group. All rights reserved, including rights for text and data mining and training of artificial technologies or similar technologies.