1. Introduction

Imaging is employed for various applications within both defense and commercial domains. Depending on the application or task, imaging modalities either assume ambient illumination or rely upon active/structured illumination. Active illumination is valuable for various applications like laser radar, digital holography, 3D imaging (e.g., depth-map extraction) and microscopy (e.g., grating-based superresolution) [1–7]. The measurement space associated with such systems is in general, not an isomorphism of the object space and computation plays an important role in performing the associated task. Recently a great deal of interest has arisen in so-called computational imagers that are optimized towards a certain system goal [8–12]. Here we are interested in a form of computational imaging that is compressive or feature-specific in nature. Feature-specific imaging (FSI) measures linear projections of the object space optically and can exploit these feature measurements for reconstruction, compression and/or recognition tasks [12–16]. The projection basis is designed such that it can extract the task-related information from the scene using fewer measurements than a conventional imager. Significant benefits in terms of improved reconstruction fidelity, improved recognition performance and improved measurement signal-to-noise-ratio have been reported in the literature [12–16].

It is possible to extend the capabilities of FSI by exploiting illumination degrees of freedom. Such a feature-specific structured imaging (FSSI) system has been described in [17]. FSSI projects spatially structured illumination patterns onto an object and collects the reflected light onto a single photo-detector resulting in feature measurements. It was demonstrated that FSSI may offer benefits in terms of reduced number of measurements and hardware complexity compared to a conventional imager [17]. This previous work on FSSI makes use of detailed object prior knowledge to define the illumination patterns. In this work we generalize FSSI by making use of random projections to define the illumination patterns. Several reports exist on the use of random projections (RP) for sparse signal/image recovery [14, 18–20]. The main appeal behind using RP is that one does not require detailed prior knowledge about the object during the measurement process. Instead, sparsity (a form of object prior knowledge) is enforced during the reconstruction process through nonlinear algorithms based on L ₁-norm minimization and/or gradient-projection [18–20]. The benefits of the present work over the passive compressive imaging architecture reported in [14] are twofold: (1) the active modality enables imaging with zero ambient light levels, and (2) the active modality reduces the complexity of the light-collection hardware. The advantage of this work compared to our previous FSSI report [17] is that the measurement mechanism uses a RP basis and so does not depend on the object. We note however that the reconstruction fidelity is dependent on the incoherence between the RP basis and the basis in which the object is sparse [18, 19].

The remainder of our paper is organized as follows. In section 2 we describe the RFSSI system. Section 3 presents the experimental setup along with results. Section 4 offers concluding remarks.

2. Imager system analysis

Figure 1 depicts the RFSSI system. It consists of 1) an illuminator to project a sequence of spatially structured intensity patterns onto a two-dimensional object, 2) light collection optics focussing the reflected light onto a single photodetector and 3) a post-processing algorithm to generate an object estimate from the sequence of measurements. We assume that 1) the illuminator resolution is matched to that of the object i.e. it provides resolution at least equal to the object pixel size, and 2) the illuminator and the light collection optics share a common optical axis. The object reflectance function is of dimension on N×N and is lexicographically ordered into a N ₂-dimensional vector G. All two-dimensional quantities will be represented in this same manner throughout the paper. The illumination pattern is considered to be spatially incoherent and is represented by the vector P. Because the illuminator and light collection optics are assumed to be in a mono-static configuration the reflected irradiance pattern from the object will be a pointwise product between P and G. This reflected light is focused onto a single photodetector resulting in the feature measurement given by

The feature measurement R in Eq. (1) is simply an inner product of P and G. We now generalize the measurement model in Eq. (1) by considering a sequence of K projections. The resulting K -dimensional measurement vector R is given by

where P̿ represents the K×N ² measurement matrix, P̿=[P ₁,P ₂……P _K]^T where P _i is the i^th projection vector of dimension N ²×1.

 

Fig. 1. System diagram for the RFSSI approach.

Download Full Size | PDF

Before describing the postprocessing algorithm it is important to discuss the assumptions related to object sparsity. The object reflectance vector G can be represented as a N ²- dimensional real-valued vector α in an orthonormal basis V̿ (of dimension N ²×N ²) so that G=V̿α. We will refer to V̿ as the representation matrix and it can be expressed as ${[{\mathbf{V}}_1,{\mathbf{V}}_2......{\mathbf{V}}_{N^2}]}^T$ where ${\mathbf{V}}_i^T{\mathbf{V}}_j=1$ for i=j and equal to 0 otherwise. By sparsity of the object reflectance G we mean that the number of non-zero elements (M) in the vector α is much smaller than its dimensionality i.e. M≪N ². We will refer to such an object as “explicitly-sparse.” Consider the example in Fig. 2(a) where we show two different 32×32 binary sparse objects with M=160 non-zero values. The representation basis V̿ in this case is simply an identity matrix so that α=G and therefore these objects are “explicitly-sparse” in the space domain. However, there are scenarios where the coefficient vector α is not explicitly sparse but still compressible. This implies that α has only M≪N ² elements larger than ε in magnitude, where ε≪max(|α|). We will refer to such objects as “mostly-sparse.” Consider the example in Fig. 2(b) where we show two different 32×32 gray-scale truck objects (reflectance normalized to 1) that are “mostly-sparse” in a wavelet basis. The wavelet basis V̿ _DW is created using the Daubechies’ wavelet filters of length equal to 4 with number of levels equal to 7 [21]. Figure 2(c) plots the coefficient vector α representing the leftmost object from Fig. 2(b) in the basis V̿ _DW. Note from Fig. 2(c) that max(|α|)=7.09 and the number of elements greater than ε=0.18 is equal to 250. We create a truncated coefficient vector α _trunc by retaining these dominant 250 coefficients of α at their respective positions and equating the remaining elements to zero. The truncated coefficient vector α ^trunc is transformed back to the object space to form G _trunc=V̿ _DW α _trunc which is shown in Fig. 2(d). Note that the visual quality remains good when retaining only 250 coefficients of the wavelet expansion. We can observe that the object in Fig. 2(d) is now explicitly sparse in the wavelet domain.

Next, we consider the design of the measurement matrix and the associated postprocessing required for the RFSSI system discussed above. The postprocessing algorithm is to be designed such that it yields an estimate (Ĝ) of the object reflectance (G). The reconstruction quality is evaluated using the root-mean-squared-error (RMSE) criterion:

where ${\parallel \mathbf{x}\parallel }_{L_2}$ denotes the L ₂-norm of the real-valued vector x and is given by ${\left({\sum }_{i=1}^{N^2}{\mid {\mathbf{x}}_i\mid }^2\right)}^{\frac{1}{2}}$ . Here we are interested in the case where the number of measurements K≪N ² and therefore solving for Ĝ becomes an underdetermined problem. We use the result by Candes et al. in [18] where it is shown that if (1) G is M-sparse, (2) the elements of P̿ are independently chosen from the symmetric Bernoulli distribution Pr(P̿ _ij=0 or 1)=1/2 and, (3) K $\widetilde{>}$ M·log(N ²/M) then G can be reconstructed exactly with high probability by solving the convex program

 

Fig. 2. (a) Two example 32×32 binary sparse objects with M=160. (b) Two example 32×32 gray-scale truck objects. (c) Plot of coefficient vector α representing leftmost object from (b) in wavelet basis V̿ _DW. (d) Estimated object reflectance G _trunc obtained by retaining the dominant 250 coefficients from (c).

Download Full Size | PDF

Note that ${\parallel \mathbf{x}\parallel }_{L_1}$ denotes the L ₁-norm of the real-valued vector x and is given by ${\sum }_{i=1}^{N^2}\mid {\mathbf{x}}_i\mid$ . We will refer to this approach as L1. Candes’ result essentially says that the measurement process need not rely on detailed object prior knowledge. The “cost” of our ignorance will be the need to take roughly log(N ²/M) times more measurements as compared to the case in which object prior knowledge (i.e., the specific directions containing maximum object information) is available during the measurement. Next, we will present simulation results to demonstrate the operation of the reconstruction algorithm in Eq. (4). Figure 3(a) shows a 32×32 binary object reflectance pattern with M=160 non-zero elements. Therefore, from the above mentioned result of Candes, we would need K≥log(N ²/M)=300 measurements to obtain RMSE=0. We use L1 to reconstruct the reflectance estimate and fig. 3(b) shows the estimate Ĝ for K=350 and K=450. The solid curve with ‘circles’ in Fig. 3(e) shows the resulting reconstruction RMSE, for the binary object in Fig. 3(a), as a function of K. The RMSE trend with respect to K is monotonically decreasing eventually going to zero. Note that the resulting RMSE values for K=350 and K=450 are 0.133 and 0 respectively. Figure 3(c) shows a 32×32 gray-scale object that is “mostly-sparse” in the wavelet basis V̿ _DW. Although this object is not explicitly sparse in the V̿ _DW basis we can still apply the L1 algorithm because of its “mostly-sparse” nature. Figure 3(d) shows the estimate Ĝ for K=350 and K=450. Note that the resulting RMSE values for K=350 and K=450 are 0.19 and 0.164 respectively. The solid curve with ‘squares’ in Fig. 3(e) represents the reconstruction RMSE for this “mostly-sparse” object.

Note that the reconstruction is not exact because the object is not explicitly-sparse as required by the Candes’ result in Eq. (4).

We will now extend the model in Eq. (2) to the case where our observations are noisy. Consider the new measurement model for RFSSI given by

where n represents the K-dimensional error/perturbation vector. We will assume that this perturbation term has bounded energy i.e. ${\parallel \mathbf{n}\parallel }_{L_2}\le \sigma$ . For example, if each element of n is a zero-mean white Gaussian (WG) random variable with variance σ ² _o then σ=σ _o √K. Candes et al. extends the L1 approach in [19] by incorporating knowledge about the noise. The estimate Ĝ can now be obtained by solving the modified convex program

 

Fig. 3. (a) Example of 32×32 binary sparse object with M=160. (b) L1 reconstructed estimate of the object in (a) for K=350 and K=450 (Simulation data). (c) Example of a 32×32 gray-scale truck object that is “mostly-sparse” in the basis V̿ _DW. (d) L1 reconstructed estimate of the object in (d) for K=350 and K=450 (Simulation data). (e) Plot of RMSE versus K resulting from applying L1 to the random projection measurements (curves with ‘circles’ and ‘squares’ represent the objects in figs. 4(a) and 4(c) respectively).

Download Full Size | PDF

Candes et al. solve this convex program by recasting it as a second order cone program (SOCP) [19]. Recently Figueiredo et al. has proposed a gradient-projection based approach to solve a slight variant of the problem in Eq. (6) [20]. This approach attempts to minimize the weighted sum of the data fidelity term ${\parallel \stackrel{=}{\mathbf{P}}\hat{\mathbf{G}}-\mathbf{R}\parallel }_{L_2}$ and the L ₁-norm ${\Vert {\stackrel{̿}{\mathbf{V}}}^T\hat{\mathbf{G}}\Vert }_{L_1}$ under no constraints. Figueiredo’s approach is shown to be computationally faster than SOCP in [20] with no sacrifice in the reconstruction quality. This method is referred to as gradient projection for sparse reconstruction (GPSR) in [20]. We make use of the MATLAB routine for GPSR publicly available at www.lx.it.pt/~mtf/GPSR. We now consider a simulation example based on the use of GPSR. The error term n in Eq. (5) will be modeled as additive WG noise with σ_o=2 which is equivalent to 7% of the dynamic range of the corresponding error-free measurement in Eq. (2). The measurement matrix P̿ is created using random binary patterns as explained above. Figure 4(a) shows the estimates Ĝ of the object in fig. 3(a) for K=450 and K=600. The resulting RMSE for this object is plotted in Fig. 4(b). Observe that RMSE is monotonically decreasing with respect to K. As opposed to the trend in Fig. 3(c) RMSE does not converge to zero due to measurement noise. Using the knowledge that M=160 we retain the dominant 160 elements of the estimates Ĝ in Fig. 4(a) resulting in the modified estimates shown in Fig. 4(c). This approach is however not optimal and therefore, we did not compare it with the GPSR method.

 

Fig. 4. Simulation example: (a) GPSR reconstructed estimate of the object in fig. 3(a), with noise (σ_o=2) added to the projection measurements), for K=450 and K=600. (b) Plot of RMSE versus K resulting from applying GPSR to the noisy random projection measurements (σ_o=2). (c) Estimates obtained by retaining only M=160 dominant values of the object estimates in fig. 4(a).

Download Full Size | PDF

3. Experimental setup

Figure 5 depicts the experimental setup of the RFSSI system. The illuminator used for the experiment is an Epson PowerLite S1+ projector. It is controlled by a computer that provides a sequence of random binary-valued images. The light collection optics is implemented using a f/1 biconvex lens with aperture size equal to 25.4 mm. The photodetector used in the experiment is a Newport 818-SL connected to a Newport 1830-C power meter. The integration time used for each feature measurement is equal to 250 ms. Both the illuminator and light collection optics are placed at a distance of 95 cm from the 2D object. The 2D object has 32×32 pixels with scene dimensions equal to 18 cm by 18 cm, and is implemented through a poster-board image. Therefore, the scene defines a 10.7° field of view as seen from the detector. The physical pixel size of the object is 5.6 mm and is larger than the projected pixel size which is roughly 3 mm.

Before describing the calibration procedure we discuss the kinds of sparse objects (of dimension 32×32; N ²=1024) being used for testing purposes. For experimental validation of the proposed RFSSI system we consider three kinds of objects: 1) Binary objects (“explicitly-sparse” in the space domain) with M=160, 2) Gray-scale truck objects “explicitly-sparse” in a principal component (PC) basis with M=160, and 3) Gray-scale truck objects “mostly-sparse” in a PC or wavelet basis. We have already explained the first and the third kind of objects in Section 2. Here we describe the procedure of generating objects that are “explicitly-sparse” in a PC basis. The PC basis is determined by the statistics of an object ensemble [21]. For a set of objects S, the PC projections are defined as the eigenvectors of the object autocorrelation matrix R_S=E{GG ^T}, where G belongs to S and E{·} denotes the expectation operator. Here we consider 4000 truck objects to define S and subsequently generate the PC basis V̿ _PC. Each object reflectance G will therefore be represented as a N ²- dimensional real-valued vector α in the PC basis V̿ _PC (of dimension N ²×N ²) i.e. G=V̿ _PC α. The “explicitly-sparse” objects (with M=160) in the PC basis can thus be generated by retaining only the largest 160 coefficients of |α|. Figure 6(a) shows two example truck objects used for generating R_s and fig. 6(b) shows their respective “explicitly-sparse” (with M=160) versions.

 

Fig. 5. Experimental setup for the RFSSI system.

Download Full Size | PDF

 

Fig. 6. (a) Two of 4000 example truck objects used to generate a PC basis. (b) Explicitly-sparse in the PC basis (with M=160) versions of the objects in (a).

Download Full Size | PDF

Next, we describe the calibration procedure used in our experiment. The idea is to perform curve-fitting between the theoretical and experimental feature measurements for a set of five known training objects. Figure 7(a) shows the five training objects. The number of measurements K for each training object is taken to be 600 i.e. the measurement matrix P̿ is of dimension 600×1024. The K-dimensional theoretical measurement vector is given by R in Eq. (2). We will denote the K-dimensional experimental measurement vector as R _exp. Figure 7 (b) plots the first 100 entries of R and R _exp corresponding to the leftmost object in Fig. 7(a).

We fit an affine-linear model given by R _calib=a·R _exp+b such that the root-least-squares error $\mathrm{RLSE}=\left(\frac{1}{K}\right)·{\parallel {\mathbf{R}}_{\mathrm{calib}}-\mathbf{R}\parallel }_{L_2}$ for each training object is minimized. The values for a and b determined using the training data are 0.93 and -157.7 respectively. The resulting residual error after fitting the model is found to be RLSE≈2. This residual error is equivalent to 7% of the dynamic range of the theoretical measurement vector R, and can be modeled as the perturbation term defined in Eq. (5) because it is bounded. Various factors that could be contributing to this error are detector noise or read-out noise, illuminator temperature variations and improper reflection model. Figure 7(c) plots the first 100 entries of R and R _calib corresponding to the leftmost object in Fig. 7(a).

Now we will use the calibration parameters obtained above to generate experimental reconstructions for several test objects. The steps involved in experimental reconstruction are: 1) measure the readings associated with K (equal to 600 for this study) illumination patterns defined by P̿ and store them in vector R _exp, 2) transform R _exp into R _calib by using the calibration parameters derived above and, 3) use the GPSR approach to generate the object estimate Ĝ from R _calib. Figure 8(a) shows two binary test objects along with their respective estimates obtained from the experiment. Figure 8(b) shows two explicitly-sparse objects in the V̿ _PC basis along with their respective estimates. Figure 8(c) shows two “mostly-sparse” truck objects in the V̿ _PC basis along with their respective estimates. Figure 8(d) shows two “mostly-sparse” truck objects in the V̿ _DW basis along with their respective estimates. These experimental reconstructions demonstrate the feasibility of RFSSI by using only 600 measurements i.e. 58% of the object dimensionality. The experimental RMSE, obtained by averaging over the test objects in fig. 8, for K=600 (or equivalently 58%) is equal to 0.17. Figure 9 shows the RMSE performance, averaged over the test objects in fig. 8, as a function of the number of feature measurements K. The curve with circles represents theory and the curve with squares represents our experiment performance. The data in fig. 9 shows excellent agreement between experiment and simulation.

 

Fig. 7. (a) Five training objects used for calibration. (b) Plot of first 100 entries of R and R _exp corresponding to the leftmost object in (a). (c) Plot of first 100 entries of R and R _calib corresponding to the leftmost object in (a).

Download Full Size | PDF

 

Fig. 8. Experimental results: (a) Two binary objects along with their respective estimates. (b) Two explicitly-sparse objects in V̿ _PC along with their estimates. (c) Two mostly-sparse truck objects in V̿ _PC along with their estimates. d) Two mostly-sparse truck objects in V̿ _DW along with their respective estimates.

Download Full Size | PDF

Lastly we will compare the RFSSI and FSSI [17] systems in terms of the number of measurements required to achieve a specified RMSE. The number of measurements required by RFSSI and FSSI to achieve a specified RMSE will be denoted by K_RFSSI and K_FSSI respectively. As noted earlier in Section 1, FSSI makes use of substantial object prior knowledge (i.e. object autocorrelation matrix R_S and PC basis V̿ _PC) during the measurement process. From [17] we find that the reconstruction RMSE for FSSI is lower-bounded by

where Tr{·} denotes the trace operator, P̿ _PC denotes the measurement matrix (of dimension K_FSSI×N ²) and [I] denotes an identity matrix of dimension K_FSSI×K_FSSI. The measurement matrix P̿ _PC is created by retaining the K_FSSI dominant eigenvectors from V̿ _PC. We will denote the experimental RMSE performance of the RFSSI system, averaged over the gray-scale test objects, as RMSE_RFSSI. To ensure a fair comparison between RFSSI and FSSI, we make sure that both systems use the same illumination energy for each pattern. This is done by enforcing ${\sum }_{j=1}^{N^2}\mid {[{\stackrel{=}{\mathbf{P}}}_{\mathrm{PC}}]}_{\mathrm{ij}}\mid ={\sum }_{j=1}^{N^2}\mid {[\stackrel{=}{\mathbf{P}}]}_{\mathrm{ij}}]$ for i =1…N ². Table 1 compares K_RFSSI and K_FSSI for several values of RMSE. It can be seen that K_FSSI≪K_RFSSI as expected.

 

Fig. 9. RMSE comparison between theory and experiment for RFSSI system.

Download Full Size | PDF

Table 1. Comparison of K_RFSSI and K_FSSI for several values of RMSE (σ_o=2)

View Table

4. Conclusions

We have presented a feature-specific structured imaging system that uses binary random projections to define the illumination patterns. The feature measurements represent the inner products of the illumination patterns with the object reflectance and are measured optically onto a single photodetector. The object reflectance estimates were generated by using nonlinear algorithms based on L ₁-norm minimization and/or gradient-projection sparse reconstruction. We consider two kinds of “explicitly-sparse” and “mostly-sparse” objects for testing the RFSSI system. We have presented an experimental validation of the RFSSI approach. The experimental reconstructions utilize only 600 measurements i.e. 58% of the object dimensionality.