Open Access
Add to BibSonomy
Abstract
Computational ghost imaging (CGI) can reconstruct the pixelated image of a target without lenses and image sensors. In almost all spatial CGI systems using various patterns reported in the past, people often only focus on the distribution of patterns in the spatial dimension but ignore the possibility of encoding in the time dimension or even the space-time dimension. Although the random illumination pattern in CGI always brings some inevitable background noise to the recovered image, it has considerable advantages in optical encryption, authentication, and watermarking technologies. In this paper, we focus on stimulating the potential of random lighting patterns in the space-time dimension for embedding large amounts of information. Inspired by binary CGI and second-order correlation operations, we design two novel generation schemes of pseudo-random patterns for information embedding that are suitable for different scenarios. Specifically, we embed a total of 10,000 ghost images (64 × 64 pixels) of the designed Hadamard-matrix-based data container patterns in the framework of CGI, and these ghost images can be quantitatively decoded to two 8-bit standard grayscale images, with a total data volume of 1, 280, 000 bits. Our scheme has good noise resistance and a low symbol error rate. One can design the number of lighting patterns and the information capacity of the design patterns according to the trade-off between accuracy and efficiency. Our scheme, therefore, paves the way for CGI using random lighting patterns to embed large amounts of information and provides new insights into CGI-based encryption, authentication, and watermarking technologies.
© 2020 Optical Society of America under the terms of the OSA Open Access Publishing Agreement
1. Introduction
Computational ghost imaging (CGI) [1], also known as single-pixel imaging [2,3], constructs a novel imaging framework in which neither a lens group nor an image sensor is needed. Therefore, these interesting features have opened up new paths for imaging in some unconventional wavebands [4–9] where lenses and image sensors are very expensive or even unavailable at all.
As a trade-off in spatial resolution, the CGI system often sacrifices temporal resolution as a cost; so it generally requires thousands of different lighting patterns in the spatial dimension to sequentially illuminate the target and a single-pixel bucket detector (BD) to record back-scattered (or transmitted) light synchronously. A second-order intensity correlation operation is then used to reconstruct the pixelated image of the target. Actually, CGI can be viewed as a space-to-time transformation or encoding using a spatiotemporal fluctuating illumination, in which the intensity fluctuation of each point or pixel in the time dimension is statistically independent. Recently, the principle of CGI has been expanded to three dimensions [10–12] or other domains [13–15] as well.
Instead of the lens group and the image sensor in classic photography, the two central feature units of CGI are structured illumination and the BD. Typically, a spatial light modulator (SLM) or a digital micromirror device (DMD) is utilized to modulate the wavefront to produce structured illumination, the latter being preferred for it can generate binary patterns at high speed (22kHz). Besides, some works have achieved ultra-fast frame-rate through LED array [16] or frequency-time-division multiplexing technology [17].
Since people get access to pre-programming lighting patterns, the design of lighting patterns for CGI has been a valuable research hotspot. When realizing that the orthogonality of the pattern directly affects the signal-to-noise ratio (SNR) of CGI [18], people focus their attention on a series of orthogonal basis patterns, such as Hadamard basis [19–21], Fourier/sinusoidal basis [22,23], wavelet basis [24–27], etc. In addition to improving imaging quality, some works have achieved dynamic sampling through non-uniformly distributed illumination patterns [28,29], thereby improving imaging efficiency. On the other hand, some patterns have better anti-scattering ability [27], and some have a greater depth of field [30,31], and one can choose carefully engineered lighting patterns according to their needs.
Although random illumination patterns bring more noise to the reconstructed image and require large memory for its storage, they have a natural advantage in optical encryption [32–42] and watermarking [43–49] schemes that others do not have. Since the spatial distribution of the illumination patterns of other modes is determined, to increase the channel capacity, more degrees of freedom must be considered, such as time and spectrum [47]. It is noted that CGI can complete high-quality image transmission under atmospheric turbulence [50], strongly scattering environment [51], and even extreme magnetic field environment [52], so it is valuable to design the secure communication system and watermarking scheme based on the principle of CGI to fill the gaps in high-quality communications in some extreme situations.
In almost all spatial CGI systems using various patterns reported in the past, people often only focus on the distribution of patterns in the spatial dimension but ignore the possibility of encoding in the time dimension or even the space-time dimension. From our perspectives, random illumination patterns in space-time dimension have great potential in information embedding, and encoding pseudo-random patterns essentially belong to a space-division multiplexing technique [13,53], which can therefore embed data that increases as spatial pixels increase. In this paper, we propose two novel schemes to generate pseudo-random patterns for large-capacity information embedding and efficient decoding. While retrieving the image of the real scene in CGI system, users can also extract information in the lighting patterns from the receiving end. The core idea of this paper is that each pixel of the pseudo-random lighting patterns in the spatial dimension is multiplexed to be a set of binary bucket signal sequence in the time dimension, and this bucket signal array can obtain the decrypted information with the second-order correlation operation of the external measurement matrix (first scheme), and can even obtain additional information by mutually second-order correlation with the internal part of the pseudo-random patterns themselves (second scheme). Through a space-division multiplexing technique, the two schemes can be applied simultaneously to a set of coded pseudo-random lighting patterns.
In order to quantify the amount of information in ghost images, we have also designed a data container pattern based on the Hadamard matrix, which utilizes the orthogonality of the Hadamard matrix to efficiently and accurately decode. As a result, we embed a total of 10,000 ghost images of the data container patterns in the framework of CGI using $100\times 100$-pixel pseudo-random binary lighting patterns. These ghost images can be decoded to obtain two 8-bit standard grayscale images, with a total data volume of $1,280,000$ bits. One can use the trade-off between accuracy and sampling rate to reasonably design the number of pseudo-random lighting patterns and the information capacity of a single pattern. Our scheme has good anti-noise performance, allowing global image information to be retrieved when some signals are unavailable. We believe our scheme paves the way for CGI using random lighting patterns to embed large amounts of information and provides new insights into CGI-based encryption, authentication, and watermarking technologies. Here we open source the simulation process of the pseudo-random lighting pattern generation process (See Supplement 1 for the m code).
2. Encoding and decoding schemes
2.1 Principle of CGI
For the convenience of description, we introduce the mathematical principle of CGI from the perspective of information theory. CGI requires $N$ different patterns of $m \times n$ pixels to illuminate the target and a single-pixel BD to record total light intensity signals of backscattered or transmitted light. Since the resolution of the illumination pattern determines the resolution of the imaging, the size of the target is the same as that of the illumination pattern. Here we resize the target into an $M$-dimensional column vector $\mathbf {o}$ ($M = m \times n$), each lighting pattern is resized into an $M$-dimensional row vector $\mathbf {r_i}$ ($i=1,2,\ldots ,N$), and $N$ lighting patterns are combined into an $N$-by-$M$ measurement matrix $\mathbf {R}$. The measurement matrix $\mathbf {R}$ represents the spatiotemporal fluctuation of the preset light field in CGI system, where each row of $\mathbf {R}$ represents the fluctuation of the spatial dimension at each moment, and each column represents the fluctuation of a single pixel in the time dimension. Therefore, the light intensity sequence collected by the BD can be represented by an $N$-dimensional column vector $\mathbf {b}$, and the corresponding physical process can be represented by the following equation:
A second-order intensity correlation algorithm is harnessed to retrieve the ghost image $\hat {\textbf {o}}$ of the target:
where $\langle {\cdots }\rangle$ denotes ensemble average, and $\mathbf {R}^{T}$ represents the transpose of $\mathbf {R}$. Interestingly, L. A. Wu et al. proposed a positive and negative GI strategy in 2012 [54], in which they set a threshold for the bucket signal sequence $\mathbf {b}$, and the average of the illumination patterns corresponding to the signals greater than the threshold will produce a positive ghost image, and vice versa. This is mathematically equivalent to the binarization of the bucket signal:2.2 Design of data container based on Hadamard matrix
For the first optical encryption scheme based on CGI [32], $\mathbf {R}$ is set as the key to retrieve the ghost image $\hat {\textbf {o}}$ according to Eq. (2). Here we adopt a similar idea, while the difference is that the binary measurement matrix $\textbf {R}$ is not only used as a key, but also as a set of bucket signals. Each binary bucket signal can retrieve a different ghost image according to Eq. (2), meaning that we can hide large-scale image information in the $\textbf {R}$, that is, the number of pixels in the illumination pattern determines the number of image subsets.
Sui et al. introduced a customized data container [42] to implement CGI-based information encryption, where an average of $8\times 8$ pixels represents a binary number (1 bit). If we follow their coding scheme, then a single ghost image ($64\times 64$ pixels) can represent a total of $64$ binary numbers (64 bits). However, this scheme is not efficient without considering the symbol error rate (SER) of the decoding process. Here we propose a data container based on the Hadamard matrix, whose capacity is 2 times that of Sui et al.’s scheme. In other words, we can embed 524,288 bits of information in a set of $64$-by-$64$ pseudo-random lighting patterns.
The Hadamard matrix is an orthogonal matrix composed of only 1 and −1, and its order can only be a power of 2. For a $16\times 16$ Hadamard matrix $\mathbf {h}$, we can use the following equation to generate 256 binary $16\times 16$ patterns that are orthogonal to each other:
where $\mathbf {h}(:,i)$ and $\mathbf {h}(j,:)$ respectively represent the $i$-th column and the $j$-th row of the $\mathbf {h}$, and the decimal number of $p$ ranges from 0 to 255. As shown in Fig. 1(a), each pattern corresponds to a unique decimal number $p$, the user can set the corresponding relationship from $p$ to the pattern. As shown in Fig. 1(b), we resize each $\mathbf {H}_p$ into a $256$-dimensional row vector and put together a $256$-by-$256$ matrix $\mathbf {H}$. Figure 1(c) shows the orthogonality map of $\mathbf {H}$, namely $\mathbf {H}\mathbf {H}^{T}$. The orthogonality between the patterns can greatly reduce the possibility of "cross-talk", thereby improving the accuracy of decoding or recognition. Specifically, we divide a $64$-by-$64$ area into $4\times 4$ cells, where each cell consists of a pattern corresponding to a decimal number. As shown in Fig. 2, each pattern represents 16 decimal numbers (128 bits), and users can encode information into 8-bit pixels or Ascii codes in a custom way.Next, we will introduce two different CGI-based encoding schemes to generate random binary measurement matrix $\mathbf {R}$ using the Hadamard-matrix-based data container, and at the same time, the two encoding schemes can be combined through a space-division multiplexing technique. As shown in Fig. 3, the blue Region1 (4096 pixels) is used for the first information embedding scheme (Section 2.3), and the red Region2 (5904 pixels) is used for the second information embedding scheme (Section 2.4). Therefore, after a certain coordinate mapping transformation according to the mask of the space-division multiplexing technique, $\mathbf {R}$ can be divided into two matrices, $\mathbf {R_1}$($4096\times 10000$) and $\mathbf {R_2}$($5904\times 10000$) as follow:
Note that users can customize the area division of the mask (including shape and proportion), or even random division, then this can be used as an additional key. Here we have chosen a simple division scheme in order to conveniently describe our scheme.
Fig. 1. Coded pattern based on Hadamard matrix. (a)The coded pattern of a decimal number from 0 to 255. (b) Hadamard matrix $\mathbf {H}$. (c) The orthogonality map $\mathbf {H}\mathbf {H}^{T}$.
2.3 Coded pseudo-random patterns using external measurement matrix
According to the mask, we demonstrate this scheme in the central area $\mathbf {R_1}$ of $64\times 64$ pixels, and these 4096 binary bucket signals can get a total of 4096 ghost images, so this scheme requires an additional random $N$-by-$M$ measurement matrix $\mathbf {R_3}$, which can be distributed as an additional key to a trusted third party to retrieve deep information or to verify the correctness (authentication) of the original measurement matrix $\mathbf {R}$. As shown in Fig. 4(a), we encode the information of a $256\times 256$ grayscale standard image into $4096$ data container patterns ($64\times 64$ pixels) in order of column (each 16 decimal numbers are encoded into one pattern). Here we resize every pattern to a 4096-dimensional column vector $\mathbf {d}_i$. As shown in Fig. 4(b), the BCGI code obtained by each column vector constitutes a certain column vector of $\mathbf {R}_1(:,i)$ as follows:
where the function $\mathrm {Binary()}$ means to binarize each column of the input matrix separately according to Eq. (3), and the function $\mathrm {Circshift}(\mathbf {d}_i,i)$ dramatically increases the statistical independence among every pixel by circularly shifting the input vector $\mathbf {d}_i$ $i$ units to the bottom. It should be noted that binary operations will inevitably bring about the loss of information. In this case, GI appears to be less sensitive to the binary operation in the bucket signal [54,55]. Considering the data container pattern used in this paper is a simple binary target, the information itself can be regarded as partially redundant, so the effect of the binary operations might not be very influential. In general, we need an $N$-by-$M$ measurement matrix $\mathbf {R_3}$ and 4096 $N$-dimensional bucket signals (i.e., $\mathbf {R_1}$) to form 4096 ghost images. Hence, the first scheme essentially does not improve the efficiency of information utilization but uses a strategy of spatial multiplexing to make it possible to encode large-scale data into the random and binary lighting patterns.Fig. 4. Overview of generation of the coded pseudo-random patterns using external measurement matrix. (a) The process of encoding 8-bit standard images with a series of data container patterns. (b) The generation process of matrix $\mathbf {R_1}$ using $\mathbf {R_3}$.
2.4 Mutually-correlated pseudo-random patterns
In the previous subsection we have discussed an encoding scheme that requires an external reference matrix or an additional key $\mathbf {R_3}$. In this subsection we propose a method for generating a mutually-correlated random measurement matrix $\mathbf {R_2}$ that does not require an external reference. According to the mask shown in Fig. 3, we select the red area $\mathbf {R_2}$ to demonstrate this scheme.
As shown in Fig. 5(a), we encode another $8$-bit standard image ($246\times 384$ pixels, 755,712 bits) into 5904 patterns ($64\times 64$ pixels) according to the coding strategy shown in Fig. 1. The $\mathbf {R_1}$ generated by Eq. (6) is used as the measurement matrix in this scheme, while the other areas in timescale are encoded to BCGI codes corresponding to the data container patterns. Therefore, this scheme does not require an external reference matrix, but only needs to allocate the measurement matrix $\mathbf {R}$ reasonably: partly as an internal reference matrix in the spatial dimension, partly as a bucket signal array in the time dimension, and the mutual second-order correlation between the two regions can retrieve a large amount of ghost images. Similar to Eq. (6), we pass each data container pattern through the measurement matrix $\mathbf {R_1}$ to obtain a BCGI code $\mathbf {e}_i$ shown in Fig. 5(b), and then each BCGI code forms a column of matrix $\mathbf {R_2}$ as follows:
It should be noted that the fluctuation of each pixel in the time dimension needs to satisfy a Gaussian distribution in CGI modality, and the intensity fluctuations of pixels are statistically independent of each other. The engineered matrix in this scheme is pseudo-random. If the $\mathrm {Circshift()}$ function is not used, it will lead to the possibility of a statistical correlation among the pixels, which might lead to a decline in imaging quality. Of course, one can also design other simplified functions to increase statistical independence among pixels.Fig. 5. Overview of generation of the mutually-correlated pseudo-random patterns. (a) The process of encoding 8-bit standard images with a series of data container patterns. (b) The generation process of matrix $\mathbf {R_2}$ using $\mathbf {R_1}$.
2.5 Experiment
As shown in Fig. 6(a), we finally integrate the generated $\mathbf {R_1}$ and $\mathbf {R_2}$ into a measurement matrix $\mathbf {R}$ according to the mask of the space-division multiplexing technique. In this way, we have finished coding the random lighting pattern, and we have embedded a total of 1,280,000 bits of data through two schemes above.
Fig. 6. (a) Schematic diagram of the generated pseudo-random lighting patterns. (b) Schematic diagram of CGI. (c) Overview of the decoding process.
Next we can directly use the measurement matrix $\mathbf {R}$ for CGI as shown in Fig. 6(b). Figure 7(a) shows one of the generated lighting patterns ($100\times 100$ pixels) and Fig. 7(b) shows the average of all lighting patterns. From the perspective of individual pattern composition or statistical characteristics, we cannot see any outlines of the mask, data container patterns, or the grayscale image. CGI randomly encodes spatial information into binary code in the time dimension, which plays a vital role in encryption. On the other hand, bucket signals in CGI can be proved to satisfy the Gaussian distribution, so the ratio of 1 and 0 fluctuates around $1:1$ after binarization.
Fig. 7. Experimental results of CGI using the coded pseudo-random lighting patterns. (a) One of generated lighting patterns. (b) The average of all lighting patterns. (c) The target. (d) The ghost image of the target using the second correlation algorithm. (e) The ghost image of the target using compressed sensing.
In the actual experiment, a commercial digital projector (Epson, EP-970) is used to project these binary illumination patterns onto the target. As shown in Fig. 7(c), we print the pattern on an A4 white paper. A single-pixel photodetector (Thorlabs, PDA100A2) is used to collect bucket signal synchronously, and a computer-hosted data acquisition card (NI, PCIe-6251) records all electric signals for post-processing. Figures 7(d) and 7(e) show the experimental results of ghost images obtained by second-order correlation and compressed sensing [2], respectively. From the different imaging results, we have not found any "cross-talk" as well.
One may ask whether the engineered pseudo-random matrix $\mathbf {R}$ will affect the imaging efficiency and quality compared to the traditional random measurement matrix. Our scheme does not increase the number of additional samples, nor does it require additional optimization algorithms, so it is equivalent to traditional CGI in terms of imaging efficiency. When it comes to the image quality, we introduce the contrast-to-noise ratio (CNR) [59] as an indicator of image evaluation:
2.6 Decoding
Next, we explain how to extract the coded information from $\mathbf {R}$. The decoding process is actually the reverse of the encoding process. The receiver divides $\mathbf {R}$ into $\mathbf {R_1}$ and $\mathbf {R_2}$ according to the mask, and the user assigned to the key $\mathbf {R_3}$ can retrieve $4096$ ghost images as follows:
All users assigned the key $\mathbf {R}$ can retrieve 5,904 ghost images through mutual second-order correlation operations as follows:
The final key step is to decode the quantitative information from these ghost images. As shown in Figs. 6(c) and 8(a), the imaging results using the second-order correlation algorithm are difficult to obtain high-quality images at this stage, which is the reason why we use a specific coding method to quantify the amount of information. As shown in Fig. 8(b), we binarized the retrieved ghost image, where the white area represents +1 and the black area represents −1. Compared with the ground truth of the data container pattern shown in Fig. 8(c), we can find that some pixels in the binarized image are wrong. In fact, these few wrong pixels will not affect the recognition process.
Fig. 8. The process of decoding information from ghost images. (a) The ghost image of a data container pattern. (b) The binarized ghost image. (c) The ground truth of the data container pattern. (d) The identification signal map of 16 cells. (e) The waveforms of the first four identification signals.
Like the design of data container pattern in Section 2.2, we divide the binarized ghost image into $4\times 4$ cells. Each cell ($16\times 16$ pixels) theoretically corresponds to a pattern of Hadamard matrix according to Eq. (4), and each pattern corresponds to a decimal number ranging from 0 to 255 (8 bits). We use the orthogonality of the Hadamard matrix to efficiently identify the closest pattern to the cell, and extract a coded decimal number. We resize each cell into a 16-dimensional column vector $\mathbf {c}$, and quickly match the corresponding pattern by the following equation:
If the match is successful, there will be only one peak in the waveform of $c_1$. Figure 8(d) shows the results of the identification signals of 16 cells corresponding to Fig. 8(b) and Eq. (11). Figure 8(e) plots the identification signals of the first four cells in Fig. 8(b) (corresponding to the four dotted lines in Fig. 8(d)). We extract the decimal number by retrieving the peak’s abscissa, each decimal number represents a pixel of the 8-bit image, and we can finally get the embedded grayscale image information by stitching these pixels in columns shown in Fig. 6(c).The application scenario of our scheme can be summarized by the schematic diagram shown in Fig. 9. User A (central system) distributes three different types of information $\mathbf {b}$, $\mathbf {R}$, and $\mathbf {R_3}$ to users B, C, and D, respectively, where the generation of $\mathbf {b}$ is usually obtained by real physical measurement, that is, the bucket signal of CGI. The information of the user B ($\mathbf {b}$) and the information of the user C ($\mathbf {R}$) are subjected to second-order intensity correlation to obtain a ghost image of the real scene; the information of the user C ($\mathbf {R_1}$) and the information of the user D ($\mathbf {R_3}$) are subjected to second-order intensity correlation to obtain a standard 8-bit image; Both of the above processes require two users to participate, so they can be used to encrypt information. Besides, user B can obtain another standard 8-bit image through the second-order mutually-correlation operation of $\mathbf {R_1}$ and $\mathbf {R_2}$ in $\mathbf {R}$. Since a single user can complete the decoding process, this process can be used to identify whether the source of $\mathbf {R}$ is reliable, and at the same time, user A can use this process to embed watermarking information into $\mathbf {R}$ for copyright protection as well.
Fig. 9. Schematic diagram of the application scenario: User A (central system) distributes three different types of information $\mathbf {b}$, $\mathbf {R}$, and $\mathbf {R_3}$ to users B, C, and D, respectively. The information of the user B ($\mathbf {b}$) and the information of the user C ($\mathbf {R}$) are subjected to second-order intensity correlation to obtain a ghost image of the real scene; the information of the user C ($\mathbf {R_1}$) and the information of the user D ($\mathbf {R_3}$) are subjected to second-order intensity correlation to obtain a standard 8-bit image; user B can obtain another standard 8-bit image through the second-order mutually-correlation operation of $\mathbf {R_1}$ and $\mathbf {R_2}$ in $\mathbf {R}$.
3. Discussion on the trade-off between accuracy and efficiency
In previous sections, through two coding schemes, we have successfully demonstrated the embedding of a total of $10,000$ ghost images of data container patterns in $10,000$ binary pseudo-random lighting patterns. These ghost images can be decoded to obtain two 8-bit images. In the decoding process, we use the orthogonality of the Hadamard matrix to quickly identify.
Next, let’s discuss the performance (accuracy and efficiency) of our scheme. When it comes to accuracy, we use $SER$ as an evaluation indicator, which is calculated as follows:
where $S_e$ and $S_n$ represent the number of error code and the total number, respectively. The total amount of data we embed each time is 160,000 decimal numbers, that is, $S_n=160,000$.Then we discuss the anti-interference performance in the encoding process from the view of $SER$. In Eqs. (6) and (7), we introduce Gaussian white noise before the binarization operation. As shown in Fig. 10(a), as the $SNR$ decreases, the $SER$ also rises further. Note that when the noise amplitude is less than the signal amplitude ($SNR>0dB$), the BCGI code is affected very little, because the bucket signal that contributes a lot to the image composition in the positive and negative GI is far from the average [57,58], and it is more difficult for smaller additive noise to generate error codes in areas with large fluctuation ranges.
Fig. 10. Performance of our scheme. The $SER$ curve as (a) $SNR$ and (b) $N$ change. 8-bit standard images extracted from the first embedding scheme under different (c) SNRs and (d) number $N$ of lighting patterns.
The spatial distribution of pseudo-random lighting patterns is determined, but the number of lighting patterns can be customized. In other words, if only part of lighting patterns are available, we can still retrieve the ghost image and embedded information, although the accuracy will inevitably decrease to some extent. In CGI, the number of lighting patterns, namely the sampling rate, directly determines the $SNR$ of the ghost image. Similarly, in the BCGI code, the encoding length $N$ in the time dimension directly affects the accuracy of decoding. When $N=10000$ (the initial value in the case in the previous sections), the average value of $SER$ nearly goes to $0$. From the perspective of the sampling rate, for a $64$-by-$64$-pixel data container pattern, $N=10000$ means that the sampling rate is about $244\%$. Especially, when the sampling rate is $100\%$ ($N=4096$), the value of $SER$ is $7.5\times 10^{-4}\pm 5\times 10^{-4}\%$; when the sampling rate is about $24.4\%$ ($N=1000$), the value of $SER$ is $2.71\pm 0.04\%$. As shown in Fig. 10(b), we plot the $SER$ curve as the value of $N$ changes. We can find that there is a trade-off relationship between $N$ and $SER$; the longer the BCGI code, the higher the accuracy of the decoding. Figures 10(c) and 10(d) show the 8-bit standard images extracted from the first generation scheme of the lighting pattern under different SNRs and sampling rates, respectively. Note the reduction of $N$ here means the reduction of the sampling pattern in the time dimension. According to the characteristics of CGI, it can still reconstruct the global image but $SNR$ will decrease; if the data in some regions is unavailable or tampered in the spatial dimension, however, its influence is broad, and it will not only cause the reconstruction of the corresponding region of the image of the real scene to fail but also directly lose part of the embedded information.
If one wants to further increase the total data capacity, there are two main methods to consider: first, modify the mask of the space-division multiplexing technique to make the area of the two regions of the mutually-correlated pseudo-random pattern equal (that is, both occupy 5000 pixels) , so the data volume of the second generation scheme can be increased from 755,712 bits to 781,250 bits ($5000\times 5000\times 1/32$). Second, one can increase efficiency by reducing the cell size of the Hadamard-matrix-based data container pattern. In the case shown above, we embed 8 bits of information in 256 pixels (the size of a cell is $16\times 16$ pixels), and the bit per pixel (bpp) of the data container pattern is only $1/32$, i.e., each pixel contains $1/32$ bit of information on average.
Figure 11(a) shows 5 data container patterns at different cell sizes. For example, when the size of a single cell is $4\times 4$ pixels, there are a total of $16$ Hadamard patterns, that is, $4$ bits of information is embedded in $16$ pixels, and the $bpp$ is $1/4$ (increased by seven times), so the data volume of a single data container pattern is $1024$ bits. However, as shown in Fig. 11(b), as the amount of data of a single pattern increases, the accuracy of recognition will decrease accordingly, that is, the value of $SER$ will increase. We also note that when $N$ is small, the $SER$ curve seems to contradict the conclusion. In fact, the reason for this result is that the upper limit of the $SER$ of the data container patterns of different cell sizes is quite different; for example, the maximum of $SER$ approaches $50\%$ for the data container pattern with a cell size of $1\times 1$ ($bpp=1$). Therefore, users must balance accuracy and efficiency according to specific needs. If users want to take into account both efficiency and a certain degree of accuracy at the same time, they can consider using the compressed sensing algorithm to reconstruct ghost images with better quality at sub-sampling rates. However, post-processing of data will take a lot of time (tens of thousands of images need to be retrieved), which is unacceptable at this stage. If necessary, users can use wavelength-division and time-division multiplexing techniques [44,47] to further increase channel capacity. In the future, we will consider the use of deep learning schemes [60–63] to simultaneously improve the accuracy and efficiency of recognition using binary bucket signals at sub-sampling rates.
Fig. 11. (a) The five data container patterns at different cell sizes. (b) The $SER$ curves of different amount of data as $N$ changes.
The quantitative description of the accuracy or precision of CGI-based encoding and decoding in terms of the amount of information is still very vague. People tend to directly transmit image information and describe performance indirectly through some image evaluation indicators. Therefore, if according to this standard, our scheme can claim to embed tens of thousands of ghost images, however, this description is a little exaggerated, because we have to consider the amount of information of the single so-called ghost image as well. Taking into account the non-orthogonality of the random measurement matrix, and after the bucket signal is binarized, we cannot directly reconstruct the high-quality and complex grayscale image at this stage. That is to say, the actual amount of information of the ghost image is much smaller than that of the original image, which is exactly the reason why we use the Hadamard-matrix-based data container pattern. On the one hand, it is efficient to use the orthogonality to identify the type of signal, that is, it is not necessary to scan each pixel of the pattern to determine whether it is 1 or 0 (not sensitive to the threshold of binarization), but to quickly find the position of the peak value through matrix operation position (see Eq. (11)). On the other hand, the accuracy is very high, which can allow bit errors in some pixels shown in Figs. 8(b) and 8(c). The CGI-based binary pseudo-random coding scheme inevitably brings background noise, which further causes bit errors in pixels. The reduction of $N$ or $SNR$ will further increase the background noise of the reconstructed image, resulting in more pixel errors, thereby reducing the recognition accuracy. As shown in Fig. 12, a quantitative comparison between two different data container patterns concerning $SER$ is presented. Figure 12(a) shows another data container pattern ($64\times 64$ pixels) discussed in Ref. [42], in which each pixel in the white area is activated to represent a binary number and a total of 128 pixels are activated ($bpp=1/32$). Figure 12(b) plots the $SER$ curves of two types of data container pattern at the same $bpp$ with the change of $N$. As expected, the result demonstrates that our scheme using the Hadamard-matrix-based data container pattern converges faster as $N$ increases. When $N$ is small, however, both schemes may face serious challenges as most information is unavailable. As shown in Fig. 13, we have also quantitatively drawn a map of the recognition accuracy of single cell ($16\times 16$ pixels) under different numbers of error pixels varying from 0 to 128. According to Fig. 1(a), there are 256 types of cells, representing 256 decimal integers. Here we define accuracy as the number of successful identifications divided by the total number. In our $1,000$ random tests for each type, we find that when the number of error pixels is approximately less than 96, the accuracy rate goes to 1, which means that when less than $75\%$ of the pixels are modified or tampered (here we think that at most half of the pixels are modified or tampered), we can still recognize patterns efficiently and accurately. However, once more than $75\%$ of error pixels, the accuracy of recognition quickly decreases, which in turn produces "cross-talk".
Fig. 12. (a) Another data container pattern ($64\times 64$ pixels) shown in Ref [42]: each pixel in the white area is activated to represent a binary number, $bpp=1/32$. (b) The $SER$ curves using two different data container patterns at the same $bpp$ as $N$ changes.
Fig. 13. The recognition accuracy map of single cell ($16\times 16$ pixels) in the Hadamard-matrix-based data container pattern in the case of different numbers of error pixels.
When it comes to safety performance, we have previously discussed potential noise attack and occlusion attack. Finally, we summarize the encryption process that can be introduced in our scheme. In the first pseudo-random pattern generation scheme, the randomly generated $\mathbf {R_3}$ is distributed as a key to a third party. It is impossible for users without a key to decode the information. In addition, the loss of $\mathbf {R_3}$ in the time dimension will bring the improvement of $SER$, while the loss of spatial dimension will directly lose part of the embedded data. In the second pseudo-random pattern generation scheme, the mask can be assigned in advance to a user as the key, and the users without the assignment will not be able to identify which region used as a bucket signal array and which one used as reference spatial signals. The process of encoding valid information into a series of decimal numbers from 0 to 255 can be regarded as an additional key as well. Even if the hacker cracks the previous series of obstacles, he cannot obtain valid information if the decoding method is unknown. In order to facilitate the demonstration of our scheme, we directly embed the pixel values of the 8-bit standard gray-scale image in columns as information. In fact, we can encode some sparse coefficients of the image for embedding much more compressed images.
4. Conclusion
In conclusion, we propose two novel generation schemes of pseudo-random lighting patterns and a design method of data container patterns based on the Hadamard matrix. With our schemes, users can extract a total of $10,000$ ghost images ($64\times 64$ pixels) under the framework of CGI, and these ghost images can be quantitatively decoded to obtain $1,280,000$ bits of data with very low SER. We believe our schemes pave the way to CGI-based large-capacity information embedding, and have potential applications in technologies such as optical encryption, authentication, and watermarking.
Funding
National Natural Science Foundation of China (11735005,11474027); Start-up Research Grant of University of Macau (SRG2019-00174-IAPME); Interdiscipline Research Funds of Beijing Normal University.
Disclosures
The authors declare no conflicts of interest.
See Supplement 1 for supporting content.
References
1. J. H. Shapiro, “Computational ghost imaging,” Phys. Rev. A: At., Mol., Opt. Phys. 78(6), 061802 (2008). [CrossRef]
2. M. F. Duarte, M. A. Davenport, D. Takhar, J. N. Laska, T. Sun, K. F. Kelly, and R. G. Baraniuk, “Single-pixel imaging via compressive sampling,” IEEE Signal Process. Mag. 25(2), 83–91 (2008). [CrossRef]
3. M. P. Edgar, G. M. Gibson, and M. J. Padgett, “Principles and prospects for single-pixel imaging,” Nat. Photonics 13(1), 13–20 (2019). [CrossRef]
4. W. L. Chan, K. Charan, D. Takhar, K. F. Kelly, R. G. Baraniuk, and D. M. Mittleman, “A single-pixel terahertz imaging system based on compressed sensing,” Appl. Phys. Lett. 93(12), 121105 (2008). [CrossRef]
5. N. Radwell, K. J. Mitchell, G. M. Gibson, M. P. Edgar, R. Bowman, and M. J. Padgett, “Single-pixel infrared and visible microscope,” Optica 1(5), 285 (2014). [CrossRef]
6. H. Yu, R. Lu, S. Han, H. Xie, G. Du, T. Xiao, and D. Zhu, “Fourier-transform ghost imaging with hard X rays,” Phys. Rev. Lett. 117(11), 113901 (2016). [CrossRef]
7. H. C. Liu and S. Zhang, “Computational ghost imaging with thermal objects in long-wave infrared range,” Appl. Phys. Lett. 111(3), 031110 (2017). [CrossRef]
8. L. Olivieri, J. S. T. Gongora, L. Peters, V. Cecconi, A. Cutrona, J. Tunesi, R. Tucker, A. Pasquazi, and M. Peccianti, “Hyperspectral terahertz microscopy via nonlinear ghost imaging,” Optica 7(2), 186 (2020). [CrossRef]
9. T. J. Lane and D. Ratner, “What are the advantages of ghost imaging? Multiplexing for x-ray and electron imaging,” Opt. Express 28(5), 5898 (2020). [CrossRef]
10. B. Sun, M. P. Edgar, R. Bowman, L. E. Vittert, S. Welsh, A. Bowman, and M. J. Padgett, “3D computational imaging with single-pixel detectors,” Science 340(6134), 844–847 (2013). [CrossRef]
11. M. J. Sun, M. P. Edgar, G. M. Gibson, B. Sun, N. Radwell, R. Lamb, and M. J. Padgett, “Single-pixel three-dimensional imaging with time-based depth resolution,” Nat. Commun. 7(1), 12010 (2016). [CrossRef]
12. M. J. Sun and J. M. Zhang, “Single-pixel imaging and its application in three-dimensional reconstruction: a brief review,” Sensors 19(3), 732 (2019). [CrossRef]
13. F. Devaux, P. A. Moreau, S. Denis, and E. Lantz, “Computational temporal ghost imaging,” Optica 3(7), 698 (2016). [CrossRef]
14. P. Janassek, S. Blumenstein, and W. Elsäßer, “Ghost Spectroscopy with Classical Thermal Light Emitted by a Superluminescent Diode,” Phys. Rev. Appl. 9(2), 021001 (2018). [CrossRef]
15. A. Abbas, C. Xu, and L. G. Wang, “Spatiotemporal ghost imaging and interference,” Phys. Rev. A 101(4), 043805 (2020). [CrossRef]
16. Z. H. Xu, W. Chen, J. Penuelas, M. Padgett, and M. J. Sun, “1000 fps computational ghost imaging using LED-based structured illumination,” Opt. Express 26(3), 2427 (2018). [CrossRef]
17. H. Kanno, H. Mikami, and K. Goda, “High-speed single-pixel imaging by frequency-time-division multiplexing,” Opt. Lett. 45(8), 2339 (2020). [CrossRef]
18. C. Zhang, S. Guo, J. Cao, J. Guan, and F. Gao, “Object reconstitution using pseudo-inverse for ghost imaging,” Opt. Express 22(24), 30063 (2014). [CrossRef]
19. M. J. Sun, L. T. Meng, M. P. Edgar, M. J. Padgett, and N. Radwell, “A Russian Dolls ordering of the Hadamard basis for compressive single-pixel imaging,” Sci. Rep. 7(1), 3464 (2017). [CrossRef]
20. S. Tetsuno, K. Shibuya, and T. Iwata, “Subpixel-shift cyclic-Hadamard microscopic imaging using a pseudo-inverse-matrix procedure,” Opt. Express 25(4), 3420 (2017). [CrossRef]
21. Z. Ye, H. Wang, J. Xiong, and K. Wang, “Simultaneous full-color single-pixel imaging and visible watermarking using Hadamard-Bayer illumination patterns,” Opt. Laser Eng. 127, 105955 (2020). [CrossRef]
22. Z. Zhang, X. Ma, and J. Zhong, “Single-pixel imaging by means of Fourier spectrum acquisition,” Nat. Commun. 6(1), 6225 (2015). [CrossRef]
23. S. M. Khamoushi, Y. Nosrati, and S. H. Tavassoli, “Sinusoidal ghost imaging,” Opt. Lett. 40(15), 3452 (2015). [CrossRef]
24. W. K. Yu, M. F. Li, X. R. Yao, X. F. Liu, L. A. Wu, and G. J. Zhai, “Adaptive compressive ghost imaging based on wavelet trees and sparse representation,” Opt. Express 22(6), 7133 (2014). [CrossRef]
25. F. Rousset, N. Ducros, A. Farina, G. Valentini, C. D’Andrea, and F. Peyrin, “Adaptive basis scan by wavelet prediction for single-pixel imaging,” IEEE Trans. Comput. Imaging 3(1), 36–46 (2017). [CrossRef]
26. M. Xi, H. Chen, Y. Yuan, G. Wang, Y. He, Y. Liang, J. Liu, H. Zheng, and Z. Xu, “Bi-frequency 3D ghost imaging with Haar wavelet transform,” Opt. Express 27(22), 32349 (2019). [CrossRef]
27. P. Wijesinghe, A. Escobet-Montalbán, M. Chen, P. R. Munro, and K. Dholakia, “Optimal compressive multiphoton imaging at depth using single-pixel detection,” Opt. Lett. 44(20), 4981 (2019). [CrossRef]
28. D. B. Phillips, M. J. Sun, J. M. Taylor, M. P. Edgar, S. M. Barnett, G. M. Gibson, and M. J. Padgett, “Adaptive foveated single-pixel imaging with dynamic supersampling,” Sci. Adv. 3(4), e1601782 (2017). [CrossRef]
29. Z. Ye, B. Su, P. Qiu, and W. Gao, “Secured regions of interest (SROIs) in single-pixel imaging,” Sci. Rep. 9(1), 12782 (2019). [CrossRef]
30. D. B. Phillips, R. He, Q. Chen, G. M. Gibson, and M. J. Padgett, “Non-diffractive computational ghost imaging,” Opt. Express 24(13), 14172 (2016). [CrossRef]
31. E. Toninelli, D. Stellinga, B. Sephton, A. Forbes, and M. J. Padgett, “Single-pixel imaging using caustic patterns,” Sci. Rep. 10(1), 2281 (2020). [CrossRef]
32. P. Clemente, V. Duran, E. Tajahuerce, and J. Lancis, “Optical encryption based on computational ghost imaging,” Opt. Lett. 35(14), 2391 (2010). [CrossRef]
33. M. Tanha, R. Kheradmand, and S. Ahmadi-Kandjani, “Gray-scale and color optical encryption based on computational ghost imaging,” Appl. Phys. Lett. 101(10), 101108 (2012). [CrossRef]
34. M. J. Sun, J. Shi, H. Li, and G. Zeng, “A simple optical encryption based on shape merging technique in periodic diffraction correlation imaging,” Opt. Express 21(16), 19395 (2013). [CrossRef]
35. S. Li, X.-R. Yao, W.-K. Yu, L.-A. Wu, and G.-J. Zhai, “High-speed secure key distribution over an optical network based on computational correlation imaging,” Opt. Lett. 38(12), 2144 (2013). [CrossRef]
36. W. Chen and X. Chen, “Ghost imaging for three-dimensional optical security,” Appl. Phys. Lett. 103(22), 221106 (2013). [CrossRef]
37. W. Chen and X. Chen, “Marked ghost imaging,” Appl. Phys. Lett. 104(25), 251109 (2014). [CrossRef]
38. S. Zhao, L. Wang, W. Liang, W. Cheng, and L. Gong, “High performance optical encryption based on computational ghost imaging with QR code and compressive sensing technique,” Opt. Commun. 353, 90–95 (2015). [CrossRef]
39. J. Wu, Z. Xie, Z. Liu, W. Liu, Y. Zhang, and S. Liu, “Multiple-image encryption based on computational ghost imaging,” Opt. Commun. 359, 38–43 (2016). [CrossRef]
40. H. C. Liu, B. Yang, Q. Guo, J. Shi, C. Guan, G. Zheng, H. Mühlenbernd, G. Li, T. Zentgraf, and S. Zhang, “Single-pixel computational ghost imaging with helicity-dependent metasurface hologram,” Sci. Adv. 3(9), e1701477 (2017). [CrossRef]
41. Z. Zhang, S. Jiao, M. Yao, X. Li, and J. Zhong, “Secured single-pixel broadcast imaging,” Opt. Express 26(11), 14578 (2018). [CrossRef]
42. L. Sui, C. Du, M. Xu, A. Tian, and A. Anand, “Information encryption based on the customized data container under the framework of computational ghost imaging,” Opt. Express 27(12), 16493 (2019). [CrossRef]
43. L. Wang, S. Zhao, W. Cheng, L. Gong, and H. Chen, “Optical image hiding based on computational ghost imaging,” Opt. Commun. 366, 314–320 (2016). [CrossRef]
44. Z. Ye, P. Qiu, H. Wang, J. Xiong, and K. Wang, “Image watermarking and fusion based on Fourier single-pixel imaging with weighed light source,” Opt. Express 27(25), 36505 (2019). [CrossRef]
45. L. Sui, J. Wang, A. Tian, and A. Asundi, “Optical image hiding under framework of computational ghost imaging based on an expansion strategy,” Opt. Express 27(5), 7213 (2019). [CrossRef]
46. C. Zhang, W. He, B. Han, M. Liao, D. Lu, X. Peng, and C. Xu, “Compressive optical steganography via single-pixel imaging,” Opt. Express 27(9), 13469 (2019). [CrossRef]
47. Z. Ye, J. Pan, H.-B. Wang, and J. Xiong, “Embedding and transmitting multi-dimensional optical information through noisy environments using a single-pixel detector,” Opt. Commun. 467, 125726 (2020). [CrossRef]
48. S. Jiao, J. Feng, Y. Gao, T. Lei, and X. Yuan, “Visual cryptography in single-pixel imaging,” Opt. Express 28(5), 7301 (2020). [CrossRef]
49. H. C. Liu and W. Chen, “Optical ghost cryptography and steganography,” Opt. Laser Eng. 130, 106094 (2020). [CrossRef]
50. J. Cheng, “Ghost imaging through turbulent atmosphere,” Opt. Express 17(10), 7916 (2009). [CrossRef]
51. Y. K. Xu, W. T. Liu, E. F. Zhang, Q. Li, H. Y. Dai, and P. X. Chen, “Is ghost imaging intrinsically more powerful against scattering?” Opt. Express 23(26), 32993 (2015). [CrossRef]
52. A. Meda, A. Caprile, A. Avella, I. Ruo Berchera, I. P. Degiovanni, A. Magni, and M. Genovese, “Magneto-optical imaging technique for hostile environments: The ghost imaging approach,” Appl. Phys. Lett. 106(26), 262405 (2015). [CrossRef]
53. G. Li, N. Bai, N. Zhao, and C. Xia, “Space-division multiplexing: the next frontier in optical communication,” Adv. Opt. Photonics 6(4), 413 (2014). [CrossRef]
54. K. H. Luo., B. Q. Huang, W. M. Zheng, and L. A. Wu, “Nonlocal imaging by conditional averaging of random reference measurements,” Chin. Phys. Lett. 29(7), 074216 (2012). [CrossRef]
55. J. Li, D. Yang, B. Luo, G. Wu, L. Yin, and H. Guo, “Image quality recovery in binary ghost imaging by adding random noise,” Opt. Lett. 42(8), 1640 (2017). [CrossRef]
56. H. Yang, S. Wu, H. B. Wang, D. Z. Cao, S. H. Zhang, J. Xiong, and K. Wang, “Probability theory in conditional-averaging ghost imaging with thermal light,” Phys. Rev. A 98(5), 053853 (2018). [CrossRef]
57. J. Leng, W. K. Yu, and S. F. Wang, “Formation mechanism of correspondence imaging with thermal light,” Phys. Rev. A 101(3), 033835 (2020). [CrossRef]
58. H. C. Liu, H. Yang, J. Xiong, and S. Zhang, “Positive and negative ghost imaging,” Phys. Rev. Appl. 12(3), 034019 (2019). [CrossRef]
59. K. W. C. Chan, M. N. O’Sullivan, and R. W. Boyd, “Optimization of thermal ghost imaging: high-order correlations vs. background subtraction,” Opt. Express 18(6), 5562 (2010). [CrossRef]
60. N. Radwell, S. D. Johnson, M. P. Edgar, C. F. Higham, R. Murray-Smith, and M. J. Padgett, “Deep learning optimized single-pixel LiDAR,” Appl. Phys. Lett. 115(23), 231101 (2019). [CrossRef]
61. F. Wang, H. Wang, H. Wang, G. Li, and G. Situ, “Learning from simulation: An end-to-end deep-learning approach for computational ghost imaging,” Opt. Express 27(18), 25560 (2019). [CrossRef]
62. S. Jiao, Y. Gao, J. Feng, T. Lei, and X. Yuan, “Does deep learning always outperform simple linear regression in optical imaging?” Opt. Express 28(3), 3717 (2020). [CrossRef]
63. H. Wu, R. Wang, G. Zhao, H. Xiao, D. Wang, J. Liang, X. Tian, L. Cheng, and X. Zhang, “Sub-Nyquist computational ghost imaging with deep learning,” Opt. Express 28(3), 3846 (2020). [CrossRef]
