Open Access
Add to BibSonomyAbstract
In this paper, compressive sensing based high-speed time-stretch optical microscopy for two-dimensional (2D) image acquisition is proposed and experimentally demonstrated for the first time. A section of dispersion compensating fiber (DCF) is used to perform wavelength-to-time conversion and then ultrafast spectral shaping of broadband optical pulses can be achieved via high-speed intensity modulation. A 2D spatial disperser comprising a pair of orthogonally oriented dispersers is employed to produce spatially structured illumination for 2D image acquisition and a section of single mode fiber (SMF) is utilized for pulse compression in the optical domain. In our scheme, a 1.2-GHz photodetector and a 50-MHz analog-to-digital converter (ADC) are used to acquire the energy of the compressed pulses. Image reconstructions are demonstrated at a frame rate of 500 kHz and a sixteen-fold image compression is achieved in our proof-of-concept demonstration.
© 2015 Optical Society of America
1. Introduction
High-throughput microscopy (HTM) has become a cornerstone technology in such diverse fields as detection of occult tumor cells (OTCs), drug discovery and phenotypic screening of small molecule libraries [1–4 ]. In blood, the tumor cells circulate at extremely low concentrations, estimated to be in the range of one tumor cell in the background of 106–107 normal blood cells [5]. Therefore, high-throughput instruments are needed to detect and classify such an extremely rare event. A preferred method for detection of OTCs is automated digital microscopy (ADM) using image analysis for recognition of specifically labeled tumor cells [6]. However, the scan speed of ADM is limited by the substantial latency associated with stepping the sample and a long exposure time, a typical scan rate of 800 cells/s makes it difficult to detect such rare tumor cells in blood. To overcome this limitation, a rare-cell detector using fiber-optic array scanning technology (FAST) is proposed, enabling a 500-fold speed-up over ADM with comparable sensitivity and superior specificity [7]. Recently, another high-throughput flow analyzer that builds on the serial time-encoded amplified microscopy (STEAM) technology has been reported [8], enabling high-throughput screening of rare tumor cells in blood with an unprecedented throughput of 100,000 cells/s and a false positive rate of approximately 10−6. However, these proposed high-throughput real-time instruments will produce massive amounts of data which is a great challenge for data acquisition, storage, and processing. Consequently, data compression should be taken into consideration to relieve the load on the back-end electronics.
Recently, a new data compression technology named anamorphic stretch transform (AST) has been proposed to achieve warped stretch imaging by employing nonlinear group delay dispersion [9]. The amount of data produced by the time stretch camera is reduced by about three times. Although optical data compression can be achieved with this technology, an analog-to-digital converter (ADC) with a high sampling rate and a large bandwidth is still necessary (a 20-GS/s sampling rate and a 7-GHz bandwidth in [10]), which will hinder the practical application of warped time stretch imaging. In addition to AST, another novel sampling approach to data compression known as compressive sensing (CS) asserts that most natural signals are sparse in a certain orthogonal basis and can be acquired at a rate much lower than the Nyquist rate [11–15 ]. With CS, data compression and bandwidth reduction of the back-end ADCs can be simultaneously achieved. Currently, multiple groups have shown great interest in high-speed imaging based on CS [16–19 ]. The first high-speed CS-based imaging system was reported by Hongwei Chen and Bryan T. Bosworth simultaneously [16,17 ], achieving a frame rate of several MHz and a high compression ratio. Then its capability of enabling high-speed flow microscopy is experimentally demonstrated in [18] for the first time. Different from the optical microscope in [8], only the total energy of the pulses is needed for image reconstruction in these systems. A high enough signal-to-noise ratio (SNR) is guaranteed at the digitizer and the amount of data captured is significantly decreased. In addition, the constraint on the dispersion is relieved because the overlap of adjacent pulses does not affect image reconstructions. However, these systems are one-dimensional (1D) imaging devices whose field of view (FOV) is limited to a line, and thus an additional mechanical scanning in the second dimension is required to capture a two-dimensional (2D) image. To eliminate the need for mechanical scanning which is too slow for many applications, a new approach to building a 2D imaging system is proposed in [20]. Such a 2D imaging system is promising for observation of fast dynamic phenomena where there is no relative motion between the illumination source and the target object, such as the typical morphological changes of cells undergoing apoptosis and some dynamic phenomena happening in live cells.
In this paper, high-speed time-stretch optical microscopy based on CS for 2D image acquisition is proposed and experimentally demonstrated for the first time [21]. In our scheme, a temporal disperser is used to convert the spectrum of a broadband optical pulse to a temporal waveform and ultrafast spectral shaping is performed via high-speed intensity modulation. Then we employ a 2D spatial disperser comprising a pair of orthogonally oriented dispersers to map the spectrum of the optical pulse into a 2D spectral pattern in space, enabling ultrafast structured illumination of objects. Then the spectrally encoded pulses are compressed in the optical domain to perform the inner product between the spatial profile of the object and the unique pseudo-random pattern. A low-speed photodetector and the corresponding ADC synchronized with the optical pulse source are used to capture the energy of the compressed pulses. Image reconstructions are demonstrated at a frame rate of 500 kHz with a sixteen-fold image compression.
2. Principle
The high-speed compressive cameras proposed in [16–19 ] are all line scan imaging systems which cannot capture some fast dynamic phenomena. To achieve 2D imaging, a 2D spatial disperser is employed in time-stretch imaging systems instead of a 1D spatial disperser. It is composed of a pair of orthogonally oriented spatial dispersers (a diffraction grating and a virtually imaged phased array (VIPA)) that can produce a 2D spectral pattern in space. The VIPA consists of a semi-cylindrical lens and a thin glass plate with a highly reflective coating (almost 100%) on one surface except for an uncoated window area and a partially reflective coating (e.g., >95%) on the other surface [22]. A collimated pulse beam is focused to a line with a cylindrical lens and enters the glass plate through the uncoated window area on the front surface as shown in Fig. 1 . Multiple reflections occur inside the VIPA and multiple diverging beams pass through the back surface of the glass plate. All the output beams interfere and then form collimated beams with different angles determined by the wavelength. The VIPA with a free spectral range (FSR) of 60 GHz spatially disperse the spectrum of the incident pulse in one direction. If the spectral bandwidth is larger than 60 GHz, the output collimated beams will be spatially superimposed on each other. Therefore, a diffraction grating with a groove density of 1200 lines/mm, whose dispersion direction is orthogonal to that of the VIPA, is employed to separate this degeneracy in the other direction. Consequently, as shown in Fig. 1, the spectrum of the incident beam is mapped into a two-dimensional array of frequency comb modes in space where each ‘dot’ represents an individual mode. The dots within a row which is tilted by the grating dispersion are separated by the VIPA FSR, and within a column which is tilted by the VIPA dispersion, the dots are separated by the comb spacing that is related to the repetition rate of optical pulses. The manner in which successive modes are indexed and counted is indicated by the arrows in the leftmost two columns.
Fig. 1 The architecture of the 2D spatial disperser and the produced two-dimensional array of frequency comb modes. Each ‘dot’ represents an individual mode. The dots within a row which is tilted by the grating dispersion are separated by the VIPA FSR, and the dots within a column which is tilted by the VIPA dispersion are separated by the comb spacing. The manner in which successive modes are indexed and counted is indicated by the arrows in the leftmost two columns.
By performing the dispersive Fourier transform (DFT), the 2D spectral pattern after reflection is mapped into a temporal waveform carrying the spatial information of the object. As the resultant temporal waveform can be sparsely represented in discrete cosine transform (DCT) domain, CS can be employed to reconstruct the 2D images. A vector of dimension N is used to denote the serial time-domain waveform. It can be sparsely expressed as follows with respect to the DCT basis .
where is a sparse vector with at most K () nonzero entries. A measurement matrix of size () whose entries follow a Gaussian or Bernoulli distribution is used to sample , which can be written as follows:where is a vector with M compressive measurements and is a noise vector bounded by a known amount . It has been demonstrated that the sensing matrix () satisfies the restricted isometry property (RIP) and can be faithfully reconstructed with only measurements. The reconstruction process is to solve the solution to the problem posed in Eq. (3).3. Experimental setup
The experimental architecture of the proposed CS based high-speed time-stretch optical microscope is shown in Fig. 2 . A mode-locked laser (MLL) (Pritel HPPRR-FFL-50MHz) with a 50-MHz repetition rate and a spectral width of 15 nm is used to generate a train of optical pulses whose full spectral width is approximately 23 nm. The optical pulses pass through a section of dispersion compensating fiber (DCF) with a group velocity dispersion (GVD) of −1368 ps/nm, then the pulse duration is stretched to greater than 32 ns. A high-power erbium-doped fiber amplifier (EDFA) (AEDFA-33-B-FA) whose operating wavelength is 1540nm-1565nm amplifies the stretched pulses with an average output power of 20 dBm. A pulse pattern generator (PPG) (Agilent N4951A-P32) synchronized to the MLL provides a 10-Gb/s pseudo-random binary sequence (PRBS) signal to modulate the amplified optical pulses by employing a 12.5 Gb/s Mach-Zehnder modulator (MZM) (PHOTLINE MX-LN-10). The reason why we use PRBS instead of the Gaussian or Bernoulli matrices (two typical measurement matrices in CS) is that binary modulation (with 0 and 1) can be easily achieved in the optical domain and the corresponding recovery performance is similar to that by using the Gaussian or Bernoulli matrices. After random modulation, a circulator directs the PRBS-modulated optical pulses into a two-dimensional (2D) spatial disperser. The 2D disperser comprises a pair of orthogonally oriented dispersers, including a diffraction grating with a groove density of 1200 lines/mm and a virtually imaged phased array (VIPA) with a free spectral range (FSR) of 60 GHz and a target finesse of 110. After passing through the 2D disperser, the spectrum of the incident pulse is mapped into a 2D spectral pattern in space which is then focused onto a USAF 1951 resolution target by using an objective lens with a focal length of 100 mm. When the spatially dispersed pulses reflect off the object, the spatial information of this target is encoded onto the spectrum of the pulses. Then the spectrally encoded pulses re-enter the 2D disperser followed by the circulator. The optical pulses are directed into another EDFA with an output power of 12 dBm. To perform the inner product between the spatial profile of the object and the unique PRBS pattern, a section of 80-km single mode fiber (SMF) with a complementary GVD of + 1360 ps/nm is used to temporally compress the optical pulses. Only the energy of each compressed pulse is required for image reconstruction as the compressive measurements. Therefore, the compressed pulses with an average power of −7 dBm are detected by a 1.2-GHz photodiode (THORLABS DET01CFC/M) and sampled by an externally-clocked analog-to-digital converter (ADC) synchronized with a 50-MHz sampling clock derived from the PPG. The phase of the sampling clock should be calibrated to keep the alignment between the sampling window and the peak of the detected pulse.
Fig. 2 Experimental setup of the proposed CS based high-speed time-stretch optical microscope. MLL: mode-locked laser, DCF: dispersion compensating fiber, EDFA: Erbium-doped fiber amplifier, MZM: Mach-Zehnder modulator, PPG: pulse pattern generator, PRBS: pseudo-random binary sequence, Cir: circulator, VIPA: virtually-imaged phased array, SMF: single mode fiber, PD: photo-detector, DSP: digital signal processor.
4. Experimental results
To evaluate the recovery performance of the proposed compressive imaging system, image acquisition without CS is also performed. Since the 2D image is acquired by performing space-wavelength-time mapping, the number of the captured temporal samples should be larger than that of the resolved pixels which is related to the parameters of the 2D spatial disperser to get all the image features. In the non-CS system, a digital phosphor oscilloscope (DSO) with a sampling rate of 50 GS/s (Tektronix DPO72004B) is used to capture the data stream, permitting 1600 samples within one pulse duration (32 ns). Since the full spectral width of the optical pulses is about 23 nm and the FSR of the VIPA is 60 GHz, approximately 46 FSRs occupy the entire pulse spectrum. The number of rows of the 2D image is equal to the number of FSRs. Consequently, a 2D image of size 46 × 35 is reconstructed from the captured data stream by simply sorting the 1D vector into a 2D matrix as shown in the inset of Fig. 3(d) . More image features and larger linear dimensions can be sought by increasing the spectral bandwidth of optical pulses or the spatial resolution determined by the parameters of the 2D disperser. When CS is applied, the size of the image vector to be reconstructed should also be 1600 × 1. The total number of pixels in the 2D image is expressed as follows:
where is the total fiber dispersion, is the spectral bandwidth of the optical pulses and is the bit rate of the PRBS signal. When N is equal to 1600, a 50-Gb/s PRBS signal is needed to perform random modulation. However, only a 10-Gb/s PRBS signal is generated due to the bandwidth constraint of the PPG. To solve this problem, a method to virtually increase the PRBS rate is proposed in our previous work [19]. The PRBS rate can be increased from 10 Gb/s to 50 Gb/s by considering a 1-ns PRBS bit as five 200-ps PRBS bits, which does not badly affect the randomness of the PRBS signal.Fig. 3 (a) The temporal waveform of the PRBS-modulated optical pulses. (b) The temporal waveform of the optical pulses after compression. (c) The spectrum of the optical pulses after passing through the 2D spatial disperser. (d) The spectrum of the optical pulses after reflecting off the object. Inset: the reconstructed 2D image of size 46 × 35.
Figure 3(a) shows the temporal waveform of the PRBS-modulated optical pulses. The pulse period is 20 ns and the duration of the stretched pulse is 32 ns, leading to an overlap between adjacent stretched pulses as shown in Fig. 3(a). Since only the energy of the compressed pulses needs to be acquired as the measurements, this overlap will not hinder image reconstruction. The temporal waveform of the optical pulses after compression is shown in Fig. 3(b) and a 50-MHz ADC synchronized to the MLL is used to acquire the compressive measurements for image reconstruction. Figure 3(c) depicts the spectrum of the optical pulses after passing through the 2D spatial disperser and about 46 FSRs can be clearly observed over the full spectrum. Figure 3(d) gives the spectrum of the optical pulses after reflecting off the object. The spatial information of the object is encoded onto the spectrum of the spatially dispersed pulses. Then a temporal waveform which mimics the spectrum of the spectrally encoded pulses is generated via DFT.
The adopted algorithm for image reconstruction is two-step iterative shrinkage/thresholding (TwIST) [23], which aims to solve the following problem of Eq. (5).
where () is the linear operator, is the regularization function (here we use total-variation (TV) regularization) and is the regularization parameter. In TwIST, the minimization of the first term is achieved when the observed data closely matches the estimated data and a piecewise constant vector helps to minimize the second term . The regularization parameter controls the relative weight of the two terms to obtain the best recovery results. To reconstruct the image vector of size 1600 × 1, approximately 0.4 s is required on a computer with Intel i3-3220CPU (3.3 GHz) and 4 GBRAM.Figure 4(a) shows the captured 2D images without CS and Figs. 4(b)-4(d) give the reconstructed images at different compression ratios (5%, 6.25%, 12.5% and 18.75%). The recovery accuracy at different compression ratios is evaluated by the peak-signal-to-noise ratios (PSNRs) of the reconstructed images. The mean squared error (MSE) and the corresponding peak signal-to-noise ratio (PSNR) are calculated from Eqs. (6) and (7) , respectively.
where denotes the captured image of size without CS and is the reconstructed image at a certain compression ratio. denotes the maximum possible pixel value of the image. The PSNRs of the reconstructed images in Figs. 4(b)-4(d) (top) are 18.75 dB, 19.89 dB, 20.77 dB and 21.56 dB, respectively. It is obvious that the recovery accuracy is improved when more compressive measurements are used for image restoration. However, the frame rate of the proposed compressive imaging system is expressed as frep / M, where frep denotes the pulse repetition rate and M is the number of measurements. Therefore, the imaging speed of our system will slow down with the number of measurements increasing. A trade-off between the frame rate and the recovery accuracy should be taken into consideration. In our system, the repetition rate of the optical pulses is 50 MHz and the 2D images can be reconstructed only from 100 measurements (corresponding to a compression ratio of 6.25%). Compared with the results obtained without CS, the amount of data acquired by our system is significantly decreased, resulting in a sixteen-fold data compression. In addition, the speed and resolution of the non-CS setup is limited by the sampling rate of electronic digitizers. However, in our system, since the measurements for image reconstruction are only the energy of the pulses after optical compression, the load on the sampling the requirement can be greatly relieved.Fig. 4 (a) The reconstructed 2D images without CS. Image reconstructions at compression ratios of (b) 5%, (c) 6.25%, (d) 12.5% and (e) 18.75%.
5. Conclusion
High-speed time-stretch optical microscopy based on CS for 2D image acquisition is proposed and experimentally demonstrated for the first time, achieving a frame rate of 500 kHz and a sixteen-fold image compression. By using a 2D spatial disperser and a temporal disperser, a 2D image can be mapped into a serial temporal waveform. Random sampling is performed via high-speed intensity modulation and pulse compression is achieved in the optical domain. Only a low-speed ADC with a sampling rate of 50 MS/s is required to acquire the compressive measurements for image reconstruction. The key features of the proposed compressive imaging system have revealed its potential applications on biomedical imaging and surface inspection.
Acknowledgments
This work is supported by NSFC under Contracts 61120106001, 61322113, 61271134; by the young top-notch talent program sponsored by Ministry of Organization, China; by Tsinghua University Initiative Scientific Research Program; by State Environmental Protection Key Laboratory of Sources and Control of Air Pollution Complex under Contracts SCAPC201407 and by special fund of State Key Joint Laboratory of Environment Simulation and Pollution Control under Contracts 14K10ESPCT.
References and links
1. R. Pepperkok and J. Ellenberg, “High-throughput fluorescence microscopy for systems biology,” Nat. Rev. Mol. Cell Biol. 7(9), 690–696 (2006). [CrossRef] [PubMed]
2. X. Zhou and S. T. Wong, “Informatics challenges of high-throughput microscopy,” IEEE Signal Process. Mag. 23(3), 63–72 (2006). [CrossRef]
3. V. Starkuviene and R. Pepperkok, “The potential of high-content high-throughput microscopy in drug discovery,” Br. J. Pharmacol. 152(1), 62–71 (2007). [CrossRef] [PubMed]
4. R. Wollman and N. Stuurman, “High throughput microscopy: from raw images to discoveries,” J. Cell Sci. 120(21), 3715–3722 (2007). [CrossRef] [PubMed]
5. K. Pantel and M. Otte, “Occult micrometastasis: enrichment, identification and characterization of single disseminated tumour cells,” Semin. Cancer Biol. 11(5), 327–337 (2001). [CrossRef] [PubMed]
6. S. K. Kraeft, A. Ladanyi, K. Galiger, A. Herlitz, A. C. Sher, D. E. Bergsrud, G. Even, S. Brunelle, L. Harris, R. Salgia, T. Dahl, J. Kesterson, and L. B. Chen, “Reliable and sensitive identification of occult tumor cells using the improved rare event imaging system,” Clin. Cancer Res. 10(9), 3020–3028 (2004). [CrossRef] [PubMed]
7. R. T. Krivacic, A. Ladanyi, D. N. Curry, H. B. Hsieh, P. Kuhn, D. E. Bergsrud, J. F. Kepros, T. Barbera, M. Y. Ho, L. B. Chen, R. A. Lerner, and R. H. Bruce, “A rare-cell detector for cancer,” Proc. Natl. Acad. Sci. U.S.A. 101(29), 10501–10504 (2004). [CrossRef] [PubMed]
8. K. Goda, A. Ayazi, D. R. Gossett, J. Sadasivam, C. K. Lonappan, E. Sollier, A. M. Fard, S. C. Hur, J. Adam, C. Murray, C. Wang, N. Brackbill, D. Di Carlo, and B. Jalali, “High-throughput single-microparticle imaging flow analyzer,” Proc. Natl. Acad. Sci. U.S.A. 109(29), 11630–11635 (2012). [CrossRef] [PubMed]
9. M. H. Asghari and B. Jalali, “Experimental demonstration of optical real-time data compression,” Appl. Phys. Lett. 104(11), 111101 (2014). [CrossRef]
10. C. L. Chen, A. Mahjoubfar, and B. Jalali, “Optical data compression in time stretch imaging,” PLoS One 10(4), e0125106 (2015). [CrossRef] [PubMed]
11. D. L. Donoho, “Compressed sensing,” IEEE Trans. Inf. Theory 52(4), 1289–1306 (2006). [CrossRef]
12. E. J. Candes, J. Romberg, and T. Tao, “Robust uncertainty principles: exact signal reconstruction from highly incomplete frequency information,” IEEE Trans. Inf. Theory 52(2), 489–509 (2006). [CrossRef]
13. E. J. Candès, “The restricted isometry property and its implications for compressed sensing,” C. R. Math. 346(9-10), 589–592 (2008). [CrossRef]
14. E. J. Candès and M. B. Wakin, “An introduction to compressive sampling,” IEEE Signal Process. Mag. 25(2), 21–30 (2008). [CrossRef]
15. J. Romberg, “Imaging via compressive sampling [introduction to compressive sampling and recovery via convex programming],” IEEE Signal Process. Mag. 25(2), 14–20 (2008). [CrossRef]
16. H. Chen, Z. Weng, Y. Liang, C. Lei, F. Xing, M. Chen, and S. Xie, “High speed single-pixel imaging via time domain compressive sampling,” in CLEO 2014, OSA Technical Digest (Optical Society of America, 2014), paper JTh2A.132.
17. B. T. Bosworth and M. A. Foster, “High-speed flow imaging utilizing spectral-encoding of ultrafast pulses and compressed sensing,” in CLEO 2014, OSA Technical Digest (Optical Society of America, 2014), paper ATh4P.3.
18. B. T. Bosworth, J. R. Stroud, D. N. Tran, T. D. Tran, S. Chin, and M. A. Foster, “High-speed flow microscopy using compressed sensing with ultrafast laser pulses,” Opt. Express 23(8), 10521–10532 (2015). [CrossRef] [PubMed]
19. Q. Guo, H. Chen, Z. Weng, M. Chen, S. Yang, and S. Xie, “Fast time-lens-based line-scan single-pixel camera with multi-wavelength source,” Biomed. Opt. Express 6(9), 3610–3617 (2015). [CrossRef] [PubMed]
20. K. Goda, K. K. Tsia, and B. Jalali, “Serial time-encoded amplified imaging for real-time observation of fast dynamic phenomena,” Nature 458(7242), 1145–1149 (2009). [CrossRef] [PubMed]
21. Q. Guo, H. Chen, M. Chen, Z. Weng, Y. Liang, F. Xing, and S. Xie, “High speed two-dimensional temporal compressive sampling microscopic camera,” in CLEO 2015, OSA Technical Digest (Optical Society of America, 2015), paper ATu1M–2.
22. M. Shirasaki, “Large angular dispersion by a virtually imaged phased array and its application to a wavelength demultiplexer,” Opt. Lett. 21(5), 366–368 (1996). [CrossRef] [PubMed]
23. J. M. Bioucas-Dias and M. A. T. Figueiredo, “A new twIst: two-step iterative shrinkage/thresholding algorithms for image restoration,” IEEE Trans. Image Process. 16(12), 2992–3004 (2007). [CrossRef] [PubMed]
