1. Introduction

In general, commercial linear and nonlinear optical microscopes are restricted to acquire images from only one focal plane at a time. Using piezoelectric translators, it is possible to rapidly change the position of the focal plane in the z-direction (axial). However, for studying phenomena such as blood flow, intracellular transport, out-of-plane organism motion, and other dynamical processes it is desirable to have the ability to access images from different focal planes taken at exactly the same time. There have been few reports of multidepth imaging in the literature [1–3]. We recently introduced a simultaneous, multidepth, two-photon absorption (TPA) fluorescence microscopic imaging method based on temporal demultiplexing of the signal coming from different focal planes [3]. Here we report on significant advances to this work by 1) demonstrating simultaneous, dynamic image capture from different focal planes, 2) demonstrating simultaneous image acquisition for orthogonal excitation polarizations for the first time, and 3) acquiring simultaneous, multimodal images in both the transmission and excitation (or “epi”) directions.

Figure 1 shows a schematic of the experimental setup. Briefly, the light source consists of a home-built, p-polarized, extended cavity Ti:Sapphire laser that produces 70 fs pulses, at 23 MHz centered at 808 nm, and 200 mW average power. We split the fundamental beam using a polarizing beamsplitter; 50% of the power is delayed in time by one-half of the laser period while the remaining 50% is retro-reflected on a deformable mirror (DM) (Agiloptics, 25 mm aperture, 37 actuators). As shown in Fig. 1 the beam is expanded to fill the DM aperture. The DM is used to change the divergence of the beam, making it possible to actively focus this arm to different depths. There is a single quarter waveplate (zero-order, centered at 800 nm) located in each of the individual arms that enables the two beams to be recombined at the polarizing beamsplitter. After the beamsplitter, the pulse train measures 46 MHz, with consecutive pulses having alternate polarizations. The recombined beam is directed to the scan mirrors and through a one-to-one telescope that forms a telecentric imaging system which relays the plane located mid-way between the scanners onto the entrance pupil of the excitation objective. The images presented in this work were taken with two different excitation objectives: a Zeiss A-plan 40×, 0.65 NA, and a Zeiss Achroplan 100×, 1.25NA, oil immersion. A dichroic mirror is used to separate the signal collected in the epi direction from the excitation beam. Two different dichroic mirrors were used to acquire the epi images presented below: 1) for second harmonic generation (SHG) epi imaging, we used a 3 mm thick dichroic that reflects the wavelength band from 360 nm to 460 nm and transmits 800 nm, 2) for TPA fluorescence images in the epi configuration, a 3 mm thick dichroic that reflects the wavelength band from 580 nm to 700 nm and transmits 800 nm was used.

We further modified the previously published setup by adding a collection imaging system for capturing nonlinear optical signals in transmission. In all cases where simultaneous imaging in the transmission and epi direction are reported a matched set of excitation and collection optics was used, namely Zeiss A-plan 40×, 0.65 NA objectives. A cooled photomultiplier tube (Hamamatsu H7422P-40) is used to detect photons in the epi direction, and an uncooled photomultiplier tube (Hamamatsu R7400-U4) to detect light in transmission. The signal processing electronics for each PMT are identical and are described in Refs. [3,5]. For collection of all SHG signals we use an additional colored glass filter (3 mm, BG39) and an interference filter (centered at 405 nm) in front of the PMT to prevent TPA fluorescence and fundamental laser light from reaching the detector. For TPA fluorescence detection in the epi configuration a 3 mm BG39 glass filter was used in front of the PMT (in combination with the second dichroic described above).

 

Fig. 1. Schematic representation of the experimental setup (not to scale): D, dichroic mirror; DM, deformable mirror; F, interference and/or color glass filters; FM, flipper mirrors; HWP, half wave plate; L, lenses (L1=200 mm; L2, L5, L6=100 mm; L3, L4=160 mm); PBS, polarizing beam splitter; PD, photodiode; PMT, photomultiplier tube; QWP, quarter wave plate. The CCD camera is a Hamamatsu chilled CCD model C5985. The pulse train is divided in two parts at the beamsplitter, one delayed by one-half the laser period and the other one reflected from a deformable mirror to change its divergence properties. Both parts are recombined after double passing quarter wave plates to form a 46MHz pulse train. The photodiode is used to provide a laser clock for demultiplexing. Note the alternating polarizations of the pulses after the beamsplitter. By means of the flipper mirrors we can obtain white light images of the region of interest.

Download Full Size | PDF

Since the two interlaced pulse trains from each of the two different paths have different divergences, they are focused by the objective to different focal planes within the sample. Explicit details on the DM can be found in Ref. [3], where measurements of depth vs. voltage and of axial resolution for a pair of focal planes separated by 8 µm are provided. These are the exact image conditions used in this work for separated image planes. For the axial resolution of each of the two focal volumes we measured a full width at half-maximum (FWHM) of 3.5 µm for the delayed path and 4.6 µm for the DM path. This compares favorably to the calculated FWHM for a 0.65 NA optic (used at 800 nm) which is 3.2 µm. Ultimately the DM can be used for aberration control, though only a simple defocus phase term was applied for this work. The signal produced in each of the two focal planes is separated based on its relative timing with respect to the laser pulses (monitored with a photodiode). A demultiplexer and two counters are programmed into a field programmable gate array (FPGA, Altera DE2) card [4]. Each counter is 16 bits deep, i.e. capable of up to 65536 counts thus providing excellent dynamic range for varied pixel dwell times. With every laser pulse from the photodiode, the demultiplexer toggles between each counter. This results in each counter storing the information from a single depth. When the FPGA card receives a pixel clock tick, the final value of each counter is recorded to onboard memory and reset to zero. The stored values are sent to a computer over USB, where the intensity images coming from each depth are generated and displayed. Notably, this detection system is an exceptionally cost-effective technology that can be used to significantly extend the capability of existing multiphoton microscopes. A more detailed description of the FPGA software and hardware architecture is given in Ref. [5].

It is important to emphasize that the system does not rely on a CCD camera for image acquisition, as many multi-foci techniques do [6–8]. As a result this “nonimaging” multifocal image acquisition scheme can be used with scattering specimens. The average power at the sample for each of the images reported here was approximately 5 mW in each one of the two focal spots. For all the images presented in Figs. 3–6 the DM voltage was set to zero in order to acquire simultaneous images from the same plane; however, it is straightforward to offset the two image planes in z by several microns, if needed. Notably, the use of photon counting techniques allows the direct retrieval of quantitative information from the images once the throughput of the system is calibrated.

2. Methods and results

2.1 Demonstration of dynamic image acquisition

In our first generation multifocal system, we were limited to a simultaneous image frame rate of 0.1 frames/s. The limiting factor was that the scanners were running in a step-and-go mode rather than in true scanning mode. This introduced many delays associated mainly with communicating with the computer over an RS-232 serial bus, thus limiting the speed. In order to image at much faster frame rates, we are now running the scanners in raster mode. With this change we have achieved rates of approximately 8 frames/s. To implement the raster scan, the scanners are programmed through the same serial bus but only at the beginning of the imaging process. A line synchronization signal from the scan controller was implemented to tell the FPGA when a line scan is finished. The line synchronization signal is produced by the scanner controller directly in TTL form, as required by the FPGA. The code loaded into the FPGA card had to be modified in order to account for this new signal [5].

An example of the dynamic imaging capability is illustrated in Fig. 2. The microorganism Euglena exhibits a strong endogenous TPA fluorescence signal and rapid, constant movement. It is thus an excellent specimen for demonstrating the utility of the system - dynamic imaging of multiple focal planes enabling tracking of out of plane motion.

 

Fig. 2. Series of two-images taken simultaneously in the epi configuration using a 0.65 NA objective. The images show Euglena swimming from the top focal plane and crossing into the lower plane. The two depths are separated by 8 µm. Each set of depth 1-depth 2 images was taken simultaneously with a single detector, the time separation between adjacent frames is ~650 ms.

Download Full Size | PDF

Sample preparation consisted of simply mounting a drop of water containing Euglena between two microscope coverslips. Figure 2 shows selected frames of a series of 100 sets of two simultaneous TPA fluorescence images taken in the epi direction at a frame rate of 3 frames/s. This was found to be the optimal frame rate for this specimen — freezing the Euglena motion, while providing sufficient photon counts that structure within the Euglena is readily resolved. To acquire these images the DM voltage was set so that the two focal planes are separated by 8 µm in the axial direction. The original 128×128 pixel images were cropped to show only the area of interest. The panels clearly show Euglena swimming from the top row panels into the bottom ones. In particular, it is possible to see the Euglena in both planes in the second and third sets of images. These images clearly illustrate the ability of the imaging system to track dynamical processes. The exposure time (frame rate) is set from software control, and thus is easily optimized for specimen conditions. Line scan rates have been tested to 14 kHz and the counting-demultiplexing electronics to 167 MHz, so that true video rate imaging is possible. Even these rates do not push the FPGA capabilities, which the manufacturer specifies to 401 MHz for 16 bit counters. Presently the frame rate is limited by the use of galvanometric mirrors. A high speed scanning system incorporating a rotating polygonal mirror is being designed and will be implemented in the next system upgrade.

2.2 Simultaneous second harmonic imaging (SHG) with two different polarizations

One of the distinctive features of this imaging system is that not only can the spatial properties of the beams be independently controlled; the two beams are also at orthogonal polarizations which leads to the interesting (and useful) possibility of simultaneously imaging systems as a function of the two separate polarizations. We demonstrate how this feature can be readily exploited through several examples of second harmonic generation imaging. The sample for simultaneous SHG imaging at orthogonal polarizations is cornstarch in water mounted between two microscope coverslips [9,10]. We used cornstarch because of its large polarization dependent SHG yield. Starch granules are semi crystalline, radially arranged spherical structures of amylose and amylopectin. Due to this radial arrangement of the starch granule, the SHG image recorded with a linearly polarized laser beam appears as a two-lobe structure oriented parallel to the polarization direction of the laser beam. The SHG intensity scales with cos²θ, where θ is the angle of the SHG radiating dipole orientation with respect to polarization of the laser beam. The SHG intensity is at a minimum when the laser polarization and SHG emitting dipoles are oriented perpendicularly [11–13].

 

Fig. 3. Second harmonic generation (SHG) images of cornstarch in water taken simultaneously with different linear excitation polarizations of the fundamental light. (a) White light image of the region of interest. Panels (b) and (c) show SHG images of the same area and same depth taken simultaneously with two linear polarizations perpendicular to each other. Images are 128×128 pixels, 0.65 NA objectives were used for excitation and collection; the acquisition time was 12 s.

Download Full Size | PDF

Figure 3(a) shows a white light image of the region of interest. Figures 3 (b) and (c) show the SHG images of the same area at the same depth taken simultaneously with two different polarizations, the lobe structure is clearly visible, and tracks the polarization of the excitation beam. The images were acquired in transmission. The color scale accompanying each SHG figure is given in photons per pixel. In this case the pixel acquisition time is approximately 730 µs, in contrast with the approximately 20 µs pixel dwell time used in Fig. 1. This also shows the capability of the system to accommodate a broad range of exposure conditions.

 

Fig. 4. Second harmonic generation (SHG) images of cornstarch in water taken with different linear polarizations of the fundamental light. (a) White light image. Panels (b) and (c) show SHG images taken simultaneously with two linear polarizations perpendicular to each other. Images are 128×128 pixels, a 1.25 NA objective was used for excitation and a 0.65 NA objective was used for collection; the acquisition time was 12 s.

Download Full Size | PDF

Figure 4 shows a second example of SHG imaging of cornstarch in water using a 1.25 NA objective (oil immersion 100×, 1.25 NA, Zeiss Achroplan) for excitation.

2.3 Multimodal microscopy

As a final demonstration of the flexibility of the system, we demonstrate simultaneous imaging in the epi and transmission directions, acquiring a set of two images (one for each orthogonal linear polarization state) in each direction. In order to obtain two sets of images simultaneously (two in the transmission direction, orthogonal polarizations, and two in the epi direction, orthogonal polarization, four images total) we split the laser clock, pixel clock and line synchronization TTL signals to two identical FPGA’s, and connect one PMT to each card (after converting its signal to TTL [5]). The computer program that handles the communications with the FPGA board was modified so that two independent boards could be controlled simultaneously. Panels (a) and (c) of Fig. 5 show SHG images recorded in the epi direction, while panels (b) and (d) show the SHG images acquired in transmission. Once again, the images in all directions clearly show the polarization dependence of the cornstarch. The intensity difference between images taken in the same modality, epi [(a) and (c)] or transmission [(b) and (d)] is due to a slight intensity imbalance between the two-excitation beams. The field of view for these simultaneously recorded images was reduced with respect to that of Fig. 3 to isolate a single cornstarch granule.

 

Fig. 5. Simultaneous SHG images of cornstarch in water. Focal planes are the same only the polarization differs. Panels (a) and (c) were taken in the epi configuration (orthogonal polarizations); (b) and (d) were taken in transmission (orthogonal polarizations). Images are 128×128 pixels, 0.65 NA objectives were used for excitation and collection; the acquisition time was 12 s.

Download Full Size | PDF

Figure 6 shows the multimodal capability of the microscope, this time simultaneously detecting TPA fluorescence in the epi configuration and SHG in transmission. The sample consisted of a mixture of cornstarch granules (for strong, polarization dependent SHG), and 10 µm diameter polystyrene crimson fluorescent microspheres (Invitrogen; Carlsbad CA, USA) in water mounted between coverslips. Panel (a) shows an image of the region of interest, the morphology allows for easy identification of both sample components. Panels (b) and (d) were taken in the epi configuration using the dichroic optimized for the TPA fluorescence (580–700 nm reflection bandpass). Panels (c) and (e) show SHG images acquired in transmission. Clearly, images on panels (b) and (d) are complementary to those in (c) and (e), thus demonstrating a high degree of discrimination between TPA fluorescence and SHG signals. All four of these images were taken simultaneously with an acquisition time of 8 s.

 

Fig. 6. Simultaneous TPA fluorescence and SHG images from a mixture of cornstarch and fluorescent microspheres in water. Panel (a) shows a white light image of the region of interest. (b) and (d) are TPA fluorescence images obtained in epi configuration (orthogonal polarizations). (d) and (e) are SHG images obtained in transmission (orthogonal polarizations). Images are 128×128 pixels, 0.65 NA objectives were used for excitation and collection; the acquisition time was 8 s.

Download Full Size | PDF

Finally we want to draw attention to the fact that all the images that were presented in this report are raw data. The only modification to the image data was cropping the area of interest in Fig. 2. No background subtraction was performed in any of the images. The use of photon counting, as opposed to analogue temporal integration of the signal, allowed for low background, high contrast images. Further, as discussed before, photon counting also allows for a straightforward quantification of the image intensity once the system throughput has been calibrated.

3. Conclusion

In conclusion, we have developed a novel, photon counting, multifocal, multiphoton imaging system. In contrast to most multifocal imaging systems, it uses nonimaging detection so that it can be used with scattering specimens. The image demultiplexing is performed with inexpensive field programmable gate arrays, enabling this technology to be employed in existing multiphoton microscopes. The resultant images are quantitative. We have, for the first time, demonstrated dynamic imaging from multiple focal planes, simultaneous acquisition of SHG images from orthogonally polarized excitation beams, and simultaneous multimodal imaging (TPA fluorescence in epi and SHG in transmission).