1. Introduction

Confocal microscopy is proving increasingly popular amongst semiconductor and life scientists thanks to its unique depth discrimination ability when imaging an object. This ensures that only the thin section that lies in the focal plane of the objective is imaged efficiently, opening up the capability to reconstruct high-resolution three-dimensional images of volume structures [1]. The optical sectioning facility lies in the use of a point light source to illuminate only a single point in the focal plane, and a pinhole aperture in front of a photodetector in order to physically reject the light originating from the out-of-focus parts of the object. The illumination point is then scanned across the sample in order to build up an optically sectioned image.

Using similar confocal illumination and detection principles several confocal-like systems have emerged such as line-scanning confocal microscopes [2,3], where the object is illuminated using a slit rather than a pinhole and the light coming back from the sample is detected through a slit. The structured illumination technique has been introduced as a widefield alternative to laser scanning confocal microscopy [4] and has now become an established method for obtaining optically sectioned images in conventional microscopes [5,6]. Traditionally this technique is based on a modification of the illumination path in order to project a single frequency optical grid pattern onto the sample and acquiring images with the grid in three complementary positions. A sectioned image is then calculated from these images by looking for the modulation in each pixel as the grid is moved. All these techniques, however, still rely on mechanical scanning of some sort, in order to move an illumination pattern across the object.

Recent technological advances in ultraviolet/visible light emitting diode (LED) technology have made these light sources increasingly popular due to their high efficiency, low coherence, low-cost, uniform illumination, and long life-span. In particular, their high brightness means that they now offer a realistic alternative to arc and filament based light sources for fluorescence microscopy [7,8,9]. An additional advantage is that standard semiconductor processing techniques can be used to produce patterned LED structures where individual micro-emitter elements can be addressed electrically [10,11,12].

In this paper, we describe the implementation of a variety of optical sectioning microscopy techniques using a single micro-stripe array LED [13]. The use of this microstructured light source enables both structured illumination and line scanning confocal microscopy techniques to be realised in a flexible design with no need to modify the microscope setup to switch from one optical sectioning technique to the other. In addition the whole system contains no moving parts and offers great promise for miniaturisation in endoscopic applications.

2. Micro-structured light source and driver

2.1 Light source

The micro-structured light source is an InGaN LED consisting of 120 side-by-side and individually addressable micro-stripe elements (Fig. 1). Each stripe of the device is 17 microns wide and 3600 microns long, with a centre-to-centre spacing between stripes of 34 microns, giving an overall diode structure size of 3.6x4.08 mm. The device used here had an emission wavelength centered on 470 nm with a spectral FWHM of 18nm. However, the device structure is generic and green and UV micro-stripe devices have also been reported [13]. Each individual micro-stripe has an n-electrode rail alongside a light emitting mesa running to a common broad area n-electrode contact and each mesa has an individual pelectrode running along its length. A particular stripe is then addressed by sourcing a constant current to its corresponding p-contact with the common cathode held at ground. The optical output power of a single blue stripe was measured to be 219 µW when driven at 20 mA under a 5 V forward voltage in CW mode.

 

Fig. 1. Picture of the micro-stripe LED

Download Full Size | PDF

 

Fig. 2. Driver board

Download Full Size | PDF

2.2 Driver

A dedicated electrical driver was constructed to allow arbitrary combinations of the stripes to be driven simultaneously to produce programmable line patterns (Fig. 2). Constant currents are produced using 15 common cathode TB62710F drivers controlled by a PIC18F4550 microcontroller and the whole system allowed complete reconfiguration of line patterns in 20 microseconds. The driver board, addressed from a host computer using a LabView^™ interface communicating over a USB connection, was designed to control the board parameters and displayed patterns. The constant current value is set using external digital potentiometers on each driver chip, allowing current values from 3 to 90 mA and reconfigurable in 1.5 µs. A further 10-bit PWM modulation enables a precise brightness control of the emitters. Overall, the electrical driver is able to display up to 50,000 independent line patterns per second.

2.3 Microscope setup

Fig. 3 shows the experimental microscope configuration used to produce optically-sectioned images. The micro-stripe array is located in the illumination path of an upright Olympus BX41 microscope in a critical illumination configuration so that the illumination pattern of the LED array was imaged directly onto the sample.

 

Fig. 3. Microscope setup

Download Full Size | PDF

 

Fig. 4. scanning scheme used for grid-projection

Download Full Size | PDF

The optics in the illumination path were removed and replaced by a standard 180mm Olympus tube lens in order to avoid aberrations when projecting the LED pattern onto the object. The LED device would thus illuminate a field of 3.6/M×4.08/M mm on the sample, where M is the magnification of the microscope objective used. Fluorescence gathered by the objective forms an image on an Orca ER CCD camera. Because the objective is used both to illuminate and to image the sample, the magnification of the pattern image onto the camera does not vary when microscope objective is changed but only depends on the ratio of the focal lengths of the illumination and imaging tube lenses. In our actual setup both tube lenses are conventional Olympus 180 mm tube lenses so this ratio is 1.

3. Structured illumination microscopy

Grid-projection structured illumination microscopy relies on the fact that higher spatial frequencies attenuate quicker with defocus than lower ones. If we incoherently project a grid pattern onto an object, the modulation in the illumination will attenuate with defocus [14] and hence will only be significant in the thin section of the sample that is in focus. By looking for the modulated part in the image of the illuminated object, we can extract an optically sectioned image. The simplest way to do this is to move the grid pattern and look for changes in the image. This can be achieved minimally by shifting the grid laterally to three equally spaced positions separated by one third of the grid pitch, acquiring an image, I ₁, I ₂ and I ₃, for each position respectively and then calculating the sectioned image according to the equation:

A conventional wide-field image can similarly be retrieved by calculating

Structured illumination microscopy has previously been demonstrated in brightfield and fluorescence modes by projecting a fixed mask pattern [4] or by interfering laser two beams onto the sample [15]. In either case moving parts were used to either shift the mask spatially or the phase of one laser beam relative to the other.

Here, we demonstrate a non-moving part alternative to the piezo-electrically driven grid pattern technique by projecting a programmable LED micro-stripe grid pattern onto the object to be imaged. Fig. 4 shows how the three different grating positions were be achieved by successively shifting a three-stripe repeat grid pattern by one stripe at a time. The grid pitch used is the smallest possible with this device at 102 µm, although more complex scanning schemes can also be achieved using more than three LED stripes and hence longer periods. This solid-state scanning approach has the advantage of removing the possible inaccurate grating shifts that give rise to artefacts in the sectioned image [16], but does suffer currently from introducing higher grid harmonics into the illumination pattern that can also result in artefacts in the sectioned image.

 

Fig. 5. 20x images of stained pollen grains acquired with grid-projection structured illumination. (a) Modulated raw image [Media 1], (b) Sectioned image, (c) Conventional image

Download Full Size | PDF

Fig. 5 illustrates the structured illumination technique applied to optically section stained pollen grain. The pollen grain images were acquired using a 20x 0.5NA Olympus objective, with 500 ms exposure per image and 3 mA current per LED stripe. In the image sequences shown in Fig. 5(a) we can see that light from pollen grains lying out-of-focus is not modulated whereas that from the in-focus grains is strongly modulated. In the sectioned image, Fig. 5(b), the out-of-focus objects are clearly suppressed compared to the conventional image, Fig. 5(c). Residual artefacts are presenting the form of a high frequency modulation across the image. This can potentially be reduced by using a finer LED line array structure with more LED elements per grid period. Other artefacts are visible due to imperfections in the array such as missing and shorted elements, which can obviously be removed by improved device fabrication.

4. Line scanning confocal microscope

In a conventional point scanning confocal microscope, the object is illuminated with the focused image of a point source, or laser, and the reflected or fluorescence light intensity is measured with a point detector, or pinhole, focused into the same point on the sample. Optical sectioning is achieved because the out-of-focus light is blurred at the pinhole and so only a small fraction passes, while the in-focus light is not blurred and can pass through the pinhole to be detected. Using slit illumination and slit detection instead of a point source and pinhole can similarly achieve optical sectioning and has advantages in terms of light budget when using spatially incoherent light sources (such as LEDs) and scanning speed, although with some loss of axial resolution in particular. In practice an arrangement of slits and cylindrical lenses forms an illumination line, which is projected onto the object to be imaged and the confocal detection slit, physically rejecting the out-of-focus light, is placed in front of a linear array detector. In order to build up a complete image, the illumination line and slit detector are scanned across the sample or the object is scanned across the line field of view.

Rather than scanning the line pattern or the object, we used the micro-stripe LED to illuminate the object one line at a time. For each line position, an image is captured on a 2-dimensional array detector and the light originating from the out-of-focus region of the object is rejected by post-processing the stack of N=120 images corresponding to the N stripe positions. Several potential detection schemes can yield optically sectioned images similar to those of a line-scanning confocal microscope. We describe two such methods: the first using using a virtual slit and the second a simple maximum projection calculation.

4.1 Confocal detection using a virtual slit aperture

Because a CCD camera is an array of point detectors, a simple way of implementing the slit confocal detection is to create a binary mask for each stripe position. Then by multiplying the image captured by the corresponding mask image, we can physically reject the out-of-focus light using this virtual slit. The confocal image is finally calculated by summing the N=120 masked images. This reconstruction can be described by the equation:

where I_i is the image captured when illuminating the sample with the i ^th LED stripe and Mask_i is the corresponding binary mask.

In order to create the set of binary masks, we acquire a set of N reference images obtained by imaging a thin and homogenous polymer film or a mirror in focus under single stripe illuminations. From this set of reference images, several methods can be used to work out the corresponding set of binary masks, including thresholding and edge detection techniques. The particular method we use is to detect for each pixel the position of the maximum intensity through the stack of N reference images. The i ^th mask can then be described as the group of pixels having the same maximum position through the stack of reference images, that is to say

Where MAXPOS_z(x) denotes the index of the maximum along the stack z-direction for the pixel x. This method is insensitive to sharp brightness variations along a single stripe, is computationally fast and attributes each pixel in the image to a slit mask. For our devices with a 17 µm LED width on a 34 µm centre to centre spacing this will result in an effective 17µm illumination slit and, on average, a 34 µm detector slit width. We note that method of generating confocal detection slits is independent of the precise array detector geometry, providing the detector pixel size is fine enough, and can be applied whatever the orientation of the illumination stripe array with respect to the array detector.

4.2 Maximum intensity confocal detection

A post-processing confocal reconstruction method [17] has was also implemented in which a confocal image is calculated from a stack of N object images without passing the detected fluorescence light through a detection slit, virtual or otherwise. In this method N wide field images of the object are acquire, one for each stripe position as before, and the maximum intensity from those N images is chosen for each image pixel. The sectioned image can thus be described by the equation:

where MAX[] denotes the maximum value from a set of pixels. Similarly, a wide field image I_WF can be calculated by averaging the N images.

Although the maximum intensity projection technique is rigorously equivalent to the confocal detection technique when imaging an in-focus object, contribution of out of focus light and out-of-focus light from ‘neighbouring stripes’ can lead to a distortion when the amount of outof- focus light is significant. However, the intensity of the distorted part of the image is usually very weak compared to the in-focus contribution and is barely noticeable in practice.

4.3 Results

Finally, we show the same cluster of stained pollen grains imaged with both these linescanning techniques. In both cases the same set of 120 images was used, each of which was acquired for approximately 10ms at an LED stripe current of 30 mA. Fig. 6(a) is a reconstructed conventional image (calculated by summing all 120 images) clearly showing large out-of-focus pollen grains. Fig. 6(b) and Fig. 6(c) show two optically sectioned confocal images processed using the maximum projection and the automatic slit detection methods respectively. The animated sequences show how the conventional and maximum projection sectioned image are built up as the line illuminated images are acquired.

In the maximum projection technique no detection slit is required making the implementation straightforward. The sectioning capability comes here from the fact that infocus parts of the object are usually brighter than those out-of-focus when illuminated with patterned light. Because the light originating from an out-of-focus object is not physically rejected, it will still contribute to the maximum projection image when there is no in-focus signal. Also, this technique tends to have a larger background than that of the auto-slit confocal as it will always choose the brightest value from a set of noisy pixel samples.

 

Fig. 6. 20x magnification images of stained pollen grains (a) conventional image, (b) maximum projection confocal image and (c) automatic slit confocal image. Movies of (a) and (b) show how the images evolve as the line is scanned across the sample [Media 2] [Media 3]

Download Full Size | PDF

5. Sectioning strength results

In order to demonstrate and quantify the sectioning ability of our microscopes, we simulated and then measured the detected signal in the confocal images as a thin fluorescent sheet is scanned through the focus of the microscope objective. Such an experiment provides a good indication of the ability of the system to reject out-of-focus light as it is only this zero spatial frequency that is not sectioned in even conventional microscopes, and which contributes most to out-of-focus blur. For these simulations and measurements we used a 40X Olympus UPlanFl 0.75 NA air microscope objective together with the corresponding Olympus tube lens and an appropriate Olympus blue filter cube. The images were captured with a Hamamatsu firewire ORCA ER camera triggered by the LED driver board, and the fluorescent sheet was moved from -10µm to +10µm in 0.2µm steps. Experimental axial response curves were calculated by integrating the total signal across the sectioned image as a function of defocus. The simulations are based on paraxial imaging theory and are only an approximation for such a high NA, but still show good agreement with experimental measurement.

5.1 Grid projection structured illumination microscopy

We model the system performance as a case of incoherent imaging of the grid illumination onto the object followed by incoherent imaging of the resulting fluorescence emission onto the detector. The sectioned image signal strength for such a uniform sheet object is thus the product of the illumination modulation transfer function, MTF_i, and the detection modulation transfer function, MTF_d, at the normalised grid spatial frequency, ν′, as a function of normalised defocus, u :

Where for small Stokes shift between excitation and emission wavelengths we assume that MTF_i(u,ν′)=MTF_d(u,ν′)=MTF(u,ν′) and u is related to the actual defocus, z, and ν′ to the the projected grid pattern pitch, Λ, by the equations:

The numerical aperture of the system is given by NA=n sin (α)and the free space wavelength is λ. We further use Stokseth’s[18] approximation to the MTF:

To give a sectioning strength Ĩ _sec (u) normalized to unity at zero defocus:

The results of this calculation are plotted in Fig. 7 along with measured values from the microscope as described above. Good agreement between the experimental and the theoretical defocus response can be seen clearly. There is an offset in the measured signal which can be attributed to device imperfections such as missing and shorted lines. The defocus response half width half maximum was measured to be 0.94 µm while the theory was predicts a HWHM width of 0.83µm for this 102µm grid and blue illumination, λ=470nm.

 

Fig. 7. Theoretical and experimental axial responses of the structured illumination system

Download Full Size | PDF

5.2 Line scanning confocal

For both line scanning schemes described above, the sectioning strength for a fluorescent sheet should be identical and can be modelled by considering the incoherent imaging of the illumination slit onto the sheet object followed by the incoherent imaging of the resulting fluorescent image onto the detector slit. The sectioning strength can then be calculated by summing all the light passing through the detector slit. This calculation can be calculated by an integral in Fourier space across the product of the illumination and detection lens 2-D incoherent optical transfer functions multiplied by the Fourier transform of the illumination and detection slits. Assuming an aberration-free system where both illumination and detection lenses are identical and excitation and emission wavelengths are the same, the defocus response of a fluorescence slit microscope under incoherent illumination can thus be written as:

where d_i and d_d are the normalized width of the illumination and detection slits, respectively:

The OTF can be approximated again from Stokseth by:

The results of this calculation are plotted in Fig. 8 along with experimental data recorded using automatic virtual slit detection. Again good agreement between theory and experiment are seen apart from a small offset (~5%) in the experiment, which in this case can be attributed to the light scattered from the back of the LED device, that has no structure in the image plane.

 

Fig. 8. Theoretical and experimental sectioning strengths of a slit scanning confocal microscope.

Download Full Size | PDF

The experimental measurements indicate a FWHM response of 3.0µm, while the theory suggests a FWHM of 2.6µm. Both graphs also show significant out-of-focus tails, which, along with the large FWHM, indicate the poorer sectioning strength of line scanning techniques compared to grid projection. This borne out in the images of Fig. 6(b) and Fig. 6(c) compared to Fig. 5(b). We also processed the z stack of raw confocal data both using the maximum intensity method confocal slit and, as expected, this showed a similar response.

6. Discussion

6.1 Device emission uniformity

As the LED device is placed in critical illumination, any non-uniformity in the device emission results in artefacts in the confocal images. As seen in Fig. 1, non-uniformities such as missing stripes, non-uniform areas or emission attenuation along a single emitter are still present in this initial generation of LED devices. Grid-projection structured illumination is a sectioning technique that uses differences to generate image contrast. Missing stripes or stripes that are shorted together can result in a dramatic degradation of the local performance of the microscope. This degradation includes loss of in-focus signal and appearance of out-off-ocus artefacts, both of which can be seen in Fig. 5(b).

Emission non-uniformity is also present along the length of individual emitters. Stripes appear brighter at their electrical p-connection end and get gradually dimmer away from that connection. This is due to the series resistance of the p-connector increasing along the length of the emitter. To compound the problem the p-connections come from different ends at the left and right of the device as seen in Fig 1. As a result, the object to be imaged looks brighter at the connection end of the stripes and other artefacts appear in the middle of the device when the connection end changes from top to bottom. In order to tackle this problem, a double ended connection device is being developed along with improved p-connection drive-line resistance.

Finally, a residual modulation can be seen, particularly in the grid-projection structured illumination images, which results from the 50 percent fill factor of the LED array. This too can be improved by the use of cylindrical micro-lens structures integrated on top of the devices that will both improve the fill factor and allow the use of even lower mark-to-space ratio emitters, which in turn will reduce heat production on the device.

6.2 Relative merits of grid-projection and line scanning techniques

Grid-projection structured illumination microscopy clearly has the advantage of superior sectioning strength over that of the line scanning techniques but at the disadvantage of poorer resilience to device imperfections in terms of artefacts. The images obtained above for gridprojection structured illumination were obtained with 3 images of total 1.5 seconds exposure time at a total LED current of 240mA. The line scanning images were obtained with 120 images for a total exposure of approximately 1.2 seconds at a total LED current of 30mA. The total light exposure of the object in the line scanning case is thus approximately a factor of 10 times less than that for grid-projection. This is due to a combination of the fact that the superior sectioning strength of grid-projection requires more light to get a better signal form a thinner section. It is also more susceptible to noise because it relies on calculating differences between images to reject out-of-focus light, whereas the line-scanning techniques merely reject that light by ignoring it. The differencing operation of the grid-projection technique also makes it more susceptible to motion artefacts when the object moves between acquired images. Both sets of images were obtained with approximately the same exposure time, but the overhead associated with reading out each image into the host computer meant that the line-scanning techniques took approximately 10 seconds to acquire the full set of 120 images.

6.3 Advantages and prospects for the micro-stripe array microscope

Because of the camera readout time, the speed our system does not compare yet with fast sectioning system such as spinning disks or laser line scanning microscopes, which can acquire up to hundreds of images per second. However, there is significant scope for improvement using systems of the kind we have described here for microscopy applications. As CCD camera technology continuously improves, faster cameras will eventually reduce dramatically the read-out time (currently a significant limitation). It is envisaged, in particular that selective region-of-interest read-out such as is currently possible with CMOS cameras, would significantly increase the speed of the line scanning techniques. The light emitting diodes diodes used here are currently first generation devices, and brighter and more uniform devices are currently under development using flip-chip semi-conductor techniques. We also expect the light efficiency of such devices to improve significantly, with integrated structures such as micro-lenses playing a key role. Finally, variations on our techniques, such as scanning more than one stripe at a time will reduce the total acquisition time.

A clear advantage of our microscope design is the possibility to switch from one optically sectionning imaging technique to the other without changing the microscope setup. For instance profiling reflective surfaces or imaging thin fluorescent samples will give better results using the grid-projection technique due to the inherently better sectioning strength. However, due to the high susceptibility of the structured illumination technique to photon scattering and noise from out-of-focus light, thick specimen and biological tissues will probably be imaged better with the line scanning techniques, increasing the signal to noise ratio at the expense of a lower sectioning capability. As a wide-field, single-photon technique, these imaging modalities are relatively harmless to the sample in terms of photobleaching. While they do still produce out-of-focus photobleaching compared to twophoton techniques, they do not result in high illumination intensities that can cause significantly higher in-focus photobleaching in single-point techniques such as two-photon and single-photon laser scanning confocal microscopy.

7. Conclusions

Using a micro-stripe LED to obtain optical sectioning in a widefield microscope is a light efficient and flexible technique that allows switching between several imaging methods, such as structured illumination and line scanning microscopy, with no need to change the microscope setup. A user can therefore choose the imaging technique that best suits the specimen, and important facility when imaging cells and tissues, objects with very different optical properties.

Finally, the micro-stripe LED has a potential for optically sectioned imaging in endoscopy applications because the ability to generate and scan the grid or line pattern is intrinsically combined with the light source in a compact device and needs no external mechanical scanning system.