1. Introduction

In recent years, optically variable devices (OVDs) [1], such as holograms and diffraction gratings, have been widely used in security documents, such as banknotes, passports, credit cards, and driver's licenses. OVDs are devices whose color and appearance drastically change according to the illumination and observation directions. These gonio properties of OVDs are important in their role as anti-counterfeiting devices for security documents, preventing fraud and counterfeiting using conventional photocopiers, image scanners, and printers.

The gonio properties of OVDs are important in the authentication of security documents, and they are also crucial in the field of forensic document examination. The difference in gonio properties between a questioned sample and reference materials often gives us effective information about whether or not the sample is genuine. The optical phenomena involved in the gonio properties of OVDs are diffraction and interference in many cases. The light diffracted by gratings and the interference light in thin layers in OVDs show drastic changes according to slight changes in their microscopic structures or optical parameters.

Machine-based gonio property analysis will be necessary as we move toward secure authentication of OVDs. In many cases, the inspection of gonio properties of OVDs is based on visual inspection by human eyes. It is typically performed by holding a document and flipping and rotating it to observe the appearance under various illumination and observation conditions. However, it is quite difficult to detect and understand the complicated gonio properties of OVDs only with the human eye.

Efficient gonio imaging is important in machine inspection of OVDs' gonio properties. In general, gonio imaging is considered time-consuming and cumbersome since it requires a large number of image acquisition procedures under varying illumination and observation geometries. It also tends to require large, complicated equipment [2,3].

There are several studies concerning efficient measurement of gonio reflection properties of objects. Dana et al. [4] proposed an efficient acquisition method based on a bidirectional reflectance distribution function (BRDF) [5] by using a concave parabolic mirror. This method measures the BRDF of one point on the object at a time; however, it still requires two dimensional scanning to obtain gonio images. Han et al. [6] used a method involving simultaneous acquisition of bidirectional texture reflectance by using a kaleidoscope. This research is in the context of measurement of a bidirectional texture function (BTF) [7]. Efficient measurement is achieved by skillfully expanding the observation angle via multiple reflections in the kaleidoscope. However, the obtained image is not rectangular. In addition, a large kaleidoscope is required in order to enlarge the range of observation directions.

Multichannel devices also provide efficient gonio imaging methods. Müller et al. [8] proposed a system with a dense array of digital still cameras uniformly mounted on a hemispherical structure. A similar gantry was used by Malzbender et al. to capture Polynomial Texture Maps [9]. These systems enabled us to capture gonio images without scanning the image sensors and the light sources. However the size of these systems is still large.

A microlens array has the ability to acquire a light-field including directional information of the light wave, such as in 3D imaging by integral photography [10,11] and a plenoptic camera [12]. Tanida et al. proposed a compound-eye image-capturing system called TOMBO (Thin Observation Module by Bound Optics) [13]. The TOMBO system is an optoelectronic hybrid system for computational imaging with a thin optical system. It has been applied to a variety of applications, including color imaging [14], spectral imaging [15] and high-resolution image reconstruction [16]. Recently, the framework for generalized sampling approaches for measuring multi-dimensional objects by using TOMBO was proposed [17].

In this paper, we propose an efficient method of gonio-imaging of OVDs by using compound-eye optics and a convex lens. The TOMBO system is used as the compound-eye optics. After the principle of the proposed method is described, an experiment involving gonio image acquisition of a holographic grating is demonstrated, and then we discuss issues related to the proposed method.

2. Method

Figure 1 shows the principle of the proposed method. The optical element used in this method is composed of a convex lens and a microlens array. First, the light from a sample is collimated by the convex lens, which is located away from the sample by the focal length f of the lens. The collimated light proceeds to the microlens array composed of multiple convex lenses. The light is focused by each convex lens onto the image plane, and therefore, multiple images are obtained.

 

Fig. 1 Principle of the gonio-imaging by using compound-eye optics.

Download Full Size | PDF

The key point of this method is that angle-resolved light from the sample is focused individually by each microlens. As seen from Fig. 1, the position of each convex lens in the microlens array corresponds to a different angle of the light from the sample. Therefore, the image focused by each convex lens corresponds to an image of the sample taken from a different observation direction.

Now, let us formulate the image formation in the proposed optical configuration shown in Fig. 2 . In Fig. 2, P₁ is the object plane, L is the convex lens, M is the microlens array, and P₂ is the image plane. For simplicity, assume that L and the center microlens in M are coaxial. The focusing condition is derived from the lens equation under the condition of the two-lens combination, as follows:

 

Fig. 2 Schematic diagram of image formation in the proposed optical configuration.

Download Full Size | PDF

The observation direction at each microlens in M is introduced. Figure 3 is a schematic diagram of geometrical variables defining the measurement geometry. The vectors i and r are the directions of illumination and observation, respectively, and the vector n is the surface normal of the measurement sample. The angles ${\theta }_i$ and ${\phi }_i$ are the polar and azimuthal angles of the illumination direction, respectively. The observation angle is represented in the same manner as the incident direction by replacing suffix i with r. The polar angle ${\theta }_r$ of observation direction r at a microlens whose center is located at $\left(x\text{'},y\text{'}\right)$ in plane M is represented by

 

Fig. 3 Schematic diagram of geometrical variables defining the measurement geometry.

Download Full Size | PDF

3. Experiments

The feasibility of proposed method was demonstrated by an experiment. Figure 4 shows a schematic diagram of the experimental setup. The distances corresponding to a, d, and b in Fig. 2 were 25.5 mm, 7 mm, and 1.3 mm, respectively.

 

Fig. 4 Schematic diagram of the experimental setup.

Download Full Size | PDF

The TOMBO system was used as the compound-eye optics. Figure 5 shows a schematic diagram of TOMBO. It consists of a microlens array, a signal separator, and an image sensor. Multiple images are captured by the individual imaging units. A small image captured by one unit is called a unit image. Table 1 summarizes the specifications of the TOMBO system used in this study. A CCD color image sensor with a Bayer filter array was used. The number of units was 10 × 10. A color image of 80 × 80 pixels was captured in each unit image. The illumination was white light from a quartz halogen light source (Dolan-Jenner Industries, DC-950H). The light was guided by a glass fiber bundle, and shaped by a holographic diffuser (Edmund, 54499) with diffuse angle of 25 deg. The polar angle ${\theta }_i$ and the azimuthal angle ${\phi }_i$ of the illumination direction i were approximately 67 deg and 0 deg, respectively. The objective lens was an achromatic lens (Edmund, 45399-F) with a focal length $f_1$ of 18 mm and a diameter of 9 mm. The magnification K of this imaging setup was 0.06, and measurement region on the sample was approximately 9 mm × 9 mm.

 

Fig. 5 Schematic diagram of TOMBO.

Download Full Size | PDF

Table 1. Specifications of the TOMBO System

View Table | View all tables in this article

Figure 6 shows a schematic diagram of the compound-eye image consisting of unit images in the x”-y” plane. The row and column number of each unit image are defined as shown in this figure. Tables 2 and 3 list the polar angle ${\theta }_r$ and azimuthal angle ${\phi }_r$ of the observation direction r at the center of each unit image. These were calculated by using Eqs. (3) and 4. The range of polar angles projected on the x-z plane was 11 deg. The difference of the projected polar angle between adjacent units along a row of the compound image was approximately 1.2 deg.

 

Fig. 6 Schematic diagram of compound-eye image consisting of unit images in the x”-y” plane.

Download Full Size | PDF

Table 2. Polar Angle ${\theta }_r$ of Observation Direction r at Center of Each Unit Image

View Table | View all tables in this article

Table 3. Azimuthal Angle ${\phi }_r$ of Observation Direction r at Center of Each Unit Image

View Table | View all tables in this article

Figure 7 is an image of an experimental sample for gonio-imaging. The sample was a holographic grating (Edmund, 43775) with a groove number of 1800 lines per millimeter. On　the surface of the grating, five pieces of sticker was attached in order to demonstrate the efficiency of imaging. The size of the holographic grating was 12.7 mm square, and the grating grooves were in vertical direction of this figure.

 

Fig. 7 Overview of a holographic grating used for gonio-imaging experiment.

Download Full Size | PDF

4. Results

Figure 8 shows the result of gonio-imaging of the experimental sample. One hundred images from different observation angles were captured in a single shot without geometrical scanning. The gonio-image consisted of 10 × 10 unit images of the TOMBO system. In each unit image, diffracted light from the grating was focused, and a sample image was captured. The row number and column number of the unit images are shown around the image.

 

Fig. 8 Result of gonio-imaging of a holographic grating.

Download Full Size | PDF

The TOMBO system acquired a compound-eye image of high quality. There was no cross-talk between adjacent unit images due to the separating walls between unit images. The resolution was enough to show the details of the sticker on the grating. Significant distortions were not observed. The imaging region was almost the same, though slightly different, between neighboring unit images. The effect of vignetting became significant in unit images distant from the center of the compound-eye image. In such unit images, the edge of the convex lens came into the image region, limiting the imaging range.

In the gonio-image, gradual color changes of the sample grating according to the observation direction were successfully captured. The color of the sample grating was different between unit images. It was red in the left part, green in the middle part, and blue in the right part. That is, the color changed along the horizontal direction, which corresponds to the x”-axis in Fig. 2. Also, the color gradually changed in each unit image, from a longer wavelength color in the left part to a shorter wavelength color in the right part. On the other hand, the color was almost constant along the vertical direction.

Color changing of the gonio-image was consistent with gonio properties of the sample grating. In a holographic grating, the wavelength of the diffracted light becomes longer as the angle from the direction of specular reflection increases. In the left part of the gonio-image, the observation direction was farther from the specular direction than in the right part. According to the calculation using grating equation, the wavelength at the center of individual eye at column 1 in Fig. 8 was 565 nm. It was consistent with the color of captured image, which is yellowish green at this part. Also in the same way, the wavelength 500 nm at column 6 was consistent with the bluish green color, and the wavelength 455 nm at column 10 was consistent with the blue color in captured gonio images. The same behavior was observed also within each unit image.

5. Discussions

The proposed method is considered to be effective for image-based inspection of OVDs. At first, the gonio image information was obtained efficiently in quite short time. In this experiment, the gonio images were captured simultaneously within a frame time less than one second. Real time monitoring of gonio images is also available by showing the image information from TOMBO system on a display device. It is at least 6400 times shorter than the method proposed by Dana et al. [4]. Compared with the method capturing images by scanning the geometries one by one, it is at least 100 times shorter even if ignoring the time for geometrical scanning.

Secondly, the proposed system successfully captured sensitive color changes of the diffracted light from holographic grating according to observation angle. In this experiment, the difference of the wavelength of diffracted light between adjacent individual eyes in row direction of TOMBO system was 12 nm according to the calculation using grating equation. Now suppose the case of grating images, which are representatives of OVDs. The groove numbers of their diffractive grating range from 760 to 1700 lines per millimeter according to an author’s previous study [18]. The difference of wavelength in first-order diffracted light between adjacent individual eyes ranges from 13 nm to 29 nm in the case that the grating direction is parallel to horizontal or vertical axis of TOMBO system. Therefore the proposed system will be able to detect gradual color change of the diffracted light from OVDs.

The proposed system also covered wide range of diffracted light. In this experiment, the range of observation direction along horizontal axis of TOMBO system was 11.4 deg, which covered 27% of diffracted light in visible wavelength region from 380 to 780 nm. In the case of grating images stated above, the expanses of first-order diffracted light in visible wavelength region range from 17 deg to 40 deg, therefore from 29% to 65% of them is able to be covered with the observation angle range of the proposed system.

The proposed system balanced between efficiency in the imaging and the resolution of observation angle. It is advantageous feature to perform rough classification of OVDs in the first step of inspection efficiently. Combining with the conventional method with higher resolution in observation angle or imaging, efficient and effective inspection scheme of OVDs will be achieved.

The proposed method will simplify the comparison of gonio reflection properties of OVDs. The comparison between a sample and reference material is achieved by using the same apparatus based on the proposed method, since the measurement geometries are considered to be constant between two image acquisition procedures. Especially, relative relationships of observation directions between individual eyes are rigid since the microlens array is fixed on an area image sensor. Therefore one can easily detect the differences between a sample and reference material by the difference of these two gonio-images. The decision whether or not the sample is genuine will be also achieved by setting a threshold.

The proposed method is now effective in the scheme of comparison between two or more samples, even if absolute value concerning the angle of measurement geometries is not determined. In future work, it should be improved into an absolute measurement method by confirming the angular accuracy in measurement geometries.

Furthermore, gonio-images of OVDs may give us information about the spatial distribution of structural parameters. This will allow structural parameters, such as the pitch of a grating and the thicknesses of thin films or multiple layers, to be used as quantitative indices for the examination of OVDs.

The proposed method is also advantageous for constructing compact, hand-held authentication devices for OVDs. The optical elements of the proposed experimental setup are small because of the integrated optical elements provided in the TOMBO system. Moreover, the absence of any mechanical scanning devices is also advantageous in reducing the size of the system.

However, there are some problems. First, the difference of the reflection angle within each unit image should be considered. As seen from the obtained gonio-image, the reflection angle was not constant within each unit image. Calibration of the reflection angle at each pixel in the unit image should be performed in future work.

Another problem is the specifications of the objective lens. In this experimental setup, the range of observation angles was not so large. In order to solve this problem, an objective lens with small F-number, i.e. the lens with larger diameter in the case of the same focal length, will be of help. The diameter of objective lens is also important factor. In order to avoid the problems of vignetting as seen in the obtained gonio-image, it must be at least larger than the size of microlens array. The larger diameter will be required as the distance between objective lens and microlens array increases.

The magnification K of this system is not adjustable in the proposed optical configurations. In future work, it should be optimized for the inspection of OVDs by investigating the size and resolution of OVDs. The magnification K has relationship with F-number of imaging system. The F-number of this experimental setup was 3.8, which may be slightly larger than conventional camera system. However this drawback is able to be compensated for by sufficient amount of illumination.

The number of compound eyes is also an important factor in this method. It affects the observation angle resolution and the number of pixels in each unit image. These parameters are in the a trade-off relationship. Suppose the system has N × N individual eyes, the number of pixels in each unit image is in proportion to ${1/N}^2$. The half angle of the light field cone captured by an elementary channel is proportional to ${\tan }^{-1}\left(1/N\right)$. The illumination angle and the range of observation angles are also crucial. In future work, it will be important to optimize the configuration of the system based on the properties of the target OVDs.

6. Conclusion

In this paper, we proposed a method for efficient gonio-imaging of OVDs by combining compound-eye optics and a convex lens. A compound-eye image-capturing device called TOMBO and an achromatic lens were used for the experiment. Experimental results for a holographic grating showed that 100 gonio images from different observation angles were successfully captured. The gonio images showed sensitive color changes of the diffracted light from the holographic grating according to the observation angle. The gonio images were captured in a single shot without geometrical scanning. The proposed method is considered to be effective for efficient gonio-imaging of OVDs. In future work, a gonio-imaging system based on this method will be constructed. Optimization of the optical configuration, such as the measurement geometry and the number of compound eyes, will be required.