Expand this Topic clickable element to expand a topic
Skip to content
Optica Publishing Group

Three-directional motion compensation-based novel-look-up-table for video hologram generation of three-dimensional objects freely maneuvering in space

Open Access Open Access

Abstract

A new three-directional motion compensation-based novel-look-up-table (3DMC-NLUT) based on its shift-invariance and thin-lens properties, is proposed for video hologram generation of three-dimensional (3-D) objects moving with large depth variations in space. The input 3-D video frames are grouped into a set of eight in sequence, where the first and remaining seven frames in each set become the reference frame (RF) and general frames (GFs), respectively. Hence, each 3-D video frame is segmented into a set of depth-sliced object images (DOIs). Then x, y, and z-directional motion vectors are estimated from blocks and DOIs between the RF and each of the GFs, respectively. With these motion vectors, object motions in space are compensated. Then, only the difference images between the 3-directionally motion-compensated RF and each of the GFs are applied to the NLUT for hologram calculation. Experimental results reveal that the average number of calculated object points and the average calculation time of the proposed method have been reduced compared to those of the conventional NLUT, TR-NLUT and MPEG-NLUT by 38.14%, 69.48%, and 67.41% and 35.30%, 66.39%, and 64.46%, respectively.

© 2014 Optical Society of America

1. Introduction

Thus far, a number of approaches for generating the computer-generated holograms (CGHs) of three-dimensional (3-D) objects have been proposed [1,2]. One of them is the classical ray-tracing method [13], in which a 3-D object image to be generated is modeled as a collection of self-luminous points of light. The fringe patterns for each object point are then calculated by using the optical diffraction and interference equations, and added up together to obtain the whole CGH pattern of a 3-D object. However, this approach has an unavoidable drawback of the computational complexity because it requires one-by-one calculation of the fringe patterns for all object points.

To solve this drawback, a look-up-table (LUT) method was proposed [4], in which all fringe patterns corresponding to point source contributions from each of the possible locations in an object volume are pre-calculated and stored in the LUT. Therefore, with this method, a great increase in the computational speed could be obtained, but it suffers from a critical shortcoming of the enormous memory size for the LUT.

For accelerating the computation, other approaches have been also proposed [512]. They include the image hologram method in which a 3-D object to be generated is placed close to the CGH to accelerate the calculation [5], the wave-front recording plane (WRP) [68] method where a virtual plane is introduced between the 3-D object and the CGH for accelerated diffraction-based calculation, as well as the recurrence relation method in which the light waves on a CGH are calculated using recurrence relations instead of time-consuming direct calculation of the optical path [11,12].

Recently, a novel-look-up-table (NLUT) method to highly enhance the computational speed as well as to massively reduce the total number of pre-calculated fringe patterns required for CGHs generation of 3-D objects was also proposed [13]. In this method, a 3-D object is approximated as a set of discretely sliced image planes having different depth, and only the fringe patterns of the center-located object points on each image plane, which are called principal fringe patterns (PFPs), are pre-calculated and stored.

Since this NLUT method was proposed, several modified versions of the NLUT [1422] for further enhancing its performance have been suggested based on its unique shift-invariance property, which include temporal redundancy-based NLUT method (TR-NLUT) [16], motion compensation-based NLUT method (MC-NLUT) [21] and MPEG-based NLUT method (MPEG-NLUT) [22].

In the TR-NLUT, the temporal redundancy between the consecutive video frames of a 3-D moving object are removed by using the differential pulse-code modulation (DPCM) algorithm and only the residual images are, then, applied to the NLUT for CGH calculation [16]. Thus, the number of calculated object points was reduced and the resultant CGH calculation time of the TR-NLUT could be shortened compared to the original NLUT.

However, this method has a critical drawback-its data compression rate highly depends on the 3-D object’s motion. That is, if the 3-D object moves faster, the difference in object data between the two consecutive video frames gets bigger, which results in a reduction of its compression efficiency. Furthermore, if the difference in object data increases higher than 50% of the original object data, the TR-NLUT cannot be used for data compression any more. More precisely, with this method, any improvements in computational speed in the case of 50% or higher image difference between the two consecutive video frames cannot be attained.

On the other hand, in the MC-NLUT, a simple motion estimation and compensation method is employed for enhanced compression of 3-D object data. That is, x and y-directional motion vectors of the object are extracted between the two consecutive video frames, and with these the object’s motions are compensated. Therefore, this method can obtain a much higher compression rate than through the TR-NLUT, which extracts just the absolute differences between the two consecutive video frames without using the motion compensation concept.

However, this MC-NLUT also has a couple of drawbacks. For motion compensation, it requires a time-consuming segmentation process of the input 3-D object [21]. In addition, since only the displaced distance of the 3-D object between the two consecutive video frames is extracted, the accuracy of the extracted motion vectors becomes very low, and its accuracy decreases as the 3-D object moves faster. In particular, for the 3-D object with several parts moving differently, it may have multiple moving vectors and each part of this object must be compensated with their estimated motion vectors. However, the MC-NLUT uses only one motion vector for each segmented object, so that the object’s motions cannot be precisely compensated.

To circumvent these drawbacks of the TR-NLUT and MC-NLUT, the MPEG-NLUT employing a sophisticated motion estimation and compensation algorithm has been proposed [22]. Thus far, the MPEG has been known as a popular compression algorithm for video images because of its excellent ability to exploit high temporal correlation between the successive video frames. In the meantime, this MPEG has been used for compression of 3-D video images or holographic video data in the electro-holographic television system [2325]. These systems used the MPEG for reduction of 3-D image or hologram data to be transmitted. On the other hand, in the NLUT, the MPEG is employed for removing the temporally redundant 3-D object data [22].

Since the MPEG algorithm performs a block-based motion estimation and compensation process based on a strict mathematical model, as much of redundant object data between the consecutive video frames as possible can be removed, the computational speed has been highly enhanced. Furthermore, data compression performance of the MPEG-NLUT may not depend on the temporal redundancy of the 3-D moving object, so it can be effectively used for video compression of fast-moving 3-D objects. In other words, unlike the TR-NLUT and MC-NLUT, the MPEG-NLUT shows an excellent compression performance even in the case of 50% or higher image differences between the consecutive video frames.

Thus far, the MPEG-NLUT, as well as the TR-NLUT and MC-NLUT, has used the shift-invariance property of the NLUT to compress the object data. This property, however, allows only the x- and y-directional motion vectors of the 3-D object to be estimated and compensated. Therefore, this MPEG-NLUT cannot be applied for a fast-moving 3-D object with a large depth variation. That is, this method can only be effective in situations where the 3-D object moves along the lateral direction with small depth variations. Thus, a new type of NLUT generating video holograms of 3-D objects which are freely maneuvering in space with large depth variations, by simultaneously compensating the z-directional motion vectors as well as the x- and y-directional motion vectors, is required to be developed.

It must be noted here that the NLUT has the thin-lens property as well as the shift-invariance property. In the NLUT, one principal fringe pattern (PFP) is calculated as a form of Fresnel-zone-plate (FZP) with a fixed focal length [13,26]. Consequently, if two kinds of PFPs with different focal lengths are sandwiched together, the composite PFP has a new focal length, which can be derived from the combined transmittance functions. That is, a PFP with a specifically-focused depth plane can be shifted to a newly-focused depth plane just by being multiplied with a depth-compensating PFP. Therefore, by using this thin-lens property of the NLUT, the z-directional motion vectors of the 3-D objects can be estimated and compensated.

Accordingly, in this paper, a new type of the NLUT employing three (x,y,z)-directional motion estimation and compensation scheme based on its shift-invariance and thin-lens properties, which is called three-directional motion compensation-based NLUT method (3DMC-NLUT), is being proposed for expeditiously generating holographic videos of 3-D objects with large depth variations that are freely maneuvering in space. In this proposed method, the input 3-D video frames are sequentially grouped into a set of eight, in which the first frame and remaining seven frames in each set become the reference frame (RF) and the general frames (GFs), respectively. Hence, each video frame is segmented into a collection of depth-sliced object images (DOIs), in which each object image is divided into blocks. The x and y-directional motion vectors are, then, estimated from the blocks between the RF and each of the GFs based on the shift-invariance property of the NLUT. At the same time, the z-directional motion vectors are estimated from the DOIs between the RF and each of the GFs based on the thin-lens property of the NLUT. With these estimated three-directional motion vectors, object motions in space are precisely compensated, and only a small portion of difference images between these three-directionally motion-compensated RFs and each of the GFs are, then, applied to the NLUT for CGHs calculation.

To confirm the feasibility of the proposed method, experiments with three test 3-D video scenarios are carried out and the results are compared to those of the conventional NLUT, TR-NLUT, and MPEG-NLT in terms of the number of object points to be calculated and the calculation time for one video frame.

2. Limitations of the MPEG-NLUT

In essence, the MPEG-NLUT is composed of six processes [22]. First, the input 3-D video frames are sequentially grouped into groups of pictures (GOPs) and each GOP is composed of one RF and several GFs. Then, all video frames in each GOP are divided into 2-D arrays of image blocks with a fixed size. Second, the CGH pattern of the RF in a GOP is generated by calculating the block-CGHs (B-CGHs) for each block of the RF by using the TR-NLUT, and by adding them all. Third, the x and y-direction motion vectors are estimated from the blocks between the RF and each of the GFs by using the block-matching motion-estimation algorithm. Fourth, all object motions on each block of the GFs are compensated just by shifting the image blocks and their corresponding B-CGH patterns according to the estimated motion vectors, from which the motion-compensated RFs (MC-RFs) for each of the GFs are obtained. Fifth, the difference images are extracted from the blocks between the MC-RFs and each of the GFs in a GOP. Lastly only the B-CGHs for these difference images are calculated and added to those of each block of the MC-RFs to generate the final CGH patterns for the GFs. In the MPEG-NLUT, a block matching-based motion-estimation algorithm is used for extraction of the x- and y-directional motion vectors in each block. In other words, a process of motion estimation and compensation of the object has been carried out only along the horizontal (x and y) plane. Therefore, as the 3-D object moves faster in space, especially with a large variation along the z direction, data compression ratio of the MPEG-NLUT sharply drops.

For example, Fig. 1(a) shows the 1st, 30th, 60th and 90th frames of the 3-D video scenario of ‘Type-1’, in which an airplane simply flies from right to left without any variation in the z-direction. That is, the airplane just moves only along the horizontal plane. Figure 1(d) shows the difference images extracted between the MC-RFs and the GFs in the MPEG-NLUT for this ‘Type-1’. As seen in Fig. 1(d), redundant object data have been massively removed due to simple lateral shifting motion of the 3-D object, which means that the numbers of object points to be involved in CGH calculation of the GFs have been considerably reduced compared to that of the RF.

 figure: Fig. 1

Fig. 1 Comparison of object motion compensations for three 3-D video test scenarios: (a), (b), (c) Object images of the 1st, 30th, 60th and 90th frames for each scenario. (d), (e), (f) Difference images between the MC-RFs and the GFs for each scenario obtained with the MEPG-NLUT.

Download Full Size | PDF

Figure 1(b) shows the 1st, 30th, 60th and 90th frames of the video scenario of ‘Type-2’, where an airplane flies from back to front with a small z-depth variation. The difference images extracted between the MC-RFs and the GFs for this scenario are shown in Fig. 1(e). As seen in Fig. 1(e), a large portion of the object data has not been removed due to the z-directional motion of the 3-D object unlike in ‘Type-1’. In addition, Fig. 1(c) shows the video scenario of ‘Type-3′, in which an airplane flies from back to front with a large depth variation, and Fig. 1(f) shows the extracted difference images, respectively. For this scenario, most of the original object data still remain in the difference images as seen in Fig. 1(f) because the z-directional motion vectors have not been compensated in the MPEG-NLUT. Thus, the difference images may contain almost all of the original object data even though the block-based motion estimation and compensation process has been carried out. Here, the larger the z-directional motion of the 3-D object gets, the higher the number of object points to be involved in CGH calculation.

Therefore, a new type of the NLUT which is able to simultaneously compensate three-directional motions of the 3-D object needs to be developed for its practical application to video hologram generation of 3-D objects moving freely in space with large depth variations.

3. Thin-lens property of the NLUT

In the NLUT, a 3-D object can be treated as a set of discretely-sliced image planes along the z-direction. Here, each image plane having a specific depth is approximated as a collection of self-luminous object points of light wave. Then, only the PFPs for the object points located on each center of the depth planes are pre-calculated and stored in the memory [13]. Based on the Fourier-optical analysis, a PFP can be represented by the FZP with the transmittance function shown in Eq. (1), in which λ denotes the wavelength of the reference beam.

f(x,y)=exp[jπ(x2+y2)λz]
The transmittance function of Eq. (1) shows that the FZP acts as a thin-lens with the focal length of z. That is, when a collimated reference beam passes through the FZP, the outgoing light wave is converged to the point at the distance of z from the FZP plane. This focused light point becomes the same object point reconstructed from the corresponding PFP.

Figures 2(a) and 2(b) show the PFP1 and PFPc with the focused depth planes of z1 and zc, respectively. That is, their transmittance functions, g1(x, y) and gc(x, y) are given by Eqs. (2) and (3), respectively.

g1(x,y)=exp[jπ(x2+y2)λz1]
gc(x,y)=exp[jπ(x2+y2)λzc]
If the PFP1 and PFPc are sandwiched together to make a new composite PFP2 as seen in Fig. 2(c), the transmittance function and focal length of the PFP2, g2(x, y) and z2 can be represented by Eqs. (4) and (5), respectively.
g2(x,y)=exp[jπ(x2+y2)λz2]=g1(x,y)gc(x,y)
1/z2=1/z1+1/zc
Equation (5) shows that the original PFP1 with the focused depth plane of z1 is shifted to the new focused depth plane of z2 just by being attached with the PFPc having the focused depth plane of zc. Thus, a PFP in the NLUT can be considered as the hologram pattern for one object point with a fixed depth value.

 figure: Fig. 2

Fig. 2 Conceptual diagram of a thin-lens property of the PFP: (a) PFP1 with the focal length of z1, (b) PFPc with the focal length of zc, (c) PFP2 with the focal length of z2, (d) Hologram pattern I generated with three object points having the same depth of z1, (e) Composite hologram with the new depth of z2.

Download Full Size | PDF

Here, a hologram pattern I is assumed to be generated with three object points of A1(xa,ya,z1), B1(0,0,z1) and C1(xc,yc,z1) located at the same depth plane of z1, which then can be regarded as a superposed function of three shifted versions of PFP1 acting just like a thin-lens with the focal length of z1 as shown in Fig. 2(d). If this hologram pattern I is sandwiched with another PFPc having the focal length of zc, which is shown in Fig. 2(e), three object points reconstructed from the hologram I are moved to the depth plane of z2 from z1 as shown in Fig. 2(e).

Hence, we consider a 3-D object moving along the z-direction. The z-directional motion vector of the object between two depth planes of z1 and z2 is assumed to be Δz, which makes z2 to be equal to z1 + Δz (z2 = z1 + Δz). Therefore, zc can be obtained using Eq. (5), in which z1 and z2 represent the previous depth plane and its motion-compensated depth plane of the object, respectively. In other words, the transmittance function of the FZP2 with the focal length z2, g2(x,y) can be generated just by multiplying the transmittance function of the FZPc with the focal length of zc, gc(x,y) to that of the current depth plane, g1(x,y). Therefore, the CGH pattern of a 3-D moving object at each depth plane can be computed only by one-step multiplication of the transmittance function of the FZPc by the corresponding focal length of zc according to Eq. (4).

This thin-lens property of the NLUT can be used for shifting the hologram pattern of one depth plane to those of other depth planes only by one-step multiplication process. Here in this paper, a new z-directional motion compensation method is proposed based on this thin-lens property of the NLUT, in which the z-directional motion vectors for each DOI of the video frame can be extracted and compensated through calculating the cost functions of the DOIs in the RF and GFs. Therefore, just by combined use of the z-directional motion compensation process based on the thin-lens property of the NLUT, and the x and y-directional motion compensation process based on the shift-invariance property of the NLUT, three-directional motion vectors of 3-D objects moving fast in space with large depth variations can be effectively estimated and compensated. As a result, the number of object points to be calculated for CGH generation is expected to be massively reduced, which ultimately causes the computational speed of the proposed method to be sharply increased.

While in the conventional methods such as the NLUT, TR-NLUT and MPEG-NLUT, only the intensity images are used in the motion estimation and compensation process, in the proposed method, a new type of the 3-D image model composed of a set of DOIs of the 3-D object is used for x, y and z-directional motion estimation and compensation. Figure 3 shows two types of motion estimation models for each of the conventional MPEG-NLUT and proposed 3DC-NLUT methods. The MPEG-NLUT, as well as the NLUT and TR-NLUT, uses the object image model of Fig. 3(b) for motion estimation, in which only the intensity image of the 3-D object without considering the depth data is employed and the intensity object image is segmented into the 2-D array of blocks. That is, only the x and y-directional motion vectors for each block are extracted from this image.

 figure: Fig. 3

Fig. 3 3-D image models for motion estimation and compensation: (a) Intensity and depth images of a 3-D object, (b) Intensity image used in the MPEG-NLUT, (c) A set of DOIs used in the proposed method, (d) An example of the DOI with a specific depth.

Download Full Size | PDF

Unlike the conventional motion estimation methods, each 3-D video image is divided into a set of DOIs along the depth direction in the proposed method as shown in Fig. 3(c). Figure 3(d) shows an example of the DOI with a specific depth, in which only a small set of object points of the ‘Airplane’ having the same depth values exists.

4. Proposed method

Figure 4 shows an overall block-diagram of the proposed method, which is called 3DMC-NLUT (3-directional motion compensation-based NLUT) as mentioned above. Here in the propose method, a new concept of three-directional motion estimation and compensation is employed for efficient generation of holographic videos of fast-moving 3-D objects in space with large depth variations. Moreover, in the proposed method, a 3-D object model composed of a set of DOIs is used for x, y and z-directional motion estimation and compensation unlike in the conventional NLUT methods where only the intensity images of the 3-D object are used for x and y-directional motion estimation and compensation.

 figure: Fig. 4

Fig. 4 Block diagram of the proposed 3DMC-NLUT for generation of holographic videos of a 3-D object moving fast in space with a large depth variation.

Download Full Size | PDF

As seen in Fig. 4, the proposed method largely consists of three steps. In the first step, x and y-directional motion vectors of each block of the intensity image and z-directional motion vectors of each DOI are extracted from the RF and each of the GFs. In the second step, object motions are compensated not only by using the x and y-directional motion vectors of each block, but also by using the z-directional motion vectors of each DOI. Here, this three-directionally motion-compensated RF is simply denoted as MCx,y,z-RF. The difference images both in intensity and depth are then extracted from the MCx,y,z-RFs and each of the GFs. In the third step, the CGH pattern of the RF is calculated with a newly defined PFP look-up-table (LUT) T'(x,y;zp), which is the z-motion compensation form of the PFP LUT, compared to the original LUT T(x,y;zp) of the NLUT. And then, the CGH patterns of the MCx,y,z-RFs are calculated based on the shift-invariance and thin-lens properties of the NLUT. Finally, the CGH patterns for each of the GFs are generated by combined use of the CGH patterns for both of the MCx,y,z-RFs and the difference images between the MCx,y,z-RFs and each of the GFs.

For the experiments, a personal computer system composed of a CPU of Intel Core i7-4930K operating at 3.4GHz, a main memory of 8GB and an operating system of Microsoft Windows 7 is employed and only one thread of the CPU is used for the computation.

4.1 Three-directional motion estimation and compensation

For x and y-directional motion estimation, the object intensity image is divided into 2-D array of ‘blocks’ as shown in Fig. 5(a). And then motion vectors for each block are extracted by calculating the MAD (mean absolute difference) cost functions between the block image of the RF and those in the searching area of the GF, which is given by Eq. (6).

MADx,y=1P2x=1Py=1P|N(x,y)M(x,y)|
Where P denotes the block length, and N(x,y) and M(x,y) represent the pixels being compared in the blocks of the GF and the RF, respectively. As shown in Fig. 5(a), M(xa,ya;ta) and N(xb,yb;tb) denote the block in the RF with the center-point coordinates of (xa,ya) at the moment of ta, and the block in the GF with the center-point coordinates of (xb,yb) at the moment of tb = ta + ∆t, respectively. Here, the block N which has the smallest MAD value in the searching area is the most matching block for the block M of the RF. Then, the x and y-directional motion vectors, x and y can be obtained according to Eq. (7)

 figure: Fig. 5

Fig. 5 A 3-directional motion estimation procedure between the RF and the GF: (a) X and y-directional motion estimation from each block between the RF and the GF, (b) z-directional motion estimation from each DOI between the RF and the GF.

Download Full Size | PDF

(Δx,Δy)=(xbxa,ybya)

Moreover, for the z-directional motion estimation, the 3-D object is modeled as a collection of 2-D DOIs with different depths. If the 3-D object moves along the z-direction, DOIs on each depth plane may be shifted to other depth planes. Here, as the depth values of the DOIs get smaller, the distance to the observer becomes closer. As seen in Fig. 5(b), the ‘Airplane’ is assumed to fly along the z-direction. If the DOI of A(x,y,za) designated to the za-depth image of the RF is shifted to another zb-depth plane of the GF, which is represented by B(x,y,zb), the shifted DOI can be, then, expressed as Eq. (8).

B(x,y,zb;t+Δt)=A(x,y,za+Δz;t)
Where z and ∆t represent the motion vector of the DOI with the za-depth value of the RF and the time interval between that RF and the GF, respectively. Thus, the depth motion-compensated DOI of the RF can be obtained by combined use of the DOI in the RF and the estimated z-motion vector.

In the proposed method, in order to perform the matching process for the DOI of A(x,y,z) in the RF, a searching area is defined by a number of DOIs around the corresponding DOI of A'(x,y,z) in the GF. As seen in Fig. 5(b), the number of DOIs in the searching area is assumed to be L. And then, the MAD cost function given by Eq. (9) is calculated between the DOI in the RF and each of the DOIs within the searching area in the GF.

MADz=1vhx=1vy=1h|B(x,y,z)A(x,y,z)|
Where v and h mean the length and width of the DOI, respectively, and A(x,y,z) and B(x,y,z) represent the pixels of the DOIs in the RF and GF being compared with, respectively. The DOI in the searching area with the smallest MAD value is the DOI that mostly matches the DOI of A(x,y,z). As seen in Fig. 5(b), the DOI of A(x,y,za) of the RF is shifted along the z-direction. After finding the most matching DOI in the GF, B(x,y,zb), the z-depth motion vector z is given by Eq. (10).

Δz=zbza

For every DOI of the RF, its best-matched DOIs in the GF are obtained and corresponding z-directional motion vectors are extracted, with which object motions of all DOIs of the RF are compensated.

In this paper, three test 3-D video scenarios, each composed of 100 frames of a 3-D object moving fast with a large depth variation, are computationally generated. Each frame has a resolution of 512 × 512 × 256 pixels. Figure 6 shows the object images of the 1st and 100th frames on their corresponding depth planes for each of the test scenarios.

 figure: Fig. 6

Fig. 6 Object images of the 1st and 100th frames with shifted depth ranges for each of the (a) Case I (Media 1), (b) Case II (Media 2) and (c) Case III (Media 3).

Download Full Size | PDF

In ‘Case I’, the ‘Airplane’ is assumed to fly from far right to near left along the x, y and z directions as shown in Fig. 6(a). That is, as the time goes on, it gets closer to the observer at the speed of 1-depth per frame. The depth of the airplane ranges from 200 to 225 at the 1st frame and ranges from 100 to 125 at the 100th frame. In ‘Case II’ as shown in Fig. 6(b), a ‘Car’ moves from far right to near left in space at the speed of 2-depth per frame, in which the depth of the car at the 1st and 100th frames ranges from 200 to 225 and from 1 to 25, respectively, moving two times faster than the ‘Airplane’ in ‘Case I’. On the other hand, in ‘Case III’, which is composed of a fixed ‘House’ and a moving ‘Car’ as shown in Fig. 6(c), the ‘Car’ also moves along the x, y and z axes. Here, the ‘Car’ moves at a speed of 1.25-depth per frame.

Figure 7 shows an example of the z-directional motion estimation and compensation process. As shown in Fig. 7(a), there are seven DOIs on which different object points of the ‘Airplane’ are located. The depth values of these object planes lie from z = −6 to z = 0. The nearest point Q and the farthest points P are located at the depth planes of (0,0,-6) and (0,0,0), respectively. Every DOI has its own z vectors. In the example, all DOIs have the same motion vector value of ‘-1’ which means the ‘Airplane’ flies from far to near with a constant speed of 1-depth per frame. Figure 7(b) shows the z-directional motion compensation result for this example. That is, seven DOIs are shifted along the -z direction with the ‘unit depth interval’, so these shifted DOIs have the depth range from z = −7 to z = −1. The original nearest and farthest points of Q(0,0,-6) and P(0,0,0) move to Q'(0,0,-7) and P'(0,0,-1), respectively.

 figure: Fig. 7

Fig. 7 An example of the z-directional motion estimation and compensation process (a) Motion vectors of DOIs (b) Motion compensated DOIs.

Download Full Size | PDF

Figure 8 shows 3-directional motion-compensated intensity and depth images of the 2nd, 50th and 100th frames for each of the test scenarios. Here, the 3-D video frames are grouped into a set of GOPs in sequence, in which every GOP consists of one RF and seven GFs. For each frame, the intensity image is divided into 8 × 8 blocks and each block is given by 64 × 64 pixels. For each block, x and y-directional motion vectors are extracted between the RF and each of the GFs. Moreover, each frame is segmented as a set of 256 DOIs and the z-directional motion vectors for each DOI are extracted between the RF and each of the GFs. Then, the x, y and z-directionally motion-compensated RF, MCx,y,z-RFs are obtained through the motion compensation process with these extracted motion vectors.

 figure: Fig. 8

Fig. 8 3-directional motion compensated object images of the 2nd, 50th, and 100th frames for each test video scenario: (a), (c), (e) Intensity images, (b), (d), (f) Depth images for each of the ‘Case I’ (Media 4), ‘Case II’ (Media 5), and ‘Case III’ (Media 6) scenarios, respectively.

Download Full Size | PDF

Figures 9(b), 9(d) and 9(f) show the difference images extracted between the MCx,y,z-RFs of the 2nd, 50th, and 100th frames and the corresponding GFs for each of the test scenarios in the proposed method. Figures 9(a), 9(c) and 9(e) also show the same results of the conventional MPEG-NLUT, which are included for comparison. As seen in Figs. 9(a), 9(c) and 9(e), almost all of the object points must be calculated for CGH generation because the z-directional motions have not been compensated in the MPEG-NLUT. On the other hand, as seen in Figs. 9(b), 9(d) and 9(f), in the proposed method, the number of calculated object points has been greatly decreased compared to the conventional MPEG-NLUT. In other words, the total number of object points in each difference image has been dramatically reduced because the x, y and z-directional object motions have been compensated contrary to the conventional MPEG-NLUT method in which only the x and y-directional object motions have been compensated.

 figure: Fig. 9

Fig. 9 Difference images between the motion-compensated RFs of the 2nd, 50th, and 100th frames and the corresponding GFs for each video scenario: (a), (c), (e) Difference images between the MCx,y-RFs and the GFs in the conventional MPEG-NLUT, (b), (d), (f) Difference images between the MCx,y,z-RFs and the GFs in the proposed method for each of the ‘Case I’, (Media 7), ‘Case II’ (Media 8) and ‘Case III’ (Media 9) scenarios, respectively.

Download Full Size | PDF

According to the video processing structure, the 2nd, 50th, 100th frames of the input 3-D video are the 2nd, 2nd and 4th frames in their corresponding GOPs, respectively. That is, both of the 2nd and 50th frames become the first GFs in its GOPs, whereas the 100th frame is the third GF in its GOP. In other words, the 2nd and 50th frames are located closer to their corresponding RFs than the 100th frame is. In general, as the GF gets closer to the RF, the temporal redundancy between them increases. Thus, the difference images shown in Figs. 9(b), 9(d) and 9(f) reveal that the number of calculated object points of the 100th frame looks bigger than those of the 2nd and 50th frames.

4.2 CGHs generation with estimated motion vectors in the x, y and z-directions

With these three-directional motion vectors of x, y and z estimated above, the CGH patterns can be generated according to the following three-step process: 1) Construction of the depth-compensation LUT, 2) Generation of hologram patterns of the RFs and MCx,y,z-RFs based on the x, y, z-directional hologram compensation processes and 3) CGHs generation for each of the GFs by combined use of the hologram patterns for the difference images between the MCx,y,z-RFs and each of the GFs, and the hologram patterns of the MCx,y,z-RFs.

4.2.1 Construction of the depth-compensation LUT

Figure 10 shows the geometry for generating the Fresnel hologram pattern of a 3-D object in the conventional NLUT, in which its LUT, T(x,y;zp) is defined as:

T(x,y;zp)=1zpexp[jπ(x2+y2)λzp]
Where λ, p and zp represent the free-space wavelength of the light, ordering number of depth, and depth value from the hologram plane to the pth depth plane, respectively.

 figure: Fig. 10

Fig. 10 Geometry for generating the Fresnel hologram pattern of a 3-D object.

Download Full Size | PDF

In the proposed method, the LUT is newly defined as a depth-compensation T', which is expressed by Eq. (12).

T'(x,y;zp)={exp[jπ(x2+y2)λzc]forp1exp[jπ(x2+y2)λz1]forp=1
Where the PFP for the first depth plane of z1, T'(x,y;z1) is the same as that of the conventional NLUT. On the other hand, the PFPs for other depth planes are changed into the depth-compensation PFPs represented by T'(x,y;zp) for p>1. Here, the compensated depth, zc can be obtained according to Eq. (13), which is identical in form as Eq. (5).

1/zp=1/z1+1/zc

That is, by multiplying the depth-compensation PFP, T'(x,y;zp(p>1)) to the PFP for the first depth plane of z1, T'(x,y;z1), the PFP for the depth plane of zp, T(x,y;zp) can be obtained. Moreover, by multiplying the T'(x,y;zp) to the hologram pattern generated with the T'(x,y;z1), the hologram pattern of the first depth plane can be converted into those for the depth plane of zp based on the thin-lens property of the NLUT,

4.2.2 CGHs generation of the RFs and MCx,y,z-RFs

Since each 3-D video image is modeled as a collection of DOIs, the CGH patterns of the RF is calculated with T'(x,y;zp). In addition, the hologram pattern of the MCx,y,z-RF is obtained by compensating the hologram pattern of the RF by using three-directional motion vectors of ∆x, ∆y and ∆z. Figure 11 shows this CGHs generation process.

 figure: Fig. 11

Fig. 11 CGH generation process of the RF and MCx,y,z-RF with three-directional motion vectors of ∆x,y,z and T'(x,y;zp).

Download Full Size | PDF

As seen in Fig. 11, the block-CGHs for the DOI of Ip, which is located at the arbitrary depth of zp, are calculated only by using the T'(x,y;z1) and they are added together to get the whole CGH pattern for the object points located on the Ip at the depth plane of z1. And then, by multiplying the depth-compensation PFP of T'(x,y;zp) to this calculated CGH pattern of Ip at the depth plane of z1, the final CGH pattern of Ip on the depth plane of zp is obtained based on the thin-lens property of the NLUT. Once the above-mentioned steps are complete for all DOIs, the final CGH pattern of the 3-D RF is obtained. On the other hand, for the MCx,y,z-RF, the block-CGHs of Ip are shifted according to the motion vectors of ∆x and ∆y, and they are added together to obtain the x and y-directionally motion-compensated CGH pattern of Ip, which is multiplied by T'(x,y;zp + ∆z) for its z-directional hologram compensation process. Accordingly, the x, y, z-directionally motion-compensated CGH pattern of Ip can be obtained. Finally, the CGH pattern of the MCx,y,z-RF, IC(x,y) is generated by adding all of the x, y and z-directionally motion-compensated CGH patterns for each of the DOIs altogether.

4.2.3 CGHs generation of the GFs

For the generation of the CGH patterns for each of the GFs, the hologram patterns for the difference images between the MCx,y,z-RF and each of the GFs are calculated, which is called ID(x, y). Then, the CGH pattern for each of the GFs, I(x, y) can be obtained just by adding the hologram pattern for the difference image of ID(x, y) to the CGH pattern for the MCx,y,z-RF of IC(x, y) as shown in Eq. (14)

I(x,y)=IC(x,y)+ID(x,y)

In this paper, the CGH patterns with the resolution of 1,600 × 1,600 pixels, in which each pixel size of the hologram pattern is 10μm × 10μm, are generated with 3-D data of the test videos of Fig. 6. Moreover, the viewing-distance and the discretization step in the horizontal and vertical directions are set to be 600mm and 30μm, respectively.

Accordingly, to fully display the hologram patterns, the PFP must be shifted by 1,536 pixels (512 × 3 pixels) both horizontally and vertically [10]. Thus, the total resolution of the PFP becomes 3,136 (1,600 + 1,536) × 3,136 (1,600 + 1,536) pixels.

5. Performance analysis of the proposed method

As seen in Fig. 12, the 3-D object images at different focusing depth of the 1st, 25th, 50th, 75th and 100th frames have been successfully reconstructed from the corresponding CGH patterns generated with the proposed method. In ‘Case I’, the ‘Airplane’ moving in space has been reconstructed to be focused at the depth planes of 810mm, 785mm, 760mm, 735mm, and 710mm, respectively for each of the 1st, 25th, 50th, 75th and 100th frames as shown in Fig. 12(a).

 figure: Fig. 12

Fig. 12 Reconstructed 3-D object images of the 1st, 25th, 50th, 75th and 100th frames for each test video of the (a) ‘Case I’ (Media 10), (b) ‘Case II’ (Media 11) and (c) ‘Case III’ (Media 12 and Media 13).

Download Full Size | PDF

On the other hand, the ‘Car’ in ‘Case II’ moves with a higher speed than that of the ‘Aircraft’ in ‘Case I’, so the reconstruction distances of the ‘Car’ get closer than those of the ‘Aircraft’ in ‘Case I’ at the same video frames. As seen in Fig. 12(b), the ‘Car’ images have been reconstructed to be focused at the depth planes of 810mm, 760mm, 710mm, 660mm, and 610mm, respectively for each of the 1st, 25th, 50th, 75th and 100th frames in ‘Case II’.

In addition, Fig. 12(c) shows the reconstructed 3-D object images of the fixed ‘House’ and the moving ‘Car’ in ‘Case III’. In Fig. 12(c), the images in the upper row have been reconstructed to be focused on the ‘House’. All those images have been well focused at the depth plane of 838mm because the ‘House’ was fixed without any motion in ‘Case III’. However, the ‘Car’ images are out of focused and blurred at the depth distance of 838mm and the size of the blurred ‘Car’ gets bigger as it moves from the 1st to 100th frames because it comes closer to observer.

Morever, the images in the lower row in Fig. 12(c) show the reconstructed ‘Car’ images at the depth plane of 825mm, 796mm, 768mm, 739mm, and 710mm, respectively for each of the 1st, 25th, 50th, 75th and 100th frames. All images of ‘Car’ have been clearly reconstructed to be focused, while the images of ‘House’ have been reconstructed to be considerably blurred. Likewise, the images of ‘House’ get smaller as it moves from the 1st to 100th frames.

Furthermore, Fig. 13 shows the computed comparison results of the number of calculated object points and the calculation time for one-object point of the conventional NLUT, TR-NLUT, MPEG-NLUT and proposed method for each of the ‘Case I’, ‘Case II’ and ‘Case III’.

 figure: Fig. 13

Fig. 13 Comparison results: (a), (c), (e) Numbers of calculated object points, (b), (d), (f) Calculation times for one-object point in the conventional NLUT, TR-NLUT, MPEG-NLUT and proposed methods for each of the ‘Case I’, ‘Case II’ and ‘Case III’, respectively.

Download Full Size | PDF

Table 1 also summarizes the quantitative comparison results on the numbers of calculated object points and the calculation times for each case of the conventional NLUT, TR-NLUT, MPEG-NLUT and proposed 3DMC-NLUT. As seen in Table 1, in ‘Case I’ and ‘Case II’, the numbers of calculated object points and the calculation times of the TR-NLUT and MPEG-NLUT have been increased almost two times than those of the original NLUT because the z-directional motions of the 3-D objects moving in space have not been compensated even though the x and y-directional motions have been compensated.

Tables Icon

Table 1. Average calculation times per one-frame, average calculation times per one-object point and average numbers of calculated object points in the conventional NLUT, TR-NLUT, MPEG-NLUT and proposed methods for each of the ‘Case I’, ‘Case II’ and ‘Case III’

On the other hand, the number of calculated object points and the calculation time of the ‘Case III’ look different from those of the ‘Case I’ and ‘Case II’ because the ‘Case III’ scenario is composed of a large fixed ‘House’ and a small moving ‘Car’. Therefore, the overall numbers of calculated object points and calculation times have been considerably reduced in all methods because a large portion of object data for the fixed ‘House’ have been removed just by lateral motion compensation processes. However, as seen in Table 1, only for the moving ‘Car’ in ‘Case III’, the numbers of calculated object points and the calculation times of the TR-NLUT and MPEG-NLUT have been grown up almost two times than those of the original NLUT just like the ‘Case I and Case II’ since the z-directional motions have not also been compensated.

In ‘Case I’, the ‘Airplane’ flies in space with a large motion along the z-direction, therefore the differences, which include both of intensity and depth image differences, in the objects points between the previous and current frames become much larger than 50% even up to 100% of the original number of object points. Thus, the numbers of calculated object points for these difference images get much larger than the original object points of the current frames. In other words, the numbers of calculated object points in the TR-NLUT and MPEG-NLUT get increased much more than that of the original NLUT.

As seen in Fig. 13(a) and Table 1, the average number of calculated object points of the original NLUT has been calculated to be 10,049, and those of the TR-NLUT, MPEG-NLUT have been estimated to be 21,386 and 19,816, respectively. In other words, the average numbers of calculated object points of the TR-NLUT and MPEG-NLUT methods get rather increased by 112.82% and 97.19%, respectively compared that of the original NLUT, which means that the TR-NLUT and MPEG-NLUT cannot be used for compression of the 3-D object data any more for this 3-D video scenario of the ‘Case I’.

On the other hand, as seen in Fig. 13(a), the numbers of calculated object points for each of the GFs have been massively reduced in proposed method, compared to those of the conventional methods because the z-directional object motions have been compensated in the proposed method. Here, the average number of calculated object points of the proposed method has been estimated to be 6,226, which means that the proposed method has obtained 38.04%, 70.89%, and 68.58% reduction of the number of calculated object points, respectively, compared to those of the conventional NLUT, TR-NLUT and MPEG-NLUT.

Furthermore, the average calculation times per one-frame and the average calculation times per one-object point have been estimated to be 250.927sec, 498.867sec, 468.381sec, 141.833sec and 24.969ms, 49.642ms, 46.613ms, 14.106ms, respectively for each of the conventional NLUT, TR-NLUT, MPEG-NLUT and proposed methods as seen in Fig. 13(b) and Table 1. That is, the average calculation time of the proposed method has been reduced by 43.51%, 71.58% and 69.74%, respectively compared to those of the conventional NLUT, TR-NLUT, and MPEG-NLUT.

In ‘Case II’, the ‘Car’ moves with a much higher speed along the z-direction than that of the ‘Airplane’ in ‘Case I’. In this case, the searching area in the z-directional motion estimation process gets enlarged so that the z-motion vectors could be more accurately estimated. Here, the preprocessing time for one-object point for estimation and compensation of the object motions has been calculated to be 0.692ms. But, this time looks much smaller than the CGH generation time of 14.256ms. That is, even though the searching area becomes enlarged as the z-directional object motion gets larger, the corresponding preprocessing time still remains much smaller than that of the CGH calculation time as shown in Table 1. Therefore, it can be ignored in estimation of the overall CGH calculation time.

In ‘Case II’, the average number of calculated object points of the proposed method has been estimated to be 9,295 whereas those of the conventional NLUT, TR-NLUT and MPEG-NLUT have been calculated to be 16,756, 33,976 and 31,701, respectively. That is, the average number of calculated object points of the proposed method has been reduced by 44.53%, 72.64% and 70.68%, respectively compared to those of the NLUT, TR-NLUT and MC-NLUT.

Moreover, the average calculation times per one-frame and the average calculation times per one-object point of the conventional NLUT, TR-NLUT, MPEG-NLUT and proposed methods have been also estimated to be 434.436sec, 833.187sec, 784.371sec, 250.610sec and 25.925ms, 49.721ms, 46.816ms, 14.948ms, respectively as shown in Fig. 13(d) and Table 1. That is, the average calculation time of the proposed method has been reduced by 42.34%, 69.94% and 68.07%, respectively compared to those of the conventional NLUT, TR-NLUT and MPEG-NLUT.

In ‘Case III’, since there is a fixed ‘House’, as well as a moving ‘Car’, the total number of objects point reaches up to 75,000. In this case, the differences in object points between the previous and current frames become lower than 50% because of the large fixed ‘House’. Therefore, the TR-NLUT and MPEG-NLUT can be used for reduction of the object data as shown in Figs. 13(e) and 13(f) by compensating the object motions in the x and y directions, which means almost all of the object data for the large fixed ‘House’ can be removed. Here, the average number of calculated object points of the proposed method has been calculated to be 5,961, whereas those of the NLUT, TR-NLUT and MPEG-NLUT have been estimated to be 75,626, 10,583 and 10,966, respectively, which means the average number of calculated object points of the proposed method has been reduced by 92.12%, 43.67% and 45.64%, respectively compared to those of the NLUT, TR-NLUT and MPEG-NLUT. In addition, the average calculation times per one-frame and the average calculation times per one-object point of the conventional NLUT, TR-NLUT, MPEG-NLUT and proposed methods have been estimated to be 1927.117sec, 262.362sec, 274.179sec, 159.642sec and 25.482ms, 3.469ms, 3.625ms, 2.111ms, respectively as shown in Fig. 13(f) and Table 1. That is, the average calculation time of the proposed method has been reduced by 91.72%, 39.15% and 41.77%, respectively compared to those of the conventional NLUT, TR-NLUT and MPEG-NLUT methods.

As mentioned above, for comparison with the results of the ‘Case I’ and ‘Case II’, the average number of calculated object points only for the moving ‘Car’ in ‘Case III’ has been also calculated and the results have been added in Table 1. As seen in Table 1, the average number of calculated object points of the original NLUT has been calculated to be, 5,375, and those of the TR-NLUT, MPEG-NLUT have been estimated to be 9,877 and 9,568, respectively. That is, the average numbers of calculated object points of the TR-NLUT and MPEG-NLUT have also been increased by 83.76% and 78.01%, respectively compared to that of the original NLUT because of a large z-directional motion of the object.

However, the average number of calculated object points of the proposed method for the moving ‘Car’ has been decreased down to 4,386 because the z-directional object motion have been compensated contrary to the conventional TR-NLUT and MPEG-NLUT methods. That is, the average number of calculated object points of the proposed method has been reduced by 18.40%, 55.59% and 54.16%, respectively compared to those of the NLUT, TR-NLUT and MPEG-NLUT. In addition, the average calculation times per one-frame and the average calculation time per one-object point of the conventional NLUT, TR-NLUT, MPEG-NLUT and proposed methods have been estimated to be 136.391sec, 255.117sec, 244.275sec, 109.101sec and 25.375ms, 47.446ms, 45.438ms, 20.294ms, respectively as shown in Table 1. Thus, the average calculation time of the proposed method has been reduced by 20.02%, 57.23% and 55.34%, respectively compared to those of the conventional NLUT, TR-NLUT and MPEG-NLUT methods.

In brief, these experimental results show that the number of calculated object points and the calculation time of the proposed method for three 3-D video scenarios of ‘Case I’, ‘Case II’ and ‘Case III’, have found to be reduced, on the average, by 38.14%, 69.48%, 67.41% and 35.30%, 66.39%, 64.46%, respectively compared to those of the conventional NLUT, TR-NLUT and MPEG-NLUT.

Table 1 also shows the computation results of the preprocessing time for 3-directional motion estimation and motion compensation and the CGH generation time in the proposed method. The average preprocessing and CGH calculation times for one object point have been found to be 0.676ms and 9.712ms, respectively, which means that a portion of the preprocessing time in the total CGH calculation time has been found to be a small value of 6.51%. By the way, there might be a tradeoff between the preprocessing time and the CGH generation time. That is, the searching area in the x, y and z directions can be enlarged for more accurate motion estimation and compensation. With this process, the difference image between the MCx,y,z-RF and the GF gets smaller than before and the following CGH generation time can be much reduced. However, the motion estimation time gets increased because searching the motion vectors in the larger searching area may cost much more time than before.

Finally, good experimental results with three types of test 3-D video scenarios mentioned above, confirm the application possibility of the proposed method in the practical fields by showing its performance superiority over the conventional methods in terms of the number of object points to be calculated and the calculation time.

Here in this paper, experiments have been done for the 3-D objects moving with large depth movements but small shape variations just for proving the feasibility of the proposed method. However, in the real environments, 3-D objects happen to be rotated, scaled and occluded while they are moving in space. Therefore, the performance dependence of the proposed method on those variations needs to be analyzed for its practical application, which is now considered as the future research topic.

6. Conclusions

In this paper, a new 3DMC-NLUT has been proposed for fast generation of holographic videos of the 3-D object freely maneuvering in space with a large depth variation. With this method, three-directional object motions have been compensated, which results in a massive reduction of object data in difference images extracted between the MCx,y,z-RF and each of the GFs. Experimental results with three kinds of test 3-D videos reveal that the average number of calculated object points and the average calculation time of the proposed method have been reduced by 38.14%, 69.48%, 67.41% and 35.30%, 66.39%, 64.46%, respectively compared to those of the conventional NLUT, TR-NLUT and MPEG-NLUT methods. From these successful experimental results, the feasibility of the proposed method in the practical application has been verified.

Acknowledgments

This work was supported by the National Research Foundation of Korea (NRF) grant funded by the Korea government (MSIP) (No. 2013-067321). This work was partly supported by GigaKOREA project, [GK13D0100, Development of Telecommunications Terminal with Digital Holographic Table-top Display].

References and links

1. C. J. Kuo and M. H. Tsai, Three-Dimensional Holographic Imaging (John Wiley, 2002).

2. T.-C. Poon, Digital Holography and Three-Dimensional Display (Springer, 2007).

3. R. Oi, K. Yamamoto, and M. Okui, “Electronic generation of holograms by using depth maps of real scenes,” Proc. SPIE 6912, 69120M (2008). [CrossRef]  

4. M. Lucente, “Interactive computation of holograms using a look-up table,” J. Electron. Imaging 2(1), 28–34 (1993). [CrossRef]  

5. T. Yamaguchi and H. Yoshikawa, “Computer-generated image hologram,” Chin. Opt. Lett. 9(12), 120006 (2011). [CrossRef]  

6. T. Shimobaba, N. Masuda, and T. Ito, “Simple and fast calculation algorithm for computer-generated hologram with wavefront recording plane,” Opt. Lett. 34(20), 3133–3135 (2009). [CrossRef]   [PubMed]  

7. T. Shimobaba, H. Nakayama, N. Masuda, and T. Ito, “Rapid calculation algorithm of Fresnel computer-generated-hologram using look-up table and wavefront-recording plane methods for three-dimensional display,” Opt. Express 18(19), 19504–19509 (2010). [CrossRef]   [PubMed]  

8. J. Weng, T. Shimobaba, N. Okada, H. Nakayama, M. Oikawa, N. Masuda, and T. Ito, “Generation of real-time large computer generated hologram using wavefront recording method,” Opt. Express 20(4), 4018–4023 (2012). [CrossRef]   [PubMed]  

9. N. Okada, T. Shimobaba, Y. Ichihashi, R. Oi, K. Yamamoto, M. Oikawa, T. Kakue, N. Masuda, and T. Ito, “Band-limited double-step Fresnel diffraction and its application to computer-generated holograms,” Opt. Express 21(7), 9192–9197 (2013). [CrossRef]   [PubMed]  

10. T. Shimobaba, T. Kakue, and T. Ito, “Acceleration of color computer-generated hologram from three-dimensional scenes with texture and depth information,” Proc. SPIE 9117, 91170B (2014). [CrossRef]  

11. K. Matsushima and M. Takai, “Recurrence formulas for fast creation of synthetic three-dimensional holograms,” Appl. Opt. 39(35), 6587–6594 (2000). [CrossRef]   [PubMed]  

12. K. Muranoa, T. Shimobaba, A. Sugiyama, N. Takada, T. Kakue, M. Oikawa, and T. Ito, “Fast computation of computer-generated hologram using Xeon Phi coprocessor,” Physics.comp-ph 11, Sep (2013).

13. S.-C. Kim and E.-S. Kim, “Effective generation of digital holograms of three-dimensional objects using a novel look-up table method,” Appl. Opt. 47, D55–D62 (2008). [CrossRef]   [PubMed]  

14. S. C. Kim, J. M. Kim, and E.-S. Kim, “Effective memory reduction of the novel look-up table with one-dimensional sub-principle fringe patterns in computer-generated holograms,” Opt. Express 20(11), 12021–12034 (2012). [CrossRef]   [PubMed]  

15. S.-C. Kim, J.-H. Kim, and E.-S. Kim, “Effective reduction of the novel look-up table memory size based on a relationship between the pixel pitch and reconstruction distance of a computer-generated hologram,” Appl. Opt. 50(19), 3375–3382 (2011). [CrossRef]   [PubMed]  

16. S.-C. Kim, J.-H. Yoon, and E.-S. Kim, “Fast generation of three-dimensional video holograms by combined use of data compression and lookup table techniques,” Appl. Opt. 47, 5986–5995 (2009). [CrossRef]   [PubMed]  

17. S.-C. Kim, W.-Y. Choe, and E.-S. Kim, “Accelerated computation of hologram patterns by use of interline redundancy of 3-D object images,” Opt. Eng. 50(9), 091305 (2011). [CrossRef]  

18. D.-W. Kwon, S.-C. Kim, and E.-S. Kim, “Memory size reduction of the novel look-up-table method using symmetry of Fresnel zone plate,” Proc. SPIE 7957, 79571B (2011). [CrossRef]  

19. D.-W. Kwon, S.-C. Kim, and E.-S. Kim, “Hardware implementation of N-LUT method using Field Programmable Gate Array technology,” Proc. SPIE 7957, 79571C (2011). [CrossRef]  

20. S.-C. Kim, K.-D. Na, and E.-S. Kim, “Accelerated computation of computer-generated holograms of a 3-D object with N×N-point principle fringe patterns in the novel look-up table method,” Opt. Lasers Eng. 51(3), 185–196 (2013). [CrossRef]  

21. S.-C. Kim, X.-B. Dong, M.-W. Kwon, and E.-S. Kim, “Fast generation of video holograms of three-dimensional moving objects using a motion compensation-based novel look-up table,” Opt. Express 21(9), 11568–11584 (2013). [CrossRef]   [PubMed]  

22. X.-B. Dong, S.-C. Kim, and E.-S. Kim, “MPEG-based novel look-up table for rapid generation of video holograms of fast-moving three-dimensional objects,” Opt. Express 22(7), 8047–8067 (2014). [CrossRef]   [PubMed]  

23. H. Yoshikawa and J. Tamai, “Holographic image compression by motion picture coding,” Proc. SPIE 2652, 2–9 (1996). [CrossRef]  

24. E. Darakis and T. J. Naughton, “Compression of digital hologram sequences using MPEG-4,” Proc. SPIE 7358, 735811 (2009). [CrossRef]  

25. T. Senoh, K. Wakunami, Y. Ichihashi, H. Sasaki, R. Oi, and K. Yamamoto, “Multiview image and depth map coding for holographic TV system,” Opt. Eng. 53(11), 112302 (2014). [CrossRef]  

26. B. Z. Zhang and D.-M. Zhao, “Focusing properties of Fresnel zone plates with spiral phase,” Opt. Express 18(12), 12818–12823 (2010). [CrossRef]   [PubMed]  

Supplementary Material (13)

Media 1: AVI (1891 KB)     
Media 2: AVI (2478 KB)     
Media 3: AVI (4932 KB)     
Media 4: AVI (1857 KB)     
Media 5: AVI (2223 KB)     
Media 6: AVI (4651 KB)     
Media 7: AVI (1672 KB)     
Media 8: AVI (1744 KB)     
Media 9: AVI (1389 KB)     
Media 10: AVI (4853 KB)     
Media 11: AVI (5681 KB)     
Media 12: AVI (15471 KB)     
Media 13: AVI (13830 KB)     

Cited By

Optica participates in Crossref's Cited-By Linking service. Citing articles from Optica Publishing Group journals and other participating publishers are listed here.

Alert me when this article is cited.


Figures (13)

Fig. 1
Fig. 1 Comparison of object motion compensations for three 3-D video test scenarios: (a), (b), (c) Object images of the 1st, 30th, 60th and 90th frames for each scenario. (d), (e), (f) Difference images between the MC-RFs and the GFs for each scenario obtained with the MEPG-NLUT.
Fig. 2
Fig. 2 Conceptual diagram of a thin-lens property of the PFP: (a) PFP1 with the focal length of z1, (b) PFPc with the focal length of zc, (c) PFP2 with the focal length of z2, (d) Hologram pattern I generated with three object points having the same depth of z1, (e) Composite hologram with the new depth of z2.
Fig. 3
Fig. 3 3-D image models for motion estimation and compensation: (a) Intensity and depth images of a 3-D object, (b) Intensity image used in the MPEG-NLUT, (c) A set of DOIs used in the proposed method, (d) An example of the DOI with a specific depth.
Fig. 4
Fig. 4 Block diagram of the proposed 3DMC-NLUT for generation of holographic videos of a 3-D object moving fast in space with a large depth variation.
Fig. 5
Fig. 5 A 3-directional motion estimation procedure between the RF and the GF: (a) X and y-directional motion estimation from each block between the RF and the GF, (b) z-directional motion estimation from each DOI between the RF and the GF.
Fig. 6
Fig. 6 Object images of the 1st and 100th frames with shifted depth ranges for each of the (a) Case I (Media 1), (b) Case II (Media 2) and (c) Case III (Media 3).
Fig. 7
Fig. 7 An example of the z-directional motion estimation and compensation process (a) Motion vectors of DOIs (b) Motion compensated DOIs.
Fig. 8
Fig. 8 3-directional motion compensated object images of the 2nd, 50th, and 100th frames for each test video scenario: (a), (c), (e) Intensity images, (b), (d), (f) Depth images for each of the ‘Case I’ (Media 4), ‘Case II’ (Media 5), and ‘Case III’ (Media 6) scenarios, respectively.
Fig. 9
Fig. 9 Difference images between the motion-compensated RFs of the 2nd, 50th, and 100th frames and the corresponding GFs for each video scenario: (a), (c), (e) Difference images between the MCx,y-RFs and the GFs in the conventional MPEG-NLUT, (b), (d), (f) Difference images between the MCx,y,z-RFs and the GFs in the proposed method for each of the ‘Case I’, (Media 7), ‘Case II’ (Media 8) and ‘Case III’ (Media 9) scenarios, respectively.
Fig. 10
Fig. 10 Geometry for generating the Fresnel hologram pattern of a 3-D object.
Fig. 11
Fig. 11 CGH generation process of the RF and MCx,y,z-RF with three-directional motion vectors of ∆x,y,z and T'(x,y;zp).
Fig. 12
Fig. 12 Reconstructed 3-D object images of the 1st, 25th, 50th, 75th and 100th frames for each test video of the (a) ‘Case I’ (Media 10), (b) ‘Case II’ (Media 11) and (c) ‘Case III’ (Media 12 and Media 13).
Fig. 13
Fig. 13 Comparison results: (a), (c), (e) Numbers of calculated object points, (b), (d), (f) Calculation times for one-object point in the conventional NLUT, TR-NLUT, MPEG-NLUT and proposed methods for each of the ‘Case I’, ‘Case II’ and ‘Case III’, respectively.

Tables (1)

Tables Icon

Table 1 Average calculation times per one-frame, average calculation times per one-object point and average numbers of calculated object points in the conventional NLUT, TR-NLUT, MPEG-NLUT and proposed methods for each of the ‘Case I’, ‘Case II’ and ‘Case III’

Equations (14)

Equations on this page are rendered with MathJax. Learn more.

f( x,y )=exp[ jπ ( x 2 + y 2 ) λz ]
g 1 ( x , y ) = exp [ j π ( x 2 + y 2 ) λ z 1 ]
g c ( x , y ) = exp [ j π ( x 2 + y 2 ) λ z c ]
g 2 ( x , y ) = exp [ j π ( x 2 + y 2 ) λ z 2 ] = g 1 ( x , y ) g c ( x , y )
1 / z 2 = 1 / z 1 + 1 / z c
M A D x , y = 1 P 2 x = 1 P y = 1 P | N ( x , y ) M ( x , y ) |
( Δ x , Δ y )=( x b x a , y b y a )
B( x,y, z b ;t+Δt )=A( x,y, z a + Δ z ;t )
MA D z = 1 vh x=1 v y=1 h | B( x,y,z )A( x,y,z ) |
Δ z = z b z a
T ( x , y ; z p ) = 1 z p exp [ j π ( x 2 + y 2 ) λ z p ]
T ' ( x,y; z p )={ exp[jπ ( x 2 + y 2 ) λ z c ] for p1 exp[jπ ( x 2 + y 2 ) λ z 1 ] for p=1
1/ z p =1/ z 1 +1/ z c
I( x,y )= I C ( x,y )+ I D ( x,y )
Select as filters


Select Topics Cancel
© Copyright 2024 | Optica Publishing Group. All rights reserved, including rights for text and data mining and training of artificial technologies or similar technologies.