Open Access
Add to BibSonomyAbstract
In an IR proximity-based motion gesture sensor (MGS) mainly composed of an IR LED and photodiodes (PDs), the IR LED is the most power-hungry component. For reducing its power, both the size and the field-of-view (FOV) of each PD can be increased instead. However, it cannot be adapted to conventional MGSs due to the geometrical limitation of their optical structures. In this paper, the optical structure of the proposed MGS with a partially open cavity package allows for power reduction by at least 70.9% compared with conventional MGSs. Optical simulation and test results validate the theoretical analysis presented.
© 2016 Optical Society of America
1. Introduction and motivation
The interactions between users and machines have become more intuitive as human machine interfaces have evolved. In this regard, various touchless interfaces, including speech recognition, eye gaze, and gesture recognition, have been widely studied [1–6]. Especially for portable applications, hand motion (or gesture) recognition is one of the best solutions among other touchless interfaces as it allows for more intuitive, more stable, and user-independent inputs [7,8]. Motion gesture sensors (MGSs) offer touchless interfaces by detecting a user’s swiping motions. Among several types of MGSs, IR proximity-based MGSs that are mainly composed of an IR LED and photodiodes (PDs) are mostly preferred for low-power applications [9]. In early development of conventional IR proximity-based MGSs, the large form factor cannot be avoided due to the lower recognition rate as the space in between external PDs becomes shorter [10,11]. To establish a small form factor, authors in [9] developed an MGS with an external optical block for detecting two-directional (2D) motion: left-right and push-pull. The optical block differentiates the region such that each PD receives the reflected IR light from an object and thus enables the PDs located adjacent to each other. Nonetheless the height of the optical block is as high as 5.3 mm due to the discrete PDs used, which make it difficult to be utilized in portable applications. The MGS in [12] overcomes this problem by integrating two PDs and their supporting circuitry on a single chip and by assembling the MGS, an external LED, and an optical block into a single package. For three-directional (3D) motion (left-right, top-bottom, and push-pull), the MGS in [13] includes a cross-shaped optical block for separating the FOV of three PDs as shown in Fig. 1.
Fig. 1 An IR proximity-based Motion Gesture Sensor with an optical block [13]: (a) basic configuration and (b) trimetric view.
Considering power dissipation in an IR proximity-based MGS, an IR LED is the most power-hungry component. Thus, reducing the power in an MGS can be achieved mainly by reducing the power in the IR LED. To compensate for the reduced power in the IR LED, especially without using a lens, there can be two possible methods: increasing either the size or the FOV of each PD. Note that enhancing the photo efficiency in each PD is not considered, since the photo efficiency cannot be optically controlled. However, both the size and the FOV of each PD in conventional MGSs are limited by their own optical block, which prevents the power consumption from being reduced further. In this paper, an IR proximity-based 3D MGS with a partially open cavity package that eliminates the use of an optical block is proposed. The proposed optical structure allows for the increased size and FOV of each PD, which leads to reduced power in the IR LED.
2. System overview
2.1 Conventional MGSs
In conventional imaging systems, FOV can be controlled by multiple optics [14]. However, in conventional MGSs, the FOV of the PD has been controlled by simple optical blocks, not complex optics. The optical blocks are located in between the PDs to differentiate the region in which each PD accepts the reflected IR light from an object. For reducing the power consumption of the IR LED, the peak current of the IR LED can be reduced. In this case, the size of each PD should be increased to compensate for the reduced input signal. The geometry of the 3D MGS in [13] is shown in Fig. 2(a), where the size of each integrated PD, APDC, can be maximized as
where lchip is the length of chip, loffset the space between the PD and the optical block considering the alignment offset, and wOB the minimum width of the optical block for fabrication. Consequentially, the size of the PD is limited by the optical block positioned between PDs. Note that the space outside the chip is disregarded for simplicity. As an example, if lchip is 1 mm, wOB is 0.4 mm, and loffset is 0.1 mm, then APD is 0.04 mm2.Fig. 2 The conventional 3D MGS in [13]: (a) top-view and (b) cross-sectional view (only the receiver part is shown for simplicity).
The optical block also causes the limited FOV of each PD. Figure 2(b) depicts a cross-sectional view of the 3D MGS in [13]: θ is the view angle limited by the near package wall, and θOB is the view angle limited by the optical block. Then, θ and θOB can be expressed as
where lPC is the distance between a PD and the nearest side wall, hW the height of the package side wall, lOC the distance between a PD and the optical block, and hOB the height of the optical block. Since the FOV due to θOB cause the ambiguous zone that cannot detect motions, θOB needs to be minimized. As a result, θOB is much smaller than θ. And cross-sectional views of PQ and P´Q´ are identical, resulting in the X-directional FOV, FOVX, is the same as the Y- directional FOV, FOVY. Then the FOVX and the FOVY of each PD can be defined asThe FOVX and the FOVY are limited by the optical block; therefore, the FOV is limited by the quarter plane.A dual-axis solar tracking sensor can be composed of four PDs located on the four different surfaces of a hexahedron shape, as shown in Fig. 3(a). The FOV of each PD is limited by a facet of the hexahedron shape, which results in the FOV of each PD being limited by the half plane. The FOV of the PD in a dual-axis solar tracking sensor is roughly twice than that of th PD in Fig. 2(a). To apply this optical structure to an integrated chip into a single package, a transparent package and an optical block are required, as shown in Fig. 3(b). The optical structure allows for increasing not only the FOV but also the size of each PD. However, the optical block is an isolated structure, which is hard to build without a supporting structure that causes a limited size of the PD because it covers some part of the PD. In addition, precise alignment of the optical block causes increasing fabrication costs.
Fig. 3 (a) Top-view of the dual-axis solar tracking sensor and (b) top-view of the MGS applied to the optical structure using a dual-axis solar tracking sensor.
2.2 Proposed MGS
Figure 4(a) depicts the top view of the proposed MGS, which mainly shows four on-chip PDs in a partially open cavity package for simplicity. The partially open cavity package of the proposed MGS works as an optical block itself. The PQ cross-sectional view of the proposed MGS is shown in Fig. 4(b). The width and the length of a PD, wPD and lPD, are limited by the chip size as
Consequently, the area of PD in proposed MGS can be defined asThe area of each PD is unconcerned with an optical structure (or optical block), but it is related to only the chip size. Note that the space outside the chip is disregarded for simplicity. As an example, if lchip is 1 mm and wPD 0.1 mm, lPD is 0.8 mm and APD results in 0.08 mm2. Since wPD can be longer, APD can also be larger. However, that area is already twice as large as the area of the PD in the MGS with optical block. In the conventional MGS, the area of the PD is limited by an optical block. But in the proposed MGS, the area of the PD is limited only by the chip size; as a result, the area of the PD can be increased.Fig. 4 A 3D MGS with the proposed optical structure: (a) top view, (b) PQ cross-sectional view, and (c) P´Q´ cross-sectional view.
The FOV of each PD and the amount of IR light received onto each PD are determined by the geometry of the package itself, unlike other MGSs with optical blocks in between PDs. The FOVXL and the FOVXR represent the X- directional FOV of PDL and PDR, respectively, and can be defined as
where lPD is the length of PD, wPD the width of PD, and hW the height of the side wall. For the same package height, the FOVX of the proposed MGS is larger than that of the conventional MGS in Fig. 2. Figure 4(c) shows the P´Q´ cross-sectional view of the proposed MGS. FOVYL is the Y-directional FOV of PDL and can be defined asUnlike the conventional MGS in Fig. 2, the FOVY of PDs for X-directional motion detection is larger than the FOVX. As a result, the FOV of each PD is increased by a half plane. If lPD is much larger than wPD/2, (lPD >> wPD/2), The FOVY is twice as large as the FOVX. In addition the FOVXL of the proposed MGS is larger than that of the conventional MGS, because the lPC is smaller than lPD + wPD. This means that the FOV of the proposed MGS is larger than that of the conventional MGS. Since the proposed MGS can increase the area and FOV of the PD, the power dissipation of the IR LED can be decreased.
3. Simulation results
For optical simulation, LightTools software with 10 million different rays is used for ray tracing and luminance distribution. Even though an LED can be a light source with 10 million rays, the number of received rays on a PD once reflected from an object can be small. Also, for eliminating the object dependency, each PD is assumed to be a light source as shown in Fig. 5, where the PQ cross-sectional view is depicted. Then 10 million rays out of each PD are then received onto a plane located 10 cm away from the proposed MGS. FOVXR indicates the X-directional FOV of PDR. The size of each PD is given by 0.11 mm × 0.76 mm.
The 2D luminance distribution in the receiver plane of four PDs (PDL, PDR, PDT, and PDB) are defined as LUL(x, y), LUR(x, y), LUT(x, y), and LUB(x, y), respectively. The difference in 2D luminance distribution in the receiver plane given by PDL and PDR is defined as
Correspondingly, the difference in 2D luminance distribution, LUTB(x, y) = LUT(x, y) – LUB(x, y), is determined by PDT and PDB. In Fig. 6, the 2D luminance distributions of the proposed MGS and MGS with a crossbar-type optical block are depicted. The two figures on the left depict LULR(x, y), on the other hand and the two figures on the right depict LUTB(x, y). The top two figures are the results of the MGS with a crossbar-type optical block, and the bottom two figures are the results of the proposed MGS. Each luminance value is normalized to the maximum value from the four figures. While the 2D luminance distribution of the MGS with a crossbar-type optical block is dispersed narrowly and finely, that of the proposed MGS is dispersed widely and strongly because the FOV and size of the PD in the proposed MGS is larger than those in the MGS with a crossbar-type optical block.For quantifying 1D motion detection, 2D luminance matrixes of LULR(x, y) and LUTB(x, y) are converted into 1D luminance distributions, LULR(x) and LUTB(y) as
As the luminance distribution of the x-axis is identical to that of the y-axis, the method of detecting Y-directional motion is identical to that for detection X-directional motion. Figure 7 shows 1D luminance distributions of the proposed MGS and the conventional MGS with the two type optical blocks used in [13]. For comparison, LULR(x) is normalized to its maximum value obtained during simulation. Simulation results show that the PDs of proposed MGS receive amount of reflected IR light that was four times larger than that of the MGS with an optical block.Fig. 7 One-dimensional distributions of the proposed MGS and the conventional MGSs with the optical blocks in [13].
The controllable factors of design are the width of the PD and the open area of the package. The width of the PD is relative to the area of the PD and the open area of the package is how much of the PD width is covered by the top plate of the package. The maximum value of 1D luminance distribution, max(LULR(x)), is compared with each value of width of the PD, wPD, as shown in Fig. 8(a). The width of the PD is longer, and the area of the PD is larger. Therefore, max(LULR(x)) is larger, as the width of the PD is longer. Since other circuitry takes possession of the chip, the available maximum width of the PD is 110 um by the limited area of chip. For the optimized package design, the simulation is conducted by changing the top plate length for each PD width, as shown in Fig. 8(b). Simulation results show that max(LULR(x)) is at maximum value when the package covers some PDs.
Fig. 8 Comparisons of one-dimensional luminance distribution according to (a) the width of each PD width and (b) the covered width of each PD for the given width.
4. Experimental results
Figure 9 depicts a simplified block diagram of the proposed MGS. An external IR LED covers the FOVs of each PD. As the reflected IR light generates a photocurrent in each PD depending on the location of an object, each fully differential trans-impedance amplifier (TIA) converts the difference in the photocurrent into a form of voltage. Then, the signals are amplified. An analog to digital converter (ADC) is used for X- and Y-directional motions, which is followed by motion detection logic. Figure 10(a) shows the cross-sectional view of the proposed 3-D-rendered MGS with partially open cavity package. The package dimension in Fig. 10(a) is 4.0 mm × 2.4 mm × 0.8 mm. The core chip size of the proposed MGS is 1.2 mm × 1.2 mm. Each PD is sized at 0.11 mm × 0.76 mm. The package photograph is depicted in Fig. 10(b). Figure 10(c) shows the top view of the proposed MGS. TIA and other circuitry are located in the top of the core. A bare chip in a package can be damaged by dirt, dust, or moisture. To prevent it from happening, the proposed open cavity package can be filled with epoxy to protect the devices from moisture [12]. Unlike other types of 3D MGSs, the proposed MGS eliminates the use of an optical block located above the bare chip and makes the epoxy filling process much simpler.
Fig. 10 The proposed MGS: (a) a cross-sectional view of 3D rendering with the partially open cavity package, (b) a photograph of the fabricated MGS, and (c) a photograph of the proposed MGS chip.
To quantify the performance of the proposed MGS independent of the objects speed, DC static positioning tests were conducted for two axes by positioning the object horizontally from −10 cm to 10 cm, with the negative value indication to the left (or bottom) of the sensor. The DC static performance was also measured by changing the vertical position of the object from 6 cm to 12 cm. A 20-mm-wide white bar was used as an object. Figure 11 shows the DC static output voltage, (Voutp - Voutn) in Fig. 9, of the proposed MGS when changing the position of the object in an X- and Y-direction.
Figure 12 shows the analog outputs Voutp and Voutn in Fig. 9 when the white bar is positioned on the plane 8 cm away from an MGS and moves from left to right. ILED is the IR LED modulation signal. Since the IR LED is modulated and the output signal is sampled at the LED on time, the noise at the LED off time is not considered.
Fig. 12 The analog outputs, Voutp and Voutn, for the proposed MGS with a left-to-right swipe. An object is located on (a) the left and (b) the right side of the MGS.
The recognition rate is defined as the ratio the number of pass trials and total trials and evaluated for the proposed MGS based on X-, Y-, and Z-directional swiping motions. The white-colored bar with the width of 2 cm is used as an object and mimics a finger. Each swiping is repeated 250 times by two adult males at their maximum speed, and the recognition test is also conducted by changing the distance between the MGS and the object from 6 to 14 cm. Figure 13 shows the test results for the recognition rate. At a distance under 10 cm, the recognition rates for all swipe-motions reach 100%. Note that Z-directional motions (push-pull) either starts or ends within 10 cm away from the sensor such that the recognition rate becomes 100% regardless of how far away the object ends or starts.
Fig. 13 Test results for the recognition rate of X- and Y- directional motions with various distance.
Table 1 compares the proposed MGS with various MGSs in [9,12,13]. If the conventional MGSs and the proposed MGS operate at the same supply voltage, power reduction ratio PR can be expressed as
where min(Iconv) is the lowest current consumption of the conventional MGSs and Iprop the current consumption of the propose MGS. The MGS in [13] consumes the lowest current of 13.0 mA among the conventional MGSs and the proposed MGS consumes 3.78 mA. Thus, the power consumption of the proposed MGS is reduced at least by 70.9% compared with other MGSs.
Table 1. Comparisons
5. Conclusion
This paper presents a low-power IR proximity-based MGS mainly composed of an IR LED and PDs. In conventional single package MGSs, it is inevitable to use an optical block that limits the size and the FOV of each PD and leads to larger power dissipation in an IR LED. The proposed MGS with a partially open cavity package eliminates the use of an optical block resulting in larger size and FOV of each PD. As a result, the power consumption of the proposed MGS can be reduced by at least 70.9% compared with other IR proximity-based MGSs.
Acknowledgments
This research was supported by Basic Science Research Program through the National Research Foundation of Korea (NRF) funded by the Ministry of Science, ICT & Future Planning (NRF-2014R1A1A1003771). This research was also supported by Samsung Electronics.
References and links
1. O. Abdel-Hamid, A.-R. Mohnmed, H. Jiang, L. Deng, G. Penn, and D. Yu, “Convolutional neural networks for speech recognition,” IEEE/ACM Trans. Audio, Speech, and Language Processing 22(10), 1533–1545 (2014).
2. H. Sak, A. Senior, K. Rao, O. Irsoy, A. Graves, F. Beaufays, and J. Schalkwyk, “Learning acoustic frame labeling for speech recognition with recurrent neural networks,” in Proceedings of IEEE International Conference on Acoustics, Speech and Signal Processing (IEEE, 2015), pp.4280–4284. [CrossRef]
3. W. Zhang, M. L. Smith, L. N. Smith, and A. Farooq, “Eye center localization and gaze gesture recognition for human-computer interaction,” J. Opt. Soc. Am. A 33(3), 314–325 (2016). [CrossRef] [PubMed]
4. I. Hong, K. Bong, D. Shin, S. Park, K. Lee, Y. Kim, and H.-J. Yoo, “A 2.71nJ/pixel 3D-stacked gaze-activated object-recognition system for low-power mobile HMD applications”, in in Proceedings of IEEE International Conference on Solid- State Circuit Conference (IEEE, 2015), pp.1–3. [CrossRef]
5. A. Sewaiwar, S. V. Tiwari, and Y.-H. Chung, “Visible light communication based motion detection,” Opt. Express 23(14), 18769–18776 (2015). [CrossRef] [PubMed]
6. C. Wang, Z. Liu, and S.-C. Chan, “Superpixel-based hand gesture recognition with Kinect depth camera,” IEEE Trans. Multimed. 17(1), 29–39 (2015). [CrossRef]
7. D. Ryu, D. Um, P. Tanofsky, D. H. Koh, Y. S. Ryu, and S. Kang, “T-less: a novel touchless human-machine interface based on infrared proximity sensing”, in Proceedings of IEEE/RSJ International Conference on Intelligent Robots and Systems (IEEE, 2010), pp.5220–5225.
8. S. Lian, W. Hu, and K. Wang, “Automatic user state recognition for hand gesture based low-cost television control system,” IEEE Trans. Consum. Electron. 60(1), 107–115 (2014). [CrossRef]
9. Y. S. Kim and K.-H. Baek, “A motion gesture sensor using photodiodes with limited field-of-view,” Opt. Express 21(8), 9206–9214 (2013). [CrossRef] [PubMed]
10. H.-T. Cheng, A. M. Chen, A. Razdan, and E. Buller, “Contactless gesture recognition system using proximity sensors”, in Proceedings of IEEE International Conference on Consumer Electronics (IEEE, 2011), pp.149–150. [CrossRef]
11. C.-T. Chuang, T. Chnag, P.-H. Jau, and F.-R. Chang, “Touchless positioning system using infrared LED sensors”, in Proceedings of IEEE International Conference on System Science and Engineering (IEEE, 2014), pp.261–266. [CrossRef]
12. K.-H. Kong, Y. S. Kim, J. E. Kim, S. Kim, and K.-H. Baek, “Single-package motion gesture sensor for portable applications,” IEEE Trans. Consum. Electron. 59(4), 848–853 (2013). [CrossRef]
13. J. S. Kim, S. J. Yun, D. J. Seol, H. J. Park, and Y. S. Kim, “An IR proximity-based 3D motion gesture sensor for low-power portable applications,” IEEE Sens. J. 15(12), 7009–7016 (2015). [CrossRef]
14. T. Martinez, D. Wick, and S. Restaino, “Foveated, wide field-of-view imaging system using a liquid crystal spatial light modulator,” Opt. Express 8(10), 555–560 (2001). [CrossRef] [PubMed]
