Open Access
Add to BibSonomyAbstract
In order to increase the data rate of the camera-based visible light communication (VLC) system, using rolling shutter effect has been demonstrated successfully, in which the pixel rows of the complementary-metal-oxide-semiconductor (CMOS) image sensor are activated sequentially. Previous camera-based VLCs focused on using a stable LED light source, and its illumination area is positioned at the center of an image frame. In this work, we investigate the performance of a camera-based VLC with light source at different parts of an image frame. We propose and demonstrate using region-grow algorithm to track the light source. We also evaluate and discuss different scenarios when the light source is moved. Besides, a recorded > 5 kbit/s net data rate can be achieved by using only a single phosphor-based white-light LED source. Here, we demonstrate that 4.502 pixel/bit can be achieved.
© 2016 Optical Society of America
1. Introduction
Visible light communication (VLC) has attracted much attention. It is considered as a possible way to offer high speed and secure wireless communication for the future 5th generation (5G) wireless communications [1]. VLC uses the visible electromagnetic (EM) spectrum; hence it relieves the congested radio-frequency (RF) spectrum in traditional wireless communications. VLC can be transmitted via the lighting systems; hence little extra cost is needed for the deployment. It is very directional and the communication zone is confined in the illumination area; hence it is secure and can provide high wireless communication density since the interference among neighbor communication zones is small. Recent studies of VLC mainly focus on using positive-intrinsic-negative (PIN) and avalanche photodiode (APD) as receivers (Rxs) [2–7]. As nearly all mobile-phones and vehicles are embedded with complementary-metal-oxide-semiconductor (CMOS) cameras and image sensors nowadays, it is important if these cameras can be used as the VLC Rxs.
However, using these cameras as the VLC Rxs is challenging, since the data rate is limited by the frame rate of the camera. A car-to-car VLC using camera has been reported [8], but the data rate is limited to 3 x 50 bit/s using red-green-blue (RGB) light emitting diodes (LEDs). In order to increase the data rate, using the rolling shutter effect has been reported [9], in which the pixel rows of the CMOS image sensor are activated sequentially. By demodulating this rolling shutter pattern, ~1 kbit/s data rate can be achieved [10]. Schemes for improving the demodulation of the rolling shutter pattern were reported in [10, 11].
Previous camera-based VLCs focused on using a stable LED light source, and its illumination area is positioned at the center of an image frame. However, in most cases, the LED light source is unstable and will be moved to different parts of an image frame. In this work, we investigate for the first time up to our knowledge, the performance of a camera-based VLC with light source at different parts of an image frame. We propose and demonstrate using region-grow algorithm to track the light source; and we evaluate and discuss different scenarios when the light source is moved vertically and horizontally. A recorded 5760 bit/s net data rate is achieved.
2. Algorithms and experiment
Figure 1(a) shows a proof-of-concept camera-based VLC experiment. Matlab program is used to generate the data packets, which are applied to a phosphor-based white-light LED (Cree XLamp XR-E, with direct modulation bandwidth of ~1.2 MHz) via an arbitrary waveform generator (AWG, Tektronix, AFG 3252C). The AWG has a sampling rate and bandwidth of 2 GSample/s and 240 MHz respectively. The modulated VLC signal is then received by a mobile-phone camera (iPhone6) through different distances and offsets of free space channel. The iPhone6 CMOS camera has a resolution and frame rate of 1080 x 1920 and 60 fps respectively. The recorded files are then transferred to a computer for demodulation. During the data recoding, the pixel row in the CMOS camera is activated sequentially. Bright and dark fringes will be captured in different pixel rows when the LED light source is intensity modulated on and off respectively. After the complete activation of all the pixel rows in an image frame, the signals obtained from different pixels will be combined to form one image. During this frame-to-frame processing time, the CMOS image sensor is “blind” and cannot detect any light. This processing time is about 40% of the frame duration. Hence, each packet sending from the LED light source is transmitted successively for 3 times to guarantee a complete data packet can be detected in an image frame for successful data demodulation. In the experiment, each packet includes a 12-bit header consists of logic “111100001111” and a 32-bit to 96-bit payload data in non-return-to-zero (NRZ) format.
Fig. 1 (a) Proof-of-concept camera-based VLC experiment. (b) Histogram of the grayscale values of a rolling shutter pattern obtained in an CMOS image sensor. Inset: rolling shutter pattern without and with the region-growing.
In the demodulation process, the video is first converted into grayscale, in which 0 means completely darkness and 255 means completely brightness. Then a vertical column pixel will be selected for the demodulation. As discussed in [11], “blooming effect” is usually observed in the CMOS image sensor, and it is due to the charge overflow from the saturated pixels into the neighboring pixels. Hence, many originally dark fringes will be received as bright fringes due to the “blooming”. Ref [11]. reported an effective way to mitigate the “blooming effect”. Each element in a row pixel of the image sensor can be represented in (xi, yi), where xi is the x-coordinate (i.e. ith pixel), and yi is the grayscale value of that pixel, i = 1, 2, … 1920. By using a second order polynomial fitting curve f(xi) to these grayscale values; and setting |yi - f(xi)| < 5, we can obtain several yi satisfying the above condition. Then, these values are arranged in ascending order according to their grayscale values, and 20% value is chosen. As a result, when processing a dark fringe with “blooming”, the selected 20% grayscale value will be approaching to zero. When processing a bright fringe with “blooming”, as the 20% value is selected (avoiding the too bright grayscale value), the “blooming” can be mitigated [11]. However, this scheme can only work well when the LED source is positioned at the center of the CMOS sensor. If the LED source is moved to the edge of the sensor leaving behind a dark screen in most part of the image sensor, grayscale value of 0 will be obtained when even processing a bright fringe.
In order to mitigate the “blooming” with light source at the edge of the image sensor, a region-grow algorithm [12] is used. This scheme is to select an object (in this case is the light source) from an image by defining a suitable threshold. The algorithm is: assume a is part of the object iff the grayscale image f(a) > T ; otherwise it is part of the background. A good way to select the suitable threshold T is to search the histogram of the grayscale values. Figure 1(b) shows the grayscale histogram of the rolling shutter pattern obtained in the CMOS image sensor. By selecting a threshold T, the light source can be distinguished and other pixels, as shown in the inset of Fig. 1(b). By change the T to T’, the region of the light source will become smaller as also shown in the inset of Fig. 1(b). After the light source has been tracked by the region-grow algorithm, the pixel xi at the center of the light source will become the reference, then the left and right margins can be set. In our case, we use the xi – 320 and xi + 320. This can avoid 0 grayscale level is obtained for processing a bright fringe with light source at the edge of the image sensor. After this, a column matrix of grayscale values can be selected having 1080 elements. Figures 2(a)-2(c) show the original rolling shutter patterns and the light source tracked when the light source is moved vertically; and Figs. 2(d)-2(f) show the corresponding selected column matrix of grayscale values with 1080 elements. In order to facilitate deciding the data logic, a second order polynomial fitting (red curve) is applied for smoothing, as shown in Figs. 2(d)-2(f). In the smoothing process, the original grayscale values larger than the polynomial fitting curve will be assigned equal to that curve, so that the grayscale value fluctuation can be significantly reduced as shown on Figs. 2(g)-2(i). Then a thresholding scheme used to define the logic 0 and 1 is applied. Here, we use the quick adaptive thresholding (QAT) [13], and its algorithm is: let yi be the grayscale value of a pixel at point i, and s is the number of pixels adjacent to point i. The threshold value from the quick adaptive scheme is described in Eq. (1). The red curves shown in Figs. 2(g)-2(i) are the QAT curves used to define the data logic.
Fig. 2 (a)-(c) Original rolling shutter patterns and the light source tracked when the light source is moved vertically. (d)-(f) Selected column matrix of grayscale values with 1080 elements. (g)-(i) QAT curves used to define the data logic after smoothing.
3. Results and discussion
After the light source tracking, “blooming” mitigation, grayscale column matrix selection, polynomial smoothing and QAT, the bit-error-rate (BER) performance of the camera-based VLC can be evaluated. As mentioned before, each VLC packet is transmitted successively for 3 times to guarantee a complete data packet can be detected in an image frame for successful data demodulation due to frame-to-frame processing time gap. In the transmitted VLC packet, 12-bit is used for the header; and 32-bit, 48-bit, 64-bit, and 96-bit are used for the payloads. These represent the net data rates of 1920 bit/s (32 bit/frame x 60 frame/s), 2880 bit/s, 3840 bit/s and 5760 bit/s respectively without considering the header and the repeated packets. The performance depends on the illuminance at the Rx, which in turn affects the received signal-to-noise ratio (SNR). Figure 3(a) shows the BER performance of the camera-based VLC at different illuminance when the LED light source is at different vertical positions. As the image sensor contains 1080 pixel rows, we select three vertical positions for evaluation: center (540th pixel row); top (810th pixel row) and extreme top (1000th pixel row). The 7% forward error correction (FEC; BER = 3.8 x 10−3) requirement can be satisfied at 800 lux when data rates are 2880 bit/s and 1920 bit/s; and at 550 lux when data rate is 1920 bit/s. We can observe that in these two data rates, negligible BER performance variation is observed even the LED light source is moved to the extreme top case. This is because the smoothing process can still correctly reduce the data fluctuation; and the QAT can correctly defining the decision threshold. However, when the data rate is increased to 3840 bit/s; this means the average number of pixel per bit is decreased. This also means the pixel per bit of 11.13 pixel/bit @1920 bit/s is decreased to 6.406 pixel/bit @3840 bit/s. Hence the bit resolution for defining the data bit is not large enough; and the QAT defines incorrectly resulting in a high error. In this case, further signal enhancement scheme, such as the Sobel filtering signal enhancement (for enhancing the signal transition) can be used [11]. Figures 3(b) and 3(c) show the screen images at the extreme top case at data rate of 3840 bit/s before and after light source tracking. After the grayscale column pixel matrix selection and second order polynomial smoothing, we can still observe that the QAT cannot correctly define the data logic, as shown in the red curve of Fig. 3(d). Then, by applying the Sobel filtering, we can observe in Fig. 3(e) that the extinction-ratio of the data pattern can be enhanced; and now the QAT can properly defining the data logic. Hence the BER performance can be recovered as shown in Fig. 3(a).
Fig. 3 (a) BER performance of the camera-based VLC different illuminance when the LED light source is moved vertically. Screen images at the extreme top case at 3840 bit/s (b) before and (c) after light source tracking. Grayscale value of the column pixel matrix selection (d) before and (e) after the Sobel filtering signal enhancement.
Figure 4(a) shows the BER performance of the camera-based VLC at different illuminance when the LED light source is at different horizontal positions. As the image sensor contains 1920 pixel columns, we select three vertical positions for evaluation: center (960th pixel column); right (1440th pixel column) and extreme right (1700th pixel column). Figures 4(b) and 4(c) show the screen images at the right and extreme right cases; and Fig. 4(d) and 4(e) show the corresponding light sources after region-growing. Unlike the cases of light source moving vertically as discussed in Fig. 3(a), as long as the center of the light source is still at the central horizontal plane of the CMOS image senor, the BER performance is independent of the light source horizontal movement. Hence, a high data rate of 3840 bit/s can still achieve the FEC requirement even at the extreme right case without using Sobel filtering signal enhancement scheme. This is because even at the extreme right case, once the light source is tracked and the proper margin (discussed in Section 2) is defined, the column grayscale matrix can be easily defined; hence the BER performance is the same at that when the light source is positioned at the center of the image sensor. Since the performance is affected by the light source vertical movement at high data rates; and the performance is nearly unaffected by the light source horizontal movement. Hence, if the light source is off-centered in both vertical and horizontal case, the performance would be similar to that of light source vertical movement.
Fig. 4 (a) BER performance of the camera-based VLC different illuminance when the LED light source is moved horizontally. Screen images at the (b) right and (c)extreme right cases; and the corresponding light sources (d), (e) after region-growing.
Figure 5(a) shows the system performance at different data rates and different illuminance. A high data rate of 5760 bit/s (using 96 bit/frame x 60 frame/s), since the pixel per bit is very small, resulting in a lower bit resolution for defining the data bit. Hence, higher illuminance is required. Figure 5(b) show the bit resolution (i.e. pixel per bit) under different data rates. When the data rates are 1920 bit/s, 2880 bit/s, 3840 bit/s and 5760 bit/s; the bit resolutions are 11.13 pixel/bit, 8.166 pixel/bit, 6.406 pixel/bit and 4.502 pixel/bit respectively. In the rolling shutter operation of the CMOS image sensor, the pixel row is activated without waiting for exposure completion of the previous row [11]; hence, there is an overlapping of exposure time of each pixel row. This means that using one pixel to represent one bit never happens, and we believe our demonstrated 4.502 pixel/bit is quite near to the minimum bit resolution. In order to improve the data rate, higher pixel image sensor or higher frame rate can be used.
Fig. 5 (a) BER performance of the camera-based VLC different illuminance at different data rates and different illuminance. (b) Bit resolution (i.e. pixel per bit) under different data rates.
4. Conclusion
In this work, we investigated the performance of a camera-based VLC system with light source moving to different parts of an image frame. We proposed and demonstrated using region-grow algorithm to track the light source. Negligible BER performance variation was observed even the LED light source was moved to the extreme top case at low data rates. At high data rate, since the pixel per bit was decreased, the bit resolution for defining the data bit was not large enough. Hence, Sobel filtering signal enhancement was used to enhance the signal performance. On the other hand, if the center of the light source was at the central horizontal plane of the senor, the BER performance was independent of the light source horizontal movement. A recorded net data rate of 5760 bit/s was achieved with bit resolution of 4.502 pixel/bit.
Acknowledgments
This work was supported by Ministry of Science and Technology, Taiwan, ROC, MOST-104-2628-E-009-011-MY3, MOST-103-2221-E-009-030-MY3, Aim for the Top University Plan, Taiwan, and Ministry of Education, Taiwan.
References and links
1. S. Wu, H. Wang, and C. H. Youn, “Visible light communications for 5G wireless networking systems: from fixed to mobile communications,” IEEE Netw. 28(6), 41–45 (2014). [CrossRef]
2. C. W. Chow, C. H. Yeh, Y. Liu, and Y. F. Liu, “Digital signal processing for light emitting diode based visible light communication,” IEEE Photonics Soc. Newslett. 26, 9–13 (2012).
3. H. H. Lu, Y. P. Lin, P. Y. Wu, C. Y. Chen, M. C. Chen, and T. W. Jhang, “A multiple-input-multiple-output visible light communication system based on VCSELs and spatial light modulators,” Opt. Express 22(3), 3468–3474 (2014). [CrossRef] [PubMed]
4. B. Janjua, H. M. Oubei, J. R. Durán Retamal, T. K. Ng, C. T. Tsai, H. Y. Wang, Y. C. Chi, H. C. Kuo, G. R. Lin, J. H. He, and B. S. Ooi, “Going beyond 4 Gbps data rate by employing RGB laser diodes for visible light communication,” Opt. Express 23(14), 18746–18753 (2015). [CrossRef] [PubMed]
5. W. Y. Lin, C. Y. Chen, H. H. Lu, C. H. Chang, Y. P. Lin, H. C. Lin, and H. W. Wu, “10m/500 Mbps WDM visible light communication systems,” Opt. Express 20(9), 9919–9924 (2012). [CrossRef] [PubMed]
6. C. H. Chang, C. Y. Li, H. H. Lu, C. Y. Lin, J. H. Chen, Z. W. Wan, and C. J. Cheng, “A 100-Gb/s multiple-input multiple-output visible laser light communication system,” J. Lightwave Technol. 32(24), 4723–4729 (2014). [CrossRef]
7. Z. Wang, C. Yu, W. D. Zhong, J. Chen, and W. Chen, “Performance of a novel LED lamp arrangement to reduce SNR fluctuation for multi-user visible light communication systems,” Opt. Express 20(4), 4564–4573 (2012). [CrossRef] [PubMed]
8. P. Luo, M. Zhang, Z. Ghassemlooy, H. L. Minh, H. M. Tsai, X. Tang, L. C. Png, and D. Han, “Experimental demonstration of RGB LED-based optical camera communications,” IEEE Photonics J. 7, 7904242 (2015).
9. C. Danakis, M. Afgani, G. Povey, I. Underwood, and H. Haas, “Using a CMOS camera sensor for visible light communication,” in Proc. OWC’12 (2012), pp. 1244–1248. [CrossRef]
10. C. W. Chow, C. Y. Chen, and S. H. Chen, “Visible light communication using mobile-phone camera with data rate higher than frame rate,” Opt. Express 23(20), 26080–26085 (2015). [CrossRef] [PubMed]
11. C. W. Chow, C. Y. Chen, and S. H. Chen, “Enhancement of signal performance in LED visible light communications using mobile phone camera,” IEEE Photonics J. 7(5), 7903607 (2015). [CrossRef]
12. C. K. Chow and T. Kaneko, “Automatic boundary detection of the left ventricle from cineangiograms,” Comput. Biomed. Res. 5(4), 388–410 (1972). [CrossRef] [PubMed]
13. P. D. Wellner, “Adaptive thresholding for the digital desk,” Tech. Rep. EPC-93–110 (1993).
