1. Introduction
In recent years, the wide-spread LED lamps for illumination purposes provide functional devices for signal modulations and imaginations of new communication and positioning approaches. Unlike the radio frequency (RF) communication approaches, visible light has little influence on sensitive electronic equipment. Hence, visible light based systems are compatible with scenarios like aircraft, hospital, airport, underwater and underground which requires communication and positioning facilities with stability concerns. Visible light based positioning (VLP) and visible light communication (VLC) systems have been researched widely in the past two decades from the basic principle level towards hybrid system and mixed fields with deep learning, reinforcement learning, and robotics [1], [2].
On the one hand, VLC has been envisioned as one of the key enabling technologies for the six-generation (6G) communication systems, due to its inherent advantages such as huge and unregulated spectrum, low-cost front-ends, and no electromagnetic interference (EMI) radiation [3]. The research on VLC has been mainly focused on the hardware developments such as high-bandwidth LEDs [4]-[6], spectral-efficient modulation schemes like orthogonal frequency division multiplexing (OFDM) [7]-[9], multiple-input multiple-output (MIMO) transmission [10]-[12], multiple access approaches [13]-[15], multi-cell networking [16], [17], etc. On the other hand, the development on VLP systems has less concerns on bandwidth but more focuses on the positioning algorithms and hardware setups. There are many positioning algorithms adopted in VLP systems, including received signal strength (RSS) [18]-[20], time difference of arrival (TDOA) [21], [22], and phased difference of arrival (PDOA) [23]-[25]. Moreover, there are generally two types of receivers that have been widely used in VLP systems, i.e., image sensor such as mobile phone camera [26], [27] and photodetector (PD). Recently, machine learning based approaches (e.g., gradient boost tree, kernel ridge regression, etc.) have also been introduced in VLP systems to boost the position solving speed and reduce intensity model based errors [20], [28]-[30].
Besides individual VLC and VLP, the integration of VLC and VLP for simultaneous communication and positioning has also been proposed and investigated in the literature [31]-[37]. In addition, hybrid RF and VLC (RF/VLC) systems have been further designed to achieve full-coverage bidirectional communications [38]-[44].
In this paper, we present a comprehensive review of our recent works on visible light positioning and communication. The rest of the paper is organized as follows. Section 2 describes the fundamental principle of visible light positioning and communication. Our recent works are reviewed in Sect. 3, including image sensor-based VLP, PD-based VLP, integrated VLCP and heterogeneous RF/VLC systems.
2. Principle
Visible light communication and positioning systems have both shared and individual components. In one way, they are compatible with each other in sharing transmitters (LED lamps) and receivers (photo-detectors, PDs), however different in algorithms and compete in resources in the other way. VLC and VLP systems usually have limited coverage in a single unit and such small coverage requires multiple VLC/VLP units to ensure full coverage. Here we only discuss on the setup of a individual VLP unit.
2.1 Modulations on LED Lamps
LED lamps consist of LED chips, and hence the fundamental limitations in modulation are born here. Most off-the-shelf white LED chips are based on phosphor stimulated by blue LED chips. Phosphor white LED chips are cheaper in price but have much smaller bandwidth comparing with RGB LED chips. It is also worth noticing that the LED chip with a smaller size usually has a higher bandwidth. Besides the chip itself, the connection of LED chips and driving circuits is also vital for determining the modulation bandwidth.
As the signal latency of LED chips is influenced by its inherent inductance, serial connection of the LED chips shall usually induce higher latency than parallel connection as the inductance is reduced. Moreover, the serial connection of LED chips can increase the voltage operation range of the LED lamp, while parallel connection of LED chips will increases the operating current range of the LED lamp. In actual experiments, voltage is usually monitored and hence a larger range of operating voltage towards individual LED lamp is preferred. One way of increasing voltage and reducing the latency is to connect LED chips with a mixed connection by parallel several serial connected LED chips to control both voltage range and frequency response. The modulation of LED lamps is intensity based modulation on the envelope of a light-wave carrier, which is visible light in this case. The modulation of LED lamps is directly from its driving circuit, which is one kind of internal modulation.
As shown in Fig. 1, the modulation of an LED lamp requires the following components: DC source, AC source, Amplifier, Bias-Tee, and the LED lamp. In some designs, several LED lamps can share the same DC or AC sources. DC source is used to maintain the operation of modulation at the linear region of the LED lamps. The linear range of LED lamps (V-I and I-P curves) are different and have to be tested to have the exact working range. The nonlinearity of modulation will induce high harmonic components, which create interference and reduce the bandwidth of modulation. The linear range of the input voltage is usually the limit of the modulation depth, and the larger modulation depth results in higher signal quality. Signal generator provides the AC source which is usually low in power. Hence, a power amplifier is usually applied to amplify the driving power. In consideration of modulation frequency, the pass band of bias-tee and amplifier should be aligned. The frequency response of LED lamps should be considered in the design, as a lower frequency usually has a higher response to the input signal.
 |
Fig. 1 Hardware of modulation on the LED lamp. |
2.2 Visible Light Receivers
There are typically two types of sensors for visible light based systems as the receiver: one is the image sensor and the other is the PD.
Image sensors are well known and used these days, which provide a low-frequency response towards the light signal usually below 20 kHz. The frame rate of the image sensor is usually less than 120 fps, and the data collected are digital and array-like pixels. From image sensors, the shape and rough intensity of the lamps can usually be extracted. Moreover, the rolling shutter effect is conventionally utilized to enable a low-speed communication. Meanwhile, PDs can respond to a relatively higher range of frequencies from a few kHz to hundreds of MHz levels. The output of a single PD is normally an analogue serial signal. Digitization is usually required before performing digital signal processing (DSP) algorithms.
2.3 Summary of Positioning Algorithms
Image sensor is usually a cost-efficient solution for VLP systems, and imaging theory is the basic principle of image sensor-based VLP system. Conventionally, at least three LED lamps with known locations are required to uniquely define the location of the image sensor:
| \[\begin{equation*} \left[\begin{array}{c c c}{{u_{1}}}&{{u_{2}}}&{{u_{3}}} \\ {{v_{1}}}&{{v_{2}}}&{{v_{3}}} \\ {{w_{1}}}&{{w_{2}}}&{{w_{3}}}\end{array}\right] = [\mathbf{R}\ \mathbf{\theta}] \left[\begin{array}{c c c}{{x_{1}}}&{{x_{2}}}&{{x_{3}}} \\ {{y_{1}}}&{{y_{2}}}&{{y_{3}}} \\ {{z_{1}}}&{{z_{2}}}&{{z_{3}}} \\ {{1}}&{{1}}&{{1}}\end{array}\right], \tag{1} \end{equation*}\] |
where \(\mathbf{R}\) is the rotation matrix, \((x_i,y_i,z_i)\) is the location of the i-th LED lamp in the actual world, \((u_i,v_i,w_i)\) is the location of the i-th LED lamp in the image space, and \(\theta\) is the spacial transform (i.e., the position estimation) between the LED lamps in the actual world and the image space [27].
 |
Fig. 2 Overview of VLP systems. |
To solve the 3D position in the actual world, the Levenberg-Marquardt (LM) solver is conventionally used to minimize the error of the following equation:
| \[\begin{equation*} E=\sum_{i=1}^{3}\lVert(u_{i},v_{i},w_{i})^{T}-\mathrm{R}(x_{i},y_{i},z_{i})^{T}-\theta){\rVert}_{F}^{2}. \tag{2} \end{equation*}\] |
Compared with image sensor, PD can provide a much larger modulation bandwidth and intensity sensitive range, and hence it can enable time and intensity based positioning solutions. Received signal strength (RSS) is the conventional solution of intensity based VLP. The intensity is related to the distance and angle between the LED lamp and the PD:
| \[\begin{equation*} RSS_{i}=\frac{(m_i+1)A}{2\pi d_{i}^{2}} \cos^{m_i}(\varphi_{i}) \cos(\Psi_{i})P_{i}, \tag{3} \end{equation*}\] |
where \(P_i\) is the optical power of the i-th LED lamp and \(RSS_i\) is the corresponding signal strength received by the PD. \(m_i\) is the Lambertian order of the illumination model of the i-th LED lamp. \(d_i\), \(\varphi_{i}\) and \(\Psi_{i}\) are the relative distance, transmitting and receiving angles between the i-th LED lamp and the the PD, respectively. A is the effective area of the PD receiver. A set of at least three LED lamps can uniquely identify the location of the receiver (i.e., PD) relative the the LED lamps.
Besides using the intensity information, time of arrival (TOA) can also be utilized which exploits the time information contained in the transmitted optical signals. The ToA approach requires synchronization between the LED lamps and the PD. The transmission distance can be calculated by
| \[\begin{equation*} d_i = c (TOA_i+t_i), \tag{4} \end{equation*}\] |
where \(d_i\) and \(TOA_i\) are respectively the distance and time of flight (TOF) between the i-th LED lamp and the PD, while \(t_i\) is the calibration term as the time synchronization should be performed and calibrated with a constant for individual lamp. Moreover, c is the speed of light.
Both RSS and TOA approaches are aimed to obtain the absolute distance (\(d_i\)) between the LED lamps and the PD. Given the distance measurements between the LED lamps and the PD, the position of the PD can be solved as follows:
| \[\begin{equation*} d_{i}={\sqrt{(x_{i}-x)^{2}+(y_{i}-y)^{2}+(z_{i}-z)^{2}}}, \tag{5} \end{equation*}\] |
where \((x_i,y_i,z_i)\) is the pre-measured location of the i-th LED lamp and \((x,y,z)\) is the location of the PD. With the assumption that LED lamps are mounted in the ceiling, solving the position \((x,y,z)\) requires just three equations from three different LED lamps.
The TOA approach requires synchronization between the LED lamps and the PD, which introduces the extra connections between the LED lamps and the PD. Hence, time difference of arrival (TDOA) is proposed to release the situation by taking the signal transmitted as reference with each other, and one more LED lamp is usually required to solve the location of the PD:
| \[\begin{equation*} d_{i} - d_{i+1} = c (TOA_i - TOA_{i+1}) = c TDOA_i, \tag{6} \end{equation*}\] |
where \(TDOA_i\) is the TDoA measurement between the i-th LED lamp and the \(i+1\)-th LED lamp. It can be observed that, compared with TOA and RSS, the solution of \((x,y,z)\) relies on solving a fourth-order equation. Geometrically speaking, it is the intersection between three hyperboloids. Hence, four LED lamps are required in a single PD receiver setup to solve a unique solution of the PD.
Among these four major types of positioning algorithms, the least complexity is achieved by RSS. Using only intensity measurements makes it sensitive to rotations and disturbances, but also fast in calculations. The accuracy of angle-of-arrival (AOA) approach relies on the resolution of image sensor and image quality. Image processing and decoding increase the computational complexity naturally. The coverage of AOA system depends on the field-of-view (FOV) on both LED lamps and image sensor. TOA is the least recommended approach, who requires synchronization between transmitters and receiver. Synchronizations pose high hardware complexity and introduce noises to the position calculations. TDOA requires only the synchronization among the transmitters, which is much easier to achieve in practical implementations. A brief comparison of various positioning algorithms is summarized in Table 1, where stability describes the positioning robustness against various disturbances such as power variations, jolts/tilts of the receiver, partial blockages of the view, and so on.
Table 1 Comparison of VLP positioning algorithms. |
 |
3. Review of Our Recent Works
In this Section, we perform a detailed review of our recent works on visible light-based positioning and communication systems. Figure 3 illustrates the relationship of our recent works which will be discussed in the following. First of all, two categories of VLP systems are introduced, including both image-sensor-based and PD-based VLP systems. After that, an integrated VLCP system is described, which can realize simultaneous communication and positioning based on the same visible light LEDs. Last but not least, an indoor heterogeneous RF/VLC system is discussed, which can provide bidirectional full-coverage indoor connectivity by integrating RF communication and VLC together.
 |
Fig. 3 The relationship of our recent works in Sect. 3. |
3.1 Image Sensor-Based VLP
3.1.1 Image Sensor-Based VLP with Multiple LED Lamps
In conventional image sensor-based VLP systems, iteration-based solvers such as Levenberg-Marquardt (LM) and Newton solvers are generally adopted to solve the non-linear least-square (NLLS) positioning problems. Nevertheless, the iteration-based solvers are very sensitive to the initial guesses and a bad guess might result in a long response time and a large positioning error. In order to speed up the positioning process and enhance the robustness, we have proposed and experimentally demonstrated an image sensor-based VLP system with three LED lamps using a singular value decomposition (SVD)-based non-iterative positioning algorithm in [46].
Based on the LED lamps projection model illustrated in Fig. 4(a), the positioning process can be considered as a 3D point fitting problem, and hence the low-complexity SVD-based non-iterative positioning algorithm can be applied to obtain the receiver’s position and orientation. Let \(N_{LED}\) denote the number of LED lamps adopted in the VLP system. Defining the centroid coordinates of three objects and their image points as two vectors \({\bf P}_{0}\) and \({\bf P}^{'}_{0}\):
| \[\begin{eqnarray*} &&\!\!\!\!\! {\bf P}_{0}={\frac{1}{N_{LED}}}\sum_{i=1}^{N_{LED}}(x_{i},y_{i},z_{i})^{T}, \tag{7} \\ &&\!\!\!\!\! {\bf P}^{'}_{0}={\frac{1}{N_{LED}}}\sum_{i=1}^{N_{LED}}(u_{i},v_{i},w_{i})^{T}, \tag{8} \end{eqnarray*}\] |
they should also satisfy \(\mathbf{P}_{0}^{\prime}=\mathbf{RP}_{0}+\mathbf{\theta}\). Based on \({\bf P}_{0}\) and \({\bf P}^{'}_{0}\), we further define the following two matrices:
| \[\begin{equation*} {\bf Q}_{i}=(x_{i},y_{i},z_{i})^{T}-{\bf P}_{0}, \; {\bf Q}^{'}_{i}=(u_{i},v_{i},w_{i})^{T}-{\bf P}^{'}_{0} \tag{9} \end{equation*}\] |
with \(i = 1, 2,\cdots, N_{LED}\). Hence, by cancelling the translation matrix \(\mathbf{\theta}\), the square cumulative error (SCE) is calculated by
| \[\begin{equation*} E=\sum_{i=1}^{N_{LED}}\lVert\mathbf{Q}_{i}^{\prime}-\mathbf{RQ}_{i}\rVert_{F}^{2}. \tag{10} \end{equation*}\] |
 |
Fig. 4 Image sensor-based VLP with three LED lamps using a SVD-based non-iterative positioning algorithm: (a) LED lamps projection model, (b) experimental setup, (c) CDF of 3D positioning error at two layers, and (d) comparison table of accuracy and robustness of LM & SVD based positioning algorithms. |
Here, \(\mathbf{R}_{opt}\) should be found to minimize the error. Letting \(Tr(\cdot)\) denote the trace of a matrix, minimizing (10) becomes minimizing (11):
| \[\begin{equation*} E=\sum_{i=1}^{N_{LED}}\;T r(2{\bf Q}_{i}^{'T}{\bf R}{\bf Q}_{i}) = Tr(2\mathbf{RH}), \tag{11} \end{equation*}\] |
where \(\mathrm{\bf{H}}=\sum_{i=1}^{N_{LED}}{\bf{Q}}_{i}{\bf{Q}}^{\prime}i^{T}\). The SVD of \(\mathbf{H}\) can be expressed by two orthonormal matrices \(\mathbf{V}\) and \(\mathbf{U}\):
| \[\begin{equation*} \mathbf{H}\ =\ \mathbf{U}{\Omega}\mathbf{V}^{T}, \tag{12} \end{equation*}\] |
where \(\Omega\) is a diagonal matrix. Thus, \(\mathbf{R}_{opt}\) can be achieved at: \(\mathbf{R}_{opt} = \mathbf{VU}^T\) and the position can be estimated by:
| \[\begin{equation*} \mathbf{\theta}_{o p t}={\bf P}^{'}_{0}-\mathbf{R}_{o p t}\mathbf{P}_{0}. \tag{13} \end{equation*}\] |
Figure 4(b) depicts the experimental setup of the image sensor-based VLP system with three LED lamps, where the three LED lamps are mounted on the floor for convenience and the rear camera of a smartphone with a resolution of about 40 Mega pixels is adopted as the receiver. Two positioning accuracy in two planes with the heights of 120 cm and 160 cm are evaluated, where a total of 35 grid testing points are deployed on each plane. Figure 4(c) shows the CDF of the positioning errors for two heights. As we can observe, the proposed system can achieve high-accuracy 3D positioning in centimeters in a practical environment. Moreover, it can be observed from the Fig. 4(d) that the proposed SVD-based non-iterative positioning algorithm achieves better or at least the same positioning accuracy as the iteration-based LM positioning algorithm. However, the iteration-based LM positioning algorithm suffers from approximately 90 failures out of 10000 runs, where a positioning failure is defined as that the corresponding positioning error is larger than 50 cm, while the proposed SVD-based non-iterative positioning algorithm has no failures out of 10000 runs. Therefore, the proposed SVD-based non-iterative positioning algorithm has much enhanced positioning robustness in comparison to the iteration-based LM positioning algorithm.
3.1.2 Image Sensor-Based VLP with a Single LED Lamp
Due to the limited field of view (FoV) of smartphone cameras, the sparse deployment of LED lamps and the potential link blockage in practical indoor environments, it might be difficult to ensure the requirement of using at least three LED lamps for positioning. In order to address the limitation of multi-LED positioning, we have proposed and experimentally demonstrated an image sensor-based VLP system with a single LED lamp in [47].
Instead of treating the LED lamps as point sources with only location information, the geometric features of an LED image can be fully exploited to realize single-LED positioning. More specifically, we adopt a common circular LED lamp as the reference LED in the proposed image sensor-based single-LED VLP system. By utilizing the geometric features of the captured ellipse image including centroid, major/minor axis length, major axis orientation, the receiver’s pose and location relative to the reference LED lamp can be successfully determined. Figure 5(a) depicts the circular LED lamp with a a red marginal marker point on the x-axis, which is placed on the x-y plane with its centroid being coincided with the origin of the global coordinate system (GCS). Figure 5(b) shows the projected ellipse image on the i-j plane in the e receiver’s coordinate system (RCS). The point \((x_n,y_n)\) in the LED lamp plane has a corresponding point in the image sensor plane \((i_n,j_n)\):
| \[\begin{equation*} \bigl[ i_n \; \; j_n \; \; 0 \bigr] = \mathrm{diag}(s,s,0) \left\{{\bf R}\bigl\lbrack x_n\ \ \ y_n\ \ 0\bigr]^{T}+{\bf \theta}\bigr\rbrack\right\}, \tag{14} \end{equation*}\] |
where \(\mathbf{R}\) is the rotation transform matrix between the LED plane and the image plane, s is the size scaling factor between the lamp in image and the actual lamp, and \(\theta\) is the position transformation between the LED plane and the image sensor. Under the approximation of weak perspective projection, the center of the circular lamp (\(\bf{O}\)) is at the center of the eclipse lamp image (\(\bf{E}\)). The link \(\bf{OE}\) passes through the center of the camera lens and one diameter can be found in parallel to the major axis of the eclipse in the image plane. Hence, the scaling factor can be calculated by
| \[\begin{equation*} s=a/r, \tag{15} \end{equation*}\] |
where a is the length of the semi-major axis of the ellipse and r is the radius of the circular LED lamp. To avoid circular symmetry, a marker dot sticks to the lamp to determine \((r,0)\) in GCS. The rotation matrix is then \(\mathbf{R}\) from the marker, center, rightmost and upper most point pairs in both image and lamp, as in Fig. 5. The location of the receiver (\(x_{r}\; y_{r}\; z_{r}\)) can be directly obtained by compute (14) with the marker point pair. The detailed image processing workflow is given in Fig. 5(c).
 |
Fig. 5 Image sensor-based VLP with a single LED lamp: (a) the circular LED lamp with a marker in the GCS, (b) the ellipse image of the circular LED lamp in the RCS, (c) image processing workflow, (d) experimental setup, (e) circular LED lamp with a red marker, (f) captured LED image with modulated signal and a red marker, and (g) the CDF plot of the 3D positioning error. |
Figure 5(d) shows the experimental setup, where the positioning accuracy is tested in an area of 3 m \(\times\) 3 m and the height of the single LED lamp is set to 1.5 m and 2 m. Figures 5(e) and (f) show the circular LED lamp with a red marker and the captured LED image with modulated signal and a red marker, respectively. The CDF plot of the 3D positioning error is shown in Fig. 5(g). As we can see, a centimeter-level positioning accuracy can be achieved in proposed image sensor-based VLP with a single LED lamp.
3.2 PD-Based VLP
3.2.1 Reducing Complexity of TDOA-Based VLP and PDOA-Based VLP
Despite of the commonly accepted fact that TDOA and PDOA can achieve higher positioning accuracy than other methods, higher complexity contributed by the adoption of high-speed digitizer and local oscillators hinders the wide application of TDOA-based and PDOA-based VLP. In the year of 2017, we were committed to reducing the complexity of TDOA and PDOA algorithm for VLP. Two representative works were reported as below.
(1) PD-based VLP Using Low-Complexity TDOA
High complexity of TDOA-based VLP due to usage of real local oscillator (RLO) and high sampling-rate digitizer for signal processing is always a big concern. In order to address those two drawbacks, a practical TDOA-based VLP algorithm was proposed in [22] by introducing virtual local oscillator (VLO) and interpolation onto conventional correlation operation.
The diagram of the TDOA estimation is illustrated in Fig. 6(a). Different from the conventional TDOA estimator, the proposed system adopts cross-correlation followed with the cubic spline interpolation to measure the TDOA values, hence easing the strict requirement of sampling rate on ADC. Moreover, the local signal for cross-correlation is proposed to be from a software-defined VLO to further simplify the TDOA estimation. Upon mixed signals from multiple LEDs received, the narrow bandpass filters will extract the corresponding carrier signals. Then, the carrier signals are sent to the sync cutters and a peak locator simultaneously. The peak locator finds the first peak of the accumulation of the carrier signals within the first period. Then, the sync cutter behind the filters on each carrier channel receives the original carrier signal and cuts off a certain amount of sample points based on the output from the peak locator. Last but not the least, the correlator receives the trimmed signal and correlates it with the virtual local signal. The remaining steps just follow the conventional TDOA estimation. Likewise, the output of the TDOA estimator is obtained by subtracting \(t_{i+1}\) from \(t_i\) that is output by the corresponding peak detector, which is
| \[\begin{equation*} \begin{aligned} TDOA_{i}=\frac{1}{Sp}\cdot (\mathop{\mathop{\mathrm{arg max}}}_{n}{(R_{i+1}(n))}-\mathop{\mathop{\mathrm{arg max}}}_{n}{(R_{i}(n))}). \end{aligned} \tag{16} \end{equation*}\] |
The cross-correlation sequence of i-th carrier signal from the i-th LED is denoted by \(R_i(m)\) with the displacement of n, and Sp represents the sampling rate.
 |
Fig. 6 Low-complexity TDOA based VLP system: (a) proposed TDOA estimator with VLO and interpolator, (b) signal flow in the experiment, (c) experimental setup, positioning errors vs. (d) interpolation factor and (e) data length for correlation, (f) experiment results and (g) CDF of positioning errors. |
The proof-of-concept system is implemented as illustrated in Figs. 6(b) and (c), respectively. Three LED transmitters (Lumileds Rebel white LED) are mounted on fake ceiling in a height of 2.2 m. The carrier frequencies allocated to three transmitters are 4, 4.2 and 4.4 MHz, respectively. The carriers generated from a waveform generator (Tabor WW2074) are amplified before they are used to drive the LEDs. The receiver is composed of an photodetector module (Hamamatsu C12907) attached with a blue filter, a digitizer and a personal computer (PC). As shown in Fig. 6(c), the coordinate system follows the left-thumb rule with its origin on the drilling hole at the edge of the fake ceiling.
To further save consumption resources as much as possible while maintaining the system performance, parameter optimization is conducted on the data length for correlation and the interpolation factor. As shown in Fig. 6(d), the positioning error decreases only with the combination of the physical sampling rate and the interpolation factor. Therefore, the positioning accuracy under significantly different physical sampling rate can still be comparable to each other only if the multiplication of the physical sampling rate and the interpolation factor are at the same level. After the multiplication value of the physical sampling rate and the interpolation factor reaches 50 GSa/s, the decrease of the positioning error becomes minor. It can be inferred that the enhancement from interpolation has an upper limit. This is understandable because the measured time-of-flight itself inherently contains the digitization error and noise from practical hardware settings, which can not be perfectly compensated. As such, it is safe to set the multiplication of the physical sampling rate and the interpolation factor as 50 GSa/s, and under this constraint, the optimal combination of the physical sampling rate and the interpolation factor should be 500 MSa/s and 100 for the VLP system to reduce the physical sampling rate. Furthermore, we also consider the positioning error at 90% confidence versus the data length for correlation under 500 MSa/s and an interpolation factor of 100, in order to derive the acceptable minimum data length for correlation. As shown in Fig. 6(e), the positioning error increases with decrease of the data length. A critical value can also be found, below which the corresponding error is dramatically increased. As observed in Fig. 6(e), dramatic increase of positioning error occurs near 250k samples when the data length reduced from 1000k samples to 50k samples. As such, the acceptable minimum data length can be as low as 250k sample points.
Figure 6(f) depicts the overall accuracy in a coverage area of 1.2\(\times\)1.2 m\(^2\) with the sampling rate of 500 MSa/s. The average positioning accuracy of the proposed TDOA-based VLP system is less than 10 cm. Each position is measured for 25 times to assure the precision of the measurements. Since the positions of the LEDs are at the edge of the tested coverage shown in Fig. 6(f), a similar performance at the opposite area can be expected. Hence, a much larger coverage of at least 2\(\times\)1.2 m\(^2\) can be achieved for the demonstrated system. Figure 6(g) depicts the cumulative distribution function (CDF) curves of the positioning errors shown in Fig. 6(f). The positioning error is less than 18 cm at the confidence of 95% and the average positioning accuracy is 9.2 cm. In addition, the proposed system using VLO has slightly higher positioning accuracy than that using RLO, as per Fig. 6(g). The root cause is actually from the ways of experiment conduction. Since the RLO is mimicked in the demonstration by means of transmitting a real local signal via a cable from the waveform generator to the digitizer, the impedance of the cable slightly changes with the bending and rotation of the cable, hence causing different initial TDOA at different positions which are supposed to be constant. In conclusion, the proposed TDOA-based VLP can successfully remove RLOs and reduce required sampling rate, while still keeping comparable positioning accuracy.
 |
Fig. 7 RSS-based VLP system using deep learning: Workflow of (a) offline preparation and (b) online position estimation of the proposed VLP system, (c) Structure of proposed ANN, (d) Experimental set-up, 3D Position estimation at heights of (e) 1.094 m, (f) 0.794 m and (g) 0.494 m, 2D horizontal positioning error and vertical error at heights of (h) 1.094 m, (i) 0.794 m and (j) 0.494 m, (k) Computation time of position estimation for our proposition vs. the conventional 3D positioning method. |
(2) PD-based VLP using DPDOA
Differential PDOA (DPDOA) was proposed by us in the year 2017 with the initial motivation of removing local oscillators from the PDOA based VLP systems [24]. Local oscillator is initially implemented to calibrate the pseudo synchronization problem when using PDOA methods. Strict synchronization between transmitter and receiver is not required in TDOA and PDOA systems, but the combined periodical signals in the receiver end forms a periodical envelope. Also the starting point of calculating these signals has to be fixed. Hence, in TDOA and PDOA systems, a header is usually needed to distinguish the start of the received signal. However, errors in detecting the header is hard to avoid, which introduce errors in position calculations.
The introduction of the headers poses limits in the time resolution of the signals and hence the sampling rate of these systems has to be above 1 GSa/s. We introduce another frequency combined with the lowest frequency modulated to the same LED lamp to eliminate the pseudo synchronization term mathematically. Eventually, no headers and pseudo synchronization are required to perform PDOA, which reduces the hardware requirement to as low as 50 to 200 MSa/s sampling rate while maintain the same or even better positioning accuracy.
The transmitted signals are sine-waves of five frequencies \(\omega_i\)\((i=1, 2,\cdots, 5)\), and these frequencies are equally spaced (\(\omega_i = \omega_1 + (i-1)\Delta \omega\)). It is worth noticing that \(\omega_1\) and \(\omega_5\) are modulated to the same LED lamp while the other three lamps have single frequency modulation. The received signal from the PD is a summation of them based on the distance and can be expressed in the following equation:
| \[\begin{equation*} R x=\textstyle{\sum_{i=1}^{5}RSS_{i} \times s i n\bigl(\omega_{i}(t+t_{i}+T_{sample})\bigr)}, \tag{17} \end{equation*}\] |
where \(RSS_i\) is the distance related RSS measurement, \(t_i\) is the time of flight between \(L_i\) and the PD, and \(T_{sample}\) is the difference between transmitters’ clock and the receiver’s clock which is different at every collection of the signal. It is worth noticing that \(t_5=t_1\) as \(\omega_5\) is modulated to the same LED lamp with \(\omega_1\).
As \(T_{sample}\) is different at every measurement, PDOA and TDOA based methods use headers to neutralize the term by fixing the start point of sampling. Our proposed DPDOA method neutralizes it theoretically through a double differential process. After filtering, the received sine-wave signals of different frequencies are separated:
| \[\begin{equation*} Rx_i=RSS_{i} \; s i n(\omega_{i}(t+t_{i}+T_{sample})). \tag{18} \end{equation*}\] |
It is known that multiplying two sinewaves of different frequencies results in their frequency difference after filtering. We perform the first difference using \(D x_{i}=R x_{i}\times R x_{i+1}\). After filtering with a band-pass filter at \(\Delta \omega\), we have:
| \[\begin{equation*} \begin{split} D x_{i}(t)=\frac{RSS_{i}RSS_{i+1}}{2}\mathrm{cos}(\Delta\omega(t+T_{sample}) \\ -\omega_{i}t_{i}+\omega_{i+1}t_{i+1}). \end{split} \tag{19} \end{equation*}\] |
Then we perform another differential using \(DD x_{i}=D x_{i}\times D x_{i+1}\). This time a low pass filter is taken to obtain the term without \(T_{sample}\):
| \[\begin{equation*} D D x_{i}=\frac{A_{i}A_{i+1}^{2}A_{i+2}}{8}\cos(\omega_{i}t_{i}+\omega_{i+2}t_{i+2}-2\omega_{i+1}t_{i+1}). \tag{20} \end{equation*}\] |
With the use of Hilbert transform, similarly \(DD s_{i}=D x_{i}\times Hilb(D x_{i+1})\) can be obtained. Hence, we can obtain the distance differences from phase differences without the need of accurate header extraction:
| \[\begin{equation*} \left(\begin{array}{c c c}{\omega_1}&{-\omega_3}&{0} \\ {0}&{\omega_2}&{-\omega_4} \\ {\omega_5}&{\omega_5}&{\omega_5+\omega_3}\end{array}\right) \left(\begin{array}{c} d_1-d_2 \\d_2-d_3 \\d_3-d_4\end{array}\right) = c \left(\begin{array}{c}Dph_1 \\Dph_2 \\Dph_3\end{array}\right), \tag{21} \end{equation*}\] |
where \(D p h_{i}=\arctan\biggl(\frac{D D s_{i}}{D D x _{i}}\biggr)\) and \(t_i=d_i/c\). The \(D p h_{i}\) can be obtained from the received signals, and then the distance differences are computed for the final calculation of the 3D position. With the reduced sample rate requirement, more periods of signals can be obtained to further improve the positioning accuracy.
The final position of the receiver (PD) can be calculated from classical ways. The position estimations of DPDOA in the quarter of our testbed is shown in Fig. 8(c). Large shifting errors occur in the Y direction as shown in Fig. 8(d) up to 40 cm at the far edge of the testbed. Such distortions of the positioning space is concluded to be due to the imperfect modelling of the phase and intensity distributions. Hence, we further introduce the NN based solution to perform distortion calibrations. A NN of two hidden layers is implemented to perform both the position estimation and distortion correction. The NN based DPDOA VLP experimental results are shown in the bottom two sub-figures in Fig. 8(c). It can be observed that the distortions of the testbed are largely corrected especially at the edges of the testbed. To further reduce the variation of DPDOA based position estimations, we implement Kalman filter to converge the positioning errors based on the variation observed in the former estimations. The converged position estimations reduce the overall errors to as low as 5 cm, as shown in Figs. 8(d) and (e).
 |
Fig. 8 Differential PDOA based VLP system: (a) DPDOA signal flow and a individual VLP unit, (b) illustration of received positioning signals, (c) experimental DPDOA positioning results with and without neural network (NN) based shifting corrections and Kalman Filter convergence, (d) shifting errors, (e) evaluation of positioning RMS errors. |
3.2.2 Leveraging Machine Learning (ML) to Improve VLP Performance
With rapid development of machine-learning-based classification and regression that are able to benefit signal processing in terms of both accuracy and processing time, we have been striving to apply the latest machine-learning algorithm into VLP since the year of 2018. Such data-driven approaches can fill in the gap between the theoretical model of PDOA or RSS and the complex actual environment, hence advancing the practical usage of VLP.
(1) 3D RSS VLP using deep-learning-assisted trilateration
3D VLP using RSS currently has bottlenecks in two-fold. The first is tedious calibration during deployment, the other is lengthy computation time of position estimation. In order to address those two issues, a practical 3D VLP system, which retains the RSS-based ranging and further integrates deep learning to simplify the trilateration, is proposed in [20]. In this way, the computation speed of the proposed RSS-based VLP can be substantially increased with minimum offline preparation work. The proposed RSS-based VLP system is in the form of atto-cellular at the transmitter side to realize a full coverage indoors where a certain number of neighboring LED lamps are grouped into one atto-cellular unit, adopting the frequency division multiple access scheme. Unfortunately, the actual Lambertian order \(m_i\) is not close to the theoretical value. Particularly, each LED lamp has slightly different values for \(m_i\). Hence, it is necessary to perform the offline preparation to measure \(m_i\) using the reference positions with known coordinates. Differing from the previous work [48]-[50], our proposed offline preparation needs as minimum as two reference positions without details about the number or the transmitted power of LEDs. The only condition is that the span of the two reference positions should cover the localization area.
The workflow of the offline preparation is depicted in Fig. 7(a). As we can see in Fig. 7(a), there are three steps to fulfill the offline preparation, and the ultimate goal is to obtain the corrected Lambertian order. In [20], we theoretically derived the formula and proved that only two reference positions can help to calibrate the channel modelling well. In this way, the corrected channel model can be very close to the practical scenario. As such, the online position estimation depicted in Fig. 7(b) can be more precise.
The deep-learning techniques are adopted to replace the conventional process of trilateration. The structure of the proposed artificial NN (ANN) is illustrated in Fig. 7(c). The input of ANN is the set of RSS values measured from all LED lamps, and the output is the desired position coordinates. As shown in Fig. 7(c), there are three inputs of RSS and three outputs corresponding to the coordinate \((r_1, r_2, r_3)\). There are n hidden layers, and the activation function is denoted by f. The offline preparation is twofold. The first stage is to use the two reference positions to obtain Lambertian orders as in Fig. 7(b) to derive a calibrated channel model. The second stage is to train the ANN using the theoretical RSS values derived from the calibrated channel model with known positions. The dataset for training herein contains 80631 RSS samples along with the respective position coordinates. The locations of the samples form a grid with 5-cm resolution in the space of 2.5\(\times\)2.5\(\times\)1.5 \(\mathrm{m}^3\). The class of MLPRegressor from Scikit-learn library [51] and the backpropagation algorithm is adopted for training. 70% of the dataset is allocated for training, while 10% and 20% of the dataset is used for validating and testing, respectively. The learning rate is set as a constant value of 0.001.
Figure 7(d) is the photograph of the experimental setup. Three LED (Lumiled LXK8-PW50-0016) lamps were used as transmitters. They were installed on a horizontal panel whose height is 2.2 m. The power of each LED lamp was 9 W (27 W for one cell unit). The divergence angle was 120\(^\circ\), and the modulation index was set as 0.5 to assure LED modulation operated in the linear region. Provided that the frequency response of the LED chips in our works is less than 7 MHz, we allocated carrier frequencies of 1 MHz, 1.2 MHz and 1.4 MHz to three LEDs, respectively. The carriers are generated from a signal generator (Spectrum M4x.6622-x4) and amplified by the current boosters (Analog Devices Inc. AD811 and Burr-Brown BUF634) before they are used to drive the LEDs. The receiver was mounted on a trolley, which is consisted of an avalanche photo-diode module (Hamamatsu S8664-50K), an oscilloscope (Tektronix MSO3102) and a laptop. As limited by the experimental environment, we only select one quarter of the illumination coverage of LED lamps to test. The area is 1.2\(\times\)2 \(\mathrm{m}^2\). Considering the ceiling height, the 3D positioning space is 1.2\(\times\)1.2\(\times\)2 \(\mathrm{m}^3\), and the actual coverage for practical usage will be more than 3 times larger than tested herein. During the experiment, \(6\times6\times3\) locations were estimated (marked by red crosses in Figs. 7(e)–(g)). The heights of the vertical layers in Fig. 7(e), (f), (g) were 0.494 m, 0.794 m and 1.094 m, respectively.
It can be seen from Figs. 7(e)–(g) that most of the position estimations are close to the true positions. The positioning errors of those three heights are summarized by Fig. 7(h), (i), (j) in terms of horizontal errors and vertical errors. It is found that Z-coordinate is mostly worse than the X-Y coordinate. 90% vertical error is not more than 12 cm, which is still in an acceptable range. More importantly, the processing of position estimation achieves 50 times faster computing speed than the conventional system, as illustrated by Figs. 7(k). In conclusion, the deep-learning-based position estimator can easily fulfil the task of traditional estimator with significantly faster speed. Regarding the computation time, the deep-learning technique is superior over the traditional methods, but we need to acknowledge that more time is involved during offline preparation. Fortunately, the extra offline training hardly adds extra labor intensity, due to that the training is autonomous and the parameters of the ANN are constant for a completed deployment. In general, our proposition is especially suitable for those scenarios where zero-latency of positioning is needed.
(2) Hybrid RSS and DPDOA based VLP system using ANN
The RSS-based VLC systems usually have good SNR but are sensitive to all kinds of intensity disturbances from power-line/driver of the LED transmitters, indoor channel and receiver power supplies. These disturbances will induce higher positioning errors or even shifting.
As addressed formerly, we have implemented neural networks for both RSS and DPDOA based positioning algorithms with much faster position calculation speed, enabling the computation for both RSS and DPDOA with the same received signal at real time. Hence the variance based integration has been introduced to combine these results and yield enhanced positioning accuracy and stability [52].
The positioning results are based on statistical selections. We propose a T-selection and a V-selection approach for the hybrid system. The V-selection is based on the measured RSS variations as shown in the equation below:
| \[\begin{equation*} V={\frac{\sqrt{\sum_{n=1}^{N}(RSS_{k,n}-\overline{{{RSS_{k}}}})^{2}/N}}{\overline{{{RSS_{k}}}}}}, \tag{22} \end{equation*}\] |
where \(RSS_{k,n}\) is the n-th measurement of RSS from the k-th LED lamp at the total measurements of N times. The variation factor V is based on the standard deviation of the RSS measurements:
| \[\begin{equation*} P_{o u p u l s}={\left\{\begin{array}{l l}{P_{R S S}}&{V\leq p \%} \\ {P_{D P D O A}}&{V > p \%}\end{array}\right.}. \tag{23} \end{equation*}\] |
If the measured V factor is less than or equal to \(p \%\), RSS is considered to be a good choice of position estimations, otherwise the position estimations from DPDOA approach is preferred.
The T-selection is based on the criteria of student’s t-test [53]. The following equation is the calculation of the T factor:
| \[\begin{equation*} t={\frac{|P_{D P D O A}-P_{R S S}|}{s}}\times{\frac{d_{l i m}}{d}}, \tag{24} \end{equation*}\] |
where s is the standard deviation of the position estimations from DPDOA based approaches. The difference between the position estimation of DPDOA and RSS divided by the variation of DPDOA estimations is considered as the key of the t-factor. The t-factor is compensated by the distance, where the further distance the t will be reduced and positioning estimation from RSS is more trusted as better SNR is obtained by RSS measurements. The selection criteria of t-factor is shown as follows:
| \[\begin{equation*} P_{o u p u l s}={\left\{\begin{array}{l l}{P_{D P D O A}}&{T(t)\leq \alpha \%} \\ {P_{R S S}}&{T(t) > \alpha \%}\end{array}\right.}, \tag{25} \end{equation*}\] |
where \(T(t)\) is the students’ t-test distribution function and \(\alpha\) is the significance level. When t is larger, the \(T(t)\) will be smaller, and DPDOA based estimation is selected when \(T(t)\) below the significant level \(\alpha\).
The overall hybrid scheme is shown in Fig. 9(a). The proposed hybrid scheme is fully based on NN position solvers for both RSS and PDOA measurements. In the off-line process, we perform simulations to generate a large size dataset to perform pre-training to learn the basic positioning theories. With the pre-trained NN for both RSS and PDOA measurements (RSSMs and PDMs), a small dataset collected from experiments is introduced to further tune the NNs. After tuning, both NNs learn the intensity and time models in the current testbed and put in online applications. In the online part, real-time RSSMs and PDMs are feed in the RSS NN and PDOA NN separately to estimate positions \(P_{RSS}\) and \(P_{PDOA}\). Use V-selection or T-selection, the final position estimation is provided.
 |
Fig. 9 Hybrid DPDOA based VLP system: (a) flowchart of the online and offline process of the hybrid system, (b) RMS positioning error under different intensity variations, (c) CDF of different positioning approaches under different intensity variations, (d) comparison of V and T selection under different levels of random rotations. |
Under different intensity variations up to 20%, as in Fig. 9(b), T-selection achieves generally the best accuracy follows by V-selection. Pure ANN-based RSS solution suffers when variation over 8%. ANN-PDOA is not sensitive to variations but is relative low in accuracy when only small intensity variation introduced, as in Fig. 9(c). Under angular variation tests, as shown in Fig. 9(d), the stability of T-selection outperforms V-selection as more PDOA based estimations are involved.
3.3 Integrated VLCP
3.3.1 Integrated VLCP Based on FBMC-SCM and PDOA
In order to simultaneously perform communication and positioning using visible light LEDs, we have proposed and experimentally demonstrated a quasi-gapless integrated VLCP system based filter bank multicarrier-based subcarrier multiplexing (FBMC-SCM) and PDOA in [33].
Figure 10(a) shows the schematic diagram of the integrated VLCP system based on FBMC-SCM and PDOA, where three LED lamps are grouped as a unit to realize communication and positioning simultaneously. On the one hand, two-dimensional (2D) positioning is performed by applying four synchronized sinusoidal signals with different frequencies to three LED lamps and using a PD to collect the signal for PDOA calculation. On the other hand, communication is achieved by transmitting the same FBMC signal through the LED lamps so as to obtain transmit diversity and reduce implementation complexity. Due to the use of SCM to multiplex the communication and positioning signals in the frequency domain, guard bands (GBs) are generally required to eliminate the interference between each other. Compared with OFDM which usually exhibits high out-of-band interference (OOBI), FBMC has much lower OOBI and hence can substantially reduce the requirement of GBs, leading to higher spectral efficiency and improved positioning accuracy [32].
 |
Fig. 10 Integrated VLCP system based on FBMC-SCM and PDOA: (a) schematic diagram, (b) experimental setup, (c) BER distribution, (d) positioning results using FBMC-SCM, (e) CDF of positioning errors and (f) positioning error vs. GB frequency spacing. |
The experimental setup of the integrated VLCP system based on FBMC-SCM and PDOA is depicted in Fig. 10(b), where the communication and positioning performance is tested in a quarter area of 1.2 m \(\times\) 1.2 m with a vertical separation between the LED lamps and the PD of 2.1 m. Moreover, the overall modulation bandwidth of the integrated VLCP system is 10 MHz, where the frequencies of the four sinusoidal signals for PDOA positioning are within the range between 5.1 to 5.7 MHz and the frequency gap is 0.2 MHz. Figs. 10(c) and (d) show the communication and positioning results over the testing area in the integrated VLCP system. As we can observe from Fig. 10(c), the BER values for both OFDM-SCM and FBMC-SCM over the testing area are all less than the 7% forward error correction (FEC) coding limit of \(3.8 \times 10^{-3}\). Moreover, it can be seen from Fig. 10(d) that a much higher positioning accuracy can be achieved when the user is near the LED lamps, while the positioning accuracy is gradually reduced when the user is moving away from the LED lamps. Figure 10(e) compares the cumulative distribution functions (CDFs) of the positioning errors of the integrated VLCP system using OFDM-SCM and FBMC-SCM. Clearly, the system using FBMC-SCM can achieve a much enhanced positioning accuracy in comparison to that using OFDM-SCM. More specifically, the positioning error at 90% confidence for OFDM-SCM is 19.6 cm, which is reduced to 11.2 cm for FBMC-SCM. Figure 10(f) studies the impact of GB frequency spacing on the position performance of the integrated VLCP system. We can observe that a negligible GB frequency spacing of 0.1 MHz is required by the system using FBMC-SCM, while a significant 1.4 MHz GB frequency spacing is needed by the system using OFDM-SCM, suggesting a greatly improved effective bandwidth utilization ratio by using FBMC-SCM in comparison to OFDM-SCM.
3.3.2 Integrated VLCP with Intelligent Resource Allocation
The performance of a practical multi-user integrated VLCP system is largely determined by the adopted resource allocation strategy. Nevertheless, the design of an efficient resource allocation framework for practical multi-user integrated VLCP system is very challenging due to the issues of user mobility, diverse quality-of-services (QoS) requirements and distinctive user arrival and departure dynamics. In order to address the above issues when performing resource allocation, an intelligent resource allocation framework based on reinforcement learning (RL) has been proposed and investigated in [34].
Figure 11(a) shows the system model of the multi-user integrated VLCP system, where four LED lamps are mounted in the ceiling, multiple users are randomly located over the floor and each user is equipped with a PD. A central controller is adopted to connect all LED lamps and Wi-Fi links are used to provide the uplink feedback information. In the considered multi-user integrated VLCP system, RSS-based positioning is applied, and FBMC-SCM is employed to multiplex the FBMC signal and the positioning signal in the frequency domain. Considering the user mobility, diverse QoS requirements and distinctive user arrival and departure dynamics, conventional optimization technologies to perform resource allocation are usually non-convex and NP-hard, and hence the model-free RL technique is utilized to solve optimization problem in the proposed multi-user integrated VLCP system. Particularly, a Markov decision process (MDP) is used to model the decision making problem for power and subcarrier allocation, while a reward function is designed by considering users’ requirements. Moreover, a modified experience replay actor-critic (MERAC) RL approach is further proposed to enhance the the learning efficiency and convergence speed of finding the optimal solution in dynamic integrated VLCP systems, which is illustrated in Fig. 11(b).
 |
Fig. 11 Multi-user integrated VLCP system with intelligent resource allocation: (a) system model, (b) MERAC learning framework, (c) reward vs. training, (d) sum rate of users vs. minimum rate threshold, and (e) sum rate of users vs. maximum positioning error threshold. |
The performance of the multi-user integrated VLCP system with intelligent resource allocation has been evaluated via computer simulations, where an indoor room of 5 m \(\times\) 5 m \(\times\) 3 m is considered. Figure 11(c) depicts the reward versus the number of training time steps for various RL approaches. As we can see, the MERAC approach achieves the highest reward and its convergence speed is the fastest in comparison to other RL approaches. It can also be found in Figs. 11(d) and (e) that the MERAC approach always obtains the highest sum rate of users with the changes of minimum rate threshold and maximum positioning error threshold among all the benchmark approaches.
3.4 Heterogeneous RF/VLC
In practical industrial Internet of Things (IoT) networks, the devices usually have diverse QoS requirements (e.g., ultrareliable low-latency communications (URLLC) or high transmission data rates), and the spectrum and energy resources might be severely limited. In order to meet the users’ QoS requirements, we proposed and evaluated a heterogeneous RF/VLC industrial network in [41], where RF communication can offer a wide communication coverage while the VLC can provide high-speed data transmission.
Figure 12(a) shows the system model of the indoor heterogeneous RF/VLC industrial network, where multiple VLC access points (APs) are uniformly distributed in the ceiling of the room and one RF AP is located at the center of the ceiling. In the heterogeneous RF/VLC industrial network, VLC is used in the downlink, while RF is used in both the downlink and uplink. As we can see, each VLC AP only covers a small area while the RF AP can cover the whole area. In order to perform intelligent resource management, a deep post-decision state (PDS)-based experience replay and transfer (PDS-ERT) RL algorithm is proposed, and its principle is shown in Fig. 12(b).
 |
Fig. 12 Indoor heterogeneous RF/VLC industrial network: (a) system model, (b) deep PDS-ERT learning-based intelligent resource management, (c) learning process comparisons of RL algorithms, (d) energy efficiency vs. packet arrival rate, and (e) URLLC latency per packet vs. packet arrival rate. |
We evaluate the performance of the indoor heterogeneous RF/VLC industrial network through computer simulations, where an indoor industrial room with a dimension of 24 m \(\times\) 24 m \(\times\) 6 m is considered and totally 6 \(\times\) 6 VLC APs are assumed to be uniformly deployed in the network. Figure 12(c) depicts the learning process of various RL algorithms in terms of the reward. It can be clearly observed that the deep PDS-ERT learning algorithm achieves the highest reward with the fastest convergence and the best stability. It can also be seen that the deep PDS-ERT learning algorithm obtains the highest energy efficiency per device and the lowest URLLC latency per packet in comparison to other benchmark RL algorithms.
4. Conclusion and Outlook
In this paper, we have conducted a detailed review of our recent works on visible light positioning and communication systems using illumination LEDs. To provide high-accuracy indoor positioning, two types of VLP systems including image sensor-based VLP and photodetector-based VLP are developed and demonstrated. Moreover, to support simultaneous communication and positioning services for indoor users, integrated VLCP systems are further designed and evaluated. Finally, heterogeneous RF/VLC systems are also investigated to ensure the diverse QoS requirements of users. In brief, the progress in the field of visible light positioning and communication is rapid, which attracts the attention from both academia and industry.
In the future, visible light positioning and communication systems are expected to achieve much higher data rates and more accurate positioning in practical real-world application scenarios. Some of the potential research directions include the cooperation of VLC and VLP for overall performance enhancement, the application of powerful machine learning/artificial intelligence tools in visible light positioning and communication systems, etc.
References
[1] T. Komine and M. Nakagawa, “Fundamental analysis for visible-light communication system using LED lights,” IEEE Trans. Consum. Electron., vol.50, no.1, pp.100–107, 2004.
CrossRef
[2] H.S. Liu and G. Pang, “Positioning beacon system using digital camera and LEDs,” IEEE Trans. Veh. Technol., vol.52, no.2, pp.406–419, 2003.
CrossRef
[3] N. Chi, Y. Zhou, Y. Wei, and F. Hu, “Visible light communication in 6G: Advances, challenges, and prospects,” IEEE Veh. Technol. Mag., vol.15, no.4, pp.93–102, Dec. 2020.
CrossRef
[4] H. Li, X. Chen, B. Huang, D. Tang, and H. Chen, “High bandwidth visible light communications based on a post-equalization circuit,” IEEE Photon. Technol. Lett., vol.26, no.2, pp.119–122, Jan. 2014.
CrossRef
[5] D. Tsonev, H. Chun, S. Rajbhandari, J.J. McKendry, S. Videv, E. Gu, M. Haji, S. Watson, A.E. Kelly, G. Faulkner, M. Dawson, H. Haas, and D. O’Brien, “A 3-Gb/s single-LED OFDM-based wireless VLC link using a gallium nitride μLED,” IEEE Photon. Technol. Lett., vol.26, no.7, pp.637–640, April 2014.
CrossRef
[6] S. Rajbhandari, H. Chun, G. Faulkner, K. Cameron, A.V. Jalajakumari, R. Henderson, D. Tsonev, M. Ijaz, Z. Chen, H. Haas, E. Xie, J. McKendry, J. Herrnsdorf, E. Gu, M.D. Dawson, and D. O’Brien, “High-speed integrated visible light communication system: Device constraints and design considerations,” IEEE J. Sel. Areas Commun., vol.33, no.9, pp.1750–1757, Sept. 2015.
CrossRef
[7] M.Z. Afgani, H. Haas, H. Elgala, and D. Knipp, “Visible light communication using OFDM,” Int. Conf. Testbeds Research Infrastructures Development Networks Communities (TRIDENTCOM), pp.129–134, March 2006.
CrossRef
[8] G. Stepniak, M. Schüppert, and C.A. Bunge, “Advanced modulation formats in phosphorous LED VLC links and the impact of blue filtering,” J. Lightw. Technol., vol.33, no.21, pp.4413–4423, Nov. 2015.
CrossRef
[9] C. Chen, X. Zhong, S. Fu, X. Jian, M. Liu, H. Yang, A. Alphones, and H.Y. Fu, “OFDM-based generalized optical MIMO,” J. Lightw. Technol., vol.39, no.19, pp.6063–6075, Oct. 2021.
CrossRef
[10] L. Zeng, D.C. O’Brien, H. Le Minh, G.E. Faulkner, K. Lee, D. Jung, Y. Oh, and E.T. Won, “High data rate multiple input multiple output (MIMO) optical wireless communications using white LED lighting,” IEEE J. Sel. Areas Commun., vol.27, no.9, pp.1654–1662, Dec. 2009.
CrossRef
[11] T. Fath and H. Haas, “Performance comparison of MIMO techniques for optical wireless communications in indoor environments,” IEEE Trans. Commun., vol.61, no.2, pp.733–742, Feb. 2013.
CrossRef
[12] C. Chen, W.D. Zhong, and D. Wu, “On the coverage of multiple-input multiple-output visible light communications [Invited],” J. Opt. Commun. Netw., vol.9, no.9, pp.D31–D41, Sept. 2017.
CrossRef
[13] H. Marshoud, V.M. Kapinas, G.K. Karagiannidis, and S. Muhaidat, “Non-orthogonal multiple access for visible light communications,” IEEE Photon. Technol. Lett., vol.28, no.1, pp.51–54, Jan. 2016.
CrossRef
[14] C. Chen, Y. Yang, X. Deng, P. Du, and H. Yang, “Space division multiple access with distributed user grouping for multi-user MIMO-VLC systems,” IEEE Open J. Commun. Soc., vol.1, pp.943–956, July 2020.
CrossRef
[15] C. Chen, S. Fu, X. Jian, M. Liu, X. Deng, and Z. Ding, “NOMA for energy-efficient LiFi-enabled bidirectional IoT communication,” IEEE Trans. Commun., vol.69, no.3, pp.1693–1706, March 2021.
CrossRef
[16] C. Chen, D.A. Basnayaka, and H. Haas, “Downlink performance of optical attocell networks,” J. Lightw. Technol., vol.34, no.1, pp.137–156, Jan. 2016.
CrossRef
[17] H. Haas, L. Yin, C. Chen, S. Videv, D. Parol, E. Poves, H. Alshaer, and M.S. Islim, “Introduction to indoor networking concepts and challenges in LiFi,” J. Opt. Commun. Netw., vol.12, no.2, pp.A190–A203, 2020.
CrossRef
[18] H. Steendam, T.Q. Wang, and J. Armstrong, “Theoretical lower bound for indoor visible light positioning using received signal strength measurements and an aperture-based receiver,” J. Lightwave Technol., vol.35, no.2, pp.309–319, 2016.
CrossRef
[19] S. Zhang, P. Du, C. Chen, and W.D. Zhong, “3D indoor visible light positioning system using RSS ratio with neural network,” Opto-Electronics and Communications Conference (OECC), pp.1–2, IEEE, 2018.
CrossRef
[20] P. Du, S. Zhang, C. Chen, H. Yang, W.D. Zhong, R. Zhang, A. Alphones, and Y. Yang, “Experimental demonstration of 3D visible light positioning using received signal strength with low-complexity trilateration assisted by deep learning technique,” IEEE Access, vol.7, pp.93986–93997, July 2019.
CrossRef
[21] T.H. Do and M. Yoo, “TDOA-based indoor positioning using visible light,” Photon. Netw. Commun., vol.27, no.2, pp.80–88, 2014.
CrossRef
[22] P. Du, S. Zhang, C. Chen, A. Alphones, and W.D. Zhong, “Demonstration of a low-complexity indoor visible light positioning system using an enhanced TDOA scheme,” IEEE Photon. J., vol.10, no.4, pp.1–10, Aug. 2018.
CrossRef
[23] S. Zhang, Z. Wen-De, P. Du, C. Chen, and W. Dehao, “PDOA based indoor visible light positioning system without local oscillators in receiver,” Conference on Lasers and Electro-Optics Pacific Rim (CLEO-PR), pp.1–3, IEEE, 2017.
CrossRef
[24] S. Zhang, W.D. Zhong, P. Du, and C. Chen, “Experimental demonstration of indoor sub-decimeter accuracy VLP system using differential PDOA,” IEEE Photon. Technol. Lett., vol.30, no.19, pp.1703–1706, Oct. 2018.
CrossRef
[25] P. Du, S. Zhang, W.D. Zhong, C. Chen, H. Yang, A. Alphones, and R. Zhang, “Real-time indoor positioning system for a smart workshop using white LEDs and a phase-difference-of-arrival approach,” Opt. Eng., vol.58, no.8, p.084112, 2019.
CrossRef
[26] C.K. Liang, L.W. Chang, and H.H. Chen, “Analysis and compensation of rolling shutter effect,” IEEE Trans. Image Process., vol.17, no.8, pp.1323–1330, 2008.
CrossRef
[27] Y.S. Kuo, P. Pannuto, K.J. Hsiao, and P. Dutta, “Luxapose: Indoor positioning with mobile phones and visible light,” Annual International conference on Mobile computing and networking, pp.447–458, 2014.
CrossRef
[28] Y.C. Wu, C.W. Chow, Y. Liu, Y.S. Lin, C.Y. Hong, D.C. Lin, S.H. Song, and C.H. Yeh, “Received-signal-strength (RSS) based 3D visible-light-positioning (VLP) system using kernel ridge regression machine learning algorithm with sigmoid function data preprocessing method,” IEEE Access, vol.8, pp.214269–214281, 2020.
CrossRef
[29] P. Du, S. Zhang, A. Alphones, and C. Chen, “Faster deployment for indoor visible light positioning using Xgboost algorithms in industrial Internet-of-Things,” IECON–47th Annual Conference of the IEEE Industrial Electronics Society, pp.1–7, IEEE, 2021.
CrossRef
[30] L.S. Hsu, D.C. Tsai, H.M. Chen, Y.H. Chang, Y. Liu, C.W. Chow, S.H. Song, and C.H. Yeh, “Using received-signal-strength (RSS) pre-processing and convolutional neural network (CNN) to enhance position accuracy in visible light positioning (VLP),” Optical Fiber Communication Conference, p.W3I.6, Optica Publishing Group, 2022.
CrossRef
[31] J. Chen and X. You, “Visible light positioning and communication cooperative systems,” International Conference on Optical Communications and Networks (ICOCN), pp.1–3, IEEE, 2017.
CrossRef
[32] H. Yang, C. Chen, W.D. Zhong, S. Zhang, and P. Du, “An integrated indoor visible light communication and positioning system based on FBMC-SCM,” IEEE Photon. Conf. (IPC), pp.129–130, IEEE, Oct. 2017.
CrossRef
[33] H. Yang, C. Chen, W.D. Zhong, A. Alphones, S. Zhang, and P. Du, “Demonstration of a quasi-gapless integrated visible light communication and positioning system,” IEEE Photon. Technol. Lett., vol.30, no.23, pp.2001–2004, Dec. 2018.
CrossRef
[34] H. Yang, P. Du, W.D. Zhong, C. Chen, A. Alphones, and S. Zhang, “Reinforcement learning-based intelligent resource allocation for integrated VLCP systems,” IEEE Wireless Commun. Lett., vol.8, no.4, pp.1204–1207, Aug. 2019.
CrossRef
[35] H. Yang, W.D. Zhong, C. Chen, A. Alphones, and P. Du, “QoS-driven optimized design-based integrated visible light communication and positioning for indoor iot networks,” IEEE Internet Things J., vol.7, no.1, pp.269–283, 2020.
CrossRef
[36] H. Yang, W.D. Zhong, C. Chen, A. Alphones, P. Du, S. Zhang, and X. Xie, “Coordinated resource allocation-based integrated visible light communication and positioning systems for indoor IoT,” IEEE Trans. Wireless Commun., vol.19, no.7, pp.4671–4684, 2020.
CrossRef
[37] H. Yang, W.D. Zhong, C. Chen, and A. Alphones, “Integration of visible light communication and positioning within 5G networks for internet of things,” IEEE Network, vol.34, no.5, pp.134–140, 2020.
CrossRef
[38] M. Ayyash, H. Elgala, A. Khreishah, V. Jungnickel, T. Little, S. Shao, M. Rahaim, D. Schulz, J. Hilt, and R. Freund, “Coexistence of WiFi and LiFi toward 5G: Concepts, opportunities, and challenges,” IEEE Commun. Mag., vol.54, no.2, pp.64–71, 2016.
CrossRef
[39] M. Kashef, M. Ismail, M. Abdallah, K.A. Qaraqe, and E. Serpedin, “Energy efficient resource allocation for mixed RF/VLC heterogeneous wireless networks,” IEEE J. Sel. Areas Commun., vol.34, no.4, pp.883–893, 2016.
CrossRef
[40] H. Tabassum and E. Hossain, “Coverage and rate analysis for co-existing RF/VLC downlink cellular networks,” IEEE Trans. Wireless Commun., vol.17, no.4, pp.2588–2601, 2018.
CrossRef
[41] H. Yang, A. Alphones, W.D. Zhong, C. Chen, and X. Xie, “Learning-based energy-efficient resource management by heterogeneous RF/VLC for ultra-reliable low-latency industrial IoT networks,” IEEE Trans. Ind. Informat., vol.16, no.8, pp.5565–5576, Aug. 2020.
CrossRef
[42] S. Shrivastava, B. Chen, C. Chen, H. Wang, and M. Dai, “Deep q-network learning based downlink resource allocation for hybrid RF/VLC systems,” IEEE Access, vol.8, pp.149412–149434, 2020.
CrossRef
[43] X. Wu, M.D. Soltani, L. Zhou, M. Safari, and H. Haas, “Hybrid LiFi and WiFi networks: A survey,” IEEE Commun. Surveys Tuts., vol.23, no.2, pp.1398–1420, 2021.
CrossRef
[44] H. Abuella, M. Elamassie, M. Uysal, Z. Xu, E. Serpedin, K.A. Qaraqe, and S. Ekin, “Hybrid RF/VLC systems: A comprehensive survey on network topologies, performance analyses, applications, and future directions,” IEEE Access, vol.9, pp.160402–160436, 2021.
CrossRef
[45] J. Xu, C. Gong, and Z. Xu, “Experimental indoor visible light positioning systems with centimeter accuracy based on a commercial smartphone camera,” IEEE Photon. J., vol.10, no.6, pp.1–17, 2018.
CrossRef
[46] R. Zhang, W.D. Zhong, K. Qian, and D. Wu, “Image sensor based visible light positioning system with improved positioning algorithm,” IEEE Access, vol.5, pp.6087–6094, April 2017.
CrossRef
[47] R. Zhang, W.D. Zhong, Q. Kemao, and S. Zhang, “A single LED positioning system based on circle projection,” IEEE Photon. J., vol.9, no.4, p.7905209, Aug. 2017.
CrossRef
[48] H. Zheng, Z. Xu, C. Yu, and M. Gurusamy, “A 3-D high accuracy positioning system based on visible light communication with novel positioning algorithm,” Opt. Commun., vol.396, pp.160–168, 2017 [doi: 10.1016/j.optcom.2017.03.058].
CrossRef
[49] F. Alam, B. Parr, and S. Mander, “Visible light positioning based on calibrated propagation model,” IEEE Sens. Lett., vol.3, no.2, pp.1–4, 2018.
CrossRef
[50] S.W. Ho, J. Duan, and C.S. Chen, “Location-based information transmission systems using visible light communications,” Trans. Emerg. Telecommun. T., vol.28, no.1, 2017 [doi: 10.1002/ett.2922].
CrossRef
[51] L. Buitinck, G. Louppe, M. Blondel, F. Pedregosa, A. Mueller, O. Grisel, V. Niculae, P. Prettenhofer, A. Gramfort, J. Grobler, R. Layton, J. VanderPlas, A. Joly, B. Holt, and G. Varoquaux, “API design for machine learning software: Experiences from the scikit-learn project,” ECML PKDD Workshop: Languages for Data Mining and Machine Learning, pp.108–122, 2013.
[52] S. Zhang, P. Du, C. Chen, W.D. Zhong, and A. Alphones, “Robust 3D indoor VLP system based on ANN using hybrid RSS/PDOA,” IEEE Access, vol.7, pp.47769–47780, April 2019.
CrossRef
[53] Student, “The probable error of a mean,” Biometrika, pp.1–25, 1908.
CrossRef
Authors
the A*STAR's Institute of Infocomm Research
received his Ph.D. degree from Nanyang Technological University (NTU), Singapore, in 2020. He was a postdoctoral researcher, in the field of point cloud processing, in the School of Electrical and Electronic Engineering, Nanyang Technological University. He is currently a research scientist, in the field of air-traffic management, in I2R, A*STAR, Singapore. His research areas include positioning systems, automated guided vehicles, point cloud processing, and trajectory-based air-traffic management.
 |
the A*STAR's Singapore Institute of Manufacturing Technology
received the B.Eng. degree in mechanical engineering from Sichuan University, Chengdu, China, in 2011, and the Ph.D. degree in optical engineering from Tsinghua University, Beijing, China, in 2016. From 2016-2019, He joined Nanyang Technological University as a research fellow. From 2019 to now, he is a scientist with the A*STAR’s Singapore Institute of Manufacturing Technology (SIMTech). His current research interests include visible light positioning, Li-Fi, Optical IoT, Vision-based Artificial-IoT, Lidar, and quantum detection.
 |
Xiamen University
is currently an Associate Professor in the Department of Information and Communication Engineering, Xiamen University, Xiamen, China. He received the B.S. and M.S. degrees in the School of Telecommunications Information Engineering from Chongqing University of Posts and Telecommunications in 2013, and 2016, and the Ph.D. degree from School of Electrical and Electronic Engineering, Nanyang Technological University, Singapore, in 2020. His current research interests include wireless communication, visible light communication, Internet of Things and resource management.
 |
Nanyang Technological University
received the Ph.D. degree (2015-2019) from Nanyang Technological University (NTU), Singapore and worked as research fellow in NTU (2019–2020). During the time period in NTU, her research topic includes signal processing, visible light communication and positioning, IoT, deep learning for indoor localization. From 2020 to now, she is Algorithm Engineer with Alibaba Group in Singapore.
 |
Chongqing University
received the B.S. and M.Eng. degrees from the University of Electronic Science and Technology of China, Chengdu, China, in 2010 and 2013, respectively, and the Ph.D. degree from Nanyang Technological University, Singapore, in 2017. He was a Post-Doctoral Researcher with the School of Electrical and Electronic Engineering, Nanyang Technological University, from 2017 to 2019. He is currently a Research Professor with the School of Microelectronics and Communication Engineering, Chongqing University, Chongqing, China. His research interests include optical wireless communication, optical access networks, Internet of Things, and machine learning.
 |
Nanyang Technological University
received his B.Tech. from Madras Institute of Technology in 1982, M.Tech. from Indian Institute of Technology Kharagpur in 1984 and Ph.D. degree in Optically Controlled Millimeter wave Circuits from Kyoto Institute of Technology (Japan) in 1992. He was a JSPS visiting fellow from 1996–97 at Japan. During 1997–2001, he was with Centre for Wireless Communications, National University of Singapore involved in the research on optically controlled passive/active devices. Since 2001 he is with the School of Electrical and Electronic Engineering, Nanyang Technological University, Singapore. He has 30 years of research experience. He has published and presented over 300 technical papers in peer reviewed International Journals/Conferences. His current interests are electro-magnetic analysis on planar RF circuits and integrated optics, microwave photonics, visible light communication and positioning, metamaterial based leaky wave antennas and wireless power transfer technologies. He was involved many IEEE flagship conferences held in Singapore and General Chair of APMC 2009, MWP 2011, TENCON 2016 and APMC 2019. He was the chairman of IEEE Singapore section during 2015–2016, 2018 and a senior member of IEEE. He is Singapore representative of IEICE (Japan). He is also the panel member of IEEE Conference Quality Committee.
 |
