Sensor Fusion and Registration of Lidar and Stereo Camera without Calibration Objects

Vijay JOHN; Qian LONG; Yuquan XU; Zheng LIU; Seiichi MITA

doi:10.1587/transfun.E100.A.499

Sensor Fusion and Registration of Lidar and Stereo Camera without Calibration Objects

Vijay JOHN, Qian LONG, Yuquan XU, Zheng LIU, Seiichi MITA

Full Text Views

0

Cite this

Summary :

Environment perception is an important task for intelligent vehicles applications. Typically, multiple sensors with different characteristics are employed to perceive the environment. To robustly perceive the environment, the information from the different sensors are often integrated or fused. In this article, we propose to perform the sensor fusion and registration of the LIDAR and stereo camera using the particle swarm optimization algorithm, without the aid of any external calibration objects. The proposed algorithm automatically calibrates the sensors and registers the LIDAR range image with the stereo depth image. The registered LIDAR range image functions as the disparity map for the stereo disparity estimation and results in an effective sensor fusion mechanism. Additionally, we perform the image denoising using the modified non-local means filter on the input image during the stereo disparity estimation to improve the robustness, especially at night time. To evaluate our proposed algorithm, the calibration and registration algorithm is compared with baseline algorithms on multiple datasets acquired with varying illuminations. Compared to the baseline algorithms, we show that our proposed algorithm demonstrates better accuracy. We also demonstrate that integrating the LIDAR range image within the stereo's disparity estimation results in an improved disparity map with significant reduction in the computational complexity.

Publication: IEICE TRANSACTIONS on Fundamentals Vol.E100-A No.2 pp.499-509

Publication Date: 2017/02/01

Publicized

Online ISSN: 1745-1337

DOI: 10.1587/transfun.E100.A.499

Type of Manuscript: Special Section PAPER (Special Section on Intelligent Transport Systems)

Category

Authors

Vijay JOHN
  Toyota Technological Institute
Qian LONG
  Nippon Soken, Inc.
Yuquan XU
  Toyota Technological Institute
Zheng LIU
  University of British Columbia
Seiichi MITA
  Toyota Technological Institute

Keyword

sensor fusion, stereo camera, LIDAR

Cite this

Copy

Vijay JOHN, Qian LONG, Yuquan XU, Zheng LIU, Seiichi MITA, "Sensor Fusion and Registration of Lidar and Stereo Camera without Calibration Objects" in IEICE TRANSACTIONS on Fundamentals, vol. E100-A, no. 2, pp. 499-509, February 2017, doi: 10.1587/transfun.E100.A.499.
Abstract: Environment perception is an important task for intelligent vehicles applications. Typically, multiple sensors with different characteristics are employed to perceive the environment. To robustly perceive the environment, the information from the different sensors are often integrated or fused. In this article, we propose to perform the sensor fusion and registration of the LIDAR and stereo camera using the particle swarm optimization algorithm, without the aid of any external calibration objects. The proposed algorithm automatically calibrates the sensors and registers the LIDAR range image with the stereo depth image. The registered LIDAR range image functions as the disparity map for the stereo disparity estimation and results in an effective sensor fusion mechanism. Additionally, we perform the image denoising using the modified non-local means filter on the input image during the stereo disparity estimation to improve the robustness, especially at night time. To evaluate our proposed algorithm, the calibration and registration algorithm is compared with baseline algorithms on multiple datasets acquired with varying illuminations. Compared to the baseline algorithms, we show that our proposed algorithm demonstrates better accuracy. We also demonstrate that integrating the LIDAR range image within the stereo's disparity estimation results in an improved disparity map with significant reduction in the computational complexity.
URL: https://global.ieice.org/en_transactions/fundamentals/10.1587/transfun.E100.A.499/_p

Copy

@ARTICLE{e100-a_2_499,
author={Vijay JOHN, Qian LONG, Yuquan XU, Zheng LIU, Seiichi MITA, },
journal={IEICE TRANSACTIONS on Fundamentals},
title={Sensor Fusion and Registration of Lidar and Stereo Camera without Calibration Objects},
year={2017},
volume={E100-A},
number={2},
pages={499-509},
abstract={Environment perception is an important task for intelligent vehicles applications. Typically, multiple sensors with different characteristics are employed to perceive the environment. To robustly perceive the environment, the information from the different sensors are often integrated or fused. In this article, we propose to perform the sensor fusion and registration of the LIDAR and stereo camera using the particle swarm optimization algorithm, without the aid of any external calibration objects. The proposed algorithm automatically calibrates the sensors and registers the LIDAR range image with the stereo depth image. The registered LIDAR range image functions as the disparity map for the stereo disparity estimation and results in an effective sensor fusion mechanism. Additionally, we perform the image denoising using the modified non-local means filter on the input image during the stereo disparity estimation to improve the robustness, especially at night time. To evaluate our proposed algorithm, the calibration and registration algorithm is compared with baseline algorithms on multiple datasets acquired with varying illuminations. Compared to the baseline algorithms, we show that our proposed algorithm demonstrates better accuracy. We also demonstrate that integrating the LIDAR range image within the stereo's disparity estimation results in an improved disparity map with significant reduction in the computational complexity.},
keywords={},
doi={10.1587/transfun.E100.A.499},
ISSN={1745-1337},
month={February},}

Copy

TY - JOUR
TI - Sensor Fusion and Registration of Lidar and Stereo Camera without Calibration Objects
T2 - IEICE TRANSACTIONS on Fundamentals
SP - 499
EP - 509
AU - Vijay JOHN
AU - Qian LONG
AU - Yuquan XU
AU - Zheng LIU
AU - Seiichi MITA
PY - 2017
DO - 10.1587/transfun.E100.A.499
JO - IEICE TRANSACTIONS on Fundamentals
SN - 1745-1337
VL - E100-A
IS - 2
JA - IEICE TRANSACTIONS on Fundamentals
Y1 - February 2017
AB - Environment perception is an important task for intelligent vehicles applications. Typically, multiple sensors with different characteristics are employed to perceive the environment. To robustly perceive the environment, the information from the different sensors are often integrated or fused. In this article, we propose to perform the sensor fusion and registration of the LIDAR and stereo camera using the particle swarm optimization algorithm, without the aid of any external calibration objects. The proposed algorithm automatically calibrates the sensors and registers the LIDAR range image with the stereo depth image. The registered LIDAR range image functions as the disparity map for the stereo disparity estimation and results in an effective sensor fusion mechanism. Additionally, we perform the image denoising using the modified non-local means filter on the input image during the stereo disparity estimation to improve the robustness, especially at night time. To evaluate our proposed algorithm, the calibration and registration algorithm is compared with baseline algorithms on multiple datasets acquired with varying illuminations. Compared to the baseline algorithms, we show that our proposed algorithm demonstrates better accuracy. We also demonstrate that integrating the LIDAR range image within the stereo's disparity estimation results in an improved disparity map with significant reduction in the computational complexity.
ER -