Research

To construct three-dimensional shape models of large-scale architectures using a laser range finder, multiple range images must be captured from various viewpoints and aligned using post-processing procedures such as the ICP algorithm. Typically, these range images must be manually registered to positions close to the correct ones before applying the ICP algorithm to ensure precise alignment. Additionally, range images must sufficiently overlap by capturing them from dense positions. We propose a new measurement system for large-scale architectures using a group of multiple robots and an onboard laser range finder. Each measurement position is identified using the highly precise Cooperative Positioning System (CPS), which leverages the characteristics of multiple robot systems. This proposed system constructs 3D shapes of large-scale architectures without requiring post-processing procedures like the ICP algorithm or dense range measurements.

A group of robots equipped with laser range finder, CPS-VIII	3D models by automatic modeling
A group of robots equipped with laser range finder, CPS-V	3D map of large scale architecture
Introduction (19MB)	Introduction (14MB)
Dazaifu Tenmangu shrine	Dazaifu Tenmangu shrine
Seibuen Park	Kyushu University

Papers

• Ryo Kurazume, Souichiro Oshima, Shingo Nagakura, Yongjin Jeong, Yumi Iwashita, Automatic large-scale three dimensional modeling using cooperative multiple robots, Computer Vision and Image Understanding, Vol.157, pp. 25-42, 2017, doi:10.1016/j.cviu.2016.05.008
• Souichiro Oshima, Shingo Nagakura, Yongjin Jeong, Yumi Iwashita, Ryo Kurazume, Automatic planning of laser measurements for a large-scale environment using CPS-SLAM system, Proc. of IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS2015), Hamburg, Germany, Sep. 28-Oct. 2, 2015
• Shuji Oishi, Yongjin Jeong, Ryo Kurazume, Yumi Iwashita and Tsutomu Hasegawa, ND voxel localization using large-scale 3D environmental map and RGB-D camera, 2013 IEEE International Conference on Robotics and Biomimetics (ROBIO), pp.538-545, Shenzhen, Dec. 12-14, 2013(Best Paper Award Finalist)
• Yongjin Jeong, Ryo Kurazume, Yoonseok Pyo, Yumi Iwashita, Tsutomu Hasegawa, High-Precision Three-Dimensional Laser Measurement System by Cooperative Multiple Mobile Robots, 2012 IEEE/SICE International Symposium on System Integration (SII2012), pp.198-205, Fukuoka, 2012.12.16-18, 2012
• Yukihiro Tobata, Ryo Kurazume, Yusuke Noda, Kai Lingemann, YumiIwashita, Tsutomu Hasegawa, Laser-based geometrical modeling of large-scale architectural structures using co-operative multiple robots, Autonomous Robot, Vol.32, No.1, pp. 49-62, (2012)
• Yukihiro Tobata, Ryo Kurazume, Yumi Iwashita and Tsutomu Hasegawa, Automatic laser-based geometrical modeling using multiple mobile robots, 2010 IEEE International Conference on Robotics and Biomimetics (ROBIO 2010), pp.363-369, Tianjin, Dec. 14-18, 2010 (T.J.Tarn Best Paper in Robotics)
• Ryo Kurazume, Yusuke Noda, Yukihiro Tobata, Kai Lingemann, Yumi Iwashita, Tsutomu Hasegawa, Laser-based Geometric Modeling using Cooperative Multiple Mobile Robots, in Proc. IEEE International Conference on Robotics and Automation, pp.3200-3205, May 12-17, 2009.
• Ryo Kurazume, Yukihiro Tobata, Yumi Iwashita, Tsutomu Hasegawa, 3D laser measurement system for large scale architectures using multiple mobile robots, The 6th International Conference on 3-D Digital Imaging and Modeling (3DIM2007), August 2007.
• Yusuke Noda, Ryo Kurazume, Yumi Iwashita, and Tsutomu Hasegawa, Laser-based Modeling of Cultural Heritage Using Cooperative Multiple Robots, Proc. 5th Joint Workshop on Machine Perception and Robotics (MPR2009), MPR2009-ps2-4, (2009 10)
• Yukihiro Tobata, Ryo Kurazume, and Tsutomu Hasegawa, Study on CPS SLAM, Proc. The Third Joint Workshop on Machine Perception and Robotics, CD-ROM, (2007.11).(Best Poster Session Award)

The three-dimensional shape measurement platform for urban structures, such as tunnels under construction, has been developed. This system comprises a mobile platform equipped with a two-dimensional laser scanner, a three-axis fiber optic gyro, corner cubes, integrated software, and a total station. The total station is positioned on reference points in a tunnel to measure the location of the corner cubes on the mobile platform. Cross-sectional shapes of the tunnel are continuously measured by the two-dimensional laser scanner on the mobile platform and converted into a three-dimensional tunnel shape using the data from the total station and fiber optic gyro. The integrated software can compare the measured shape with the design shape from a three-dimensional CAD system via the standard LandXML format, displaying the error distribution for construction process management through a GUI in real-time and on-site.

Laser measurement system	Multiple robot system
Real-time tunnel shape measurement	Measured 3D tunnel shape

We developed two denoising techniques for noisy range images using reflectivity: range image smoothing with a trilateral filter and range image inpainting through belief propagation. The trilateral filter utilizes reflectivity, spatial, and intensity information, preserving geometric features such as jump and roof edges while smoothing. The range image inpainting technique based on belief propagation restores deteriorated range images by using not only adjacent range values but also the continuity of the reflectance image. Simulations and experiments using synthesized images and actual range images taken by a laser scanner verify that the proposed techniques effectively suppress noise while preserving jump and roof edges, and repair deteriorated range images.

Raw range image	Bilateral filter	Proposed filter
Shape recognition by slit laser using reflectivity	Reflectance image	Recognition of reflectance image

Papers

• Kazuto Nakashima, Seungwoo Nham, Hojung Jung, Yumi Iwashita, Ryo Kurazume, Oscar Mozos, Recognizing Outdoor Scenes by Convolutional Features of Omni-directional LiDAR Scans, 2017 IEEE/SICE International Symposium on System Integration (SII 2017), Taiwan, 2017.12.11-14
• Shuji Oishi, Ryo Kurazume, Yumi Iwashita, Tsutomu Hasegawa, Range Image Smoothing and Completion utilizing Laser Intensity, Advanced Robotics, Volume 27, Issue 12, pp.947-958, August 2013
• Shuji Oishi, Naoaki Kondo, Yumi Iwashita, Ryo Kurazume, Object recognition by a laser scanner using multimodal information, 11th International Conference on Quality Control by Artificial Vision (QCAV), pp.-, Fukuoka, May 30 - June 1, 2013
• Shuji Oishi, Ryo Kurazume, Yumi Iwashita, and Tsutomu Hasegawa, Colorization of 3D Geometric Model utilizing Laser Reflectivity, in Proc. IEEE International Conference on Robotics and Automation, pp.2311-2318, Karlsruhe, May 6-10, 2013 (IEEE Robotics and Automation Society Japan Chapter Young Award)<
• Shuji Oishi, Ryo Kurazume, Yumi Iwashita, and Tsutomu Hasegawa, Denoising of Range Images using a Trilateral Filter and Belief Propagation, 2011 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS2011), pp.2020--2027, San Francisco, September 25-30, 2011.
• Naoaki Kondo, Shuji Oishi, Yumi Iwashita, Ryo Kurazume, Classification of Urban Area Using Laser Scanner, Proc. The Ninth Joint Workshop on Machine Perception and Robotics (MPR13), K-P-04 , Kyoto, 2012.10.31-11.1
• Shuji Oishi, Ryo Kurazume, Yumi Iwashita, Texture Restoration Using Laser Attributes, Proc. The Ninth Joint Workshop on Machine Perception and Robotics (MPR13), K-P-03, Kyoto, 2012.10.31-11.1
• Shuji Oishi, Yumi Iwashita, Ryo Kurazume, Color Mapping on 3D Model Using Laser Reflectivity, Proc. The Eighth Joint Workshop on Machine Perception and Robotics (MPR12), PS2-4, Fukuoka, 2012.10.16
• Shuji Oishi, Ryo Kurazume, Yumi Iwashita, Tsutomu Hasegawa, Surface Smoothing using a Trilateral filter, Proc. The Seventh Joint Workshop on Machine Perception and Robotics (MPR11), OS6-2,Beijing, 2011.10.14
• Shuji Oishi, Ryo Kurazume, Yumi Iwashita, Smoothing Range Image using Reflectivity, Proc. The Sixth Joint Workshop on Machine Perception and Robotics, PS1-10, 2010.10
• Shuji Oishi, Ryo Kurazume, Yumi Iwashita, and Tsutomu Hasegawa, Development of Denoising Techniques Using Reflectivity, ENSEEIHT- Kyushu University Workshop on Data Mining, and Media Processing (EKDM 2011), Toulouse, France, Nov. 24,25, 2011

We developed a 6D global localization technique using range and reflectance data obtained from a laser scanner and the bag-of-features (BoF) technique. Global localization is crucial for recognizing the accurate global position of a mobile robot in a revisited environment. The map-based global localization provides precise positioning by calculating an accurate transformation, but comparing large 3D data sets is time-consuming. Appearance-based global localization, which determines the global position through image retrieval techniques with similar structures, operates in real-time but requires external illumination and does not function effectively in darkness. We propose combining map-based and appearance-based global localization techniques. Instead of using camera images for appearance-based global localization, we utilize reflectance images captured as a byproduct of range sensing by a laser range finder.

Proposed technique

Pairs in reflectance images using BoF

Papers

• Dong Xiang ZHANG, Ryo Kurazume, Yumi Iwashita, Tsutomu Hasegawa, Robust global localization using laser reflectivity, Journal of Robotics and Mechatronics, vol.25, no.1, 2013
• Dong Xiang Zhang, Ryo Kurazume, Yumi Iwashita, Tsutomu Hasegawa, Robot localization under perceptual aliasing conditions based on laser reflectivity using particle filter, 2011 IEEE/SICE International Symposium on System Integration (SII2011), pp.--, Kyoto, 2011.12.20-22, 2011
• DongXiang Zhang, Ryo Kurazume, Yumi Iwashita, Tsutomu Hasegawa, Appearance and map-based global localization using laser reflectivity, 2011 IEEE International Conference on Robotics and Biomimetics (ROBIO), pp.--, Phuket, 2011.12.7-11, 2011
• Dong Xiang Zhang, Ryo Kurazume, Yumi Iwashita, An effective method for robot positioning based on panoramic reflectance images, Proc. The Sixth Joint Workshop on Machine Perception and Robotics, PS1-9, 2010.10

Tokyo University, Ikeuchi Lab.

Papers

• Katsushi Ikeuchi, Takeshi Oishi, Jun Takamatsu, Ryusuke Sagawa, Atsushi Nakazawa, Ryo Kurazume, No Nishino, Mawo Kamakura and Yasuhide Okamoto, The Great Buddha Project: Digitally Archiving, Restoring, and Analyzing Cultural Heritage Objects, International Journal of Computer Vision, Vol.75, No.1, pp.189-208, (2007.2)
• Ryo Kurazume, Ko Nishino, Mark D. Wheeler, Katsushi Ikeuchi, Mapping textures on 3D geometric model using reflectance image, Systems and Computers in Japan, Vol. 36, Issue 13, pp.92-101, (2005.11)
• Takeshi Oishi, Atsushi Nakazawa, Ryo Kurazume, and Katsushi Ikeuchi, Fast Simultaneous Alignment of Multiple Range Images Using Index Images, in Proc. The 5th International Conference on 3-D Digital Imaging and Modeling (3DIM), pp.476-483,2005.
• Yuichiro Hirota, Tomohito Masuda, Ryo Kurazume, Kohichi Ogawara, Kazuhide Hasegawa, Katsushi Ikeuchi, Flying Laser Range Finder and its data registration, Proc. IEEE Int. Conf. on Robotics and Automation, pp. 3155-3160, 2004.
• Yuichiro Hirota, Tomohito Masuda, Ryo Kurazume, Koichi Ogawara, Kazuhide Hasegawa, Katsushi Ikeuchi, Designing a laser range finder which is suspended beneath a balloon, Proc. of Sixth Asian Conference on Computer Vision (ACCV), 2004.
• Takeshi Oishi, Ryusuke Sagawa, Atsushi Nakazawa, Ryo Kurazume, Katsushi Ikeuchi, Parallel Alignment of a Large Number of Range Images, Proc. The 4th International Conference on 3D Digital Imaging and Modeling (3DIM 2003) , pp.195-202, 2003
• Ryusuke Sagawa, Takeshi Oishi, Atsushi Nakazawa, Ryo Kurazume, Katsushi Ikeuchi, Iterative Refinement of Range Images with Anisotropic Error Distribution,Proceedings of the 2002 IEEE/RSJ Intl. Conference on Intelligent Robots and Systems, October 2002
• Ryo Kurazume, Ko Noshino, Zhengyou Zhang, and Katushi Ikeuchi, Simultaneous 2D images and 3D geometric model registration for texture mapping utilizing reflectance attribute, Proc. of Fifth Asian Conference on Computer Vision (ACCV), vol.1, pp.99-106, 2002.
• Katsushi Ikeuchi, Yutaka Takase, Ryo Kurazume, Takeshi Oishi, Ryusuke Sagawa, Ko Nishino, Modeling Cultural Heritage Through Observation, Int. Sympo. on Artificial Intelligence, Robotics and Human Centered Technology for Nuclear Applications, pp.26-32, 2002
• Ryo Kurazume, Mark D. Wheeler, and Katushi Ikeuchi, Mapping textures on 3D geometric model using reflectance image, Workshop for Data Fusion at IEEE Int. Conf. on Robotics and Automation, 2001.