Automated BIM Generation Based on UAV and Indoor 3D Laser Scanning Technologies (基於無人機及室內激光掃描之自動化建築信息建模)

Prof. Jack C. P. CHENG, Prof. Ming LIU, Prof. Fu ZHANG,

Chao YIN, Boyu WANG, Changhao SONG


Proposed a UAV platform to allow point cloud data collection from complex MEP scenes.

Developed algorithms for automatic BIM geneartion from complex 3D LiDAR point clouds.

Overview



Autonomous LiDAR-carrying UAV

2D sectioning method for BIM generation

3D deep learning based method (ResPointNet++)


Motivation of Research


 

Unmanned aerial vehicles (UAVs) have attracted growing attentions from the AEC/FM industry for their great potential presented in as-built data collection and 3D reconstruction. Conventionally, laser scanning is conducted by human workers using terrestrial laser scanners. This process can be time-consuming, error-prone, and potentially dangerous in some construction sites.

A multi-rotor UAV equipped with a Light Detection and Ranging (LiDAR) scanner is a versatile and efficient robot platform to collect point cloud data in various construction sites. It can cover multiple checkpoints in a single flight and outperform ground vehicles in outdoor environments, and indoor environments with high headroom and/or poor ground mobility. UAVs have been recently applied for the inspection of construction sites, building facades and civil infrastructures like bridges.


Outcome 1: Autonomous LiDAR-carrying UAV

Read the published paper



This study proposes a model-based scan planning and kinodynamic motion planning framework ‒ consisting of map construction, scan planning, path finding, trajectory generation, and flight control ‒ to facilitate fully autonomous LiDAR-carrying UAVs in AEC/FM applications. The map construction phase extracts the essential geometry and semantics from BIM to construct a probabilistic voxel map. The scan planning phase adopts a greedy algorithm to iteratively generate waypoints that maximize the expected coverage. Afterward, a path finding technique is applied to determine a safe and optimal path that traverses all the waypoints. The path is further transformed into a smooth, physically feasible trajectory comprising piecewise polynomials. The proposed flight is validated in a water treatment facility with mechanical, electrical, and plumbing (MEP) components.

The major contributions of this study are summarized as follows: (1) A BIM-supported information integration technique to construct probabilistic voxel maps for navigation and coverage planning. (2) A waypoint-based scan coverage planning technique for highly irregular MEP components. (3) A progressive motion planning technique to generate smooth and energy-efficient UAV trajectories.

Watch the project video on Youtube



 

Outcome 2: 2D sectioning method for BIM generation

Read the published paper



To adopt BIM to MEP system O&M, a BIM model with accurate and updated facility information is required. To solve this problem, 3D point cloud data with millimeter accuracy collected with terrestrial laser scanners, which are also known as Light Detection and Ranging (LiDAR), have received much more attention in recent years. These data points are often defined with X, Y, and Z coordinates and can capture the external surfaces of the objects. Sometimes, point colors and reflection intensities of the surfaces can also be collected to enrich the point cloud information. But still, how to efficiently convert the point cloud data of MEP scenes to semantically rich BIMs remains incompletely solved. There have been some research efforts to automatically detect and model MEP scene from point cloud data. However, most previous studies only focus on the modeling of pipelines and rely heavily on accurate local features such as normal vectors and curvatures, which may not be practical in environments with heavy noise and occlusions. Besides, none of these methods have achieved the modeling of a complete MEP system including MEP components with regular and irregular shapes as well as their connections. As all these components are indispensable for the MEP system, there is a strong need to automatically build the complete as-built utility system model.

To address the research gap, this study proposes a fully automatic framework to generate complete parametric BIMs for complex MEP scenarios from laser scanning data. The proposed method is able to detect and model both regular shaped components (e.g. pipes and ducts) with constant cross sections and irregular shaped components (e.g. valves and pumps) whose shapes cannot be represented by a few parameters. The proposed technique consists of three modules. In the first module, the raw point cloud data are pre-processed to remove irrelevant points and transform to the desired coordinate system. In the second module, the geometry information of MEP components is extracted from the point cloud data. The geometry information is extracted by two steps including preliminary geometry extraction based on 2D slices of point cloud data, and refined geometry extraction based on 3D point cloud data. Lastly, the third module constructs the MEP network by determining the connections between MEP components, and eventually creates the as-built parametric BIMs of the MEP scene.



 

Outcome 3: 3D Deep learning based method (ResPointNet++)

Read the published paper



The objective of this study is to develop a new automatic DL-based semantic segmentation method, which is able to segment diverse industrial components from large-scale industrial point clouds accurately and robustly. In this study, a DL-based semantic segmentation network (called ResPointNet++) is invented through the integration of deep residual learning with a point-based network for point cloud semantic segmentation. Due to the lack of industrial LiDAR dataset, an annotated industrial LiDAR dataset is established to train neural networks for interpreting industrial scenes. Inspired by conventional point-based networks, a generic local aggregation operator (LAO) module is developed to learn complex local geometric patterns for point clouds taking into consideration more types of input features and aggregation functions. Considering LAO is computationally demanding and not suitable to be used to construct deeper neural networks, this paper presents residual block modules to enable the construction of deeper point cloud networks to overcome the optimization problems (e.g., gradient vanishing issue). Based on the LAO and residual block modules, the proposed new ResPointNet++ can be constructed to underpin the semantic segmentation with a common U-Net style encoder-decoder segmentation architecture.

Our primary contributions are as follows: (1) Establish a publicly available and expert-labeled industrial indoor LiDAR dataset for 3D semantic learning (our open dataset will be released at authors’ GitHub page at https://github.com/PointC loudYC/ResPointNet2). (2) Test representative state-of-the-art DL algorithms (as baselines) on our benchmark dataset. (3) Propose two effective neural modules —local aggregation operator and residual bottleneck modules — to learn complex local structures from neighborhood regions and enable building up deeper point cloud networks with residual settings. The new development also facilitates the application of CNNs in 2D image processing to 3D point could analysis. (4) Construct a new neural network named ResPointNet++ based on the two neural modules and PointNet++ and demonstrate that our model significantly outperforms baselines on our benchmark dataset. (5) Perform ablation studies on design choices of the local aggregation operator module including input feature type and aggregation function type.



 

List of Related Publications


Dr. Jack CHENG's Google Scholar Page

Publications from this project:

1. Yin, C., Yang, B., Cheng, J. C., Gan, V. J., Wang, B., & Yang, J. (2023). Label-efficient semantic segmentation of large-scale industrial point clouds using weakly supervised learning. Automation in Construction, 148, 104757.

2. Song, C., Chen, Z., Wang, K., Luo, H., & Cheng, J. C. (2022). BIM-supported scan and flight planning for fully autonomous LiDAR-carrying UAVs. Automation in Construction, 142, 104533.

3. Wang, B., Wang, Q., Cheng, J. C., & Yin, C. (2022). Object verification based on deep learning point feature comparison for scan-to-BIM. Automation in Construction, 142, 104515.

4. Wang, B., Wang, Q., Cheng, J. C., Song, C., & Yin, C. (2022). Vision-assisted BIM reconstruction from 3D LiDAR point clouds for MEP scenes. Automation in Construction, 133, 103997.

5. Yin, C., Cheng, J. C., Wang, B., & Gan, V. J. (2022). Automated classification of piping components from 3D LiDAR point clouds using SE-PseudoGrid. Automation in Construction, 139, 104300.

6. Yin, C., Wang, B., Gan, V. J., Wang, M., & Cheng, J. C. (2021). Automated semantic segmentation of industrial point clouds using ResPointNet++. Automation in Construction, 130, 103874.

7. Wang, B., Yin, C., Luo, H., Cheng, J. C., & Wang, Q. (2021). Fully automated generation of parametric BIM for MEP scenes based on terrestrial laser scanning data. Automation in Construction, 125, 103615.

8. Song, C., Wang, K., & Cheng, J. C. (2020). BIM-aided scanning path planning for autonomous surveillance uavs with lidar. In ISARC. Proceedings of the International Symposium on Automation and Robotics in Construction (Vol. 37, pp. 1195-1202). IAARC Publications.





Meet Our Team



Jack C. P. CHENG (鄭展鵬) Ph.D., Stanford University

Associate Head and Professor

Department of Civil and Environmental Engineering

The Hong Kong University of Science and Technology

Email: cejcheng@ust.hk

 

RESEARCH INTERESTS:

• Construction robotics

• Building information modeling (BIM)

• Blockchain

• Internet of Things (IoT)

• Digital twin

 

 

Chao YIN (尹超), Ph.D., HKUST

Email: cyinac@connect.ust.hk

 

RESEARCH INTERESTS:

• 3D deep learning

• Point cloud processng

• GIS

 

 

Boyu WANG (王博宇), Ph.D., HKUST

Email: bwangbb@connect.ust.hk

 

RESEARCH INTERESTS:

• Scan to BIM

• Point cloud processing

• Computer vision

 

 

Changhao SONG (宋昌昊), Ph.D. candidate, HKUST

Email: csongae@connect.ust.hk

 

RESEARCH INTERESTS:

• Construction robotics

• UAV

• SLAM

 

 



Codes and Datasets

Click here to visit ResPointNet++


All rights reserved. Copyright @ HKUST, 2023