Automated identification of structural crack using image-processing technique and optimized machine learning
- Faculty of Civil Engineering, Ho Chi Minh City University of Technology (HCMUT)
- Faculty of Civil Engineering, Ho Chi Minh City University of Technology (HCMUT), 268 Ly Thuong Kiet Street, District 10, Ho Chi Minh City, Vietnam
Abstract
This paper explores the application of advanced machine learning and image processing methodologies, specifically targeting the detection of structural damage, particularly cracks, within the construction sector. By utilizing Convolutional Neural Network (CNN) technology on the Kaggle online platform for tasks like data analysis and model development, the study aims to automatically identify errors and defects on work surfaces, ultimately seeking to elevate the quality control processes in construction projects. The CNN architecture, tailored for image processing, plays a crucial role in effectively learning features and patterns, thereby significantly enhancing the precision in identifying errors and defects. The research employs a dataset consisting of 1000 images collected from real projects for training, validation, and testing of the proposed model. Through the integration of image processing, optimized CNN, and the resources provided by the Kaggle platform, the study strives to enhance the efficiency, accuracy, and automation of error and defect detection and classification, making notable contributions to the progression of quality management processes in the construction industry.
Introduction
In the current era of Industry 4.0, it is evident that nearly all aspects of societal life are experiencing enhancements due to the prevalence of information technology. The construction industry, which demands precision down to the millimeter, is undergoing significant transformations and advancements owing to the integration of artificial intelligence 1. The utilization of science and technology, particularly machine learning and image processing, within the construction sector is gaining increasing attention and popularity. Initiating a construction project entails not only financial investment but also meticulous management to ensure that the quality of work meets both technical and aesthetic standards2, 3. Quality management throughout the construction process is a crucial concern, demanding utmost accuracy and attention to detail 4.
Structural crack detection in construction, utilizing advanced image processing and machine learning techniques, is gaining traction as a prominent research area within the industry. These cutting-edge technologies are employed to assess and enhance construction quality in this crucial domain. Given the intricate nature of construction, encompassing factors like durability, safety, design, materials, and construction processes, traditional quality control methods reliant on manual inspection prove to be both time-consuming and labor-intensive. To address these challenges, recent research has focused on automating the quality inspection process through the utilization of image processing and machine learning technologies. This approach holds the potential to streamline testing efforts and time requirements while simultaneously enhancing accuracy and reliability5. As this technology is readily available and operational, it serves to advance construction practices, offering benefits such as improved cost-effectiveness and accelerated construction speed to the industry6.
Structural crack detection entails enhancing product quality through ongoing improvement efforts, involving the identification of issues and the implementation of solutions. Machine learning, a subset of artificial intelligence, revolves around algorithms that enable computers to carry out tasks without explicit programming. In construction quality management, image processing leverages machine learning algorithms and models to analyze and assess images captured from construction sites. With advancements in image recognition, segmentation, and classification algorithms, this technology can identify and quantify crucial factors such as surface quality, size, shape, and other characteristics of building structures depicted in the images 7.
Furthermore, the synergistic integration of machine learning enhances the predictability and optimization of the construction quality management process. Machine learning models can be trained to identify potential quality issues and provide enhanced solutions. Moreover, the judicious application of machine learning facilitates optimal planning for testing and quality management activities. By analyzing data from past construction projects, advanced machine learning techniques can construct predictive models to offer intelligent recommendations and decisions regarding inspections, thus enhancing construction quality5, 8. In essence, the exploration of construction quality management grounded in optimal image processing and machine learning holds immense potential for refining quality management practices within the construction industry. Both quality management and machine learning are highly pertinent topics in today's landscape. By embracing innovative technologies, we can drive progress and sustainability in building construction. The research will primarily focus on utilizing image processing and machine learning to detect cracks and damages in quality management processes9, 10, 11, 12.
The objective of this study is to find new solutions that utilize optimal image processing and machine learning techniques to enhance the construction quality management process. The primary aim is to reduce the cost and time required for quality inspection while ensuring adherence to technical standards throughout the construction process. The key contributions of this study are as follows: (1) Research and develop optimal image analysis techniques and machine learning models to evaluate the quality of construction works. (2) Develop new solutions using optimized image processing and machine learning to detect and resolve errors that occur during construction. (3) Optimize the construction quality management process through the application of optimal machine learning and image analysis techniques. (4) Evaluate the effectiveness of the proposed new solutions and compare them with traditional methods in quality management of construction works via a real case study. (5) Provide a more advanced and effective solution for quality control of construction works, while helping to reduce cost and time for quality inspection.
Related works
The primary benefit of utilizing image-based analysis for crack detection lies in its ability to deliver precise results when compared to traditional manual methods, owing to the application of advanced image processing techniques7. Research endeavors have concentrated on detecting cracks in engineering structures using processing techniques reliant on camera images. Adhikari, et al. 13 introduced an integrated model utilizing digital image processing to augment routine bridge inspections' core tasks. This model incorporates crack length and change detection mechanisms supported by neural networks to forecast crack depth and enable 3D visualization of crack patterns. Alam, et al. 14 introduced a detection technique that combines digital image correlation and acoustic emission to identify concrete cracking mechanisms and assess the impact of structural size.
Nguyen, et al.15 presented an automated approach for precise edge detection of concrete cracks in authentic 2D images of concrete surfaces, which often include noise and unintended objects. They devised a novel crack enhancement filter based on phase symmetry to facilitate crack edge detection. This filter takes into account the geometric characteristics of cracks, including line-like features and local symmetry along the center-lines, to accurately identify genuine crack edges within the 2D image. Lins and Givigi 16 developed a system grounded in machine vision principles aimed at automating the crack measurement process. Utilizing this method, a sequence of images is processed using only a single camera installed in a truck or even in a robot, enabling estimation of crack dimensions.
Jahanshahi and Masri 17 proposed a novel, time-saving method utilizing an autonomous robotic system with vision-based crack detection for processing 2D images. This system automatically adjusts depth parameters and employs 3D reconstruction for depth perception, effectively isolating cracks from their backgrounds for accurate analysis. Hamrat, et al.18 examined the flexural behavior of three types of concrete: normal strength concrete, high strength concrete, and high strength fiber concrete. It explores crack detection, development, width measurements, and strain components using the digital image correlation technique.
Convolutional Neural Networks (CNNs) excel in image detection due to their ability to automatically learn and extract features such as edges, textures, and shapes from raw images, and their robustness to positional changes within images thanks to translation invariance. By capturing spatial hierarchies through multiple layers, CNNs enhance the detection of complex objects, while parameter sharing in convolutional layers reduces the number of parameters, boosting computational efficiency and minimizing overfitting. End-to-end training simplifies the process from raw pixels to detection outputs, and their versatility allows application to various tasks like object detection and face recognition. Pre-trained models facilitate effective training with limited data, and their adaptability and strong community support further establish CNNs as a powerful tool in computer vision. In deep learning for crack detection, CNNs have demonstrated superior performance compared to edge detection and traditional machine learning classifiers, achieving accuracies over 98% for crack/non-crack detection19, 20. Studies by Chow, et al. 21 showcased CNNs' adaptability in detecting concrete defects, Perez, et al. 22 assessed CNNs for automated detection and localization of building defects, and Li, et al. 23 introduced an FCN-based method for precise pixel-level detection of multiple damages in concrete structures, including cracks, spalling, efflorescence, and holes, with minimal noise.
As evidenced by prior research studies, the field of structural crack detection utilizing optimal image processing and machine learning presents significant potential for enhancing quality management in the construction sector. Leveraging insights gleaned from past construction data, advanced machine learning techniques can construct predictive models to offer intelligent recommendations and decisions regarding inspections, thereby enhancing construction quality. Building on these findings, the research will concentrate on detecting cracks and damage in quality management through image processing and machine learning. The overarching goal is to facilitate proactive quality management by achieving thorough structural crack detection.
Proposal method
In construction quality management, a key challenge is detecting cracks within images. To address this, the research will first preprocess the images to standardize their sizes. Subsequently, the artificial neural network algorithm Convolutional Neural Network (CNN) will be employed to train the model, facilitating accurate detection of images containing cracks and those without. Figure 1 provides the fundamental steps involved in training a CNN model for crack detection, utilizing the Kaggle online platform.
Flowchart of CNN model for structural crack detection
Dataset preparation
The dataset consists of 1,000 images categorized into positive (crack) and negative (non-crack), with 500 images each. These images, captured at a distance of one meter and with dimensions of 227 × 227 pixels, showcase diverse surface finishes and lighting conditions. The data were randomly split into training (80%) and testing (20%) sets, resulting in 800 training images and 200 testing images. Additionally, 30% of the training set was used for validation. To mitigate data partition bias during model training, this study utilized a five-fold validation approach. The training dataset was divided into five mutually exclusive subsets. The convolutional neural network (CNN) underwent five training cycles, with each cycle using four subsets for training and the remaining subset for validation.
Import library and create CNN model
The process begins by importing essential libraries like Keras, numpy, and matplotlib, followed by constructing a CNN architecture that includes Convolutional, Pooling, and Fully Connected layers. Hyper parameters such as filter count, kernel size, and neuron quantity are then adjusted to optimize the model's performance. Finally, the algorithm is executed on the Kaggle platform to train and evaluate the CNN model for crack detection.
The research utilizes the online platform Kaggle to execute the algorithm. To generate a dataset on Kaggle, users can click on the "+ New Dataset" option, prompting a pop-up interface to appear. Within this interface, users can adjust settings to either "Upload Private" or "Public" by selecting the Settings icon located in the bottom left corner, as demonstrated in Figure 2. Figure 3 depicts the process of initiating a fresh notebook on Kaggle, offering users a platform to delve into data, construct models, acquire new proficiencies, engage in collaboration, and make contributions to the data science community.
The architecture of the Convolutional Neural Network (CNN) used in this study consists of several key components. The input layer accepts the input images, followed by 3-5 convolutional layers that extract features, each typically increasing in the number of filters from 32 to 128. Pooling layers follow each convolutional layer to reduce spatial dimensions. The network includes 1-2 fully connected layers, usually with 128 or 256 nodes, to perform the final classification based on the extracted features. The output layer provides the final classification output. The network is trained over 20 to 50 epochs, depending on the dataset size and model complexity. The ReLU (Rectified Linear Unit) activation function is used in the hidden layers to introduce non-linearity, enhancing the model's ability to learn complex patterns.
New dataset creation demonstration
New notebook creation illustration
Data preprocessing
The process involves loading images from both the train and test folders and converting them into numpy arrays. Subsequently, the data is normalized by dividing all pixel values by 255, effectively scaling them to a range between 0 and 1. To prepare the labels of the images, a binary classification approach is employed, assigning a label of 1 to images containing cracks and a label of 0 to those without cracks.
Model training
To assess the model's performance, the training data is split into a training set and a validation set. Binary cross-entropy loss functions and the Adam optimizer are utilized to optimize the model. The number of epochs and batch size are defined to adjust the training process, with the model trained using the fit() function on the training set and evaluated on the validation set after each epoch. Initially, the study aimed to investigate if image processing could enhance deep learning performance, thus an initial test was conducted with 100 epochs of training. This duration was chosen as it typically allows convergence with many pre-trained networks.
Model rating and evaluation
Various CNN models were constructed employing transfer learning to yield eight distinct models, comprising four types of images trained at two different durations. These models generated confusion matrices, enabling a comparative evaluation of their performance. The outcomes of the confusion matrices, derived from test data consisting of 150 sample images, along with associated metrics, are elucidated in this section. The validation data were employed for model training, and the training accuracy of each model was graphed at every epoch to visualize the training trajectory and illustrate the model's progression throughout training. The models exhibited significant enhancements in the initial two epochs, followed by more gradual improvements. To assess the model's performance on the test set, three commonly used evaluation metrics in classification and object detection are precision, recall, and F1-score. These metrics are derived from various error metrics and performance indicators in artificial intelligence and machine learning 24.
Precision: Measures the accuracy of positive predictions. It is calculated as true positives divided by the sum of true positives and false positives.
Precision (Pr) = TP/(TP+FP) (1)
Recall: Assesses the model's ability to capture positive instances. It is calculated as true positives divided by the sum of true positives and false negatives.
Recall (Rc)=TP/(TP+FN) (2)
F1-score: Balances precision and recall into a single metric, with scores close to one indicating high precision and effective detection.
F = (2*P*R)/(P + R) (3)
where true positive (TP) signifies a positive sample accurately predicted as positive, false positive (FP) indicates a positive sample mistakenly predicted as negative, and false negative (FN) denotes a negative sample inaccurately predicted as positive. The F-score, reflecting both Precision and Recall, is crucial in classification tasks. It reaches its highest values when both Precision and Recall are optimal, but diminishes if one of these metrics is low, dropping to 0 if either Precision or Recall is 0. Conversely, with both Precision and Recall at 1, the F-score reaches its maximum of 1.
Prediction with the trained model
After training the model, it is applied to predict outcomes on the test set or new images. This involves utilizing the predict function to generate predictions and ascertain their confidence levels. Subsequently, a threshold is applied to determine whether an image contains a crack, based on the confidence level determined through the predictions.
Evaluation and improvement
Evaluate the model's predictions against the actual labels to assess its performance accurately. Augment the dataset by applying various transformations such as rotation, flipping, zooming, or adjusting lighting conditions to enhance the model's ability to generalize to new data. Experiment with different model architectures or fine-tune hyper parameters to optimize the model's performance further. These iterative steps help refine the model and improve its accuracy in crack detection tasks.
Experimental result comparison and discussion
The structural crack detection datasets comprise 1000 images gathered from Thao Dien Luxury Villas - APSC Branch - An Phu One Member Co., Ltd. Adobe Photoshop 2021 software was utilized to resize the images to the specified dimensions and adjust lighting and color as needed.
Experimental results
The proposed model's performance was evaluated across different epochs, with Table 1 displaying the accuracy results for crack detection as assessed by the testing model. The table displays the accuracy results of the proposed model across various epochs, as evaluated by the testing model. With accuracy values ranging from 0.82 to 0.9 across different runs, the model demonstrates consistency in effectively classifying both positive and negative instances, indicative of its robust performance. This comprehensive evaluation sheds light on the model's effectiveness in crack detection across multiple iterations and dataset sizes, offering valuable insights into the trade-offs between precision, recall, and overall accuracy in the context of crack detection tasks.
After the training phase, the model is deployed to assess crack detection using real project data. The process of verifying the results against the actual project data is illustrated in Figure 4.
Accuracy of testing models
|
No. |
Run 1 |
Run 2 |
Run 3 |
Run 4 |
Run 5 | |||||
|
Negative |
Positive |
Negative |
Positive |
Negative |
Positive |
Negative |
Positive |
Negative |
Positive | |
|
Precision |
0.81 |
1 |
0.81 |
0.89 |
0.84 |
0.78 |
0.95 |
0.83 |
0.87 |
0.93 |
|
Recall |
1 |
0.82 |
0.94 |
0.67 |
0.89 |
0.7 |
0.83 |
0.95 |
0.93 |
0.87 |
|
F1-score |
0.9 |
0.9 |
0.87 |
0.76 |
0.86 |
0.74 |
0.89 |
0.89 |
0.9 |
0.9 |
|
Support |
13 |
17 |
18 |
12 |
18 |
10 |
24 |
21 |
29 |
31 |
|
Data set's size |
200 |
400 |
600 |
800 |
1000 | |||||
|
Accuracy |
0.90 |
0.83 |
0.82 |
0.89 |
0.9 | |||||
Outcome evaluation indicators
|
Precision |
Recall |
F1-Score |
Support | |
|
Negative |
0.87 |
0.93 |
0.9 |
29 |
|
Positive |
0.93 |
0.87 |
0.9 |
31 |
|
Accuracy |
0.9 |
60 |
Comparison of proposal model with direct view method
|
No. |
Image |
Outputs |
Predict |
Visual method |
Compare |
|
1 |
N1 |
0.3914 |
negative |
negative |
TRUE |
|
2 |
N5 |
0.127 |
negative |
negative |
TRUE |
|
3 |
C12 |
0.9233 |
positive |
positive |
TRUE |
|
…………………………. | |||||
|
98 |
K8 |
0.2569 |
negative |
negative |
TRUE |
|
99 |
C9 |
0.9082 |
positive |
positive |
TRUE |
|
100 |
K18 |
0.1811 |
negative |
negative |
TRUE |
|
Accuracy |
89% | ||||
Process of verifying the results with the actual project
Training and validation loss over time chart
Detection result of selected testing images (true positive values)
By monitoring the loss value and accuracy on both the training and test datasets over epochs, we can assess the model's progress and determine the optimal point to stop training. The Plotly Express library can be utilized to create a line chart that illustrates the changes in loss during training and testing across epochs. The graph, as shown in Figure 5, will display the variation in loss throughout the training and testing periods over the epochs. Analyzing this graph allows us to evaluate the model's performance, track its progress, and identify any signs of overfitting. Figure 6 displays the detection results for selected test images of wall surfaces in the building. These images illustrate the positive values achieved by the proposed model.
Table 2 presents the evaluation of the training process results, including accuracy, precision, recall, F1-score, and support. The classification report indicates the model's strong classification capability across both classes, demonstrating high accuracy and nearly equal F1-scores for both crack and non-crack categories. Through the training process, the model effectively identifies images containing cracks and those without, setting the stage for predictions with the actual dataset for testing purposes.
Results comparison and discussion
The proposed model underwent performance assessment by contrasting it with traditional detectors using the direct view method. This comparison aimed to gauge the effectiveness and potential benefits of the proposed model in crack detection tasks relative to conventional detectors. Through the direct view method, which entails directly observing and evaluating crack presence, the proposed model's outputs were juxtaposed with those of traditional detectors, facilitating a thorough evaluation of its performance compared to established detection methods. This comparative analysis sought to offer insights into the proposed model's efficiency and dependability in crack detection, especially when compared to traditional methodologies. Table 3 displays the outcomes of running the model with real project data and juxtaposing it with the direct view method for comparison.
The comparison table reveals that the model exhibits a slight variance compared to the visual method, with an accuracy rate of 89%, closely aligning with the model's training accuracy of 90%. There are 11 instances of differing results and 89 instances of consistency. Compared to the visual method, the model delivers rapid results and can be applied across a larger dataset while maintaining high accuracy.
The outcomes derived from training the Convolutional Neural Network (CNN) model to discern between cracked and intact images in construction quality management, leveraging image processing and machine learning, bear considerable significance for the administration and execution of construction projects. While the visual method relies on human perception and experience to evaluate work quality, it is susceptible to subjectivity and potential human error despite providing valuable insights. In contrast, employing image processing and machine learning models in construction quality management offers several advantages over the visual method.
• Automatic classification ability: The CNN model, having undergone training on extensive datasets, exhibits the capacity to autonomously categorize images depicting cracked and intact surfaces. This diminishes reliance on individual perception and expertise, thereby mitigating the likelihood of errors in the evaluation process.
• Uniformity and objectivity: Leveraging the CNN model ensures consistent and objective assessment of images, guided by the rules and insights acquired from the training data. This eradicates subjectivity and variability among evaluators, fostering a standardized approach to assessing work quality.
• Time and cost savings: Implementing the CNN model translates to savings in both time and expenses compared to the visual method. The automated assessment process operates swiftly and necessitates no human intervention at each stage of quality control.
• Efficient management: By automating crack detection and non-crack identification, construction quality managers gain comprehensive insights into building conditions and quality. This facilitates early identification of cracks and prompt application of remedial measures, averting the escalation of larger issues and ensuring project safety and quality. Nevertheless, while the utilization of image processing and machine learning models serves as a valuable support tool in project quality management, accurate assessment still necessitates a blend of model analysis and human judgement.
Conclusions
In summary, the integration of machine learning models for detecting cracks in construction images presents a promising advancement in quality management practices. This approach not only streamlines the detection process but also provides a more efficient alternative to traditional inspection methods, saving time and resources. This study introduced Convolutional Neural Network (CNN) technology on the Kaggle platform to automatically identify errors and defects on work surfaces, with the goal of improving quality control in construction projects. The research utilized a dataset of 1,000 images from real projects for training, validation, and testing of the proposed model. By leveraging image processing, optimized CNN, and Kaggle's resources, the study aims to enhance the efficiency, accuracy, and automation of defect detection and classification. The results demonstrate the transformative potential of image processing and machine learning technologies in the construction industry, signaling a shift towards more automated and accurate quality assessment methods.
Through the development of an automated system trained on a diverse dataset of construction images, this research has yielded tangible results in detecting errors and defects. The system's robust performance, as evidenced by high accuracy rates in identifying issues, highlights its efficacy in enhancing construction quality management. By leveraging machine learning algorithms to analyze images and classify them based on the presence of cracks, this study demonstrates the feasibility of implementing advanced technological solutions to address longstanding challenges in the construction industry.
Acknowledgement
This research is funded by Vietnam National University HoChiMinh City (VNU-HCM) under grant number B2024-20-05
Conflict of Interest
We wish to confirm that there are no known conflicts of interest associated with this publication and there has been no significant financial support for this work that could have influenced its outcome.
We confirm that the manuscript has been read and approved by all named authors and that there are no other persons who satisfied the criteria for authorship but are not listed. We further confirm that the order of authors listed in the manuscript has been approved by all of us.
We confirm that we have given due consideration to the protection of intellectual property associated with this work and that there are no impediments to publication, including the timing of publication, with respect to intellectual property. In so doing we confirm that we have followed the regulations of our institutions concerning intellectual property.
We understand that the Corresponding Author is the sole contact for the Editorial process (including Editorial Manager and direct communications with the office). He/she is responsible for communicating with the other authors about progress, submissions of revisions and final approval of proofs. We confirm that we have provided a current, correct email address which is accessible by the Corresponding Author and which has been configured to accept email from [email protected]
Author Contribution
Thi-Cam Tien Ngo: Conceptualization, Writing- Original draft preparation, Visualization, Investigation; Duc-Hoc Tran: Supervision, Validation, Writing- Reviewing and Editing, Methodology; Thi-Thom Pham: Conceptualization, Data curation, Methodology, Software.