Vision-based inspection of bolted joints: Field evaluation on a historical truss bridge in vietnam

e processing integration. A programing platform, Matlab 2020a, was used to build the 
RCNN model and to conduct all computations in this study. 
q+
ja
90o jq+
y
x
jth edge
jr
jq
Fig. 3. Estimation method of bolt angle
It is supposed that the jth true edge of the
bolt, detected by the Hough transform, is de-
scribed by the two parameters (θj, rj), as de-
scribed in Fig. 3. Thus, the angle of the jth edge
is defined as θj + 90◦, which is the angle be-
tween the jth edge and the horizontal direction.
Due to the inborn shape of the hexagon bolt,
the angle of the jth edge can be consistently ex-
pressed in the range of 0◦ − 60◦ as
αj = mod
[
(θj + 90◦)/60◦
]
(2)
in which αj is the equivalent angle of the jth edge and should be the same for all edges
(see Fig. 3); mod[.] is the remainder of a division. To consider all true edges of a bolt, the
angles are averaged as
α =
1
k
k
∑
j=1
αj (3)
Vision-based inspection of bolted joints: Field evaluation on a historical truss bridge in Vietnam 19
where α is the bolt angle, and k is the number of selected bolt edges. It is noted that k≤6
should be preset in practice.
The rotational angle of the bolt (∆α) is estimated by comparing the present angle
with the one obtained in the intact state. For detecting loosened bolts in the connection,
the absolute rotational angle |∆α| is often compared with a control limit. The bolt is ‘loos-
ened’ if the rotational angle is higher than the threshold; otherwise, the bolt is ‘tightened’.
A well-established control limit is determined by three standard deviations of the mean
with a confidence level of 99.7%.
3. VISION-BASED INSPECTION OF BOLTED JOINTS OF A HISTORICAL TRUSS
BRIDGE IN VIETNAM
3.1. Experiments on historical truss bridge
The test bridge is a historical truss bridge, the Nam O Bridge in Da Nang City (Viet-
nam). The bridge crosses Cu De River and is an important link in the national high way
1A of Vietnam, as shown in Fig. 4(a). The bridge has four spans and each span consists of
many large bolted joints, as shown in Fig. 4(b). The bridge was constructed before 1975,
the year of the reunification of Vietnam. So far, the bridge has been maintained several
times and its current structural performance is an important concern. Health monitoring
of the old bolted joints of the bridge is essential to ensure their safety and serviceability.
 7 
(a) Location of the Nam O Bridge (obtained from Google Maps) 
(b) Real view of the Nam O bridge 
Fig. 4. Nam O Bridge in Da Nang City, Viet Nam 
(a) A joint of the vertical and bottom chord 
Nam O 
Bridge
(a) Location of the Nam O Bridge (obtained from
Google Maps)
(b) Real view of the Nam O bridge
Fig. 4. Nam O Bridge in Da Nang City, Vietnam
In this study, the authors used a digital camera to shoot the images of several bolted
connections (a resolution of 4032×3024 pixels, AF f/2.8, a focal length of 7 mm). The con-
nection images w re input into the vision-based approach for bolt-loosening monitoring.
Fig. 5 shows the selected images of two bolted joints of the bridge. As shown in Fig. 5(a),
the first one is the joint between the vertical member and the bottom chord of the bridge,
which has 24 bolts. The second one is the joint between the floor beam and the stringer
of the bridge, which also consist of many bolts, see Fig. 5(b).
It is noted that both joint images have certain levels of perspective distortion, which
should be corrected to enhance the accuracy of bolt angle estimation. In the following, the
bolts in those connections are detected and their angles are estimated by the vision-based
20 Thanh-Canh Huynh, Ba-Phu Nguyen, Ananta Man Singh Pradhan, Quang-Quang Pham
(a) A joint of the vertical and bottom chord
(b) A joint of the floor beam and stringer
Fig. 5. Images of a representative bolted joint of Nam O Bridge
approach through the deep learning-image processing integration. A programming plat-
form, Matlab 2020a, was used to build the RCNN model and to conduct all computations
in this study.
3.2. Vision-based bolt-angle estimation: Vertical member-bottom chord joint
The results of bolt detection and bolt angle estimation of the vertical member-bottom
chord joint by the vision-based approach are presented in Fig. 6. It is shown in Fig.
6(a) that the perspective distortion of the joint image was corrected by the homography
algorithm [20] and all 24 bolts of the joint were successfully detected by the deep learning-
based bolt detector. After bolt detection, the detected bolts were labelled from Bolts 1 to
Bolt 24. The labelling rule is based on sorting the bolts’ coordinates from left to right and
top to bottom. The detected bolts were subsequently cropped in sub-images of single
bolts, as displayed in Fig. 6(b).
Vision-based inspection of bolted joints: Field evaluation on a historical truss bridge in Vietnam 21
Afterwards, the angles of the bolts were estimated by the Hough transform algo-
rithm. As shown in Fig. 6(c), two strongest edges of Bolts 1- Bolt 24 were well identified
(k = 2). Basically, all six edges of a bolt should be used for angle estimation. Nonethe-
less, it is hard to capture the six edges of all bolts in a realistic joint, as evidenced in
Fig. 6(c). Fig. 6(d) shows the comparison of the estimated angles of the bolts between
the vision-based approach and the manual measurement by a protractor. There exists
strong consistency between the two methods (99.3% correlation) in which the estimated
bolt angles by the vision approach are well-agreed with the measured ones by the pro-
tractor. The result evidenced the reliability of the vision-based approach for monitoring
large connections in the field.
 9 
(a) Perspective correction and bolt detection 
(b) Bolt segmentation (c) Bolt angle estimation 
(a) Perspective correction and bolt
detection
 9 
(a) Perspective correction and bolt detection 
(b) Bolt segmentation (c) Bolt angle estimation (b) Bolt segmentation
 9 
(a) Perspective correcti n and bolt detection 
(b) Bolt segmentation (c) Bolt angle estimation (c) Bolt angle estimationThanh-Canh Huynh 10 
(d) Accuracy of vision approach 
Fig. 6. Bolt angle estimation for the joint of vertical member and bottom chord 
3.3. Vision-based Bolt-Angle Estimation: Floor Beam-Stringer Joint 
As the second example, the vision-based bolt detection and bolt angle estimation are conducted 
on the vertical member-bottom chord joint, as shown in Fig. 7. After the perspective distortion by the 
homography method, all bolts of the joint were successfully identified by the bolt detector, as depicted 
in Fig. 7a. The detected bolts were then labelled by Bolts 1 - Bolt 24 and the detected bolts were cropped 
in sub-images of single bolts, as seen in Fig. 7b. Lastly, the bolt angles were estimated by the Hough 
transform algorithm [20]. Two strongest edges of Bolts 1- Bolt 24 were well identified, as shown in Fig. 
7c. 
For the accuracy evaluation, the comparison between the vision-based bolt angle estimation and 
the measurement by a protractor is shown in Fig. 7d. The protractor-based method is a kind of visual 
inspection by the human. This method is labour-intensive and dangerous in some situations. However, 
it is simple to perform do not require high computational costs. The bolt angle estimation results between 
the two methods were well-agreed with a high correlation of 99.5%. This result again proved the 
accuracy of the vision-based approach for inspecting realistic bridge joints in practice. 
y = 0.973x + 0.3503
R² = 0.993
0
10
20
30
40
50
60
0 10 20 30 40 50 60
Pr
ot
ra
ct
or
 (D
eg
)
Vision Approach (Deg)
(d) Accuracy ision approach
Fig. 6. Bolt angle estimation for the joint of vertical member and bottom chord
22 Thanh-Canh Huynh, Ba-Phu Nguyen, Ananta Man Singh Pradhan, Quang-Quang Pham
3.3. Vision-based bolt-angle estimation: Floor beam-stringer joint
As the second example, the vision-based bolt detection and bolt angle estimation
are conducted on the vertical member-bottom chord joint, as shown in Fig. 7. After the
perspective distortion by the homography method, all bolts of the joint were successfully
identified by the bolt detector, as depicted in Fig. 7(a). The detected bolts were then
labelled by Bolts 1 - Bolt 24 and the detected bolts were cropped in sub-images of single
bolts, as seen in Fig. 7(b). Lastly, the bolt angles were estimated by the Hough transform
algorithm [20]. Two strongest edges of Bolts 1- Bolt 24 were well identified, as shown in
Fig. 7(c).
 11 
(a) Perspective correction and bolt detection 
(b) Bolt segmentation (c) Bolt angle estimation 
(a) Perspective correction and bolt detec-
tion
 11 
(a) Perspective correction and bolt detection 
(b) Bolt segmentation (c) Bolt angle estimation (b) Bolt segmentation
 11 
(a) erspective correction and bolt detection 
 (c) Bolt angle estimationThanh-Canh Huynh 12 
(d) Accuracy of vision-based approach 
Fig. 7. Bolt angle estimation for the joint of floor beam and stringer 
5. SUMMARY, CONCLUSION, AND FUTURE WORK 
This study examined the applicability of the RCNN-image processing integrated method for 
monitoring large-scale bolted joints of a realistic bridge in Viet Nam. Firstly, the vision-based bolt-
loosening monitoring approach was briefly described. Secondly, field experiments on a historical truss 
bridge, the Nam O bridge in Da Nang City, was performed. A digital camera was used to capture the 
images of representative bolted joints of the bridge. Lastly, the vision-based approach was applied to 
estimate the bolt angles of the two representative joints (i.e., the vertical-bottom chord joint and the floor 
beam-stringer joint) of the test bridge. 
The bolt angles estimated by the vision approach showed a good agreement with the manual 
measurement by protractor, showing the potentials of the vision-based approach for inspecting realistic 
bridge connections in the field. So far, there exists limited research on vision-based approaches for 
loosened bolt detection in Viet Nam. As the safety of historical bridges is increasingly concerned, this 
study could provide practical value for structural health monitoring practices of ageing truss and girder 
bridges in Viet Nam. 
Nonetheless, to ensure the in-service strength of the bolted joints of the test bridge, it is necessary 
to capture the joint images and monitor the rotations of bolts over time. The long-term monitoring of 
the bridge joints by the vision-based approach is under investigating and will be presented in future 
work. The effect of the light condition, the distance from the camera to the joint, the specification of the 
camera on the image quality and detection results will be also investigated in details. 
ACKNOWLEDGEMENT 
This research is funded by Vietnam National Foundation for Science and Technology 
Development (NAFOSTED) under grant number 107.01-2019.332 
REFERENCES
y = 0.9867x - 0.0039
R² = 0.9947
0
10
20
30
40
50
60
0 10 20 30 40 50 60
Pr
ot
ra
ct
or
 (D
eg
)
Vision Approach (Deg)
(d) Accuracy of visi - ased approach
Fig. 7. Bolt angle estimation for the joint of floor beam and stringer
For the accuracy evaluation, the comparison between the vision-based bolt angle es-
timation a d th measur me t by a pro ractor is shown in ig. 7(d). The protractor-based
Vision-based inspection of bolted joints: Field evaluation on a historical truss bridge in Vietnam 23
method is a kind of visual inspection by the human. This method is labour-intensive
and dangerous in some situations. However, it is simple to perform do not require high
computational costs. The bolt angle estimation results between the two methods were
well-agreed with a high correlation of 99.5%. This result again proved the accuracy of
the vision-based approach for inspecting realistic bridge joints in practice.
4. SUMMARY, CONCLUSION, AND FUTURE WORK
This study examined the applicability of the RCNN-image processing integrated
method for monitoring large-scale bolted joints of a realistic bridge in Vietnam. Firstly,
the vision-based bolt-loosening monitoring approach was briefly described. Secondly,
field experiments on a historical truss bridge, the Nam O bridge in Da Nang City, was
performed. A digital camera was used to capture the images of representative bolted
joints of the bridge. Lastly, the vision-based approach was applied to estimate the bolt
angles of the two representative joints (i.e., the vertical-bottom chord joint and the floor
beam-stringer joint) of the test bridge.
The bolt angles estimated by the vision approach showed a good agreement with
the manual measurement by protractor, showing the potentials of the vision-based ap-
proach for inspecting realistic bridge connections in the field. So far, there exists limited
research on vision-based approaches for loosened bolt detection in Vietnam. As the safety
of historical bridges is increasingly concerned, this study could provide practical value
for structural health monitoring practices of aging truss and girder bridges in Vietnam.
Nonetheless, to ensure the in-service strength of the bolted joints of the test bridge,
it is necessary to capture the joint images and monitor the rotations of bolts over time.
The long-term monitoring of the bridge joints by the vision-based approach is under
investigating and will be presented in future work. The effect of the light condition, the
distance from the camera to the joint, the specification of the camera on the image quality
and detection results will be also investigated in details.
ACKNOWLEDGEMENT
This research is funded by Vietnam National Foundation for Science and Technology
Development (NAFOSTED) under grant number 107.01-2019.332.
REFERENCES
[1] T. Wang, G. Song, S. Liu, Y. Li, and H. Xiao. Review of bolted connection
monitoring. International Journal of Distributed Sensor Networks, 9, (12), (2013).
https://doi.org/10.1155/2013/871213.
[2] T.-C. Nguyen, T.-C. Huynh, J.-H. Yi, and J.-T. Kim. Hybrid bolt-loosening detection in wind
turbine tower structures by vibration and impedance responses. Wind and Structures, 24, (4),
(2017), pp. 385–403. https://doi.org/10.12989/was.2017.24.4.385.
[3] S. M. Y. Nikravesh and M. Goudarzi. A review paper on looseness detection methods in
bolted structures. Latin American Journal of Solids and Structures, 14, (12), (2017), pp. 2153–
2176. https://doi.org/10.1590/1679-78254231.
24 Thanh-Canh Huynh, Ba-Phu Nguyen, Ananta Man Singh Pradhan, Quang-Quang Pham
[4] M. Zhang, Y. Shen, L. Xiao, and W. Qu. Application of subharmonic resonance for the
detection of bolted joint looseness. Nonlinear Dynamics, 88, (3), (2017), pp. 1643–1653.
https://doi.org/10.1007/s11071-017-3336-1.
[5] T.-C. Huynh, N.-L. Dang, and J.-T. Kim. Preload monitoring in bolted con-
nection using piezoelectric-based smart interface. Sensors, 18, (9), (2018).
https://doi.org/10.3390/s18092766.
[6] S. G. Joshi and R. G. Pathare. Ultrasonic instrument for measuring bolt stress. Ultrasonics, 22,
(6), (1984), pp. 261–269. https://doi.org/10.1016/0041-624x(84)90043-x.
[7] M. Hirao, H. Ogi, and H. Yasui. Contactless measurement of bolt axial stress using a shear-
wave electromagnetic acoustic transducer. Ndt & E International, 34, (3), (2001), pp. 179–183.
https://doi.org/10.1016/s0963-8695(00)00055-4.
[8] T.-C. Huynh, D.-D. Ho, N.-L. Dang, and J.-T. Kim. Sensitivity of piezoelectric-based
smart interfaces to structural damage in bolted connections. Sensors, 19, (17), (2019).
https://doi.org/10.3390/s19173670.
[9] L. Huo, F. Wang, H. Li, and G. Song. A fractal contact theory based model for bolted connec-
tion looseness monitoring using piezoceramic transducers. Smart Materials and Structures, 26,
(10), (2017). https://doi.org/10.1088/1361-665x/aa6e93.
[10] T.-C. Huynh. Structural parameter identification of a bolted connection embedded with
a piezoelectric interface. Vietnam Journal of Mechanics, 42, (2), (2020), pp. 173–188.
https://doi.org/10.15625/0866-7136/14806.
[11] Y.-J. Cha, K. You, and W. Choi. Vision-based detection of loosened bolts using the Hough
transform and support vector machines. Automation in Construction, 71, (2016), pp. 181–188.
https://doi.org/10.1016/j.autcon.2016.06.008.
[12] D. Feng and M. Q. Feng. Computer vision for SHM of civil infrastructure: From dynamic
response measurement to damage detection–A review. Engineering Structures, 156, (2018),
pp. 105–117. https://doi.org/10.1016/j.engstruct.2017.11.018.
[13] C. M. Yeum and S. J. Dyke. Vision-based automated crack detection for bridge inspec-
tion. Computer-Aided Civil and Infrastructure Engineering, 30, (10), (2015), pp. 759–770.
https://doi.org/10.1111/mice.12141.
[14] J.-H. Park, T.-C. Huynh, S.-H. Choi, and J.-T. Kim. Vision-based technique for bolt-
loosening detection in wind turbine tower. Wind and Structures, 21, (6), (2015), pp. 709–726.
https://doi.org/10.12989/was.2015.21.6.709.
[15] T.-C. Nguyen, T.-C. Huynh, J.-Y. Ryu, J.-H. Park, and J.-T. Kim. Bolt-loosening identi-
fication of bolt connections by vision image-based technique. In Nondestructive Charac-
terization and Monitoring of Advanced Materials, Aerospace, and Civil Infrastructure, (2016).
https://doi.org/10.1117/12.2219055.
[16] L. Ramana, W. Choi, and Y.-J. Cha. Fully automated vision-based loosened bolt detection
using the Viola–Jones algorithm. Structural Health Monitoring, 18, (2), (2019), pp. 422–434.
https://doi.org/10.1177/1475921718757459.
[17] X. Kong and J. Li. Image registration-based bolt loosening detection of steel joints. Sensors,
18, (4), (2018). https://doi.org/10.3390/s18041000.
[18] X. Zhao, Y. Zhang, and N. Wang. Bolt loosening angle detection technology using deep learn-
ing. Structural Control and Health Monitoring, 26, (1), (2019). https://doi.org/10.1002/stc.2292.
[19] C. Wang, N. Wang, S.-C. Ho, X. Chen, and G. Song. Design of a new vision-based method for
the bolts looseness detection in flange connections. IEEE Transactions on Industrial Electronics,
67, (2), (2019), pp. 1366–1375. https://doi.org/10.1109/tie.2019.2899555.
Vision-based inspection of bolted joints: Field evaluation on a historical truss bridge in Vietnam 25
[20] T.-C. Huynh, J.-H. Park, H.-J. Jung, and J.-T. Kim. Quasi-autonomous bolt-loosening detec-
tion method using vision-based deep learning and image processing. Automation in Construc-
tion, 105, (2019). https://doi.org/10.1016/j.autcon.2019.102844.
[21] S.-Y. Lee, T.-C. Huynh, J.-H. Park, and J.-T. Kim. Bolt-loosening detection using
vision-based deep learning algorithm and image processing method. Journal of the
Computational Structural Engineering Institute of Korea, 32, (4), (2019), pp. 265–272.
https://doi.org/10.7734/coseik.2019.32.4.265.
[22] R. Girshick, J. Donahue, T. Darrell, and J. Malik. Rich feature hierarchies for accurate object
detection and semantic segmentation. In Proceedings of the IEEE Conference on Computer Vision
and Pattern Recognition, (2014), pp. 580–587. https://doi.org/10.1109/cvpr.2014.81.
[23] J. R. R. Uijlings, K. E. A. Van De Sande, T. Gevers, and A. W. M. Smeulders. Selective search
for object recognition. International Journal of Computer Vision, 104, (2), (2013), pp. 154–171.
https://doi.org/10.1007/s11263-013-0620-5.
[24] S.-J. Yang, C. C. Ho, J.-Y. Chen, and C.-Y. Chang. Practical homography-based per-
spective correction method for license plate recognition. In 2012 International Confer-
ence on Information Security and Intelligent Control, IEEE, IEEE, (2012), pp. 198–201.
https://doi.org/10.1109/isic.2012.6449740.
[25] J. Canny. A computational approach to edge detection. IEEE Transactions on Pattern Analysis
and Machine Intelligence, (6), (1986), pp. 679–698. https://doi.org/10.1016/b978-0-08-051581-
6.50024-6.