The International Arab Journal of Information Technology (IAJIT)


Deep Learning Based Feature Discriminability Boosted Concurrent Metal Surface Defect Detection System Using YOLOv-5s-FRN

Computer vision and deep learning techniques are the most emerging technologies in this era. Both of these can greatly raise the rate at which defects on metal surfaces are identified while performing industrial quality checks. The identification of faults over metal surfaces can be viewed as a significant challenge since they are easily impacted by ambient factors including illumination and light reflections. This paper proposes novel metal surface defect detection network called as YOLOv-5s-FRN in response to the problems of ineffective detection brought by the conventional manual inspection system. The proposed system is developed through the integration of a novel architectural module called as Feature Recalibration Network (FRN) to the You Only Look Once-version-5 small network )YOLOv-5s(. In order to extract the global feature information from the provided image, FRN is able to evaluate the interdependencies between the channels. This improves the feature discrimination capability and prediction accuracy of the defect detection system. The incorporation of FRN structure makes YOLOv-5s architecture to selectively enhance the necessary features and discard the unwanted ones. Therefore, the proposed novel method will efficiently detect and classify the metal surface defects such as crazing, patches, inclusions, scratches, pitted surfaces and rolled in scale. North Eastern University surface defect database (NEU-DET) has been used to train and test the proposed architectural model. The suggested system has been compared with alternative models based on several performance matrices such as precision, recall and Mean Average Precision (mAP). It is observed that the proposed YOLOv-5s-FRN architecture provides significant performance improvement than state-of-the-art methods. The proposed system has been provided satisfactory results by means of improvement in mAP and time consumption. The proposed model has delivered value of mAP_0.5 as 98.05% and that of mAP_0.5:0.95 as 89.03%.

[1] Bayraktar E., Tosun B., Altintas B., and Celebi N., “Combined GANs and Classical Methods for Surface Defect Detection,” in Proceedings of the Signal Processing and Communications Applications Conference, Safranbolu, pp. 1-4, 2022. Doi: 10.1109/SIU55565.2022.9864705.

[2] Fu J., Zhu X., and Li Y., “Recognition of Surface Defects on Steel Sheet Using Transfer Learning,” arXiv Preprint, arXiv:1909.03258v2, 2019.

[3] Gyimah N., Girma A., Mahmoud M., Nateghi S., Homaifar A., and Opoku D., “Robust Completed Local Binary Pattern (RCLBP) for Surface Defect Detection,” in Proceedings of the IEEE International Conference on Systems, Man, and Cybernetics, Melbourne, pp. 1927-1934, 2021. DOI: 10.1109/SMC52423.2021.9659140

[4] Hatab M., Malekmohamadi H., and Amira A., “Surface defect detection using YOLO network,” in Proceedings of the SAI Intelligent Systems Conference, London, pp. 505-515, 2020.

[5] He K., Zhang X., Ren S., and Sun J., “Deep Residual Learning for Image Recognition,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, USA, pp. 770- 778, 2016.

[6] He K., Zhang X., Ren S., and Sun J., “Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 37, pp. 1904-1916. 2015. Doi:

[7] He Y., Song K., Meng Q., and Yan Y., “An End-to- End Steel Surface Defect Detection Approach Via Fusing Multiple Hierarchical Features,” IEEE Transactions on Instrumentation and Measurement, vol. 69, no. 4, pp. 1493-1504, 2019. Doi: 10.1109/TIM.2019.2915404

[8] Kim M., Park T., and Park P., “Classification of Steel Surface Defect Using Convolutional Neural Network with Few Images,” in Proceedings of the 12th Asian Control Conference, Kitakyushu, pp. 1398–1401, 2019.

[9] Komijani A., Vafaeinezhad F., Khoramdel J., Borhani Y., and Najafi E., “Multi-Label Classification of Steel Surface Defects Using Transfer Learning and Vision Transformer,” in Proceedings of the International Conference on Information and Knowledge Technology, Karaj, pp. 1-5, 2022. doi: 10.1109/IKT57960.2022.10039038

[10] Kou X., Liu S., Cheng K., and Qian Y., “Development of a YOLO-V3-based Model for Detecting Defects on Steel Strip Surface,” Measurement, vol. 182, pp. 109454, 2021. 54.

[11] Liang F., Zhou Y., Chen X., Liu F., and Zhang C., “Review of Target Detection Technology Based on Deep Learning,” in Proceedings of the 5th International Conference on Control Engineering and Artificial Intelligence, Sanya, pp. 132-135, 2021.

[12] Lin H. and Wibowo F., “Image Data Assessment Approach for Deep Learning-Based Metal Surface Defect-Detection Systems,” IEEE Access, vol. 9, pp. 47621-47638, 2021. doi: 10.1109/ACCESS.2021.3068256.

[13] Lin Z., Ye H., Zhan B., and Huang X., “An Efficient Network for Surface Defect Detection,” journal of Applied Sciences, vol. 10, no. 17, pp. 6085, 2020.

[14] Lv X., Duan F., Jiang J., Fu X., and Gan L., “Deep Metallic Surface Defect Detection: The New Benchmark and Detection Network,” Sensors Journal, vol. 20, no. 6, pp. 1562, 2020.

[15] Nair V. and Hinton G., “Rectified Linear Units Improve Restricted Boltzmann Machines,” in Proceedings of the 27th International Conference on International Conference on Machine Learning, Haifa, pp. 807-814, 2010.

[16] Narayanan L., Krishnan S., and Robinson H., “A Hybrid Deep Learning Based Assist System for Detection and Classification of Breast Cancer from Mammogram Images,” The International Arab Journal of Information Technology, vol. 19, no. 6, pp. 965-974, 2022.

[17] Neu Surface Defect Database, Available: defect_database.html, Last Visited, 2023.

[18] Ning Z. and Mi Z., “Research on Surface Defect Detection Algorithm of Strip Steel Based on Improved YOLOV3,” in Proceedings of the International Conference on Electronic Materials and Information Engineering, Xain, pp. 1-6, 2021. 10.1088/1742-6596/1907/1/012015

[19] Niu Z., Zhong G., and Yu H., “A Review on the Attention Mechanism of Deep Learning,” Neurocomputing Journal, vol. 452, pp. 48–62, 2021.

[20] Prihatno A., Utama I., Kim J., and Jang Y., “Metal Defect Classification Using Deep Learning,” in Proceedings of the International Conference on Ubiquitous and Future Networks, Jeju Island, pp. 106 The International Arab Journal of Information Technology, Vol. 21, No. 1, January 2024 389-393, 2021. Doi: 10.1109/ICUFN49451.2021.9528702

[21] Qing Y., Fing I., Tang I., Xu W., and Zho X., “Development of an Automatic Monitoring System for Rice Light-Trap Pests Based on Machine Vision,” Journal of Integrative Agriculture, vol. 19, no. 10, pp. 2500-2513, 2020.

[22] Tabernik D., Sela S., Skvarc J., and Skocaj D., “Segmentation-Based Deep-Learning Approach for Surface-Defect Detection,” Journal of Intelligent Manufacturing, vol. 31, pp. 759-776, 2019. x

[23] Truong Duy T. and Natori N., “Efficient Defect Detection from Consecutive Monocular Images by Deep Learning,” in Proceedings of the IEEE International Conference on Machine Learning and Applications, Miami, pp. 473-478, 2020. Doi: 10.1109/ICMLA51294.2020.00080.

[24] Vengaloor P. and Muralidhar R., “Deep Learning Based Semantic Segmentation Technique for Anomaly Detection on Metal Surfaces using High Calibre U-Shaped Network,” International Information and Engineering Technology Association, vol. 39, no. 6, pp. 2023-2031, 2022.

[25] Vengaloor R. and Muralidhar R., “Smart Farming Using IOT: The Surveillance and the Regulation of Agricultural Area,” Bulletin of Environment, Pharmacology and Life Sciences, vol. 10, no. 11, pp. 206-212, 2021.

[26] Wang C., Yu Y., Yu J., Zhang Y., and Zhao Y., “Microstructure Evolution and Corrosion Behavior of Dissimilar 304/430 Stainless Steel Welded Joints,” Journal of Manufacturing Process, vol. 50, pp. 183-191, 2020.

[27] Wang C. and Xie H., “MeDERT: A Metal Surface Defect Detection Model,” IEEE Access, vol. 11, pp. 35469-35478, 2023. Doi: 10.1109/ACCESS.2023.3262264.

[28] Wang C., Xu J., Liang X., and Yin D., “Metal Surface Defect Detection Based on Weighted Fusion,” in Proceedings of the International Conference on Virtual Reality and Visualization, Recife, pp. 179-184, 2020. doi: 10.1109/ICVRV51359.2020.00044

[29] Wang C., Zhou Z., and Chen Z., “An Enhanced YOLOv4 Model with Self-Dependent Attentive Fusion and Component Randomized Mosaic Augmentation for Metal Surface Defect Detection,” IEEE Access, vol. 10, pp. 97758- 97766, 2022. Doi: 10.1109/ACCESS.2022.3203198.

[30] Wang H., Li M., Gao P., and Zhang S., “YOLOXD: A New Network for Metal Surface Defect Detection,” in Proceedings of the IEEE 2nd International Conference on Computer Systems, Qingdao, pp. 75-79, 2022. Doi: 10.1109/ICCS56273.2022.9988327.

[31] Woo S., Park J., Lee J., and Kweon I., “CBAM: Convolutional Block Attention Module,” in Proceedings of the 15th European Conference on Computer Vision, Munich, pp. 3-19, 2018.

[32] Ye Z., Liu M., Zhang S., and Wei P., “Dual-Path GAN: A Method for Enhancing Small-Scale Defect Detection on Metal Images,” in Proceedings of the 41st Chinese Control Conference, Hefei, pp. 6292-6297, 2022. Doi: 10.23919/CCC55666.2022.9902599

[33] Yu R., Guo B., and Yang K., “Selective Prototype Network for Few-Shot Metal Surface Defect Segmentation,” in Proceedings of the IEEE Transactions on Instrumentation and Measurement, vol. 71, pp. 1-10, 2022. Doi: 10.1109/TIM.2022.3196447.

[34] Zhao H., Wan F., Lei G., Xiong Y., and Xu L., “LSD-YOLOv5: A Steel Strip Surface Defect Detection Algorithm Based on Lightweight Network and Enhanced Feature Fusion Mode,” Sensors Journal, vol. 23, pp. 6558, 2023.

[35] Zhou X., Nie Y., Wang Y., Pingguo C., and Ye M., “A Real-time and High-Efficiency Surface Defect Detection Method for Metal Sheets Based on Compact CNN,” in Proceedings of the International Symposium on Computational Intelligence and Design, Hangzhou, pp. 259-264, 2020. Doi: 10.1109/ISCID51228.2020.00064.