Intrusion Detection Systems (IDS) are one of the major research application problems in the computer security domain. With the increasing number of advanced network attacks, the improvement of the traditional IDS techniques become a challenge. Efficient ways and methods of identifying, protecting, and analyzing data are needed. In this paper, a comprehensive survey on the application of Machine Learning (ML) and Deep Learning (DL) methods on the IDS to increase detection accuracy and reduce error rate is proposed. The recent research papers that have been published between 2018 and 2021 in the area of applying ML and DL in the IDS are analyzed and summarized. Four main analyzing aspects are presented as follows: (1) IDS concepts and taxonomy. (2) The strength and weaknesses of ML and DL methods. (3) IDS benchmark datasets. (4) Comprehensive review of the most recent articles that used ML and DL to improve IDS with highlighting the strengths and weaknesses of each work. Based on the analysis of the literature review papers, a framework for the application of ML and DL in the IDS is proposed. Finally, the current limitations are discussed and future research directions are provided.

[1] Abubakar A., Chiroma H., Muaz S., and Ila L., “A Review of the Advances in Cyber Security Benchmark Datasets for Evaluating Data-Driven based Intrusion Detection Systems,” in Proceedings of the International Conference on Soft Computing and Software Engineering, California, pp. 221-227, 2015.

[2] Ahmad Z., Shahid Khan A., Wai Shiang C., Abdullah J., and Ahmad F., “Network Intrusion Detection System: A Systematic Study of Machine Learning and Deep Learning Approaches,” Transactions on Emerging Telecommunications Technologies, vol. 32, no. 1, pp. 1-29, 2021.

[3] Ahmim A., Derdour M., and Ferrag M., “An Intrusion Detection System Based on Combining Probability Predictions of a Tree of Classifiers,” International Journal of Communication Systems, vol. 31, no. 9, pp. 1-17, 2018.

[4] Almomani I., Al-Kasasbeh B., and AL-Akhras M., “WSN-DS: A Dataset for Intrusion Detection Systems in Wireless Sensor Networks,” Journal of Sensors, pp. 1-16, 2016.

[5] Alzahrani A. and Alenazi M., “Designing a Network Intrusion Detection System Based on Machine Learning for Software Defined Networks,” Future Internet, vol. 13, no. 5, pp. 1- 18, 2021.

[6] Anderson J., Computer Security Threat Monitoring and Surveillance, Technical Report, James P. Anderson Company, 1980.

[7] Chandola V., Banerjee A., and Kumar V., “Anomaly Detection: a Survey,” ACM Computing Surveys, vol. 41, no. 3, pp. 1-58, 2009.

[8] Chary S. and Rama B., “A Survey on Comparative Analysis of Decision Tree Algorithms in Data Mining,” International Journal of Advanced Scientific Technologies, Engineering and Management Sciences, vol. 3, no. 1, pp. 91-95, 2017.

[9] CSE-CIC-IDS2018, https://www.unb.ca/cic/datasets/ids-2018.html Last Visited, 2022.

[10] Denning D., “An Intrusion-Detection Model,” IEEE Transactions on Software Engineering, no. 2, pp. 222-232, 1987.

[11] Ding Y. and Zhai Y., “Intrusion Detection System for NSL-KDD Dataset Using Convolutional Neural Networks,” in Proceedings of 2nd International Conference on Computer Science and Artificial Intelligence, New York, pp. 81-85, 2018.

[12] Dutta V., Chora´s M., Kozik R., and Pawlicki M., “Hybrid Model for Improving the Classification Effectiveness of Network Intrusion Detection,” in Proceeding of the Computational Intelligence in Security for Information Systems Conference, 2019.

[13] Faker O. and Dogdu E., “Intrusion Detection Using Big Data and Deep Learning Techniques,” in Proceedings of the ACM Southeast Conference, New York, pp. 86-93, 2019.

[14] Fung P., Zaidan A., Surakhi O., Tarkoma S., Petäjä T., and Hussein T., “Data Imputation in Situ-Measured Particle Size Distributions by Means of Neural Networks,” Atmospheric Measurement Techniques, vol. 14, no. 8, pp. 5535- 5554, 2021. The Intrusion Detection System by Deep Learning Methods: Issues and Challenges 511

[15] Garcia-Teodoro P., Diaz-Verdejo J., Maciá- Fernández G., and Vázquez E., “Anomaly-Based Network Intrusion Detection: Techniques Systems and Challenges,” Computers Security, vol. 28, no. 1-2, pp. 18-28, 2009.

[16] Haider W., Creech G., Xie Y., and Hu J., “Windows Based Data Sets for Evaluation of Robustness of Host Based Intrusion Detection Systems (IDS) to Zero-Day and Stealth Attacks,” Future Internet, vol. 8, no. 3, pp. 1-8, 2016.

[17] Hajimirzaei B. and Navimipour N., “Intrusion Detection for Cloud Computing Using Neural Networks and Artificial Bee Colony Optimization Algorithm,” ICT Express, vol. 5, no. 1, pp. 56-59, 2019.

[18] Jan S., Ahmed S., Shakhov V., and Koo I., “Toward a Lightweight Intrusion Detection System for the Internet of Things,” IEEE Access, vol. 7, pp. 42450-42471, 2019.

[19] Kabiri P. and Ghorbani A. “Research on Intrusion Detection and Response: a Survey,” International Journal of Network Security, vol. 1, no. 2, pp. 84- 102, 2005.

[20] Kaplana M. and Alptekin S. “An Improved Bigan Based Approach for Anomaly Detection,” Procedia Computer Science, vol. 176, pp. 185- 194, 2020.

[21] Khan F., Gumaei A., Derhab A., and Hussain A., “A Novel Two-Stage Deep Learning Model for Efficient Network Intrusion Detection,” IEEE Access, vol. 7, pp. 30373-30385, 2019.

[22] Khan M., “Hybrid Convolutional Recurrent Neural Network-Based Network Intrusion Detection System,” Processes, vol. 9, no. 5, pp. 1- 14, 2021.

[23] Khater B., Abdul Wahab A., Idris M., Hussain M., and Ibrahim A., “A Lightweight Perceptron-Based Intrusion Detection System for Fog Computing,” Applied Sciences, vol. 9, no. 1, pp. 1-21, 2019.

[24] Kim J., Kim J., Kim H., Shim M., and Choi E., “CNN-Based Network Intrusion Detection against Denial-of-Service Attacks,” Electronics, vol. 9, no. 6, pp. 1-21, 2020.

[25] Koroniotis N., Moustafa N., Sitnikova E., and Turnbull B., “Towards the Development of Realistic Botnet Dataset in the Internet of Things for Network Forensic Analytics: Bot-Iot Dataset,” Future Generation Computer Systems, vol. 100, pp. 779-796, 2019.

[26] Li J., Qu Y., Chao F., Shum H., Ho E., and Yang L., “Machine Learning Algorithms For Network Intrusion Detection,” AI in Cybersecurity, pp. 151-179, 2019.

[27] Li Z. and Qin Z., “A Semantic Parsing Based LSTM Model for Intrusion Detection,” in proceedings of International Conference on Neural Information Processing, Cambodia, pp. 600-609, 2018.

[28] Liang C., Shanmugam B., Azam S., Karim A., Islam A., Zamani M., Kavianpour S., and Idris N., “Intrusion Detection System for the Internet of Things Based on Blockchain and Multi-Agent Systems,” Electronics, vol. 9, no. 7, pp. 1-27, 2020.

[29] Liu G. and Zhang J., “CNID: Research of Network Intrusion Detection Based on Convolutional Neural Network,” Discrete Dynamics in Nature and Society, vol. 2020, pp. 1-11, 2020.

[30] Liu H. and Lang B., “Machine Learning and Deep Learning Methods for Intrusion Detection Systems: A Survey,” Applied Sciences, vol. 9, no. 20, pp. 1-28, 2019.

[31] Ma W., “Analysis of Anomaly Detection Method for Internet of Things Based on Deep Learning,” Transactions on Emerging Telecommunications Technologies, vol. 31, no. 12, pp. 1-13, 2020.

[32] Mahbooba B., Sahal R, Alosaimi W., and Serrano M., “Trust in Intrusion Detection Systems: an Investigation of Performance Analysis for Machine Learning and Deep Learning Models,” Complexity, 2021.

[33] Mahmood H., “Network Intrusion Detection System (NIDS) in Cloud Environment based on Hidden Naïve Bayes Multiclass Classifier,” Al- Mustansiriyah Journal of Science, vol. 28, no. 2, pp. 134-142, 2017.

[34] Mebawondua J., Alowolodub O., Mebawondua J., and Adetunmbi A., “Network Intrusion Detection System Using Supervised Learning Paradigm,” Scientific African, vol. 9, 2020.

[35] Michie D., Spiegelhalter D., and Taylor C., Machine Learning, Neurall and Statistical Classification, Citeseer, 1994.

[36] Moustafa N. and Slay J., “UNSW-NB15: A Comprehensive Data Set for Network Intrusion Detection Systems,” in Proceedings of Military Communications and Information Systems Conference (MilCIS), Canberra, pp. 1-6, 2015.

[37] Mukkamala S., Janoski G., and Sung A., “Intrusion Detection Using Neural Networks and Support Vector Machines,” in Proceedings of the International Joint Conference on Neural Networks, Honolulu, pp. 1702-1707, 2002.

[38] Nguyen S., Nguyen V., Choi J., and Kim K., “Design and Implementation of Intrusion Detection System using Convolutional Neural Network for DoS Detection,” in Proceedings of the International Conference on Machine Learning and Soft Computing, Phu Quoc, pp. 34- 38, 2018.

[39] Pandey N. and Savakis A., “Poly-GAN: Multi- Conditioned GAN for Fashion Synthesis,” Neurocomputing, vol. 414, pp. 356-364, 2020.

[40] Pham N., Foo E., Suriadi S., Jeffery H., and Lahza H., “Improving Performance of Intrusion Detection System Using Ensemble Methods and 512 The International Arab Journal of Information Technology, Vol. 19, No. 3A, Special Issue 2022 Feature Selection,” in Proceedings of the Australasian Computer Science Week Multiconference, Brisbane, pp. 1-6, 2018.

[41] Patil R., Biradar R., Ravi V., Biradar P., and Ghosh U., “Network Traffic Anomaly Detection using PCA and BiGAN,” Internet Technology Letters, vol. 5, no. 1, pp. 1-6, 2022.

[42] Peng W., Kong X., Peng G., Li X., and Wang Z., “Network Intrusion Detection Based on Deep Learning,” in Proceedings of International Conference on Communications, Information System and Computer Engineering (CISCE), Haikou, pp. 431-435, 2019.

[43] Pu Y., Gan Z., Henao R., Yuan X., Li C., Stevens A., and Carin L., “Variational Autoencoder for Deep Learning of Images, Labels and Captions,” in Proceedings of Advances in Neural Information Processing Systems, Barcelona, 2016.

[44] Salameh A. and Surakhi O., “An Optimized Convolutional Neural Network for Handwritten Digital Recognition Classification,” Journal of Theoretical and Applied Information Technology, vol. 98, no. 21, pp. 3494-3503, 2020.

[45] Saranya T., Sridevi S., Deisy C., Chung T., and Khan M., “Performance Analysis of Machine Learning Algorithms in Intrusion Detection System: A Review,” Procedia Computer Science, vol. 171, pp. 1251-1260, 2020.

[46] School of Engineering and Information Technology, UNSW, Australia. ADFA Linux data set (ADFA-LD) cyber security benchmark dataset, http://www.cybersecurity.unsw.adfa.edu.au/ADF A%20IDS%20Datasets, 2021.

[47] Shahriar M., Haque N., Rahman M., and Alonso M., “G-IDS: Generative Adversarial Networks Assisted Intrusion Detection System,” in Proceedings of IEEE 44th Annual Computers, Software, and Applications Conference (COMPSAC), Madrid, pp. 376-385, 2020.

[48] Sharafaldin I., Lashkari A., and Ghorbani A., “Toward Generating a New Intrusion Detection Dataset and Intrusion Traffic Characterization,” in Proceedings of 4th International Conference on Information Systems Security and Privacy (ICISSP), Madeira, pp. 108-116, 2018.

[49] Stolfo S, Fan W., Lee W., and Prodromidis A., http://kdd.ics.uci.edu/databases/kddcup99/kddcup99.h tml, Last Visited, 2018.

[50] Su T., Sun H., Zhu J., Wang S., and Li Y., “BAT: Deep Learning Methods on Network Intrusion Detection Using NSL-KDD Dataset,” IEEE Access, vol. 8, pp. 29575-29585, 2020.

[51] Sumaiya I., Saira Banu J., Lavanya K., Rukunuddin M., and Abhishek K., “An Integrated Intrusion Detection System Using Correlation- Based Attribute Selection and Artificial Neural Network,” Transactions on Emerging Telecommunications Technologies, vol. 32, no. 2, pp. 1-15, 2021.

[52] Surakhi O. and AlKhanafseh M., “Review on the Application of Blockchain Technology to Compact COVID-19 Pandemic,” in Proceedings of IEEE Jordan International Joint Conference on Electrical Engineering and Information Technology (JEEIT), Amman, pp. 193-198, 2021.

[53] Surakhi O., Zaidan M., Fung P., Hossein Motlagh N., Serhan S., AlKhanafseh M., Ghoniem R., and Hussein T., “Time-Lag Selection for Time-Series Forecasting Using Neural Network and Heuristic Algorithm,” Electronics, vol. 10, no. 20, pp. 1-22, 2021.

[54] Surakhi O., García A., Jamoos M., and Alkhanafseh M., “A Comprehensive Survey for Machine Learning and Deep Learning Applications for Detecting Intrusion Detection,” in 22nd International Arab Conference on Information Technology, Muscat, pp. 1-13, 2021.

[55] Surakhi O., Serhan S., and Salah I., “On the Ensemble of Recurrent Neural Network for Air Pollution Forecasting: Issues and Challenges,” Advances in Science, Technology and Engineering Systems Journal, vol. 5, no. 2, pp. 512-526, 2020.

[56] Surakhi O., Zaidan M., Serhan S., Salah I., and Hussein T., “An Optimal Stacked Ensemble Deep Learning Model for Predicting Time-Series Data Using a Genetic Algorithm-An Application for Aerosol Particle Number Concentrations,” Computers, vol. 9, no. 4, pp. 1-26, 2020.

[57] Susilo B. and Sari R., “Intrusion Detection in IoT Networks Using Deep Learning Algorithm,” Information, vol. 11, no. 5, pp. 1-11, 2020.

[58] Tang C., Luktarhan N., and Zhao Y., “SAAE- DNN: Deep Learning Method on Intrusion Detection,” Symmetry, vol. 12, no. 10, pp. 1-20, 2020.

[59] Tarter A., Community Policing-A European Perspective: Strategies, Best Practices and Guidelines, Springer, 2017.

[60] Tavallaee M., Bagheri E., Lu W., and Ghorbani A., “A Detailed Analysis of the KDD CUP 99 Data Set,” in Proceedings of IEEE Symposium on Computational Intelligence for Security and Defense Applications, Ottawa, pp. 1-6, 2009.

[61] Thamilarasu G. and Chawla S., “Towards Deep- Learning-Driven Intrusion Detection for the Internet of Things,” Sensors, vol. 19, no. 9, pp. 1- 19, 2019.

[62] Uddin M., Rahman A., Uddin N., Memon J., Alsaqour R., and Kazi S., “Signature-Based Multi- Layer Distributed Intrusion Detection System Using Mobile Agents,” International Journal of Network Security, vol. 15, no. 2, pp. 79-87, 2013.

[63] Verwoerd T. and Hunt R., “Intrusion Detection Techniques and Approaches,” Computer The Intrusion Detection System by Deep Learning Methods: Issues and Challenges 513 Communications, vol. 25, no. 15, pp. 1356-1365, 2002.

[64] Xiao Y. and Xiao X., “An Intrusion Detection System Based on a Simplified Residual Network,” Information, vol. 10, no. 11, pp. 1-17, 2019.

[65] Yao R., Wang N., Liu Z., Chen P., and Sheng X., “Intrusion Detection System in the Advanced Metering Infrastructure: A Cross-Layer Feature- Fusion CNN-LSTM-Based Approach,” Sensors, vol. 21, no. 2, pp. 1-17, 2021.

[66] Zaidan M., Surakhi O., Fung P., and Hussein T., “Sensitivity Analysis for Predicting Sub-Micron Aerosol Concentrations Based on Meteorological Parameters,” Sensors, vol. 20, no. 10, 2020.

[67] Zhang G., Wang X., Li R., Song Y., HEe J., and Lai J., “Network Intrusion Detection Based on Conditional Wasserstein Generative Adversarial Network and Cost-Sensitive Stacked Autoencoder,” IEEE Access, vol. 8, pp. 190431- 190447, 2020.

[68] Zhang Y., Cao G., Wang B., and Li X., “A Novel Ensemble Method Ffor K-Nearest Neighbor,” Pattern Recognition, vol. 85, pp. 13-25, 2019.

[69] Zhong M., Zhou Y., and Chen G., “Sequential Model Based Intrusion Detection System for IoT Servers Using Deep Learning Methods,” Sensors, vol. 21, no. 4, pp. 1-21, 2021.