Under-Water Sensor Networks (UWSNs) are important for applications like oceanographic data collection, environmental protection, monitoring, and disaster response. These UWSN networks face challenges in energy efficiency and protocol convergence because of dynamic underwater ecosystems limited in number and low channel capacity. This paper covers and finds real-world solutions for these challenges by proposing an adaptive Q-Learning Medium Access Control (MAC) protocol for UWSNs. The methodology used in this paper involves utilizing Q-Learning, a reinforcement learning technique, to make sensor nodes autonomously refine their transmission strategies in real-time, enhancing energy consumption and improving protocol convergence. The protocol was implemented using the NS 3 network simulator, which offers a detailed and real-world environment for analyzing the protocol’s performance. Extensive simulations were conducted, and experiments were used to analyze the performance of the proposed protocol. The results showcase significant improvements over other and traditional MAC protocols, with a 13% to 19% increase in energy efficiency and channel utilization enhancement in static and mobile network scenarios. The adaptive Q-Learning MAC protocol provides a robust solution for the challenges of UWSNs, offering energy efficiency and faster convergence times. This research significantly contributes to the advancement of efficient and adaptive underwater communication protocols, paving the way for future development in the field.

[1] Agarwal M., Roy A., and Saxena N., “Next Generation 5G Wireless Networks: A Comprehensive Survey,” IEEE Communications Surveys and Tutorials, vol. 18, pp. 1617-1655, 2016. DOI: 10.1109/COMST.2016.2532458

[2] Ahmed F. and Cho H., “A Time-Slotted Data Gathering Medium Access Control Protocol Using Q-Learning for Underwater Acoustic Sensor Networks,” IEEE Access, vol. 9, pp. 48742-48752, 2021. DOI:10.1109/ACCESS.2021.3068407

[3] Alfouzan F., Shahrabi A., Ghoreyshi S., and Boutaleb T., “An Energy-Conserving Collision- Free MAC Protocol for Underwater Sensor Networks,” IEEE Access, vol. 7, pp. 27155-27171, 2019. DOI: 10.1109/ACCESS.2019.2901646

[4] Gunn M. and Koo S., “A Comparative Study of Medium Access Control Protocols for Wireless Sensor Networks,” International Journal of Communications, Network and System Sciences, vol. 2, no. 8, pp. 695-703, 2009. DOI:10.4236/ijcns.2009.28080

[5] Hsu C., Kuo M., Chou C., and Lin K., “The Elimination of Spatial-Temporal Uncertainty in Underwater Sensor Networks,” IEEE/ACM Transactions on Networking, vol. 21, no. 4, pp. 1229-1242, 2013. DOI:10.1109/TNET.2012.2220155

[6] Hsu C., Lai K., Chou C., and Lin K., “ST-MAC: Spatial-Temporal MAC Scheduling for Underwater Sensor Networks,” in Proceedings of the IEEE INFOCOM, Rio De Janeiro, pp. 1827- 1835, 2009. 168 The International Arab Journal of Information Technology, Vol. 22, No. 1, January 2025 DOI:10.1109/INFCOM.2009.5062103

[7] Jouhari M., Ibrahimi K., Tembine H., and Ben- Othman J., “Underwater Wireless Sensor Networks: A Survey on Enabling Technologies, Localization Protocols, and Internet of Underwater Things,” IEEE Access, vol. 7, pp. 96879-96899, 2019. DOI:10.1109/ACCESS.2019.2928876

[8] Kredo K., Djukic P., and Mohapatra P., “Stump: Exploiting Position Diversity in the Staggered Time Underwater MAC Protocol,” in Proceedings of the IEEE INFOCOM, pp. 2961-2965, 2009. DOI: 10.1109/INFCOM.2009.5062267

[9] Matignon L., Laurent G., and Fort-Piat N., “Hysteretic Q-Learning: An Algorithm for Decentralized Reinforcement Learning in Cooperative Multi-Agent Teams,” in Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems, San Diego, pp. 64- 69, 2007. DOI: 10.1109/IROS.2007.4399095

[10] Noh Y., Wang P., Lee U., Torres D., and Gerla M., “Dots: A Propagation Delay-Aware Opportunistic Mac Protocol for Underwater Sensor Networks,” in Proceedings of the 18th IEEE International Conference on Network Protocols, Kyoto, pp. 183-192, 2010. DOI:10.1109/ICNP.2010.5762767

[11] Park S., Mitchell P., and Grace D., “Reinforcement Learning Based MAC Protocol (UW-ALOHA-Q) for Underwater Acoustic Sensor Networks,” IEEE Access, vol. 7, pp. 165531-165542, 2019. DOI:10.1109/ACCESS.2019.2953801

[12] Peleato B. and Stojanovic M., “Distance Aware Collision Avoidance Pro-Tool for Ad-Hoc Underwater Acoustic Sensor Networks,” IEEE Communications Letters, vol. 11, no. 12, pp. 1025-1027, 2007. DOI:10.1109/LCOMM.2007.071160

[13] Ramakrishnan D. and Radhakrishnan K., “Applying Deep Convolutional Neural Network (DCNN) Algorithm in the Cloud Autonomous Vehicles Traffic,” The International Arab Journal of Information Technology, vol. 19, no. 2, pp. 186- 194, 2022. https://doi.org/10.34028/iajit/19/2/5

[14] Sendra S., Lloret J., Jimenez J., and Parra L., “Underwater Acoustic Modems,” IEEE Sensors Journal, vol. 16, no. 11, pp. 4063-4071, 2016. DOI: 10.1109/JSEN.2015.2434890

[15] Tomovic S. and Radusinovic I., “DR-ALOHA-Q: A Q-Learning-Based Adaptive MAC Protocol for Underwater Acoustic Sensor Networks,” Sensors, vol. 23, no. 9, pp. 1-22, 2023. https://doi.org/10.3390/s23094474

[16] Vieira L., Kong J., Lee U., and Gerla M., “Analysis of Aloha Protocols for Underwater Acoustic Sensor Networks,” https://www.semanticscholar.org/paper/Analysis-of- Aloha-Protocols-for-Underwater-Acoustic- VieiraKong/095862fcbbbde81ba0cbb51831276e8c0 e70accbJ, Last Visited, 2024.

[17] Yackoski J. and Shen C., “UW-Flash: Achieving High Channel Utilization in A Time-Based Acoustic Mac Protocol,” in Proceedings of the 3rd International Workshop on Underwater Networks, pp. 59-66, 2008. DOI: 10.1145/1410107.1410119

[18] Zhuo X., Qu F., Yang H., Wei Y., Wu Y., and Li J., “Delay and Queue Aware Adaptive Scheduling-Based MAC Protocol for Underwater Acoustic Sensor Networks,” IEEE Access, vol. 7, pp. 56263-56275, 2019. DOI:10.1109/ACCESS.2019.2913209