Abstract The constraints are the most important part of many optimization problems. The metaheuristic algorithms are designed for solving continuous unconstrained optimization problems initially. The constraint handling methods are integrated into these algorithms for solving constrained optimization problems. Penalty approaches are not only the simplest way but also as effective as other constraint handling techniques. In literature, there are many penalty approaches and these are grouped as static, dynamic and adaptive. In this study, we collect them and discuss the key benefits and drawbacks of these techniques. Tree-Seed Algorithm (TSA) is a recently developed metaheuristic algorithm, and in this study, nine different penalty approaches are integrated with the TSA. The performance of these approaches is analyzed on well-known thirteen constrained benchmark functions. The obtained results are compared with state-of-art algorithms like Differential Evolution (DE), Particle Swarm Optimization (PSO), Artificial Bee Colony (ABC), and Genetic Algorithm (GA). The experimental results and comparisons show that TSA outperformed all of them on these benchmark functions.

References

[1] Afshar M., “Penalty Adapting Ant Algorithm: Application to Pipe Network Optimization,” Engineering Optimization, vol. 40, no. 10, pp. 969-987, 2008.

[2] Altun A. and Şahman M., “Cost Optimization of Mixed Feeds with the Particle Swarm Optimization Method,” Neural Computing and Applications, vol. 22, no. 2, pp. 383-390, 2013.

[3] Asmaran M., Sharieh A., and Mahafzah B., “Chemical Reaction Optimization Algorithm to Find Maximum Independent Set in a Graph,” International Journal of Advanced Computer Science and Applications, vol. 10, no. 9, 2019.

[4] Babaeizadeh S. and Ahmad R., “Enhanced Constrained Artificial Bee Colony Algorithm for Optimization Problems,” The International Arab Journal of Information Technology, vol. 14, no. 2, pp. 246-253, 2017.

[5] Babalik A., Cinar A., and Kiran M., “A Modification of Tree-Seed Algorithm Using Deb’s Rules for Constrained Optimization,” Applied Soft Computing, vol. 63, pp. 289-305, 2018.

[6] Bäck T., Fogel D., and Michalewicz Z., Handbook of Evolutionary Computation, Oxford University Press, 1997.

[7] Ben Hadj-Alouane A. and Bean J., “A Genetic Algorithm for the Multiple-Choice Integer Program,” Operations Research, vol. 45, no. 1, pp. 92-101, 1997.

[8] Carlson S. and Shonkwiler R., “Annealing A Genetic Algorithm over Constraints,” in Proceedings of Systems, Man, and Cybernetics, IEEE International Conference on, San Diego, pp. 3931-3936, 1998.

[9] Chehouri A., Younes R., Perron J., and Ilinca A., “A Constraint-Handling Technique for Genetic Algorithms Using A Violation Factor,” Journal of Computer Science, vol. 12, no. 7, pp. 350-362, 2016.

[10] Cinar A. and Kiran M., “A Cuda-based Parallel Programming Approach to Tree-Seed Algorithm,” MSc Thesis, Selcuk University, 2016. 806 The International Arab Journal of Information Technology, Vol. 17, No. 5, September 2020

[11] Cinar A. and Kiran M., “Boundary Conditions In Tree-Seed Algorithm: Analysis of The Success of Search Space Limitation Techniques In Tree- Seed Algorithm,” in Proceedings of International Conference on Computer Science and Engineering, Antalya, pp. 571-576, 2017.

[12] Cinar A. and Kiran M., “A Parallel Version of Tree-Seed Algorithm (TSA) within CUDA Platform,” in Proceedings of International Scientific Conference on Applied Sciences, At Antalya, pp. 174-178, 2016.

[13] Cinar A. and Kiran M., “Similarity and Logic Gate-Based Tree-Seed Algorithms for Binary Optimization,” Computers and Industrial Engineering, vol. 115, no. pp. 631-646, 2018.

[14] Cinar A., Korkmaz S., and Kiran M., “A Discrete Tree-Seed Algorithm for Solving Symmetric Traveling Salesman Problem,” Engineering Science and Technology, an International Journal, vol. 22, no. 6, pp. 1169-1200, 2019.

[15] Coello C., “Theoretical and Numerical Constraint-Handling Techniques Used With Evolutionary Algorithms: A Survey of the State of the Art,” Computer Methods in Applied Mechanics and Engineering, vol. 191, no. 11, pp. 1245-1287, 2002.

[16] Coello C., “Use of A Self-Adaptive Penalty Approach for Engineering Optimization Problems,” Computers in Industry, vol. 41, no. 2, pp. 113-127, 2000.

[17] Coit D. and Smith A., “Penalty Guided Genetic Search for Reliability Design Optimization,” Computers and Industrial Engineering, vol. 30, no. 4, pp. 895-904, 1996.

[18] De Castro Rodrigues M., Guimarães S., and De Lima B., “E-BRM: A Constraint Handling Technique to Solve Optimization Problems With Evolutionary Algorithms,” Applied Soft Computing, vol. 72, pp. 14-29, 2018.

[19] Deb K., “An Efficient Constraint Handling Method for Genetic Algorithms,” Computer Methods in Applied Mechanics and Engineering, vol. 186, no. 2-4, pp. 311-338, 2000.

[20] Farmani R. and Wright J., “Self-Adaptive Fitness Formulation for Constrained Optimization,” IEEE Transactions on Evolutionary Computation, vol. 7, no. 5, pp. 445-455, 2003.

[21] Hamida S. and Schoenauer M., “An Adaptive Algorithm for Constrained Optimization Problems,” in Proceedings of International Conference on Parallel Problem Solving from Nature, Paris, pp. 529-538, 2000.

[22] Hamida S. and Schoenauer M., “ASCHEA: New Results Using Adaptive Segregational Constraint Handling,” in Proceedings of Evolutionary Computation, CEC'02, Honolulu, pp. 884-889, 2002.

[23] Homaifar A., Qi C., and Lai S., “Constrained Optimization Via Genetic Algorithms,” Simulation, vol. 62, no. 4, pp. 242-253, 1994.

[24] Joines J. and Houck C., “On The Use of Non- Stationary Penalty Functions To Solve Nonlinear Constrained Optimization Problems with GA's,” in Proceedings of Evolutionary Computation, IEEE World Congress on Computational Intelligence, Orlando, pp. 579-584, 1994.

[25] Kazarlis S. and Petridis V., “Varying Fitness Functions in Genetic Algorithms: Studying the Rate of Increase of the Dynamic Penalty Terms,” in Proceedings of International Conference on Parallel Problem Solving from Nature, Amsterdam, pp. 211-220, 1998.

[26] Kiran M., in Intelligent and Evolutionary Systems, Springer, 2016.

[27] Kiran M., “TSA: Tree-Seed Algorithm for Continuous Optimization,” Expert Systems with Applications, vol. 42, no. 19, pp. 6686-6698, 2015.

[28] Kuri-Morales A. and Gutiérrez-García J., “Penalty Function Methods for Constrained Optimization With Genetic Algorithms: A Statistical Analysis,” in Proceedings of Mexican International Conference on Artificial Intelligence, Yucatan, pp. 108-117, 2002.

[29] Lemonge A. and Barbosa H., “An Adaptive Penalty Scheme for Genetic Algorithms in Structural Optimization,” International Journal for Numerical Methods in Engineering, vol. 59, no. 5, pp. 703-736, 2004.

[30] Liang J., Runarsson T., Mezura-Montes E., Clerc M., Suganthan P., Coello C., and Deb K., “Problem Definitions and Evaluation Criteria for The CEC 2006 Special Session on Constrained Real-Parameter Optimization,” Journal of Applied Mechanics, vol. 41, no. 8, pp. 8-31, 2006.

[31] Liu J., Teo K., Wang X., and Wu C.,“An Exact Penalty Function-Based Differential Search Algorithm for Constrained Global Optimization,” Soft Computing, vol. 20, no. 4, pp. 1305-1313, 2016.

[32] Mallipeddi R. and Suganthan P., “Differential Evolution With Ensemble of Constraint Handling Techniques for Solving CEC 2010 Benchmark Problemsm,” in Proceedings of Evolutionary Computation, IEEE Congress on, Barcelona, pp. 1-8, 2010.

[33] Masadeh R., Mahafzah B., and Sharieh A., “Sea Lion Optimization Algorithm,” International Journal of Advanced Computer Science and Applications, vol. 10, no. 5, pp. 388-395, 2019.

[34] Masadeh R., Sharieh A., and Mahafzah B., “Humpback whale Optimization Algorithm Based on Vocal Behavior for Task Scheduling in Cloud Computing,” International Journal of Advanced Science and Technology, vol. 13, no. 3, The Performance of Penalty Methods on Tree-Seed Algorithm for Numerical ... 807 pp. 121-140, 2019.

[35] Michalewicz Z. “Genetic Algorithms, Numerical Optimization, and Constraints,” in Proceedings of 6th International Conference on Genetic Algorithms, pp. 151-158, 1995.

[36] Michalewicz Z. and Attia N., “Evolutionary optimization of constrained problems,” in Proceedings of 3rd annual conference on Evolutionary Programming, pp. 98-108, 1994.

[37] Michalewicz Z. and Schoenauer M., “Evolutionary Algorithms for Constrained Parameter Optimization Problems,” Evolutionary Computation, vol. 4, no. 1, pp. 1-32, 1996.

[38] Morales A. and Quezada C., “A Universal Eclectic Genetic Algorithm for Constrained Optimization,” in Proceedings of 6th European Congress on Intelligent Techniques and Soft Computing, pp. 518-522, 1998.

[39] Powell D. and Skolnick M., “Using Genetic Algorithms in Engineering Design Optimization with Non-Linear Constraints,” in Proceedings of 5th International Conference on Genetic Algorithms, San Francisco, pp. 424-431, 1993.

[40] Runarsson T. and Yao X., “Stochastic Ranking for Constrained Evolutionary Optimization,” IEEE Transactions on Evolutionary Computation, vol. 4, no. 3, pp. 284-294, 2000.

[41] Schoenauer M. and Xanthakis S., “Constrained GA optimization,” in Proceedings of the 5th International Conference on Genetic Algorithms Urbana Champaign, pp. 573-580, 1993.

[42] Smith A. and Coit D., C5.2 of Handbook of Evolutionary Computation in Handbook of Evolutionary Computation, A Joint Publication of Oxford University Press and Institute of Physics Publishing, 1996.

[43] Smith A. and Tate D., “Genetic Optimization Using A Penalty Function,” in Proceedings of 5th International Conference on Genetic Algorithms, San Francisco, pp. 499-505, 1993.

[44] Şahman, M., Çunkaş M., İnal Ş., İnal F., Coşkun B., and Taşkiran U., “Cost Optimization of Feed Mixes by Genetic Algorithms,” Advances in Engineering Software, vol. 40, no. 10, pp. 965- 974, 2009.

[45] Takahama T. and Sakai S., “Constrained Optimization by the Ε Constrained Differential Evolution with an Archive and Gradient-Based Mutation,” in Proceedings of IEEE Congress on Evolutionary Computation, Barcelona, pp. 1-9, 2010.

[46] Tessema B. and Yen G., “A Self Adaptive Penalty Function Based Algorithm for Constrained Optimization,” in Proceedings of International Conference on Evolutionary Computation, Vancouver, 246-253, 2006.

[47] Yeniay Ö., “Penalty Function Methods for Constrained Optimization with Genetic Algorithms,” Mathematical and computational Applications, vol. 10, no. 1, pp. 45-56, 2005.

[48] Yuchi M. and Kim J., “Evolutionary Algorithm Using Feasibility-Based Grouping for Numerical Constrained Optimization Problems,” Applied Mathematics and Computation, vol. 175, no. 2, pp. 1298-1319, 2006. Ahmet Cinar was born in Turkey, in 1986. He graduated from the Department of Computer Engineering, Selçuk University, Turkey, in 2009. He received the M.S. degree from the Computer Engineering Department, Selcuk University in 2016. He is currently pursuing his Ph.D. at the Konya Technical University. He is a Research Staff at the Department of Computer Engineering, Selçuk University. His current research interests include swarm intelligence, nature-inspired algorithms, machine learning and artificial intelligence systems. Mustafa Kiran received the B.S. and Ph.D. degrees in computer engineering from the Institute of Natural and Applied Sciences, Selcuk University, Konya, Turkey, in 2010 and 2014, respectively. He is currently an Associate Professor at the Computer Engineering Department, Konya Technical University. His current research interests include swarm intelligence, evolutionary algorithms, and their real-world applications.