The International Arab Journal of Information Technology (IAJIT)

..............................
..............................
..............................


TSRL: Boosting IoT Edge Time Series Database Scalability through Lightweight Rate Limiter

In the Internet of Things, edge computing decreases the latency between things and the central computing service commonly deployed on the cloud. However, as edge devices are located at the edge and closer to things, the type and scale of hardware capabilities are below the commodity hardware. This resulted in limited, in terms of computational power, deployments of Time-Series Databases (TSDB). Consequently, these TSDB do not have enough buffer space to handle an increase in data ingestion rates, causing a high loss rate of data points and sometimes database crashes. In this work, we introduce a lightweight Time-Series Rate Limiter (TSRL) deployed between the clients and the TSDB to boost its scalability and decrease the loss rate due to higher ingestion rates. A buffering of requests and a re-buffering of failed requests increase the chances of data points being successfully inserted. The proposed TSRL is modular, meaning it can be deployed without any changes to the existing TSDB engine. To evaluate the proposed solution, the IoTBenchmark tool is used to compare the performance of InfluxDB with and without a TSRL. The results demonstrate significant performance improvements across multiple metrics. It shows an increase in successfully ingested points and overall throughput, alongside a decrease in both median latency and peak memory usage. The system's scalability also improved with more allocated TSDB memory and with a higher number of concurrent clients. Additionally, an enhancement in query performance is observed.


[1] Abendroth D., Eckel M., and Killat U., “Solving the Trade-Off Between Fairness and Throughput: Token Bucket and Leaky Bucket-Based Weighted Fair Queueing Schedulers,” International Journal of Electronics and Communications, vol. 60, no. 5, pp. 404-407, 2006. https://doi.org/10.1016/j.aeue.2005.04.002

[2] Ahmad K. and Ansari M., Hands-on InfluxDB, NoSQL, Chapman and Hall/CRC, 2017.

[3] Alam T., “Cloud-Based IoT Applications and Their Roles in Smart Cities,” Smart Cities, vol. 4, no. 3, pp. 1196-1219, 2021. DOI: 10.3390/smartcities4030064

[4] Bader A., Kopp O., and Falkenthal M., “Survey and Comparison of Open Source Time Series Databases,” in Proceedings of the Datenbanksysteme für Business, Technologie und Web Workshopband, Bonn, pp. 249-268, 2017. https://dl.gi.de/items/1d3c3186-8d50-4b51-b097- 7695027076fd 


[5] Carvalho G., Cabral B., Pereira V., and Bernardino J., “Edge Computing: Current Trends, Research Challenges and Future Directions,” Computing, vol. 103, pp. 993-1023, 2021. DOI:10.1007/s00607-020-00896-5

[6] Chalmers K. and Pedersen J., “Analysing Concurrent Queues Using CSP: Examining Java’s ConcurrentLinkedQueue,” Software, vol. 4, no. 3, pp.15, 2025. https://doi.org/10.3390/software4030015

[7] Dhanagari M., “Scaling with MongoDB: Solutions for Handling Big Data in Real-Time,” Journal of Computer Science and Technology Studies, vol. 6, no. 5, pp. 246-264, 2024. DOI: 10.32996/jcsts.2024.6.5.20

[8] El-Naggar A., Ghanem S., and Nagi M., “Edge- Level Rate Limiting in Influx DB: Enhancing Performance and Scalability,” in Proceedings of the International Conference on Computer and Communication Systems, Chengdu, pp. 191-196, 2025. https://ieeexplore.ieee.org/document/11069899

[9] Google Cloud, Compute Engine Machine Families, https://cloud.google.com/compute/docs/machine- resource, Last Visited, 2025.

[10] Google, Firebase Realtime Database limits, https://firebase.google.com/docs/database/usage/l imits, Last Visited, 2025.

[11] Gubbi J., Buyya R., Marusic S., and Palaniswami M., “Internet of Things (IoT): A Vision, Architectural Elements, and Future Directions,” Future Generation Computer Systems, vol. 29, no. 7, pp. 1645-1660, 2013. DOI: 10.1016/j.future.2013.01.010

[12] Hadjichristofi C., Diochnos S., Andresakis K., and Vescoukis V., “Using Time-Series Databases for Energy Data Infrastructures,” Energies, vol. 17, no. 21, pp. 1-23, 2004. https://doi.org/10.3390/en17215478

[13] Hao Y., Qin X., Chen Y., Li Y., and et al., “TS- Benchmark: A Benchmark for Time Series Databases,” IEEE 37th International Conference on Data Engineering, Chania, pp. 588-599, 2021. https://doi.org/10.1109/ICDE51399.2021.00057

[14] Holopainen M., Monitoring Container Environment with Prometheus and Grafana, Bachelor’s Thesis, Metropolia University of Applied Sciences, 2021. https://www.theseus.fi/bitstream/handle/10024/4 97467/Holopainen_Matti.pdf

[15] Hou Z., Huang Q., Song Y., Fan S., and Nie X., “A Memory Optimization Method for High Cardinality Based on InfluxDB,” in Proceedings of the 7th International Conference on Big Data Technologies, New York, pp. 51-57, 2024. https://doi.org/10.1145/3698300.3698312

[16] InfluxData, InfluxDB Cloud Pricing and Plans, https://www.influxdata.com/pricing/, Last Visited, 2025.

[17] Ishtaiwi A., Nabot A., Alzubi O., Ramadan A., and et al., “HECOCP: Hybrid Edge-Cloud Optimistic Concurrency Protocol for Sensor Data Transactional Services,” The International Arab Journal of Information Technology, vol. 22, no. 4, pp. 756-768, 2025. https://doi.org/10.34028/iajit/22/4/10

[18] Jayakumar H., Lee K., Lee W., Raha A., and et al., “Powering the Internet of Things,” in Proceedings of the International Symposium on Low Power Electronics and Design, La Jolla, pp. 375-380, 2014. https://doi.org/10.1145/2627369.2631644

[19] Karkouch A., Mousannif H., Moatassime H., and Noel T., “Data Quality in Internet of Things: A State-of-the-Art Survey,” Journal of Network and Computer Applications, vol. 73, pp. 57-81, 2016. https://doi.org/10.1016/j.jnca.2016.08.002

[20] Liu R., Yuan J., and Huang X., “Benchmark Time Series Database with IoTDB-Benchmark for IoT Scenarios,” CoRR, vol. abs/1901.08304, 2019. http://arxiv.org/abs/1901.08304

[21] McBride B. and Reynolds D., Survey of Time Series Database Technology, UK Centre for Ecology and Hydrology, Wallingford, 2020. https://nora.nerc.ac.uk/id/eprint/527832/

[22] Namiot D., “Time Series Databases,” in Proceedings of the 17th International Conference Data Analytics and Management in Data Intensive Domains, Obninsk, pp. 132-137, 2015. file:///C:/Users/user/Downloads/paper20.pdf

[23] Nasar M. Abu Kausar M., “Suitability of Influxdb Database for IoT Applications,” International Journal of Innovative Technology and Exploring Engineering, vol. 8, no. 10, pp. 1850-1857, 2019. DOI: 10.35940/ijitee.J9225.0881019

[24] Nawab I. and Kulkarni S., “Toward Next- Generation Distributed Rate-Limiters,” in Proceedings of the IEEE/ACM 23rd International Symposium on Cluster, Cloud and Internet Computing Workshops, Bangalore, pp. 354-356, 2023. DOI: 10.1109/CCGridW59191.2023.00083

[25] Ng M. and Huang Z., “Data-Mining Massive Time Series Astronomical Data: Challenges, Problems and Solutions,” Information and Software Technology, vol. 41, no. 9, pp. 545-556, 1999. https://doi.org/10.1016/S0950-5849(99)00018-X

[26] Niswar M., Safruddin R., Bustamin A., and Aswad I., “Performance Evaluation of Microservices Communication with REST, GraphQL, and gRPC,” International Journal of Electronics and Telecommunication, vol. 70, no. 2, pp. 429-436, 2024. DOI: 10.24425/ijet.2024.149562

[27] Peter O., Pradhan A., and Mbohwa C., “Industrial Internet of Things (IIoT): Opportunities, Challenges, and Requirements in Manufacturing Businesses in Emerging Economies,” Procedia Computer Science, vol. 217, pp. 856-865, 2023. https://doi.org/10.1016/j.procs.2022.12.282

[28] Shah B., Jat P., and Sashidhar K., “Performance Study of Time Series Databases,” arXiv Preprint, vol. arXiv:2208.13982v1, pp. 1-14, 2022. https://arxiv.org/abs/2208.13982

[29] Tan L., Dong X., Tu N., and Hong J., “IntDB: Towards a Spatiotemporal Database for In-Band Network Telemetry,” in Proceedings of the 25th Asia-Pacific Network Operations and Management Symposium Kaohsiung, Taiwan, pp. 1-6, 2025. DOI: 10.23919/APNOMS67058.2025.11181327

[30] Tripathi P., Miraz M., and Joshi S., “Comparative Analysis of MongoDB and InfluxDB for Time Series Data Management in IoT Environments: A Study on Performance, Scalability, and Concurrency,” in Proceedings of the International Conference on Computing, Networking, Telecommunications and Engineering Sciences Applications, Zagreb, pp. 39-42, 2023. https://ieeexplore.ieee.org/document/10384962

[31] Wang C., Qiao J., Huang X., Song S., and et al., “Apache IoTDB: A Time Series Database for Large Scale IoT Applications,” ACM Transactions on Database Systems, vol. 50, no. 2, pp. 1-45, 2025. https://doi.org/10.1145/3726523

[32] Wang S., Sun Z., Hu C., Li C., and et al., “MatrixGate: A High-performance Data Ingestion Tool for Time-Series Databases,” arXiv Preprint, vol. arXiv:2406.05462, pp. 1-13, 2024. https://arxiv.org/html/2406.05462v1

[33] Wu H., Shang Z., Peng G., and Wolter K., “A Reactive Batching Strategy of Apache Kafka for Reliable Stream Processing in Real-Time,” in Proceedings of the IEEE 31st International Symposium on Software Reliability Engineering, Coimbra, pp. 207-217, 2020. DOI: 10.1109/ISSRE5003.2020.00028

[34] Zhang L., Jeong D., and Lee S., “Data Quality Management in the Internet of Things,” Sensors, vol. 21, no. 17, pp. 5834, 2021. https://doi.org/10.3390/s21175834

[35] Zhu X., Nie X., and Liu J., “Time Series Database Optimization Based on InfluxDB,” in Proceedings of the International Conference on Power, Electrical Engineering, Electronics and Control, Athens, pp. 879-885, 2023. https://doi.org/10.1109/PEEEC60561.2023.00172