Optimizing Network Traffic Classification Models with a Hybrid Approach for Large-Scale Data
DOI:
https://doi.org/10.31154/cogito.v11i2.966.281-294Keywords:
Network Traffic Classification, Hybrid, Autoencoder, CNN, XGBoostAbstract
The escalating threat of cyberattacks necessitates the development of intrusion detection models that are both accurate and computationally efficient for large-scale network traffic. To address this issue, this study proposes a hybrid approach combining Autoencoder, Convolutional Neural Network (CNN), and XGBoost as an adaptive and lightweight solution for network traffic classification. The key contribution of this research lies in the design of a multi-stage pipeline that performs dimensionality reduction, feature extraction, and final classification. The model was evaluated using the Moore Dataset, which contains complex and high-dimensional network traffic data. The experimental results indicate that the proposed hybrid model achieved a classification accuracy of 99.20% with a testing time of only 0.09 seconds. Furthermore, the pipeline significantly reduced computational load compared to single CNN or XGBoost models. These findings demonstrate that the hybrid approach not only offers high classification performance but also enhances scalability and efficiency, making it suitable for real-world implementation in modern network security systems. Overall, the proposed model presents a promising and practical solution for advancing future intrusion detection systems.References
A. Gofur, R. F. Aji, and H. Kurniawan, “Pengukuran kesadaran keamanan informasi pegawai: Studi kasus PT Meshindo Jayatama,” J. Tek. Inf. dan Ilmu Komput., vol. 11, no. 2, pp. 315–320, Apr. 2024, doi: 10.25126/jtiik.20241128106.
K. Inayah and K. Ramli, “Analisis kinerja intrusion detection system berbasis algoritma random forest menggunakan dataset unbalanced honeynet BSSN,” J. Tek. Inf. dan Ilmu Komput., vol. 4, no. 11, pp. 867–876, Aug. 2024, doi: 10.25126/jtiik1148911.
F. Hering, O. Hinz, J. Pfeiffer, and W. v. d. Aalst, “The Damocles sword of cyber attacks,” Bus. Inf. Syst. Eng., vol. 67, pp. 141–147, Feb. 2025, doi: 10.1007/s12599-025-00933-7.
R. Sham, W. Christian, D. Daud, and M. H. Miraz, “Modelling a sustainable digital economy among e-commerce users in Indonesia,” in Proc. 13th AMER Int. Conf. Quality of Life (AicQoL2025), Pangkor Island, Malaysia, 2025. [Online]. Available: www.e-iph.co.uk (accessed Jun. 15, 2025).
F. Fachri, “Optimasi keamanan web server terhadap serangan brute-force menggunakan penetration testing,” J. Tek. Inf. dan Ilmu Komput., vol. 10, no. 1, pp. 51–58, Feb. 2023, doi: 10.25126/jtiik.2023105872.
A. O. Salau and M. M. Beyene, “Software defined networking based network traffic classification using machine learning techniques,” Sci. Rep., vol. 14, no. 20060, Aug. 2024, doi: 10.1038/s41598-024-70983-6.
I. M. A. Surya, T. A. Cahyanto, and L. A. Muharom, “Deep learning dengan teknik early stopping untuk mendeteksi malware pada perangkat IoT,” J. Tek. Inf. dan Ilmu Komput., vol. 12, no. 1, pp. 21–30, Feb. 2025, doi: 10.25126/jtiik.2025128267.
J. H. Kalwar and S. Bhatti, “Deep learning approaches for network traffic classification in the Internet of Things (IoT): A survey,” arXiv preprint, arXiv:2402.00920. [Online]. Available: https://arxiv.org/abs/2402.00920 (accessed May 27, 2025).
E. Mushtaq, F. Shahid, and A. Zameer, “A comparative study of machine learning models for malware detection,” in Proc. 2022 19th Int. Bhurban Conf. Appl. Sci. Technol. (IBCAST), Islamabad, Pakistan, Aug. 16–20, 2022, pp. 677–681, doi: 10.1109/IBCAST54850.2022.999054.
W. Chimphlee and S. Chimphlee, “Hyperparameters optimization XGBoost for network intrusion detection using CSE-CICIDS 2018 dataset,” IAES Int. J. Artif. Intell., vol. 13, no. 1, pp. 817–826, 2024, doi: 10.11591/ijai.v13.i1.pp817826.
E. L. Elsedimy and S. M. M. Abohashish, “An intelligent hybrid approach combining fuzzy C-means and the sperm whale algorithm for cyber attack detection in IoT networks,” Sci. Rep., vol. 15, no. 1005, 2025, doi: 10.1038/s41598-024-79230-4.
R. Budiarto and Y. D. Kuntjoro, “Analisis perilaku pendeteksian entitas serangan untuk internal menggunakan kombinasi model prediksi memori dan metode PCA,” J. Tek. Inf. dan Ilmu Komput., vol. 10, no. 6, pp. 1223–1232, 2023, doi: 10.25126/jtiik.2023107123.
Sulandri, A. Basuki, and F. A. Bachtiar, “Metode deteksi intrusi menggunakan algoritma extreme learning machine dengan correlation-based feature selection,” J. Tek. Inf. dan Ilmu Komput., vol. 8, no. 1, pp. 103–110, 2021, doi: 10.25126/jtiik.202183358.
A. Sanmorino, S. Suryati, R. Gustriansyah, S. Puspasari, and N. Ariati, “Feature extraction vs fine-tuning for cyber intrusion detection model,” J. Infotel, vol. 16, no. 2, pp. 302–315, 2024, doi: 10.20895/infotel.v16i2.996.
R. H. Serag, M. S. Abdalzaher, H. A. A. Elsayed, and M. Sobh, “Software defined network traffic classification for QoS optimization using machine learning,” J. Netw. Syst. Manag., vol. 33, no. 41, 2025, doi: 10.1007/s10922-025-09911-6.
A. Kumar, R. Radhakrishnan, M. Sumithra, P. Kaliyaperumal, B. Balusamy, and F. Benedetto, “A scalable hybrid autoencoder–extreme learning machine framework for adaptive intrusion detection in high-dimensional networks,” Future Internet, vol. 17, no. 5, p. 221, 2025, doi: 10.3390/fi17050221.
H. Liu and H. Wang, “Real-time anomaly detection of network traffic based on CNN,” Symmetry, vol. 15, no. 6, p. 1205, 2023, doi: 10.3390/sym15061205.
A. Sharma, E. Vans, D. Shigemizu, K. A. Boroevich, and T. Tsunoda, “DeepInsight: A methodology to transform a non-image data to an image for convolution neural network architecture,” Sci. Rep., vol. 9, p. 11399, Aug. 2019, doi: 10.1038/s41598-019-47765-6.
T. Chen and C. Guestrin, “XGBoost: A scalable tree boosting system,” arXiv preprint, arXiv:1603.02754v3, Jun. 10, 2016. [Online]. Available: https://arxiv.org/abs/1603.02754 (accessed Jun. 30, 2025).
J. Cui, L. Bai, G. Li, Z. Lin, and P. Zeng, “Semi-2DCAE: A semi-supervision 2D CNN autoencoder model for feature representation and classification of encrypted traffic,” PeerJ Comput. Sci., vol. 9, p. e1635, Nov. 2023, doi: 10.7717/peerj-cs.1635.
Y. Song, S. Hyun, and Y.-G. Cheong, “Analysis of autoencoders for network intrusion detection,” Sensors, vol. 21, p. 4294, 2021, doi: 10.3390/s21134294.
W. Wenxu, F. S. Sabrina, J. Jang-Jaccard, A. Singh, and Y. Wei, “Improving performance of autoencoder-based network anomaly detection on NSL-KDD dataset,” IEEE Access, vol. 9, pp. 140136–140146, 2021, doi: 10.1109/ACCESS.2021.3116612.
B. Sae-ang, W. Kumwilaisak, and P. Kaewtrakulpong, “Semi-supervised learning for defect segmentation with autoencoder auxiliary module,” Sensors, vol. 22, p. 2915, 2022, doi: 10.3390/s22082915.
F. S. Nugraha and H. F. Pardede, “Autoencoder untuk sistem prediksi berat lahir bayi,” J. Tek. Inf. dan Ilmu Komput., vol. 9, no. 2, pp. 235–244, Apr. 2022, doi: 10.25126/jtiik.202293868.
I. Ramadhanti, A. Prasetiadi, and I. K. A. Iqsyahiro, “Clothing recommendation and face swap model based on VGG16, autoencoder, and facial landmark points,” J. Tek. Inform., vol. 5, no. 1, pp. 19–29, Feb. 2024, doi: 10.52436/1.jutif.2024.5.1.1016.
Yuliska, D. H. Qudsi, J. H. Lubis, K. U. Syaliman, and N. F. Najwa, “Analisis sentimen pada data saran mahasiswa terhadap kinerja departemen di perguruan tinggi menggunakan convolutional neural network,” J. Tek. Inf. dan Ilmu Komput., vol. 8, no. 5, pp. 1067–1076, Oct. 2021, doi: 10.25126/jtiik.202184842.
F. Y. Santoso, E. Sediyono, and H. D. Purnomo, “Optimalisasi hyper parameter convolutional neural networks menggunakan ant colony optimization,” J. Tek. Inf. dan Ilmu Komput., vol. 11, no. 2, pp. 243–248, Apr. 2024.
C. M. Bachri and W. Gunawan, “Deteksi email spam menggunakan algoritma convolutional neural network (CNN),” J. Eduk. Penelit. Inform., vol. 10, no. 1, pp. 88–94, Apr. 2024.
B. Nugroho and E. Y. Puspaningrum, “Kinerja metode CNN untuk klasifikasi pneumonia dengan variasi ukuran citra input,” J. Tek. Inf. dan Ilmu Komput., vol. 8, no. 3, pp. 533–538, Jun. 2021, doi: 10.25126/jtiik.202184515.
Z. S. Dhahir, “A hybrid approach for efficient DDoS detection in network traffic using CBLOF-based feature engineering and XGBoost,” J. Future Artif. Intell. Technol., vol. 1, no. 2, pp. 174–190, Sep. 2024, doi: 10.62411/faith.2024-33.
D. Kurnia, M. I. Mazdadi, D. Kartini, R. A. Nugroho, and F. Abadi, “Seleksi fitur dengan particle swarm optimization pada klasifikasi penyakit Parkinson menggunakan XGBoost,” J. Tek. Inf. dan Ilmu Komput., vol. 10, no. 5, pp. 1083–1094, Oct. 2023, doi: 10.25126/jtiik.2023107252.
D. Niu, J. Zhang, L. Wang, K. Yan, T. Fu, and X. Chen, “A network traffic anomaly detection method based on CNN and XGBoost,” in Proc. 2020 Chinese Autom. Congr. (CAC), 2020, pp. 1–6, doi: 10.1109/CAC51589.2020.9327030.
A. Moore, D. Zuev, and M. Crogan, “Discriminators for use in flow-based classification,” Tech. Rep. RR-05-13, Dept. Comput. Sci., Queen Mary Univ. London, Aug. 2005.
A. W. Moore and K. Papagiannaki, “Toward the accurate identification of network applications,” in Lect. Notes Comput. Sci., Mar. 2005.
A. W. Moore and D. Zuev, “Internet traffic classification using Bayesian analysis techniques,” in Proc. SIGMETRICS’05, Banff, AB, Canada, Jun. 2005, pp. 50–60.
D. Zuev and A. W. Moore, “Traffic classification using a statistical approach,” in Proc. 6th Passive and Active Meas. Workshop (PAM), Boston, MA, Mar./Apr. 2005.
R. Gustriansyah, N. Suhandi, S. Puspasari, and A. Sanmorino, “Machine learning method to predict the toddlers’ nutritional status,” J. Infotel, vol. 16, no. 1, pp. 32–43, Feb. 2024, doi: 10.20895/infotel.v15i4.988.
M. B. Umair, Z. Iqbal, M. Bilal, J. Nebhen, T. A. Almohamad, and R. M. Mehmood, “An efficient internet traffic classification system using deep learning for IoT,” Comput. Mater. Contin., vol. 71, no. 1, pp. 407–421, 2022, doi: 10.32604/cmc.2022.020727.
S. Dong, “Multi-class SVM algorithm with active learning for network traffic classification,” Expert Syst. Appl., vol. 176, p. 114885, 2021, doi: 10.1016/j.eswa.2021.114885.
S. Zhao, Y. Xiao, Y. Ning, Y. Zhou, and D. Zhang, “An optimized K-means clustering for improving accuracy in traffic classification,” Wirel. Pers. Commun., 2021, doi: 10.1007/s11277-021-08435-x.
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2025 CogITo Smart Journal
This work is licensed under a Creative Commons Attribution 4.0 International License.
Authors who publish with this journal agree to the following terms:- Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under a Creative Commons Attribution License that allows others to share the work with an acknowledgment of the work's authorship and initial publication in this journal.
- Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgment of its initial publication in this journal.
- Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (See The Effect of Open Access).
