COMPARATIVE STUDY OF CLASSIFICATION ALGORITHMS: HOLDOUTS AS ACCURACY ESTIMATION

Authors

  • Debby Erce Sondakh Universitas Klabat

DOI:

https://doi.org/10.31154/cogito.v1i1.2.13-23

Abstract

Penelitian ini bertujuan untuk mengukur dan membandingkan kinerja lima algoritma klasifikasi teks berbasis pembelajaran mesin, yaitu decision rules, decision tree, k-nearest neighbor (k-NN), naïve Bayes, dan Support Vector Machine (SVM), menggunakan dokumen teks multi-class. Perbandingan dilakukan pada efektifiatas algoritma, yaitu kemampuan untuk mengklasifikasi dokumen pada kategori yang tepat, menggunakan metode holdout atau percentage split. Ukuran efektifitas yang digunakan adalah precision, recall, F-measure, dan akurasi. Hasil eksperimen menunjukkan bahwa untuk algoritma naïve Bayes, semakin besar persentase dokumen pelatihan semakin tinggi akurasi model yang dihasilkan. Akurasi tertinggi naïve Bayes pada persentase 90/10, SVM pada 80/20, dan decision tree pada 70/30. Hasil eksperimen juga menunjukkan, algoritma naïve Bayes memiliki nilai efektifitas tertinggi di antara lima algoritma yang diuji, dan waktu membangun model klasiifikasi yang tercepat, yaitu 0.02 detik. Algoritma decision tree dapat mengklasifikasi dokumen teks dengan nilai akurasi yang lebih tinggi dibanding SVM, namun waktu membangun modelnya lebih lambat. Dalam hal waktu membangun model, k-NN adalah yang tercepat namun nilai akurasinya kurang.

Author Biography

Debby Erce Sondakh, Universitas Klabat

Fakultas Ilmu Komputer

References

H.Brucher, G. Knolmayer, and M.A. Mittermayer ., “Document Classification Methods for Organizing Explicit Knowledge”, Proceedings of the 3rd European Conference on Organizational Knowledge, Learning, and Capabilities, Athens, Greece, 2002.

F. Sebastiani, “Machine Learning in Autmated Text Categorization”, ACM Computing Surveys, Vol. 34, No. 1, pp. 1–47, March 2002,.

S. Ramasundaram & S.P. Victor, “Algorithms for Text Categorization: A Comparative Study”, World Applied Sciences Journal, Vol. 22, No.9, pp. 1232-1240, 2013.

E. Leopold & J. Kindermann, “Text Categorization with Support Vector Machines. How to Represent Texts in Input space?”, Machine Learning 46, pp. 423-444, 2002.

M. Ikonomakis, S. Kotsiantis, & V. Tampakas, “Text Classification Using Machine Learning Techniques”, WEAS Transactions on Computers, Vol. 4, No. 8, pp. 966-975, August 2005.

C. Goller, et.al., Automatic Document Classification: A Thorough Evaluation of Various Methods, Proceedings of Internationalen Symposiums Informationsgesellschaft, 2000.

C. C. Aggarwal & C. X. Zhai, “A Survey of Text Classification Algorithms”, in Mining Text Data, Springer Science Business Media, 2012.

A. Bratko & B. Filipié, A Study of Approaches to Semi-structured Document Classification, Technical Report IJS-DP 9015, Josef Stefan Institute, Slovenia, 2004.

Y. Yang. & X. Liu, “A Re-examination of Text Categorization Methods”, Proceedings of SIGIR-99, 22nd ACM International Conference on Research and Development in Information Retrieval, New York, US, pp.42-49 1999,

J.Han & M. Kamber, Data Mining Concepts and Techniques, Academic Press,USA, 2001.

I. H. Witten & Eibe Frank, Data Mining Practical Machine Learning Tools and Techniques, Edisi Kedua, Morgan Kaufmann Publishers, 2005.

Downloads

Published

2016-09-18

How to Cite

Sondakh, D. E. (2016). COMPARATIVE STUDY OF CLASSIFICATION ALGORITHMS: HOLDOUTS AS ACCURACY ESTIMATION. CogITo Smart Journal, 1(1), 13–23. https://doi.org/10.31154/cogito.v1i1.2.13-23

Issue

Vol. 1 No. 1 (2015): CogITo Smart Journal

Section

Articles

License

Authors who publish with this journal agree to the following terms:
  1. Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under a Creative Commons Attribution License that allows others to share the work with an acknowledgment of the work's authorship and initial publication in this journal.
  2. Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgment of its initial publication in this journal.
  3. Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (See The Effect of Open Access).