Comparison of Classification Methods for Email Spam Detection

Abstract

Email spam remains a persistent cybersecurity problem, since unsolicited messages waste user time and often deliver phishing or malicious content. Prior comparative studies of conventional spam classifiers rarely state clearly whether their preprocessing avoids leakage between training and test data, or whether reported metrics come from a held-out test set or from cross-validation. This study addresses that gap by comparing five conventional classifiers — Naïve Bayes, Logistic Regression, Decision Tree, Random Forest, and Support Vector Machine (SVM) — under a pipeline in which the 80:20 train-test split is performed first, TF-IDF is fit only on the training partition, and SMOTE oversampling is applied only to the training data, so the test set stays unseen during model development. A public Kaggle email dataset of 5,157 messages (4,516 legitimate, 641 spam) was used, with spam explicitly defined as the positive class. On the untouched test set, SVM achieved the best overall performance (98.55% accuracy, 98.40% precision, 90.44% recall, 94.25% F1-score), followed by Random Forest (98.16% accuracy, 97.56% precision), while Logistic Regression obtained the highest recall (95.59%). Comparing these results with five-fold cross-validation on the already-oversampled training data revealed a large optimistic gap — up to about 30 points in precision for Naïve Bayes — demonstrating why resampling must be repeated inside each fold rather than applied once beforehand. The findings show that reported performance depends strongly on how resampling interacts with data splitting, and provide a transparent, leakage-aware baseline for future spam-detection research.

GS Cited Analysis