Identification of Text Relevance in Service Desk Systems using Machine Learning Classifiers

Resumo

Service Desk systems form a wide source of useful information for organizations, which consists of
historical support requests. Such information can serve as a reference for responding future requests. Standardized
search tools, such as keyword searches in support request histories, are infeasible in large datasets and may provide
answers unrelated to a problem of interest. This manuscript aims to compare the performance of machine learning
algorithms in classifying support requests as relevant or not. We define as relevant the support requests that have

the potential to serve as a basis for responding to others. We will develop a filter to remove non-relevant informa-
tion from the dataset of historical support requests to provide a finite low-cardinality set of recommendations for

future support and assistance. In the performed tests, Naive-Bayes, Adaptive Boosting, Random Forest, Stochastic
Gradient Descent, Logistic Regression, Support Vector Machine, and Light Gradient Boosting Machine classifiers
were used. The classifier with the best performance (Random Forest) presented maximum average accuracy close
to 80%, and recall, F1-score, and AUC values all greater than 80%.