Performance Evaluation of a Pretrained BERT Model for Automatic Text Classification

Authors : Sercan ÇEPNİ, Amine Gonca TOPRAK, Aslı YATKINOĞLU, Öykü Berfin MERCAN, Şükrü OZAN

Pages : 27-35

View : 76 | Download : 46

Publication Date : 2023-06-30

Article Type : Research Paper

Abstract :This study presents a pre-trained BERT model application on texts that are extracted from website URLs automatically to classify texts according to the industry. With the aim of doing so, the related dataset is first obtained from different kinds of websites by web scraping. Then, the dataset is cleaned and labeled with the relevant industries among 42 different categories. The pre-trained BERT model which was trained on 101.000 advertisement texts in one of our previous ad text classification studies is used to classify texts. Classification performance metrics are then used to evaluate the pre-trained BERT model on the test set and 0.98 average accuracy and 0.67 average F1 score for different 12 categories are obtained. The method can be used to test the compatibility of texts to be used in online advertising networks with the advertiser\`s industry. In this way, the suitability of the texts, which is an important component in determining the quality of online advertising, within the industry will be tested automatically.
Keywords : digital marketing, ad text, natural language processing, text classification, bidirectional encoder representations from transformers