Título: Tree-Based Models Outperform Transformers for Music Engagement Prediction
Autores: João Victor de Souza Albuquerque, Amauri Holanda Souza Junior
Resumo: Youtube and Spotify have transformed media consumption by enabling creators to reach global audiences without traditional intermediaries. Music content, in particular, dominates user engagement, motivating the development of predictive tools to forecast content popularity. Despite the emergence of Transformer-based models as the state-of-the-art across various domains, their effectiveness in music engagement prediction remains unexplored. In this paper, we investigate whether Transformers for tabular data (FT-Transformer) can outperform traditional tree-based models such as XGBoost and Random Forest on a real dataset combining Spotify and YouTube metadata. Our findings reinforce prior results that highlight the dominance of tree-based models on tabular data, with the FT-Transformer failing to surpass them across all considered settings. However, we show that the FT-Transformer remains competitive against XGBoost, particularly when GPU resources are available. These results provide further empirical support for the limitations of current Transformer-based approaches in low-dimensional, heterogeneous tabular data regimes, and suggests future research combining tabular data with multimodal inputs like lyrics or video content.
Palavras-chave: FT-Transformer; Tabular Data; Music Engagement Prediction; Random Forest; XGBoost.
Páginas: 8
Código DOI: 10.21528/CBIC2025-1191740
Artigo em PDF: CBIC_2025_paper1191740.pdf
Arquivo BibTeX:
CBIC_2025_1191740.bib