Advancing Loss Functions in Recommender Systems: A Comparative Study with a Rényi Divergence-Based Solution
Abstract
Loss functions play a pivotal role in optimizing recommendation models. Among various loss functions, Softmax Loss (SL) and Cosine Contrastive Loss (CCL) are particularly effective. Their theoretical connections and differences warrant in-depth exploration. This work conducts comprehensive analyses of these losses, yielding significant insights: 1) Common strengths — both can be viewed as augmentations of traditional losses with Distributional Robust Optimization (DRO), enhancing robustness to distributional shifts; 2) Respective limitations — stemming from their use of different distribution distance metrics in DRO optimization, SL exhibits high sensitivity to false negative instances, whereas CCL suffers from low data utilization. To address these limitations, this work proposes a new loss function, DrRL, which generalizes SL and CCL by leveraging Rényi-divergence in DRO optimization. DrRL incorporates the advantageous structures of both SL and CCL, and can be demonstrated to effectively mitigate their limitations. Extensive experiments have been conducted to validate the superiority of DrRL on both recommendation accuracy and robustness. Code: https://github.com/cynthia-shengjia/AAAI-2025-DrRL
Shengjia Zhang
Student
Shengjia (Cynthia) Zhang is currently a Master student in ZLST led by Profs. Chun Chen, Can Wang and Jiawei Chen, and he is supervised by Prof. Jiawei Chen.
Jiawei Chen
陈佳伟 研究员
Changdong Li
Student
Changdong Li is currently a Master student in ZLST, where he is supervised by Profs. Can Wang, and Profs. Jiawei Chen.
Qihao Shi
史麒豪 副研究员
Yan Feng
冯雁 副教授
Chun Chen
陈纯 院士
