On the Convergence of A Family of Robust Losses for Stochastic Gradient Descent

Publisher:
Springer
Publication Type:
Conference Proceeding
Citation:
Machine Learning and Knowledge Discovery in Databases - LNCS, 2016, 9851
Issue Date:
2016-09-23
Full metadata record
Files in This Item:
Filename Description Size
paper.pdfPublished version525.69 kB
Adobe PDF
The convergence of Stochastic Gradient Descent (SGD) using convex loss functions has been widely studied. However, vanilla SGD methods using convex losses cannot perform well with noisy labels, which adversely affect the update of the primal variable in SGD methods. Unfortunately, noisy labels are ubiquitous in real world applications such as crowdsourcing. To handle noisy labels, in this paper, we present a family of robust losses for SGD methods. By employing our robust losses, SGD methods successfully reduce negative effects caused by noisy labels on each update of the primal variable. We not only reveal the convergence rate of SGD methods using robust losses, but also provide the robustness analysis on two representative robust losses. Comprehensive experimental results on six real-world datasets show that SGD methods using robust losses are obviously more robust than other baseline methods in most situations with fast convergence.
Please use this identifier to cite or link to this item: