Constrained stochastic gradient descent for large-scale least squares problem

Mu, Y; Ding, W; Zhou, T; Tao, D

Constrained stochastic gradient descent for large-scale least squares problem

Mu, Y Ding, W Zhou, T Tao, D

Permalink

Publication Type:: Conference Proceeding
Citation:: Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 2013, Part F128815 pp. 883 - 891
Issue Date:: 2013-08-11

Closed Access

	Filename	Description	Size
	2013003399OK_Mu.pdf		749.65 kB	Adobe PDF	View/Open

Copyright Clearance Process

Recently Added
In Progress
Closed Access

This item is closed access and not available.

Full metadata record

Field	Value	Language
dc.contributor.author	Mu, Y	en_US
dc.contributor.author	Ding, W	en_US
dc.contributor.author	Zhou, T	en_US
dc.contributor.author	Tao, D https://orcid.org/0000-0001-7225-5449	en_US
dc.date.issued	2013-08-11	en_US
dc.identifier.citation	Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 2013, Part F128815 pp. 883 - 891	en_US
dc.identifier.isbn	9781450321747	en_US
dc.identifier.uri	http://hdl.handle.net/10453/28349
dc.description.abstract	Copyright © 2013 ACM. The least squares problem is one of the most important regression problems in statistics, machine learning and data mining. In this paper, we present the Constrained Stochastic Gradient Descent (CSGD) algorithm to solve the largescale least squares problem. CSGD improves the Stochastic Gradient Descent (SGD) by imposing a provable constraint that the linear regression line passes through the mean point of all the data points. It results in the best regret bound O(log T), and fastest convergence speed among all first order approaches. Empirical studies justify the effectiveness of CSGD by comparing it with SGD and other state-of-Theart approaches. An example is also given to show how to use CSGD to optimize SGD based least squares problems to achieve a better performance.	en_US
dc.relation.ispartof	Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining	en_US
dc.relation.isbasedon	10.1145/2487575.2487635	en_US
dc.title	Constrained stochastic gradient descent for large-scale least squares problem	en_US
dc.type	Conference Proceeding
utslib.citation.volume	Part F128815	en_US
utslib.for	0801 Artificial Intelligence and Image Processing	en_US
dc.location.activity	Chicago, IL, USA	en_US
pubs.embargo.period	Not known	en_US
pubs.organisational-group	/University of Technology Sydney
pubs.organisational-group	/University of Technology Sydney/Faculty of Engineering and Information Technology
utslib.copyright.status	closed_access
pubs.publication-status	Published	en_US
pubs.volume	Part F128815	en_US

Abstract:

Copyright © 2013 ACM. The least squares problem is one of the most important regression problems in statistics, machine learning and data mining. In this paper, we present the Constrained Stochastic Gradient Descent (CSGD) algorithm to solve the largescale least squares problem. CSGD improves the Stochastic Gradient Descent (SGD) by imposing a provable constraint that the linear regression line passes through the mean point of all the data points. It results in the best regret bound O(log T), and fastest convergence speed among all first order approaches. Empirical studies justify the effectiveness of CSGD by comparing it with SGD and other state-of-Theart approaches. An example is also given to show how to use CSGD to optimize SGD based least squares problems to achieve a better performance.

Please use this identifier to cite or link to this item:

http://hdl.handle.net/10453/28349