Multivariate multilinear regression

Su, Y; Gao, X; Li, X; Tao, D

Multivariate multilinear regression

Su, Y Gao, X Li, X Tao, D

Permalink

Publication Type:: Journal Article
Citation:: IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics, 2012, 42 (6), pp. 1560 - 1573
Issue Date:: 2012-06-08

Closed Access

	Filename	Description	Size
	2012002999OK.pdf		1.05 MB	Adobe PDF	View/Open

Copyright Clearance Process

Recently Added
In Progress
Closed Access

This item is closed access and not available.

Full metadata record

Field	Value	Language
dc.contributor.author	Su, Y	en_US
dc.contributor.author	Gao, X	en_US
dc.contributor.author	Li, X	en_US
dc.contributor.author	Tao, D https://orcid.org/0000-0001-7225-5449	en_US
dc.date.issued	2012-06-08	en_US
dc.identifier.citation	IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics, 2012, 42 (6), pp. 1560 - 1573	en_US
dc.identifier.issn	1083-4419	en_US
dc.identifier.uri	http://hdl.handle.net/10453/22889
dc.description.abstract	Conventional regression methods, such as multivariate linear regression (MLR) and its extension principal component regression (PCR), deal well with the situations that the data are of the form of low-dimensional vector. When the dimension grows higher, it leads to the under sample problem (USP): the dimensionality of the feature space is much higher than the number of training samples. However, little attention has been paid to such a problem. This paper first adopts an in-depth investigation to the USP in PCR, which answers three questions: 1) Why is USP produced? 2) What is the condition for USP, and 3) How is the influence of USP on regression. With the help of the above analysis, the principal components selection problem of PCR is presented. Subsequently, to address the problem of PCR, a multivariate multilinear regression (MMR) model is proposed which gives a substitutive solution to MLR, under the condition of multilinear objects. The basic idea of MMR is to transfer the multilinear structure of objects into the regression coefficients as a constraint. As a result, the regression problem is reduced to find two low-dimensional coefficients so that the principal components selection problem is avoided. Moreover, the sample size needed for solving MMR is greatly reduced so that USP is alleviated. As there is no closed-form solution for MMR, an alternative projection procedure is designed to obtain the regression matrices. For the sake of completeness, the analysis of computational cost and the proof of convergence are studied subsequently. Furthermore, MMR is applied to model the fitting procedure in the active appearance model (AAM). Experiments are conducted on both the carefully designed synthesizing data set and AAM fitting databases verified the theoretical analysis. © 1996-2012 IEEE.	en_US
dc.relation	http://purl.org/au-research/grants/arc/DP120103730
dc.relation.ispartof	IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics	en_US
dc.relation.isbasedon	10.1109/TSMCB.2012.2195171	en_US
dc.subject.classification	Artificial Intelligence & Image Processing	en_US
dc.title	Multivariate multilinear regression	en_US
dc.type	Journal Article
utslib.citation.volume	6	en_US
utslib.citation.volume	42	en_US
utslib.for	0801 Artificial Intelligence and Image Processing	en_US
utslib.for	0102 Applied Mathematics	en_US
utslib.for	0906 Electrical and Electronic Engineering	en_US
pubs.embargo.period	Not known	en_US
pubs.organisational-group	/University of Technology Sydney
pubs.organisational-group	/University of Technology Sydney/Faculty of Engineering and Information Technology
utslib.copyright.status	closed_access
pubs.issue	6	en_US
pubs.publication-status	Published	en_US
pubs.volume	42	en_US

Abstract:

Conventional regression methods, such as multivariate linear regression (MLR) and its extension principal component regression (PCR), deal well with the situations that the data are of the form of low-dimensional vector. When the dimension grows higher, it leads to the under sample problem (USP): the dimensionality of the feature space is much higher than the number of training samples. However, little attention has been paid to such a problem. This paper first adopts an in-depth investigation to the USP in PCR, which answers three questions: 1) Why is USP produced? 2) What is the condition for USP, and 3) How is the influence of USP on regression. With the help of the above analysis, the principal components selection problem of PCR is presented. Subsequently, to address the problem of PCR, a multivariate multilinear regression (MMR) model is proposed which gives a substitutive solution to MLR, under the condition of multilinear objects. The basic idea of MMR is to transfer the multilinear structure of objects into the regression coefficients as a constraint. As a result, the regression problem is reduced to find two low-dimensional coefficients so that the principal components selection problem is avoided. Moreover, the sample size needed for solving MMR is greatly reduced so that USP is alleviated. As there is no closed-form solution for MMR, an alternative projection procedure is designed to obtain the regression matrices. For the sake of completeness, the analysis of computational cost and the proof of convergence are studied subsequently. Furthermore, MMR is applied to model the fitting procedure in the active appearance model (AAM). Experiments are conducted on both the carefully designed synthesizing data set and AAM fitting databases verified the theoretical analysis. © 1996-2012 IEEE.

Please use this identifier to cite or link to this item:

http://hdl.handle.net/10453/22889