image2emmet: Automatic code generation from web user interface image

Xu, Y; Bo, L; Sun, X; Li, B; Jiang, J; Zhou, W

image2emmet: Automatic code generation from web user interface image

Xu, Y Bo, L Sun, X Li, B Jiang, J

Zhou, W

Permalink

Publisher:: Wiley
Publication Type:: Journal Article
Citation:: Journal of Software: Evolution and Process, 2021, 33, (8)
Issue Date:: 2021-08-01

Closed Access

	Filename	Description	Size
	J Software Evolu Process - 2021 - Xu - image2emmet Automatic code generation from web user interface image.pdf	Published version	8.19 MB	Adobe PDF	View/Open

Copyright Clearance Process

Recently Added
In Progress
Closed Access

This item is closed access and not available.

Full metadata record

Field	Value	Language
dc.contributor.author	Xu, Y
dc.contributor.author	Bo, L
dc.contributor.author	Sun, X
dc.contributor.author	Li, B
dc.contributor.author	Jiang, J https://orcid.org/0000-0001-5301-7779
dc.contributor.author	Zhou, W
dc.date.accessioned	2022-05-03T22:45:30Z
dc.date.available	2022-05-03T22:45:30Z
dc.date.issued	2021-08-01
dc.identifier.citation	Journal of Software: Evolution and Process, 2021, 33, (8)
dc.identifier.issn	2047-7473
dc.identifier.issn	2047-7481
dc.identifier.uri	http://hdl.handle.net/10453/156980
dc.description.abstract	Web development usually follows with analyzing the functionality, designing the user interface (UI) prototype, implementing the UI by front-end (FE) developers and implementing the REpresentational State Transfer (RESTful) application programming interface (API) by back-end (BE) programmers. Unfortunately, web development is a tedious, cumbersome, and time-consuming task, which makes it a challenge for the FE programmers to work in an efficient way. In this paper, we propose an approach, image2emmet, to assist FE programmers in implementing the UI. First, we collect HyperText Markup Language, Cascading Style Sheets (HTML-CSS) dataset in an automatic and efficient way. The HTML-CSS dataset used for model training consists of HTML-CSS code and its display images. Second, the faster region-based convolutional neural network (CNN) (R-CNN) is utilized to detect the UI component. Finally, we build a model combining CNN and long short-term memory (LSTM) to transform the UI component into the HTML-CSS code. The empirical study demonstrates that image2emmet can achieve a precision of 80% on the UI component detection and 60% on the transformation of UI component into HTML-CSS code.
dc.language	en
dc.publisher	Wiley
dc.relation.ispartof	Journal of Software: Evolution and Process
dc.relation.isbasedon	10.1002/smr.2369
dc.rights	info:eu-repo/semantics/closedAccess
dc.subject	0803 Computer Software, 0806 Information Systems
dc.subject.classification	Software Engineering
dc.title	image2emmet: Automatic code generation from web user interface image
dc.type	Journal Article
utslib.citation.volume	33
utslib.for	0803 Computer Software
utslib.for	0806 Information Systems
pubs.organisational-group	/University of Technology Sydney
pubs.organisational-group	/University of Technology Sydney/Faculty of Engineering and Information Technology
pubs.organisational-group	/University of Technology Sydney/Strength - AAII - Australian Artificial Intelligence Institute
utslib.copyright.status	closed_access	*
dc.date.updated	2022-05-03T22:45:29Z
pubs.issue	8
pubs.publication-status	Published
pubs.volume	33
utslib.citation.issue	8

Abstract:

Web development usually follows with analyzing the functionality, designing the user interface (UI) prototype, implementing the UI by front-end (FE) developers and implementing the REpresentational State Transfer (RESTful) application programming interface (API) by back-end (BE) programmers. Unfortunately, web development is a tedious, cumbersome, and time-consuming task, which makes it a challenge for the FE programmers to work in an efficient way. In this paper, we propose an approach, image2emmet, to assist FE programmers in implementing the UI. First, we collect HyperText Markup Language, Cascading Style Sheets (HTML-CSS) dataset in an automatic and efficient way. The HTML-CSS dataset used for model training consists of HTML-CSS code and its display images. Second, the faster region-based convolutional neural network (CNN) (R-CNN) is utilized to detect the UI component. Finally, we build a model combining CNN and long short-term memory (LSTM) to transform the UI component into the HTML-CSS code. The empirical study demonstrates that image2emmet can achieve a precision of 80% on the UI component detection and 60% on the transformation of UI component into HTML-CSS code.

Please use this identifier to cite or link to this item:

http://hdl.handle.net/10453/156980