Demo Abstract: VGGlass - Demonstrating Visual Grounding and Localization Synergy with a LiDAR-enabled Smart-Glass

Rathnayake, D; Weerakoon, D; Radhakrishnan, M; Subbaraju, V; Hwang, I; Misra, A

Demo Abstract: VGGlass - Demonstrating Visual Grounding and Localization Synergy with a LiDAR-enabled Smart-Glass

Rathnayake, D Weerakoon, D Radhakrishnan, M

Subbaraju, V Hwang, I Misra, A

Permalink

Publisher:: ASSOC COMPUTING MACHINERY
Publication Type:: Conference Proceeding
Citation:: SenSys 2023 - Proceedings of the 21st ACM Conference on Embedded Networked Sensors Systems, 2023, pp. 492-493
Issue Date:: 2023-11-12

Open Access

Copyright Clearance Process

Recently Added
In Progress
Open Access

This item is open access.

Download Published versionAdobe PDF (8.33 MB)

View on publisher's site

View statistics

Full metadata record

Field	Value	Language
dc.contributor.author	Rathnayake, D
dc.contributor.author	Weerakoon, D
dc.contributor.author	Radhakrishnan, M https://orcid.org/0000-0002-9884-1769
dc.contributor.author	Subbaraju, V
dc.contributor.author	Hwang, I
dc.contributor.author	Misra, A
dc.date	2023-11-12
dc.date.accessioned	2024-11-15T23:35:40Z
dc.date.available	2024-11-15T23:35:40Z
dc.date.issued	2023-11-12
dc.identifier.citation	SenSys 2023 - Proceedings of the 21st ACM Conference on Embedded Networked Sensors Systems, 2023, pp. 492-493
dc.identifier.uri	http://hdl.handle.net/10453/181929
dc.description.abstract	This work demonstrates the VGGlass system, which simultaneously interprets human instructions for a target acquisition task and determines the precise 3D positions of both user and the target object. This is achieved by utilizing LiDARs mounted in the infrastructure and a smart glass device worn by the user. Key to our system is the union of LiDAR-based localization termed LiLOC and a multi-modal visual grounding approach termed RealG(2)In-Lite. To demonstrate the system, we use Intel RealSense L515 cameras and a Microsoft HoloLens 2, as the user devices. VGGlass is able to: a) track the user in real-time in a global coordinate system, and b) locate target objects referred by natural language and pointing gestures.
dc.language	en
dc.publisher	ASSOC COMPUTING MACHINERY
dc.relation.ispartof	SenSys 2023 - Proceedings of the 21st ACM Conference on Embedded Networked Sensors Systems
dc.relation.ispartof	21st ACM Conference on Embedded Networked Sensor Systems (SenSys)
dc.relation.isbasedon	10.1145/3625687.3628407
dc.rights	info:eu-repo/semantics/openAccess
dc.title	Demo Abstract: VGGlass - Demonstrating Visual Grounding and Localization Synergy with a LiDAR-enabled Smart-Glass
dc.type	Conference Proceeding
utslib.location.activity	TURKEY, Bahcesehir Univ, Istanbul
pubs.organisational-group	University of Technology Sydney
pubs.organisational-group	University of Technology Sydney/Faculty of Engineering and Information Technology
pubs.organisational-group	University of Technology Sydney/UTS Groups
pubs.organisational-group	University of Technology Sydney/Strength - DSI - Data Science Institute
pubs.organisational-group	University of Technology Sydney/UTS Groups/The Trustworthy Digital Society
utslib.copyright.status	open_access	*
dc.rights.license	This work is licensed under a Creative Commons Attribution 4.0 International License (CC BY 4.0). To view a copy of this license, visit https://creativecommons.org/licenses/by/4.0/
dc.date.updated	2024-11-15T23:35:35Z
pubs.finish-date	2023-11-17
pubs.publication-status	Published
pubs.start-date	2023-11-12

Abstract:

This work demonstrates the VGGlass system, which simultaneously interprets human instructions for a target acquisition task and determines the precise 3D positions of both user and the target object. This is achieved by utilizing LiDARs mounted in the infrastructure and a smart glass device worn by the user. Key to our system is the union of LiDAR-based localization termed LiLOC and a multi-modal visual grounding approach termed RealG(2)In-Lite. To demonstrate the system, we use Intel RealSense L515 cameras and a Microsoft HoloLens 2, as the user devices. VGGlass is able to: a) track the user in real-time in a global coordinate system, and b) locate target objects referred by natural language and pointing gestures.

Please use this identifier to cite or link to this item:

http://hdl.handle.net/10453/181929