Expanding Creative Interaction: Toward a Modular Toolkit for Audio-Visual Composition and Augmented Performance

Savery, A; Ferguson, S; Johnston, A

Expanding Creative Interaction: Toward a Modular Toolkit for Audio-Visual Composition and Augmented Performance

Savery, A

Ferguson, S

Johnston, A

Permalink

Publisher:: Association for Computing Machinery (ACM)
Publication Type:: Conference Proceeding
Citation:: Hai 2025 Proceedings of the 13th International Conference on Human Agent Interaction, 2026, pp. 439-441
Issue Date:: 2026-01-02

Open Access

Copyright Clearance Process

Recently Added
In Progress
Open Access

This item is open access.

Adobe PDF

Download Published versionAdobe PDF (1.87 MB)

View on publisher's site

View statistics

Full metadata record

Field	Value	Language
dc.contributor.author	Savery, A https://orcid.org/0009-0004-8394-6316
dc.contributor.author	Ferguson, S https://orcid.org/0000-0003-1853-3000
dc.contributor.author	Johnston, A https://orcid.org/0000-0001-8306-7507
dc.date.accessioned	2026-10-09T07:08:14Z
dc.date.available	2026-10-09T07:08:14Z
dc.date.issued	2026-01-02
dc.identifier.citation	Hai 2025 Proceedings of the 13th International Conference on Human Agent Interaction, 2026, pp. 439-441
dc.identifier.uri	http://hdl.handle.net/10453/197000
dc.description.abstract	We present a modular, sensor-based violin bow interface designed to augment musical performance and audiovisual composition. The system integrates real-time audio processing, visual feedback, and machine learning models for gesture recognition. Through an expert user study with seven violinists, we identified themes of ergonomic adaptation, co-agency, and creative potential. While results highlight the toolkit's promise as both instrument and collaborator, limitations include the absence of non-expert perspectives, training protocols, and audience evaluation. This work contributes to human-agent interaction by positioning augmented instruments as sites of improvisational dialogue and outlines future directions in unifying system components, quantitative evaluation, and broader user studies.
dc.language	en
dc.publisher	Association for Computing Machinery (ACM)
dc.relation.ispartof	Hai 2025 Proceedings of the 13th International Conference on Human Agent Interaction
dc.relation.ispartof	Proceedings of the 13th International Conference on Human-Agent Interaction
dc.relation.isbasedon	10.1145/3765766.3765845
dc.rights	info:eu-repo/semantics/openAccess
dc.title	Expanding Creative Interaction: Toward a Modular Toolkit for Audio-Visual Composition and Augmented Performance
dc.type	Conference Proceeding
pubs.organisational-group	University of Technology Sydney
pubs.organisational-group	University of Technology Sydney/Faculty of Engineering and Information Technology
pubs.organisational-group	University of Technology Sydney/UTS Groups
pubs.organisational-group	University of Technology Sydney/UTS Groups/Visualisation Institute (VI)
pubs.organisational-group	University of Technology Sydney/UTS Groups/The Trustworthy Digital Society
pubs.organisational-group	University of Technology Sydney/UTS Groups/Animal Logic Academy
utslib.copyright.status	open_access	*
dc.rights.license	This work is licensed under a Creative Commons Attribution 4.0 International License (CC BY 4.0). To view a copy of this license, visit https://creativecommons.org/licenses/by/4.0/
dc.date.updated	2026-10-09T07:08:13Z
pubs.publication-status	Published

Abstract:

We present a modular, sensor-based violin bow interface designed to augment musical performance and audiovisual composition. The system integrates real-time audio processing, visual feedback, and machine learning models for gesture recognition. Through an expert user study with seven violinists, we identified themes of ergonomic adaptation, co-agency, and creative potential. While results highlight the toolkit's promise as both instrument and collaborator, limitations include the absence of non-expert perspectives, training protocols, and audience evaluation. This work contributes to human-agent interaction by positioning augmented instruments as sites of improvisational dialogue and outlines future directions in unifying system components, quantitative evaluation, and broader user studies.

Please use this identifier to cite or link to this item:

http://hdl.handle.net/10453/197000