Machine learning-based motion capture: Exploring the application, limits and opportunities in live performance
- Publication Type:
- Thesis
- Issue Date:
- 2024
Open Access
Copyright Clearance Process
- Recently Added
- In Progress
- Open Access
This item is open access.
This practice-based research investigates how machine learning techniques can be used to generate motion capture data for abstract animations in live performance contexts. Combining a literature review, model experimentation, practitioner interviews, collaborative processes, and the production of a creative work, the study explores whether machine learning-based motion capture can enhance current practices and offer new artistic possibilities.
The literature review traces the evolution of motion capture technologies, highlighting gaps where machine learning could be effectively integrated. Interviews with experienced performance practitioners identify key needs—particularly affordability and portability—that machine learning models may address.
Experimental testing compares monocular and multi-camera machine learning models in capturing performer movements. While monocular models perform adequately in uncontrolled environments, they struggle with atypical movements, diverse body types, and unusual positions. Multi-camera setups offer smoother results but require more equipment and calibration.
In collaboration with professional choreographers, machine learning-based motion capture was integrated into a performance context. This collaboration demonstrated that machine learning models can facilitate efficient workflows and rapid iteration, though occasional imprecisions in output introduced creative opportunities rather than limitations.
The research culminates in a performance piece using a multi-camera machine learning system to generate real-time animation projected alongside live dance. This production allowed for an examination of the practical considerations involved in integrating machine learning-based animation into live performance—from the impact on artistic collaboration and the performer’s experience to audience engagement. Feedback from the audience indicated strong appreciation for the interplay of live and digital elements, while dancers responded positively to the speed and flexibility of the machine learning workflow compared to traditional systems.
Overall, the study finds that machine learning-based motion capture holds strong democratising potential by offering accessible, lower-cost alternatives to traditional systems. It shows that pose detection using ML can be effectively integrated into creative workflows and used to produce compelling performance-based animations. While there are still areas for technical refinement, the research reveals meaningful artistic and practical benefits. This positions machine learning-based motion capture as a promising tool for expanding creative expression at the intersection of technology and the performing arts.
Please use this identifier to cite or link to this item: