PhD Dissertation Proposal: Ignacio Gavier, Overcoming Data and Energy Challenges in Wearable IMU-based Learning
Content
Speaker:
Abstract:
Inertial measurement units (IMUs) are integrated into every upper-limb wearable device, enabling personalized applications such as human activity recognition and health monitoring. Although relevant information has usually been extracted using heuristic algorithms, recent success of deep neural networks (NNs) in other fields has motivated their incorporation into these sensors. However, two fundamental challenges remain unresolved. First, NNs are computationally expensive, making them unsuitable for wearables with limited energy resources. Second, unlike fields such as vision or language, large IMU datasets are difficult to collect and annotate—signals are not directly human-interpretable, and labeling requires considerable effort—, thus hindering proper NN training.
To address IMU data scarcity, I first introduce a system that leverages videos—a more abundant resource—to generate and augment IMU signals. By estimating human body motion, I sample multiple placements of a virtual sensor on the upper limbs to synthesize diverse signals. Yet, collecting and annotating videos may not be viable in some scenarios. Thus, I develop a biomechanics-informed generative model that synthesizes IMU signals based on a tiny input dataset of movements. By using a graphical environment, I simulate realistic humanoid motion using motor control principles to create diverse synthetic IMU datasets. Furthermore, to generate IMU of more complex activities, I implement a generative model that learns the sequential and compositional structure of movements during daily activities.
To address energy constraints, I develop and implement on hardware an ultra-low-power activity recognition system. I create a novel algorithm inspired by upper-limb motor control theory to decompose IMU signals into primitive movements and encode them into a sparse low-dimensional representation. I then use spiking neural networks (SNNs) for activity inference, as they enable much lower energy consumption than conventional NNs.
Advisor:
Ivan Lee