PhD Thesis Defense: Ignacio Gavier, Overcoming Data and Energy Challenges in Wearable IMU-Based Learning
Content
Speaker:
Abstract:
Inertial measurement units (IMUs) are integrated into every wearable device (e.g., watches, wristbands, rings), as they enable personalized applications that enhance users' quality of life, such as human activity recognition or health monitoring. Historically, due to strict limits of available energy that wearable devices can offer, heuristic algorithms and shallow learning models have been utilized to extract patterns from IMUs and provide relevant higher-level information. More recently, after the success of deep learning in fields like vision or language, there has been a significant effort to incorporate these computation models to perform inference on device. However, training these models requires large-scale datasets from diverse activities and individuals to learn effectively. Yet, IMU datasets are extremely difficult to collect and annotate (i.e., signals are subject to privacy concerns, and labeling requires considerable time and effort due to non-interpretability), impeding a proper training. Moreover, deep models contain at least hundreds of thousands of parameters, requiring enormous amounts of computations per inference, posing a huge burden to the limited battery capacities available on device.
This thesis addresses the aforementioned challenges of dataset scarcity and energy efficiency. For the first challenge, I introduce an algorithm that leverages the vast abundance of public videos to generate diverse IMU signals. The algorithm estimates body motion, which is used to simulate multiple realizations of virtual IMU recordings with variations in user- and sensor-dependent parameters. I then present a generative pipeline that is capable of synthesizing large-scale and diverse datasets of IMU by relying on a small set of examples. The pipeline combines learning algorithms with biomechanical principles of human arm to boost the amount of available data, which can then be used to robustly train deep learning models. For the latter challenge, I develop a novel system that allows to deploy deep models on device using ultra-low-power neuromorphic technology. The system pushes the energy efficiency to the limit by seeking sparse representations of IMU signals based on redundant patterns observed in motor behavior. These representations are processed through spiking neural network (SNN) learning models, which benefit from sparse signals for low-power inference. The system is implemented on hardware, on which I run feasibility analyses to test its potential impact in the wearable industry.
Advisor:
Ivan Lee