LifeHD: Lifelong Intelligence Beyond the Edge Using Hyperdimensional Computing
On-device learning has emerged as a prevailing trend that avoids the slow response time and costly communication of cloud-based learning. The ability to learn continuously and indefinitely in a changing environment, and with resource constraints, is critical for real sensor deployments. However, existing designs are inadequate for practical scenarios with (i) streaming data input, (ii) lack of supervision and (iii) limited on-board resources.
We designed and deployed the first on-device lifelong learning system for general Internet of Things (IoT) applications with limited supervision: LifeHD. The design of LifeHD is based on a novel neurally-inspired and lightweight learning paradigm called Hyperdimensional Computing (HDC). We utilized a two-tier associative memory organization to intelligently store and manage high-dimensional, low-precision vectors, which represent the historical patterns as cluster centroids.
Additionally, we proposed two variants of LifeHD to cope with scarce labeled inputs and power constraints. LifeHD is implemented on off-the-shelf edge platforms and performs extensive evaluations across three scenarios. The measurements show that LifeHD improves the unsupervised clustering accuracy by up to 74.8% compared to the state-of-the-art NN-based unsupervised lifelong learning baselines with as much as 34.3x better energy efficiency.
Figure 1. The design diagram of LifeHD.
Building on previous work on practical system heterogeneities in federated IoT networks, we have recently focused on the lifelong learning ability on a single IoT, i.e., the ability to learn continuously and indefinitely in a changing environment. We envision that the lifelong learning capability of an IoT device lies at the core to achieve long-term and adaptive edge intelligence. By enabling continuous adaptation, such lifelong learning capability is also a key component in deploying and testing other learning algorithms developed by other TILOS research teams.
We next plan to investigate the expansion of LifeHD to distributed IoT networks running larger-scale applications.