SonarID (2022 CHI)
Posted on April 13, 2022, 1:26 p.m.
SonarID: Using Sonar to Identify Fingers on a Smartwatch
Jiwan Kim and Ian Oakley
CHI'22: ACM Conference on Human Factors in Computing Systems
DOI: https://doi.org/10.1145/3491102.3501935
Session: Sensing
Abstract
The diminutive size of wrist wearables has prompted the design of many novel input techniques to increase expressivity. Finger identification, or assigning different functionality to different fingers, has been frequently proposed. However, while the value of the technique seems clear, its implementation remains challenging, often relying on external devices (e.g., worn magnets) or explicit instructions. Addressing these limitations, this paper explores a novel approach to natural and unencumbered finger identification on an unmodified smartwatch: sonar. To do this, we adapt an existing finger tracking smartphone sonar implementation---rather than extract finger motion, we process raw sonar fingerprints representing the complete sonar scene recorded during a touch. We capture data from 16 participants operating a smartwatch and use their sonar fingerprints to train a deep learning recognizer that identifies taps by the thumb, index, and middle fingers with an accuracy of up to 93.7%, sufficient to support meaningful application development.
Full presentation at CHI 2022
Short concept video