Xiaoran Fan
ABSTRACT
Headphones continue to become more intelligent as new functions (e.g., touch-based gesture control) appear. These functions usually rely on auxiliary sensors (e.g., accelerometer and gyroscope) that are available in smart headphones. However, for those headphones that do not have such sensors, supporting these functions becomes a daunting task. This paper presents HeadFi, a new design paradigm for bringing intelligence to headphones. Instead of adding auxiliary sensors into headphones, HeadFi turns the pair of drivers that are readily available inside all headphones into a versatile sensor to enable new applications spanning across mobile health, user-interface, and context-awareness. HeadFi works as a plug-in peripheral connecting the headphones and the pairing device (e.g., a smartphone). The simplicity (can be as simple as only two resistors) and small form factor of this design lend itself to be embedded into the pairing device as an integrated circuit. We envision HeadFi can serve as a vital supplementary solution to existing smart headphone design by directly transforming large amounts of existing "dumb" headphones into intelligent ones. We prototype HeadFi on PCB and conduct extensive experiments with 53 volunteers using 54 pairs of non-smart headphones under the institutional review board (IRB) protocols. The results show that HeadFi can achieve 97.2%--99.5% accuracy on user identification, 96.8%--99.2% accuracy on heart rate monitoring, and 97.7%--99.3% accuracy on gesture recognition.
- 2014. Gig Fix: Turn Your Headphones Into A Mic. Webpage.Google Scholar
- 2014. Voice Quality Enhancement and Echo Cancellation.Google Scholar
- 2018. B21 Super Bass Wireless Bluetooth Headphone Stereo Touch Control Headset Noise Cancelling With Micro. Webpage.Google Scholar
- 2018. Wireless Noise Canceling Stereo Headset WH-1000XM3. Webpage.Google Scholar
- 2019. Bose Noise Cancelling Headphones 700. Webpage.Google Scholar
- 2019. CMS50D1A GEHP040AHUS pulse oximeter. Webpage.Google Scholar
- 2019. Earphones and Headphones Market Size, Industry Report. Webpage.Google Scholar
- 2019. Global unit sales of headphones and headsets from 2013 to 2017. Website.Google Scholar
- 2019. P.800.1 : Mean opinion score (MOS) terminology. Webpage.Google Scholar
- 2019. Voice frequency. Webpage.Google Scholar
- 2020. BOSE QC-35 Wireless Headphones.Google Scholar
- 2020. The Differential Amplifier. Webpage.Google Scholar
- 2020. How Sound Works. Webpage.Google Scholar
- 2020. Microsoft Surface Headphones. Website.Google Scholar
- 2020. MiniDSP E.A.R.S. Webpage.Google Scholar
- 2020. A Research Space for EARABLE COMPUTING. Webpage.Google Scholar
- 2020. Samgsung Galaxy Buds. Website.Google Scholar
- 2021. Apple Airpods. Website.Google Scholar
- Anton HM Akkermans, Tom AM Kevenaar, and Daniel WE Schobben. 2005. Acoustic ear recognition for person identification. In AutoID. IEEE.Google Scholar
- Takayuki Arakawa, Takafumi Koshinaka, Shohei Yano, Hideki Irisawa, Ryoji Miyahara, and Hitoshi Imaoka. 2016. Fast and accurate personal authentication using ear acoustics. In APSIPA. IEEE.Google Scholar
- Yoshua Bengio and Yves Grandvalet. 2004. No unbiased estimator of the variance of k-fold cross-validation. Journal of machine learning research (2004).Google Scholar
- Nam Bui, Nhat Pham, Jessica Jacqueline Barnitz, Zhanan Zou, Phuc Nguyen, Hoang Truong, Taeho Kim, Nicholas Farrow, Anh Nguyen, Jianliang Xiao, et al. 2019. eBP: A Wearable System For Frequent and Comfortable Blood Pressure Monitoring From User's Ear. In MobiCom.Google Scholar
- Bruce Carter. 2009. Op Amp noise theory and applications, 12.3.2 Thermal Noise. Elsevier.Google Scholar
- Bruce Carter and Thomas R Brown. 2001. Handbook of operational amplifier applications. Texas Instruments Dallas, Tex, USA.Google Scholar
- Justin Chan, Sharat Raju, Rajalakshmi Nandakumar, Randall Bly, and Shyamnath Gollakota. 2019. Detecting middle ear fluid using smartphones. Science translational medicine 11, 492 (2019).Google Scholar
- John Clarke, Claudia D Tesche, and RP Giffard. 1979. Optimization of dc SQUID voltmeter and magnetometer circuits. Journal of Low Temperature Physics (1979).Google Scholar
- David Da He, Eric S Winokur, and Charles G Sodini. 2012. An ear-worn continuous ballistocardiogram (BCG) sensor for cardiovascular monitoring. In EMBC. IEEE.Google Scholar
- Jesse Davis and Mark Goadrich. 2006. The relationship between Precision-Recall and ROC curves. In Proceedings of the 23rd international conference on Machine learning. 233--240.Google ScholarDigital Library
- Christian C Enz and Gabor C Temes. 1996. Circuit techniques for reducing the effects of op-amp imperfections: autozeroing, correlated double sampling, and chopper stabilization. Proc. IEEE (1996).Google Scholar
- Xiaoran Fan, Daewon Lee, Yuan Chen, Colin Prepscius, Volkan Isler, Larry Jackel, H Sebastian Seung, and Daniel Lee. 2020. Acoustic collision detection and localization for robot manipulators. IROS.Google Scholar
- Joel Gak, Matías Miguez, Martín Bremermann, and Alfredo Arnaud. 2008. On the reduction of thermal and flicker noise in ENG signal recording amplifiers. Analog Integrated Circuits and Signal Processing (2008).Google Scholar
- Yang Gao, Wei Wang, Vir V Phoha, Wei Sun, and Zhanpeng Jin. 2019. EarEcho: Using Ear Canal Echo for Wearable Authentication. IMWUT (2019).Google ScholarDigital Library
- Yutaka Higashiguchi, Yoshinobu Kajikawa, and Shunsuke Kita. 2017. A personal authentication system based on pinna related transfer function. In ICBAKE. IEEE.Google Scholar
- Karl Hoffmann. 1974. Applying the Wheatstone bridge circuit. HBM Germany.Google Scholar
- Texas Instruments. 2007. Noise analysis in operational amplifier circuits. Application Report, SLVA043B (2007).Google Scholar
- Thomas M Jensen, Vladan Bajic, and Andrew P Bright. 2016. Active noise cancellation using multiple reference microphone signals. US Patent 9,330,652.Google Scholar
- Liang-Ting Jiang and Joshua R Smith. 2012. Seashell effect pretouch sensing for robotic grasping. In ICRA. IEEE.Google Scholar
- Agnès Job, Paul Grateau, and Jacques Picard. 1998. Intrinsic differences in hearing performances between ears revealed by the asymmetrical shooting posture in the army. Hearing research (1998).Google Scholar
- Ron Kapusta, Haiyang Zhu, and Colin Lyden. 2014. Sampling circuits that break the kT/C thermal noise limit. IEEE Journal of Solid-State Circuits (2014).Google ScholarCross Ref
- Ronald A Kapusta, Katsufumi Nakamura, et al. 2007. Methods and apparatus for reducing thermal noise. US Patent 7,298,151.Google Scholar
- F Laurain King and Doreen Kimura. 1972. Left-ear superiority in dichotic perception of vocal nonverbal sounds. Canadian Journal of Psychology/Revue canadienne de psychologie (1972).Google Scholar
- Shivangi Mahto, Takayuki Arakawa, and Takafumi Koshinak. 2018. Ear acoustic biometrics using inaudible signals and its application to continuous user authentication. In EUSIPCO. IEEE.Google Scholar
- Henrik Møller, Dorte Hammershøi, Clemen Boje Jensen, and Michael Friis Sørensen. 1995. Transfer characteristics of headphones measured on human ears. Journal of the Audio Engineering Society (1995).Google Scholar
- Anh Nguyen, Raghda Alqurashi, Zohreh Raghebi, Farnoush Banaei-Kashani, Ann C Halbower, and Tam Vu. 2016. A lightweight and inexpensive in-ear sensing system for automatic whole-night sleep stage monitoring. In SenSys.Google Scholar
- Viet Nguyen, Siddharth Rupavatharam, Luyang Liu, Richard Howard, and Marco Gruteser. 2019. HandSense: capacitive coupling-based dynamic, micro finger gesture recognition. In SenSys.Google Scholar
- Guy C Nicholson. 2013. Active noise cancellation decisions in a portable audio device. US Patent 8,515,089.Google Scholar
- Sean Olive, Omid Khonsaripour, and Todd Welti. 2018. A Survey and Analysis of Consumer and Professional Headphones Based on Their Objective and Subjective Performances. In Audio Engineering Society Convention 145. Audio Engineering Society.Google Scholar
- Alexandros Pantelopoulos and Nikolaos G Bourbakis. 2009. A survey on wearable sensor-based systems for health monitoring and prognosis. IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews) (2009).Google Scholar
- Melih Papila, Raphael T Haftka, Toshikazu Nishida, and Mark Sheplak. 2006. Piezoresistive microphone design pareto optimization: tradeoff between sensitivity and noise floor. Journal of microelectromechanical systems (2006).Google Scholar
- Ming-Zher Poh, Kyunghee Kim, Andrew Goessling, Nicholas Swenson, and Rosalind Picard. 2010. Cardiovascular monitoring using earphones and a mobile device. IEEE Pervasive Computing (2010).Google Scholar
- Ming-Zher Poh, Kyunghee Kim, Andrew D Goessling, Nicholas C Swenson, and Rosalind W Picard. 2009. Heartphones: Sensor earphones and mobile application for non-obtrusive health monitoring. In ISWC. IEEE.Google Scholar
- Tobias Röddiger, Daniel Wolffram, David Laubenstein, Matthias Budde, and Michael Beigl. 2019. Towards Respiration Rate Monitoring Using an In-Ear Headphone Inertial Measurement Unit. In EarComp.Google Scholar
- Peter J Rousseeuw and Annick M Leroy. 2005. Robust regression and outlier detection. John wiley & sons.Google Scholar
- Siddharth Rupavatharam and Marco Gruteser. 2019. Towards In-Ear Inertial Jaw Clenching Detection. In EarComp.Google Scholar
- Otto Stuhlman Jr. 1937. The nonlinear transmission characteristics of the auditory ossicles. The Journal of the Acoustical Society of America (1937).Google Scholar
- Frédéric E Theunissen, Kamal Sen, and Allison J Doupe. 2000. Spectral-temporal receptive fields of nonlinear auditory neurons obtained using natural sounds. Journal of Neuroscience (2000).Google Scholar
- Stefan Vogel, Markus Hülsbusch, Thomas Hennig, Vladimir Blazek, and Steffen Leonhardt. 2009. In-ear vital signs monitoring using a novel microoptic reflective sensor. IEEE Transactions on Information Technology in Biomedicine (2009).Google ScholarDigital Library
- Joseph C Volpe Jr. 2008. Heart rate monitor for controlling entertainment devices. US Patent 7,354,380.Google Scholar
- Susan E Voss and Jont B Allen. 1994. Measurement of acoustic impedance and reflectance in the human ear canal. The Journal of the Acoustical Society of America (1994).Google Scholar
- Eric S Winokur, David Da He, and Charles G Sodini. 2012. A wearable vital signs monitor at the ear for continuous heart rate and pulse transit time measurements. In EMBS. IEEE.Google Scholar
- Myung-Gyoo Won, Jae-hoon Kim, and Jong-wook Park. 2006. Temperature sensing circuit for use in semiconductor integrated circuit. US Patent 7,107,178.Google Scholar
- Daniel Yum. 1991. Bandgap voltage reference circuit. US Patent 5,053,640.Google Scholar
Index Terms
- HeadFi: bringing intelligence to all headphones
Recommendations
Semantic Hearing: Programming Acoustic Scenes with Binaural Hearables
UIST '23: Proceedings of the 36th Annual ACM Symposium on User Interface Software and TechnologyImagine being able to listen to the birds chirping in a park without hearing the chatter from other hikers, or being able to block out traffic noise on a busy street while still being able to hear emergency sirens and car honks. We introduce semantic ...
ClearBuds - wireless binaural earbuds for learning-based speech enhancement
MobiSys '22: Proceedings of the 20th Annual International Conference on Mobile Systems, Applications and ServicesWe present ClearBuds, the first end-to-end hardware and software system that utilizes a neural network to enhance speech streamed from two wireless earbuds. Real-time speech enhancement for wireless earbuds requires high-quality sound separation and ...
Leveraging Earables for Natural Calibration-Free Multi-Device Identification in Smart Environments
HotMobile '21: Proceedings of the 22nd International Workshop on Mobile Computing Systems and ApplicationsWe present Look&Lock: a novel ubiquitous system that utilizes BLE communication between smart appliances in the environment and earables worn by the user to track head orientation and determine which device she is looking at. By harnessing this ...
Comments