Yincheng Jin
ABSTRACT
With the aging of the population and the long-time wearing of earphones, hearing health has gradually emerged as a worldwide health issue. Early detection of hearing health conditions would greatly reduce potential risks with timely medical intervention. This study proposes an earphone-based ear condition monitoring system, named EarHealth, which is low-cost, non-invasive, and easily usable in daily life. It can detect three major hearing health conditions: ruptured eardrum, earwax buildup and blockage, and otitis media. By analyzing the recorded echoes evoked by a chirp sound stimulus, EarHealth recognizes the distinguishable characteristics from ear canal structure and eardrum mobility. EarHealth achieves an accuracy of 82.6% in 92 human subjects, including 27 normal subjects, 22 patients with ruptured eardrum, 25 patients with otitis media, and 18 patients with earwax blockage. EarHealth is the first earphone-based system capable of monitoring hearing health conditions by utilizing the ear canal geometry and eardrum mobility. It is anticipated that EarHealth would provide pervasive and proactive protection for hearing health.
- Penelope Abbott, Sara Rosenkranz, Wendy Hu, Hasantha Gunasekera, and Jennifer Reath. 2014. The effect and acceptability of tympanometry and pneumatic otoscopy in general practitioner diagnosis and management of childhood ear disease. BMC Family Practice 15, 181 (2014), 1--10.Google Scholar
Cross Ref
- Ashwin Ahuja, Andrea Ferlini, and Cecilia Mascolo. 2021. PilotEar: Enabling In-ear Inertial Navigation. In Adjunct Proceedings of the 2021 ACM International Joint Conference on Pervasive and Ubiquitous Computing and Proceedings of the 2021 ACM International Symposium on Wearable Computers. 139--145.Google Scholar
- Takashi Amesaka, Hiroki Watanabe, and Masanori Sugimoto. 2019. Facial expression recognition using ear canal transfer function. In Proceedings of the 23rd International Symposium on Wearable Computers. 1--9.Google ScholarDigital Library
- Oliver Amft, Mathias Stäger, Paul Lukowicz, and Gerhard Tröster. 2005. Analysis of chewing sounds for dietary monitoring. In Proceedings of the 2005 International Conference on Ubiquitous Computing. Springer, 56--72.Google ScholarDigital Library
- Michael J Babb, Raymond L Hilsinger Jr, Harold W Korol, and Robert D Wilcox. 2004. Modern acoustic reflectometry: accuracy in diagnosing otitis media with effusion. Ear, Nose & Throat Journal 83, 9 (2004), 622--624.Google ScholarCross Ref
- Elizabeth D Barnett, Jerome O Klein, Kimberly A Hawkins, Howard J Cabral, Margaret Kenna, and Gerald Healy. 1998. Comparison of spectral gradient acoustic reflectometry and other diagnostic techniques for detection of middle ear effusion in children with middle ear disease. The Pediatric Infectious Disease Journal 17, 6 (1998), 556--559.Google ScholarCross Ref
- Abdelkareem Bedri, Richard Li, Malcolm Haynes, Raj Prateek Kosaraju, Ishaan Grover, Temiloluwa Prioleau, Min Yan Beh, Mayank Goel, Thad Starner, and Gregory Abowd. 2017. EarBit: using wearable sensors to detect eating episodes in unconstrained environments. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 1, 3 (2017), 1--20.Google ScholarDigital Library
- Stan L Block, Ellen Mandel, Samuel McLinn, Michael E Pichichero, Shelly Bernstein, Sandra Kimball, and Joseph Kozikowski. 1998. Spectral gradient acoustic reflectometry for the detection of middle ear effusion by pediatricians and parents. The Pediatric Infectious Disease Journal 17, 6 (1998), 560--564.Google ScholarCross Ref
- Nam Bui, Nhat Pham, Jessica Jacqueline Barnitz, Zhanan Zou, Phuc Nguyen, Hoang Truong, Taeho Kim, Nicholas Farrow, Anh Nguyen, Jianliang Xiao, et al. 2019. eBP: A wearable system for frequent and comfortable blood pressure monitoring from user's ear. In Proceedings of the 25th Annual International Conference on Mobile Computing and Networking (MobiCom). 1--17.Google ScholarDigital Library
- Jamie Bullock. 2007. LibXtract: a Lightweight Library for Audio Feature Extraction.. In Proceedings of the International Computer Music Conference (ICMC).Google Scholar
- Kayla-Jade Butkow, Ting Dang, Andrea Ferlini, Dong Ma, and Cecilia Mascolo. 2021. Motion-resilient Heart Rate Monitoring with In-ear Microphones. arXiv preprint arXiv:2108.09393 (2021).Google Scholar
- Chao Cai, Rong Zheng, and Menglan Hu. 2019. A survey on acoustic sensing. arXiv preprint arXiv:1901.03450 (2019).Google Scholar
- Yetong Cao, Huijie Chen, Fan Li, and Yu Wang. 2021. CanalScan: Tongue-Jaw Movement Recognition via Ear Canal Deformation Sensing. In Proceedings of the 2021 IEEE Conference on Computer Communications (INFOCOM). IEEE, 1--10.Google ScholarDigital Library
- Justin Chan, Sharat Raju, Rajalakshmi Nandakumar, Randall Bly, and Shyamnath Gollakota. 2019. Detecting middle ear fluid using smartphones. Science Translational Medicine 11, 492 (2019), eaav1102.Google Scholar
- D. K. Cherry, D. A. Woodwell, and E. A. Rechtsteiner. 2005. National Ambulatory Medical Care Survey: 2005 summary. Advance Data 387 (2005), 1--39.Google Scholar
- Mayo Clinic. 2019. Hearing loss. https://www.mayoclinic.org/diseases-conditions/hearing-loss/symptoms-causes/syc-20373072.Google Scholar
- Jerome T Combs, Hugh W Busey, and Kresimir Ukraincik. 1999. Device and process for generating and measuring the shape of an acoustic reflectance curve of an ear. US Patent 5,868,682.Google Scholar
- Andrea Ferlini, Dong Ma, Robert Harle, and Cecilia Mascolo. 2021. EarGate: gait-based user identification with in-ear microphones. In Proceedings of the 27th Annual International Conference on Mobile Computing and Networking. 337--349.Google ScholarDigital Library
- Yang Gao, Yincheng Jin, Jagmohan Chauhan, Seokmin Choi, Jiyang Li, and Zhanpeng Jin. 2021. Voice In Ear: Spoofing-Resistant and Passphrase-Independent Body Sound Authentication. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 5, 1, Article 12 (2021), 25 pages.Google ScholarDigital Library
- Yang Gao, Wei Wang, Vir V. Phoha, Wei Sun, and Zhanpeng Jin. 2019. EarEcho: Using ear canal echo for wearable authentication. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 3, 3 (2019), 1--24.Google ScholarDigital Library
- Xiying Guan, Yongzheng Chen, and Rong Z. Gan. 2014. Factors Affecting Loss of Tympanic Membrane Mobility in Acute Otitis Media Model of Chinchilla. Hearing Research 309 (2014), 136--146.Google ScholarCross Ref
- Xiying Guan and Rong Z. Gan. 2013. Mechanisms of Tympanic Membrane and Incus Mobility Loss in Acute Otitis Media Model of Guinea Pig. Journal of the Association for Research in Otolaryngology 14 (2013), 295--307.Google ScholarCross Ref
- Preben Homøe, Kari Kværner, Janet R Casey, Roger AMJ Damoiseaux, Thijs MA van Dongen, Hasantha Gunasekera, Ramon G Jensen, Ellen Kvestad, Peter S Morris, and Heather M Weinreich. 2017. Panel 1: epidemiology and diagnosis. Otolaryngology-Head and Neck Surgery 156, 4_suppl (2017), S1--S21.Google Scholar
- Yincheng Jin, Yang Gao, Yanjun Zhu, Wei Wang, Jiyang Li, Seokmin Choi, Zhangyu Li, Jagmohan Chauhan, Anind K Dey, and Zhanpeng Jin. 2021. SonicASL: An Acoustic-based Sign Language Gesture Recognizer Using Earphones. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 5, 2 (2021), 1--30.Google ScholarDigital Library
- Michael Kearns and Dana Ron. 1999. Algorithmic stability and sanity-check bounds for leave-one-out cross-validation. Neural Computation 11, 6 (1999), 1427--1453.Google ScholarDigital Library
- Sandra Kimball. 1998. Acoustic reflectometry: spectral gradient analysis for improved detection of middle ear effusion in children. The Pediatric Infectious Disease Journal 17, 6 (1998), 552--555.Google ScholarCross Ref
- Helene J Krouse, Anthony E Magit, Sarah O'connor, Seth R Schwarz, and Sandra A Walsh. 2017. Plain Language Summary: Earwax (Cerumen Impaction). Otolaryngology-Head and Neck Surgery 156, 1 (2017), 30--37.Google ScholarCross Ref
- Nicholas D Lane, Petko Georgiev, and Lorena Qendro. 2015. Deepear: robust smartphone audio sensing in unconstrained acoustic environments using deep learning. In Proceedings of the 2015 ACM International Joint Conference on Pervasive and Ubiquitous Computing. 283--294.Google ScholarDigital Library
- Steven F LeBoeuf, Michael E Aumer, William E Kraus, Johanna L Johnson, and Brian Duscha. 2014. Earbud-based sensor for the assessment of energy expenditure, heart rate, and VO2max. Medicine and Science in Sports and Exercise 46, 5 (2014), 1046.Google ScholarCross Ref
- Hyewon Lee, Tae Hyun Kim, Jun Won Choi, and Sunghyun Choi. 2015. Chirp signal-based aerial acoustic communication for smart devices. In Proceedings of the 2015 IEEE Conference on Computer Communications (INFOCOM). IEEE, 2407--2415.Google ScholarCross Ref
- David B Lindell, Gordon Wetzstein, and Vladlen Koltun. 2019. Acoustic non-line-of-sight imaging. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 6780--6789.Google ScholarCross Ref
- Tal Marom, Oded Kraus, Nadeem Habashi, and Sharon Ovnat Tamir. 2019. Emerging Technologies for the Diagnosis of Otitis Media. Otolaryngology-Head and Neck Surgery 1 (2019), 10.Google Scholar
- Ritvik P. Mehta, John J. Rosowski, Susan E. Voss, Ellen O'Neil, and Saumil N. Merchant. 2006. Determinants of Hearing Loss in Perforations of the Tympanic Membrane. Otology & Neurotology 27, 2 (2006), 126--143.Google ScholarCross Ref
- Mark Mirtchouk, Christopher Merck, and Samantha Kleinberg. 2016. Automated estimation of food type and amount consumed from body-worn audio and motion sensors. In Proceedings of the 2016 ACM International Joint Conference on Pervasive and Ubiquitous Computing. 451--462.Google ScholarDigital Library
- Aaron C Moberly, Margaret Zhang, Lianbo Yu, Metin Gurcan, Caglar Senaras, Theodoros N Teknos, Charles A Elmaraghy, Nazhat Taj-Schaal, and Garth F Essig. 2018. Digital otoscopy versus microscopy: How correct and confident are ear experts in their diagnoses? J. Telemedicine and Telecare 24, 7 (2018), 453--459.Google ScholarCross Ref
- Lorenzo Monasta, Luca Ronfani, Federico Marchetti, Marcella Montico, Liza Vecchi Brumatti, Alessandro Bavcar, Domenico Grasso, Chiara Barbiero, and Giorgio Tamburlini. 2012. Burden of disease caused by otitis media: systematic review and global estimates. PloS One 7, 4 (2012), e36226.Google ScholarCross Ref
- American Academy of Pediatrics Subcommittee on Management of Acute Otitis Media et al. 2004. Diagnosis and management of acute otitis media. Pediatrics 113, 5 (2004), 1451.Google ScholarCross Ref
- World Health Organization. 2004. Chronic suppurative otitis media: burden of illness and management options. Technical Report. Geneva, Switzerland.Google Scholar
- World Health Organization. 2015. Hearing loss due to recreational exposure to loud sounds: a review. Technical Report. Geneva, Switzerland.Google Scholar
- World Health Organization. 2022. Hearing Loss. who.int/health-topics/hearing-loss#tab=tab_2.Google Scholar
- Ali Qureishi, Yan Lee, Katherine Belfield, John P Birchall, and Matija Daniel. 2014. Update on otitis media-prevention and treatment. Infection and Drug Resistance 7 (2014), 15.Google Scholar
- Valentin Radu, Catherine Tong, Sourav Bhattacharya, Nicholas D Lane, Cecilia Mascolo, Mahesh K Marina, and Fahim Kawsar. 2018. Multimodal deep learning for activity and context recognition. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 1, 4 (2018), 1--27.Google ScholarDigital Library
- Anne GM Schilder, Tasnee Chonmaitree, Allan W Cripps, Richard M Rosenfeld, Margaretha L Casselbrant, Mark P Haggard, and Roderick P Venekamp. 2016. Otitis media. Nature Reviews Disease Primers 2 (2016), 16063.Google ScholarCross Ref
- Edgar AG Shaw and Ryunen Teranishi. 1968. Sound pressure generated in an external-ear replica and real human ears by a nearby point source. The Journal of the Acoustical Society of America 44, 1 (1968), 240--249.Google ScholarCross Ref
- Ilia Shumailov, Laurent Simon, Jeff Yan, and Ross Anderson. 2019. Hearing your touch: A new acoustic side channel on smartphones. arXiv preprint arXiv:1903.11137 (2019).Google Scholar
- Hyewon Suh, Nina Shahriaree, Eric B Hekler, and Julie A Kientz. 2016. Developing and validating the user burden scale: A tool for assessing user burden in computing systems. In Proceedings of the 2016 CHI Conference on human factors in computing systems. 3988--3999.Google ScholarDigital Library
- Wei Sun, Franklin Mingzhe Li, Benjamin Steeper, Songlin Xu, Feng Tian, and Cheng Zhang. 2021. TeethTap: Recognizing Discrete Teeth Gestures Using Motion and Acoustic Sensing on an Earpiece. In Proceedings of the 26th International Conference on Intelligent User Interfaces. 161--169.Google ScholarDigital Library
- David W Teele and John Teele. 1984. Detection of middle ear effusion by acoustic reflectometry. The Journal of Pediatrics 104, 6 (1984), 832--838.Google ScholarCross Ref
- John H Teele. 1986. Ear pathology diagnosis apparatus and method. US Patent 4,601,295.Google Scholar
- Heikki Teppo, Matti Revonta, Henriikka Lindén, and Arto Palmu. 2006. Detection of middle-ear fluid in children with spectral gradient acoustic reflectometry: A screening tool for nurses? Scandinavian Journal of Primary Health Care 24, 2 (2006), 88--92.Google ScholarCross Ref
- Yanwen Wang, Jiaxing Shen, and Yuanqing Zheng. 2020. Push the limit of acoustic gesture recognition. IEEE Transactions on Mobile Computing (2020).Google ScholarDigital Library
- Zi Wang, Sheng Tan, Linghan Zhang, Yili Ren, Zhi Wang, and Jie Yang. 2021. EarDynamic: An Ear Canal Deformation Based Continuous User Authentication Using In-Ear Wearables. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 5, 1 (2021), 1--27.Google ScholarDigital Library
- Tony Wright. 2015. Ear Wax. BMJ Clinical Evidence 07, 0504 (2015), 1--24.Google Scholar
- Hanbin Zhang, Chen Song, Aosen Wang, Chenhan Xu, Dongmei Li, and Wenyao Xu. 2019. Pdvocal: Towards privacy-preserving parkinson's disease detection using non-speech body sounds. In Proceedings of the The 25th Annual International Conference on Mobile Computing and Networking. 1--16.Google ScholarDigital Library
- Qian Zhang, Dong Wang, Run Zhao, and Yinggang Yu. 2021. SoundLip: Enabling Word and Sentence-level Lip Interaction for Smart Devices. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 5, 1 (2021), 1--28.Google ScholarDigital Library
Index Terms
- EarHealth: an earphone-based acoustic otoscope for detection of multiple ear diseases in daily life
Recommendations
EarDynamic: An Ear Canal Deformation Based Continuous User Authentication Using In-Ear Wearables
Biometric-based authentication is gaining increasing attention for wearables and mobile applications. Meanwhile, the growing adoption of sensors in wearables also provides opportunities to capture novel wearable biometrics. In this work, we propose ...
An ear canal deformation based continuous user authentication using earables
MobiCom '21: Proceedings of the 27th Annual International Conference on Mobile Computing and NetworkingBiometric-based authentication is gaining increasing attention for wearables and mobile applications. Meanwhile, the growing adoption of sensors in wearables also provides opportunities to capture novel wearable biometrics. In this work, we propose ...
ToothSonic: Earable Authentication via Acoustic Toothprint
Earables (ear wearables) are rapidly emerging as a new platform encompassing a diverse range of personal applications. The traditional authentication methods hence become less applicable and inconvenient for earables due to their limited input interface. ...
Comments