Carrie J Cai
Lauren Wilcox
ABSTRACT
Deep neural networks (DNNs) routinely achieve state-of-the-art performance in a wide range of tasks, but it can often be challenging for them to meet end-user needs in practice. This case study reports on the development of human-AI onboarding materials (i.e., training materials for users prior to using an AI) for a DNN-based medical AI Assistant to aid in the grading of prostate cancer. Specifically, we describe how the process of developing these materials changed the team’s understanding of end-user requirements, contributing to modifications in the development and assessment of the underlying machine learning model. Importantly, we discovered that onboarding materials served as a useful boundary object for cross-functional teams, uncovering a new way to assess the ML model and specify its end-user requirements. We also present evidence of the utility of the onboarding materials by describing how it affected user strategies and decision-making with AI in a study deployment to pathologists.
- [n.d.]. Cancer of the Prostate - Cancer Stat Facts. https://seer.cancer.gov/statfacts/html/prost.htmlGoogle Scholar
- Simon Baron-Cohen. 1991. Precursors to a theory of mind: Understanding attention in others. Natural theories of mind: Evolution, development and simulation of everyday mindreading 1(1991), 233–251.Google Scholar
- Emma Beede, Elizabeth Baylor, Fred Hersch, Anna Iurchenko, Lauren Wilcox, Paisan Ruamviboonsuk, and Laura M Vardoulakis. 2020. A Human-Centered Evaluation of a Deep Learning System Deployed in Clinics for the Detection of Diabetic Retinopathy. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems. 1–12.Google ScholarDigital Library
- Carrie J. Cai, Samantha Winter, David Steiner, Lauren Wilcox, and Michael Terry. 2019. ”Hello AI”: Uncovering the Onboarding Needs of Medical Practitioners for Human-AI Collaborative Decision-Making. Proc. ACM Hum.-Comput. Interact. 3, CSCW, Article 104 (Nov. 2019), 24 pages. https://doi.org/10.1145/3359206Google ScholarDigital Library
- Narayan Hegde, Jason D Hipp, Yun Liu, Michael Emmert-Buck, Emily Reif, Daniel Smilkov, Michael Terry, Carrie J Cai, Mahul B Amin, Craig H Mermel, 2019. Similar image search for histopathology: SMILY. NPJ digital medicine 2, 1 (2019), 1–9.Google ScholarCross Ref
- Bonnie E John, Len Bass, Rick Kazman, and Eugene Chen. 2004. Identifying gaps between HCI, software engineering, and design, and boundary objects to bridge them. In CHI’04 extended abstracts on Human factors in computing systems. 1723–1724.Google Scholar
- Charlotte P Lee. 2007. Boundary negotiating artifacts: Unbinding the routine of boundary objects and embracing chaos in collaborative work. Computer Supported Cooperative Work (CSCW) 16, 3 (2007), 307–339.Google ScholarDigital Library
- Margaret Mitchell, Simone Wu, Andrew Zaldivar, Parker Barnes, Lucy Vasserman, Ben Hutchinson, Elena Spitzer, Inioluwa Deborah Raji, and Timnit Gebru. 2019. Model cards for model reporting. In Proceedings of the conference on fairness, accountability, and transparency. 220–229.Google ScholarDigital Library
- Kunal Nagpal, Davis Foote, Yun Liu, Po-Hsuan Cameron Chen, Ellery Wulczyn, Fraser Tan, Niels Olson, Jenny L Smith, Arash Mohtashamian, James H Wren, 2019. Development and validation of a deep learning algorithm for improving Gleason scoring of prostate cancer. NPJ digital medicine 2, 1 (2019), 1–10.Google Scholar
- Susan Leigh Star. 1989. The structure of ill-structured solutions: Boundary objects and heterogeneous distributed problem solving. In Distributed artificial intelligence. Elsevier, 37–54.Google Scholar
- [11] David F. Steiner, Kunal Nagpal, Rory Sayres, Davis J. Foote, Benjamin D. Wedin, Adam Pearce, Carrie J. Cai, Samantha R. Winter, Matthew Symonds, Liron Yatziv, Andrei Kapishnikov, Trissia Brown, Isabelle Flament-Auvigne, Fraser Tan, Martin C. Stumpe, Pan-Pan Jiang, Yun Liu, Po-Hsuan Cameron Chen, Greg S. Corrado, Michael Terry, and Craig H. Mermel.2020. (2020). Manuscript under review.Google Scholar
- Harold Thimbleby. 1996. Creating User Manuals for Using in Collaborative Design. In Conference Companion on Human Factors in Computing Systems (Vancouver, British Columbia, Canada) (CHI ’96). Association for Computing Machinery, New York, NY, USA, 279–280. https://doi.org/10.1145/257089.257321Google ScholarDigital Library
- Fumeng Yang, Zhuanyi Huang, Jean Scholtz, and Dustin L Arendt. 2020. How do visual explanations foster end users’ appropriate trust in machine learning?. In Proceedings of the 25th International Conference on Intelligent User Interfaces. 189–201.Google ScholarDigital Library
- Qian Yang, Aaron Steinfeld, and John Zimmerman. 2019. Unremarkable ai: Fitting intelligent decision support into critical, clinical decision-making processes. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems. 1–11.Google ScholarDigital Library
- Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency. 295–305.Google ScholarDigital Library
Index Terms
- Onboarding Materials as Cross-functional Boundary Objects for Developing AI Assistants
Recommendations
Artificial intelligence assistants and risk: framing a connectivity risk narrative
AbstractOur social relations are changing, we are now not just talking to each other, but we are now also talking to artificial intelligence (AI) assistants. We claim AI assistants present a new form of digital connectivity risk and a key aspect of this ...
Introduction to Explainable AI
CHI EA '20: Extended Abstracts of the 2020 CHI Conference on Human Factors in Computing SystemsAs Artificial Intelligence (AI) technologies are increasingly used to make important decisions and perform autonomous tasks, providing explanations that allow users to understand the AI has become a ubiquitous concern in human-AI interaction. Recently, a ...
Asking ‘Why’ in AI: Explainability of intelligent systems – perspectives and challenges
SummaryRecent rapid progress in machine learning (ML), particularly so‐called ‘deep learning’, has led to a resurgence in interest in explainability of artificial intelligence (AI) systems, reviving an area of research dating back to the 1970s. The aim of ...
Comments