Evaluation of Computer-aided Detection Systems in the Detection of Small Invasive Breast Carcinoma

Published Online:Oct 1 2007https://doi.org/10.1148/radiol.2451060760

Abstract

Purpose: To retrospectively compare two CAD systems for detecting invasive breast cancers manifesting as noncalcified masses smaller than 16 mm.

Materials and Methods: Waiver of informed consent was granted by the Institutional Review Board that approved this HIPAA-compliant study. Mammograms obtained from two institutions providing consecutive invasive carcinomas manifesting as noncalcified masses smaller than 16 mm were evaluated by using two commercially available CAD systems (R2 ImageChecker M1000, version 5.0A and iCAD Second Look, version 6.0 mid operating point). To provide statistical power to test for a possible 10% difference in the sensitivity performance between the systems, 192 consecutive mammographic studies (182 unifocal, six multifocal, and four bilateral cancers) were collected. Masses were characterized using the Breast Imaging Reporting and Data System (BI-RADS). Per study specificity and mass false marker rate were determined by using 51 normal four-view studies, while scoring only the mass false-positive marks for noncalcified masses. Associations between mass characteristics and supplying institution were compared by using χ² tests. A P value of .05 was considered to indicate a significant difference.

Results: The respective per study sensitivity, per image (ie, per view) sensitivity, per study specificity, and mass false-positive marker rates were 81.8%, 64.7%, 39.2%, and 1.08 for the R2 ImageChecker M1000 system, and 60.9%, 42.6%, 31.4%, and 1.41 for the iCAD Second Look system. The overall per study and per image sensitivities were significantly better for R2 than for iCAD (McNemar test, all P < .001), with a nonsignificant higher per study specificity and lower mass false marker rate on normal studies. CAD results demonstrated at least a 20% variation between BI-RADS categories 4a and 5 for per study and per image sensitivity.

Conclusion: A statistically significant difference was observed in per study and per image sensitivity in our mammography data set with small (<16 mm), noncalcified invasive breast malignancies between two CAD systems. Differences in per study specificity and mass false marker rate were noted but were not statistically significant.

References

1 Thurfjell EL, Lernevall KA, Taube AA. Benefit of independent double reading in a population-based mammography screening program. Radiology 1994; 191: 241–244.
Google Scholar
2 Anderson ED, Muir BB, Walsh JS, Kirkpatrick AE. The efficacy of double reading mammograms in breast screening. Clin Radiol 1994; 49: 248–251.
MedlineGoogle Scholar
3 Thurfjell E. Mammography screening: one versus two views and independent double reading. Acta Radiol 1994; 35: 345–350.
Google Scholar
4 Warren RM, Duffy SW. Comparison of single reading with double reading of mammograms, and change in effectiveness with experience. Br J Radiol 1995; 68: 958–962.
MedlineGoogle Scholar
5 Ciatto S, Del Turco MR, Morrone D, et al. Independent double reading for screening mammograms. J Med Screen 1995; 2: 99–101.
MedlineGoogle Scholar
6 Freer TW, Ulissey MJ. Screening mammography with computer-aided detection: prospective study of 12 860 patients in a community breast center. Radiology 2001; 220: 781–786.
Google Scholar
7 Birdwell RL, Bandodkar P, Ikeda DM. Computer-aided detection with screening mammography in a university hospital setting. Radiology 2005; 236: 451–457.
Google Scholar
8 Morton MJ, Whaley DH, Brandt KR, Amrami KK. Screening mammograms: interpretation with computer-aided detection—prospective evaluation. Radiology 2006; 239: 375–383.
Google Scholar
9 Gur D, Sumkin JH, Rockette HE, et al. Changes in breast cancer detection and mammography recall rates after the introduction of a computer-aided detection system. J Natl Cancer Inst 2004; 96: 185–190.
MedlineGoogle Scholar
10 Cupples TE, Cunningham JE, Reynolds JC. Impact of computer-aided detection in a regional screening mammography program. AJR Am J Roentgenol 2005; 185: 944–950.
MedlineGoogle Scholar
11 Feig SA, Sickles EA, Evans WP, Linver MN. Re: Changes in breast cancer detection and mammography recall rates after the introduction of a computer-aided detection system [letter]. J Natl Cancer Inst 2004; 96: 1260–1261; author reply 1261.
MedlineGoogle Scholar
12 Michaelson JS, Satija S, Kopans D, et al. Gauging the impact of breast carcinoma screening in terms of tumor size and death rate. Cancer 2003; 98: 2114–2124.
MedlineGoogle Scholar
13 Michaelson JS, Silverstein M, Wyatt J, et al. Predicting the survival of patients with breast carcinoma using tumor size. Cancer 2002; 95: 713–723.
MedlineGoogle Scholar
14 Tabar L, Duffy SW, Vitak B, Chen HH, Prevost TC. The natural history of breast carcinoma: what have we learned from screening? Cancer 1999; 86: 449–462.
MedlineGoogle Scholar
15 Tabar L, Vitak B, Chen HH, Yen MF, Duffy SW, Smith RA. Beyond randomized controlled trials: organized mammographic screening substantially reduces breast carcinoma mortality. Cancer 2001; 91: 1724–1731.
MedlineGoogle Scholar
16 Tabar L, Dean PB. Mammography and breast cancer: the new era. Int J Gynaecol Obstet 2003; 82: 319–326.
MedlineGoogle Scholar
17 Duffy SW, Tabar L, Vitak B, et al. The relative contributions of screen-detected in situ and invasive breast carcinomas in reducing mortality from the disease. Eur J Cancer 2003; 39: 1755–1760.
MedlineGoogle Scholar
18 Miller AB, To T, Baines CJ, Wall C. The Canadian National Breast Screening Study-1: breast cancer mortality after 11 to 16 years of follow-up—a randomized screening trial of mammography in women age 40 to 49 years. Ann Intern Med 2002; 137: 305–312.
MedlineGoogle Scholar
19 Miller AB, To T, Baines CJ, Wall C. Canadian National Breast Screening Study-2: 13-year results of a randomized trial in women aged 50–59 years. J Natl Cancer Inst 2000; 92: 1490–1499.
MedlineGoogle Scholar
20 Destounis SV, DiNitto P, Logan-Young W, Bonaccio E, Zuley ML, Willison KM. Can computer-aided detection with double reading of screening mammograms help decrease the false-negative rate? initial experience. Radiology 2004; 232: 578–584.
Google Scholar
21 Warren Burhenne LJ, Wood SA, D'Orsi CJ, et al. Potential contribution of computer-aided detection to the sensitivity of screening mammography. Radiology 2000; 215: 554–562.
Google Scholar
22 Birdwell RL, Ikeda DM, O'Shaughnessy KF, Sickles EA. Mammographic characteristics of 115 missed cancers later detected with screening mammography and the potential utility of computer-aided detection. Radiology 2001; 219: 192–202.
Google Scholar
23 Balleyguier C, Kinkel K, Fermanian J, et al. Computer-aided detection (CAD) in mammography: does it help the junior or the senior radiologist? Eur J Radiol 2005; 54: 90–96.
Google Scholar
24 Vyborny CJ, Doi T, O'Shaughnessy KF, Romsdahl HM, Schneider AC, Stein AA. Breast cancer: importance of spiculation in computer-aided detection. Radiology 2000; 215: 703–707.
Google Scholar
25 Gur D, Stalder JS, Hardesty LA, et al. Computer-aided detection performance in mammographic examination of masses: assessment. Radiology 2004; 233: 418–423.
Google Scholar
26 Baker JA, Rosen EL, Lo JY, Gimenez EI, Walsh R, Soo MS. Computer-aided detection (CAD) in screening mammography: sensitivity of commercial CAD systems for detecting architectural distortion. AJR Am J Roentgenol 2003; 181: 1083–1088.
MedlineGoogle Scholar
27 Soo MS, Rosen EL, Xia JQ, Ghate S, Baker JA. Computer-aided detection of amorphous calcifications. AJR Am J Roentgenol 2005; 184: 887–992.
MedlineGoogle Scholar
28 Malich A, Sauner D, Marx C, et al. Influence of breast lesion size and histologic findings on tumor detection rate of a computer-aided detection system. Radiology 2003; 228: 851–856.
Google Scholar
29 Brem RF, Hoffmeister JW, Zisman G, DeSimio MP, Rogers SK. A computer-aided detection system for the evaluation of breast cancer by mammographic appearance and lesion size. AJR Am J Roentgenol 2005; 184: 893–896.
MedlineGoogle Scholar
30 Nishikawa RM, Giger ML, Doi K, et al. Effect of case selection on the performance of computer-aided detection schemes. Med Phys 1994; 21: 265–269.
MedlineGoogle Scholar
31 Taylor P, Given-Wilson R, Champness J, Potts HW, Johnston K. Assessing the impact of CAD on the sensitivity and specificity of film readers. Clin Radiol 2004; 59: 1099–1105.
MedlineGoogle Scholar

Article History

Published in print: 2007

Vol. 245, No. 1

Metrics

Altmetric Score

PDF download