uu.seUppsala University Publications
Change search
ReferencesLink to record
Permanent link

Direct link
On the comparison of FROC curves in mammography CAD systems.
Uppsala University, Disciplinary Domain of Humanities and Social Sciences, Faculty of Social Sciences, Department of Statistics.
2005 (English)In: Medical physics (Lancaster), ISSN 0094-2405, Vol. 32, no 2, 412-7 p.Article in journal (Refereed) Published
Abstract [en]

We present a novel method for assessing the performance of computer-aided detection systems on unseen cases at a given sensitivity level. The sampling error introduced when training the system on a limited data set is captured as the uncertainty in determining the system threshold that would yield a certain predetermined sensitivity on unseen data sets. By estimating the distribution of system thresholds, we construct a confidence interval for the expected number of false positive markings per image at a given sensitivity. We present two alternative procedures for estimating the probability density functions needed for the construction of the confidence interval. The first is based on the common assumption of Poisson distributed number of false positive markings per image. This procedure also relies on the assumption of independence between false positives and sensitivity, an assumption that can be relaxed with the second procedure, which is nonparametric. The second procedure uses the bootstrap applied to the data generated in the leave-one-out construction of the FROC curve, and is a fast and robust way of obtaining the desired confidence interval. Standard FROC curve analysis does not account for the uncertainty in setting the system threshold, so this method should allow for a more fair comparison of different systems. The resulting confidence intervals are surprisingly wide. For our system a conventional FROC curve analysis yields 0.47 false positive markings per image at 90% sensitivity. The 90% confidence interval for the number of false positive markings per image is (0.28, 1.02) with the parametric procedure and (0.27, 1.04) with the nonparametric bootstrap. Due to its computational simplicity and its allowing more fair comparisons between systems, we propose this method as a complement to the traditionally presented FROC curves.

Place, publisher, year, edition, pages
2005. Vol. 32, no 2, 412-7 p.
National Category
Probability Theory and Statistics
URN: urn:nbn:se:uu:diva-225706PubMedID: 15789587OAI: oai:DiVA.org:uu-225706DiVA: diva2:722103
Available from: 2014-06-05 Created: 2014-06-05 Last updated: 2014-06-05

Open Access in DiVA

No full text


Search in DiVA

By author/editor
Hermansson, Anna Bornefalk
By organisation
Department of Statistics
In the same journal
Medical physics (Lancaster)
Probability Theory and Statistics

Search outside of DiVA

GoogleGoogle Scholar
The number of downloads is the sum of all downloads of full texts. It may include eg previous versions that are now no longer available

Altmetric score

Total: 296 hits
ReferencesLink to record
Permanent link

Direct link