Research

Improving inter-observer variability in the evaluation of ultrasonographic features of polycystic ovaries

Marla E Lujan¹ , Donna R Chizen¹ , Andrew K Peppin² , Stefan Kriegler³ , David A Leswick³ , Terri G Bloski¹ and Roger A Pierson¹

¹Obstetrics, Gynecology & Reproductive Sciences, University of Saskatchewan, Saskatoon, Canada

²Radiology & Diagnostic Imaging, University of Alberta, Edmonton, Canada

³Academic Department of Medical Imaging, University of Saskatchewan, Saskatoon, Canada

author email corresponding author email

Reproductive Biology and Endocrinology 2008, 6:30doi:10.1186/1477-7827-6-30

Published:	18 July 2008

Abstract

Background

We recently reported poor inter-observer agreement in identifying and quantifying individual ultrasonographic features of polycystic ovaries. Our objective was to determine the effect of a training workshop on reducing inter-observer variation in the ultrasonographic evaluation of polycystic ovaries.

Methods

Transvaginal ultrasound recordings from thirty women with polycystic ovary syndrome (PCOS) were evaluated by three radiologists and three reproductive endocrinologists both before and after an ultrasound workshop. The following endpoints were assessed: 1) follicle number per ovary (FNPO), 2) follicle number per single cross-section (FNPS), 3) largest follicle diameter, 4) ovarian volume, 5) follicle distribution pattern and 6) presence of a corpus luteum (CL). Lin's concordance correlation coefficients (rho) and kappa statistics for multiple raters (kappa) were used to assess level of inter-observer agreement (>0.80 good, 0.60 – 0.80 moderate/fair, <0.60 poor).

Results

Following the workshop, inter-observer agreement improved for the evaluation of FNPS (rho = 0.70, delta rho = +0.11), largest follicle diameter (rho = 0.77, delta rho = +0.10), ovarian volume (rho = 0.84, delta rho = +0.12), follicle distribution pattern (kappa = 0.80, delta kappa = +0.21) and presence of a CL (kappa = 0.87, delta kappa = +0.05). No improvement was evident for FNPO (rho = 0.54, delta rho = -0.01). Both radiologists and reproductive endocrinologists demonstrated improvement in scores (p < 0.001).

Conclusion

Reliability in evaluating ultrasonographic features of polycystic ovaries can be significantly improved following participation in a training workshop. If ultrasonographic evidence of polycystic ovaries is to be used as an objective measure in the diagnosis of PCOS, then standardized training modules should be implemented to unify the approach to evaluating polycystic ovarian morphology.