Please use this identifier to cite or link to this item:
Title: Inter-rater agreement between 13 otolaryngologists to diagnose otitis media in Aboriginal and Torres Strait Islander children using a telehealth approach.
Authors: Habib A-R
Perry C
Crossland G
Patel H
Kong K
Whitfield B
North H
Walton J
Da Cruz M
Suruliraj A
Smith M
Harris R
Hasan Z
Gunaratne DA
Sacks R
Singh N
Citation: Copyright © 2023 Elsevier B.V. All rights reserved.
Int J Pediatr Otorhinolaryngol. 2023 May;168:111494. doi: 10.1016/j.ijporl.2023.111494. Epub 2023 Mar 13.
Abstract: INTRODUCTION: Telehealth programs are important to deliver otolaryngology services for Aboriginal and Torres Strait Islander children living in rural and remote areas, where distance and access to specialists is a critical factor. OBJECTIVE: To evaluate the inter-rater agreement and value of increasing levels of clinical data (otoscopy with or without audiometry and in-field nurse impressions) to diagnose otitis media using a telehealth approach. DESIGN: Blinded, inter-rater reliability study. SETTING: Ear health and hearing assessments collected from a statewide telehealth program for Indigenous children living in rural and remote areas of Queensland, Australia. PARTICIPANTS: Thirteen board-certified otolaryngologists independently reviewed 80 telehealth assessments from 65 Indigenous children (mean age 5.7 ± 3.1 years, 33.8% female). INTERVENTIONS: Raters were provided increasing tiers of clinical data to assess concordance to the reference standard diagnosis: Tier A) otoscopic images alone, Tier B) otoscopic images plus tympanometry and category of hearing loss, and Tier C) as B plus static compliance, canal volume, pure-tone audiometry, and nurse impressions (otoscopic findings and presumed diagnosis). For each tier, raters were asked to determine which of the four diagnostic categories applied: normal aerated ear, acute otitis media (AOM), otitis media with effusion (OME), and chronic otitis media (COM). MAIN OUTCOME MEASURES: Proportion of agreement to the reference standard, prevalence-and-bias adjusted κ coefficients, mean difference in accuracy estimates between each tier of clinical data. RESULTS: Accuracy between raters and the reference standard increased with increased provision of clinical data (Tier A: 65% (95%CI: 63-68%), κ = 0.53 (95%CI: 0.48-0.57); Tier B: 77% (95%CI: 74-79%), 0.68 (95%CI: 0.65-0.72); C: 85% (95%CI: 82-87%), 0.79 (95%CI: 0.76-0.82)). Classification accuracy significantly improved between Tier A to B (mean difference:12%, p < 0.001) and between Tier B to C (mean difference: 8%, p < 0.001). The largest improvement in classification accuracy was observed between Tier A and C (mean difference: 20%, p < 0.001). Inter-rater agreement similarly improved with increasing provision of clinical data. CONCLUSIONS: There is substantial agreement between otolaryngologists to diagnose ear disease using electronically stored clinical data collected from telehealth assessments. The addition of audiometry, tympanometry and nurse impressions significantly improved expert accuracy and inter-rater agreement, compared to reviewing otoscopic images alone.
Click to open Pubmed Article:
Journal title: International journal of pediatric otorhinolaryngology
Volume: 168
Pages: 111494
Publication Date: 2023-05-01
Type: Journal Article
DOI: 10.1016/j.ijporl.2023.111494
Appears in Collections:(a) NT Health Research Collection

Files in This Item:
There are no files associated with this item.

Items in ePublications are protected by copyright, with all rights reserved, unless otherwise indicated.

Google Media

Google ScholarTM

Who's citing


PubMed References

Who's citing