Comparing the accuracy of syndrome surveillance systems in detecting influenza-like illness: GUARDIAN vs. RODS vs. electronic medical record reports

作者：Silva Julio C; Shah Shital C; Rumoro Dino P; Bayram Jamil D; Hallock Marilyn M; Gibbs Gillian S^*; Waddell Michael J

来源：Artificial Intelligence in Medicine, 2013, 59(3): 169-174.

DOI：10.1016/j.artmed.2013.09.001

• Biosurveillance
• Syndromic surveillance systems
• Influenza-like illness
• Public health informatics
• GUARDIAN
• RODS

Background: A highly sensitive real-time syndrome surveillance system is critical to detect, monitor, and control infectious disease outbreaks, such as influenza. Direct comparisons of diagnostic accuracy of various surveillance systems are scarce. %26lt;br%26gt;Objective: To statistically compare sensitivity and specificity of multiple proprietary and open source syndrome surveillance systems to detect influenza-like illness (ILI). %26lt;br%26gt;Methods: A retrospective, cross-sectional study was conducted utilizing data from 1122 patients seen during November 1-7,2009 in the emergency department of a single urban academic medical center. The study compared the Geographic Utilization of Artificial Intelligence in Real-time for Disease Identification and Alert Notification (GUARDIAN) system to the Complaint Coder (CoCo) of the Real-time Outbreak Detection System (RODS), the Symptom Coder (SyCo) of RODS, and to a standardized report generated via a proprietary electronic medical record (EMR) system. Sensitivity, specificity, and accuracy of each classifier%26apos;s ability to identify ILI cases were calculated and compared to a manual review by a board-certified emergency physician. Chi-square and McNemar%26apos;s tests were used to evaluate the statistical difference between the various surveillance systems. %26lt;br%26gt;Results: The performance of GUARDIAN in detecting ILI in terms of sensitivity, specificity, and accuracy, as compared to a physician chart review, was 95.5%, 97.6%, and 97.1%, respectively. The EMR,generated reports were the next best system at identifying disease activity with a sensitivity, specificity, and accuracy of 36.7%, 99.3%, and 83.2%, respectively. RODS (CoCo and SyCo) had similar sensitivity (35.3%) but slightly different specificity (CoCo = 98.9%; SyCo = 99.3%). The GUARDIAN surveillance system with its multiple data sources performed significantly better compared to CoCo (chi(2) = 130.6, p %26lt; 0.05), SyCo (chi(2) = 125.2,p %26lt; 0.05), and EMR-based reports (chi(2) = 121.3,p %26lt; 0.05). In addition, similar significant improvements in the accuracy (%26gt;12%) and sensitivity (%26gt;47%) were observed for GUARDIAN with only chief complaint data as compared to RODS (CoCo and SyCo) and EMR-based reports. %26lt;br%26gt;Conclusion: In our study population, the GUARDIAN surveillance system, with its ability to utilize multiple data sources from patient encounters and real-time automaticity, demonstrated a more robust performance when compared to standard EMR-based reports and the RODS systems in detecting ILI. More large-scale studies are needed to validate the study findings, and to compare the performance of GUARDIAN in detecting other infectious diseases.

• 出版日期2013-11