At-a-glance – Development and formative evaluation of the Canadian Armed Forces Surveillance and Outbreak Management System (CAF SOMS): applications for COVID-19 and beyond

The COVID-19 pandemic and the Canadian Armed Forces

During the early stages of the COVID-19 pandemic, outbreak management relied on traditional public health measures, such as case isolation and contact tracing.^{Footnote 1Footnote 2} The Canadian Forces Health Services Group (CF H Svcs Gp) operates a separate medical and public health system for Canadian Armed Forces (CAF) personnel. As part of its public health program, the CAF has developed its own contact tracing capability, which is run in coordination with local public health authorities across Canada and abroad.^{Footnote 3Footnote 4Footnote 5} To meet the growing information management needs associated with case and contact management, the CF H Svcs Gp’s National Contact Tracing Team developed the Canadian Armed Forces Surveillance and Outbreak Management System (CAF SOMS), a centralized, secure, relational database designed to facilitate contact tracing efforts while also providing real-time disease surveillance.

COVID-19 data management: systems and information gaps

Case and contact management for COVID-19 requires multipurpose health data collection tools that can be used for clinical case management, contact tracing, surveillance and epidemiological research. A comprehensive data management system can enable timely and effective follow-up, describe disease transmission dynamics, identify disease clusters and evaluate outbreak management effectiveness.

The COVID-19 pandemic highlighted existing limitations in the Canadian public health data infrastructure that impacted the efficiency and effectiveness of measures taken to control the COVID-19 outbreak.^{Footnote 6Footnote 7Footnote 8} These limitations include poorly designed data collection tools, non-interoperability of systems across jurisdictions, inadequate data sharing policies and regulations, nonintegration with electronic medical records and laboratory data, and nonstandardized data. To overcome these difficulties, several provinces, municipalities and the CAF have separately developed COVID-19 case and contact management information technology solutions.^{Footnote 6} 

During the early phases of the pandemic, an urgent need for a comprehensive data management tool specific to CAF contact tracing was identified. The main goal was to replace spreadsheets that were cumbersome to complete, file, track and collate. These issues resulted in poor data quality and limited real-time CAF COVID-19 situational awareness beyond case monitoring. Also, it was recognized that this system should collect additional demographic, clinical and risk factor information to improve understanding of disease trends among population subgroups. To overcome these limitations, the National Contact Tracing Team worked with the Health Informatics team to design a centralized, easy-to-use, secure, relational database that linked case and contact data, allowed for multiple follow-up entries, improved data quality and could streamline data collection for current and future CAF disease surveillance, beyond COVID-19.

Development and formative evaluation of the CAF SOMS

The CAF SOMS was developed, evaluated and implemented in three stages, the last of which is ongoing (Figure 1).

Figure 1 - Text description

• System development
  • Definition of business requirements
  • Definition of variables
  • Validations
  • Definition of role permissions
  • Design of forms
  • Refinement of functionality
• Formative evaluation
  • 3 phases of testing and piloting integrated with the CAF SOMS development process
  • Feedback used to iteratively refine the system throughout the process
• Implementation, refinement, and expansion
  • CAF-wide roll-out
  • Further evaluation and refinement
  • Added functionality (case follow-up form)
  • Expanded use for general CAF disease surveillance and outbreak management (additional pathogens)
  • Streamlining and integration with other CAF information management systems

Abbreviation: CAF SOMS, Canadian Armed Forces Surveillance and Outbreak Management System.

The design of the CAF SOMS included four separate data entry forms: (1) case details (e.g. demographic and contact information, symptoms, comorbidities, risk factors or exposures, isolation period); (2) contact elicitation (e.g. demographic and contact information, diagnostic status, date of last exposure, risk category); (3) contact notification (e.g. symptoms if applicable, self-isolation period, supports required if applicable, contact status); and (4) contact follow-up (e.g. symptoms if applicable, contact status, supports required if applicable, self-isolation period, isolation day). Features were embedded to allow navigation between forms for related cases and contacts using a unique identification code, as well as line listing functionality to subset case-contact relationships.

A formative evaluation was employed to inform further development and implementation of the system through two waves of testing during December 2020 and January 2021, and through the piloting of the system in February and March 2021. The first testing was completed by five members of the National Contact Tracing Team using a prespecified test plan developed by the Health Informatics team to assess the basic functioning of the database. The second wave of testing was completed by four CAF-trained contact tracers at a single CAF base using scenario data to simulate real-world contact tracing. The CAF SOMS was then piloted using real data at two bases to assess the real-world performance of the system. Contact tracers involved in the second wave of testing and the pilot of the system were emailed 22 open-ended survey questions and asked to provide feedback on the CAF SOMS functionality and data flow and the utility of an embedded user guide (Table 1). The results of the testing and piloting were used to refine the system for each subsequent stage of the formative evaluation and implementation.

Table 1. Open-ended survey questions given to contact tracers testing and piloting the CAF SOMS

Abbreviation: CAF SOMS, Canadian Armed Forces Surveillance and Outbreak Management System.

The final phase of the development and evaluation process was a national roll-out of the CAF SOMS in April 2021. A short user feedback survey was also sent to the broader CAF contact tracing community, and analyses are underway to validate the accuracy and completeness of COVID-19 cases and contact data to identify strengths and weaknesses.

Lessons learned

The preliminary results of the formative evaluation are promising and suggest that the CAF SOMS is an effective tool for contact tracing. User perceptions of the system were positive, and some suggested improvements could be acted on immediately during the phased development and implementation of the system (e.g. providing additional response options in drop-down menus, correcting the auto-calculations for self-isolation periods, using additional variables for filtering and searching for records and searching for contacts directly rather than through case files, simplifying the process to close contact records separately from cases). Although some feedback could not be addressed prior to the national roll-out (e.g. include checkboxes to indicate “no change” for contact follow-up, add case follow-up forms), these items may be addressed in future versions of the database, and the changes that were prioritized and carried out were well received.

Overall, a key lesson learned is that newly developed outbreak management information systems must be continually evaluated to proactively ensure they function adequately as new information and shifting priorities emerge. In this regard, our preliminary formative evaluation results suggest that the built-in flexibility of the CAF SOMS design supports the ability to make critical system modifications as required to meet changing outbreak management needs.

COVID-19 and beyond: recommendations and future directions

The COVID-19 pandemic has underscored the need for better disease surveillance and outbreak management systems that allow for timely, informed and agile public health management of disease threats.^{Footnote 7Footnote 9} As the emergence of COVID-19 has demonstrated, the ability to respond quickly by continually incorporating updated knowledge into systems and processes is crucial for managing a novel and emerging disease that may not be fully understood. Evidence has shown that contact tracing, when performed quickly, can be one of the most effective strategies for disease containment.^{Footnote 8} This is especially true for COVID-19, since it is estimated that approximately 40%, and possibly up to 70%, of disease transmission occurs when cases are presymptomatic or asymptomatic.^{Footnote 8Footnote 10}

While the CAF SOMS was initially designed to respond to the current public health crisis, the intent is to further scale and develop the system for future pandemic preparedness. Thus, the system was designed to be flexible and adaptable, and to allow for the easy incorporation of updated public health guidance and epidemiology regarding COVID-19, as well as for other diseases. Expansion of the system to capture other diseases is planned for the next phase of system development, as well as the inclusion of other functionality based on feedback from the formative evaluation.

Additionally, there are several challenges that need to be addressed to maximize the uptake and utility of the system moving forward. These include streamlining CAF data collection mechanisms, integrating the system with electronic medical records and improving internal and external data sharing capabilities. To streamline CAF data collection and reduce redundancies, it will be necessary to collaborate with key stakeholders and system end users. Increased promotion of the CAF SOMS, along with training and strong leadership support can accelerate this type of organizational change. Dedicated resources and continuing multilevel health informatics support are key to ensuring sustainability of the system, as are efforts to find solutions to create a seamless and connected data management system that services the entire continuum between clinical care and public health. These efforts include exploring ways to transfer critical public health data, such as case and laboratory reports, between the CAF SOMS and electronic medical records, since the latter system was not designed for public health and epidemiological purposes.

Lastly, since diseases are not constrained by jurisdictional boundaries, options should be explored to develop a consistent approach with external Canadian public health agencies. Enhanced data sharing would be beneficial for minimizing gaps in public health coverage and ensuring continuity in case and contact management. To this end, alignment with the vision of the pan-Canadian Health Data Strategy to create an effective, interoperable, integrated health data ecosystem that is continually optimized could provide additional direction for correcting some of the CAF’s current health data management limitations, and would ensure a consistent and collaborative approach with Canadian public health partners.^{Footnote 8}

Conclusion

The results of the CAF SOMS formative evaluation demonstrated the information system’s potential as a tool to improve the CAF’s disease surveillance and outbreak management capability. The system’s success will depend on effective organizational change, continued leadership support, and adequate resourcing. Further alignment with the vision of the pan-Canadian Health Data Strategy, along with improved data management and collaboration with external public health agencies, can help to increase pandemic preparedness and improve outcomes.

Acknowledgements

The authors would like to acknowledge the contributions of Dr. Maureen Carew, Kristen Simkus, Master Warrant Officer Natalie Deziel and Captain Kristian Cunningham to early discussions in the development of the Canadian Armed Forces Surveillance and Outbreak Management System (CAF SOMS). We would also like to recognize the work of Lucie Bergeron, Cheryl Ward, Paul Jacobs, Guy Lefebvre and Loan Nguyen in the development of the web application, and the contributions of Second-Lieutenant Rachel Dubé and contact tracers in the field to the testing and piloting of the system.

Conflicts of interest

The authors have no conflicts of interest to disclose.

Authors’ contributions and statement

CD and SG: conceptualization, data collection and analysis, methodol­ogy and writing—review and editing; JP: conceptualization, data collection, methodology and writing—review and editing.

The content and views expressed in this article are those of the authors and do not necessarily reflect those of the Government of Canada.

Previous | Table of Contents | Next