From paper to pixels: A collaboration to digitize Canada’s historical weather records

Public Services and Procurement Canada (PSPC) is collaborating with Environment and Climate Change Canada (ECCC) on the digitization and preservation of ECCC's unique collection of historical weather records for the study of Canada's climate history.

This project, "Preparation, Digitization and Data Capture of Observational Weather Data," requires the conversion of these paper-based documents into electronic formats. Digitizing these documents will allow anyone to access the information freely on the Internet, which will support increased exploration, discovery and knowledge of climate information at the local, national and international levels.

The collection dates as far back as 1840 and includes handwritten weather records, climate data and related information for numerous locations across Canada. In some remote places, volunteers have been observing and recording the weather and sending in paper records for generations. These records are invaluable for understanding historical weather patterns and preserving Canada's meteorological history.

The teams

The collaboration requires ECCC's Data Management and Operations team to prepare and send the paper records to PSPC's Document Imaging Solutions Centre (DISC) for digitization. DISC then returns the digitized images to ECCC, where they are checked for quality. Next, the quality-controlled images are uploaded to the National Climate Archive, and the physical copies of historical value are sent to Library and Archives Canada (LAC).

For Raphaël Lambert-Angers, DISC Project Manager, this project differs from his team's usual work, which is mostly with business value files that employees use on a day-to-day basis. "What's special about the ECCC collaboration is that we're digitizing paper documents written by volunteers hundreds of years ago. It's like holding a piece of Canada's history in your hands."

Thinesh Sornalingam, Manager of the Data Management and Operations Section of the Meteorological Service of Canada, also finds the project interesting. "There's a physical dimension to this project. You can touch these records, and there's appreciation and additional care involved since they have historical value."

The partners

The partners in the digitization project include PSPC, ECCC, LAC, Western University and the Treasury Board of Canada Secretariat, which has been providing the funding for the project. Part of the collection was housed at Western University's campus in London, Ontario, for research prior to being shipped to DISC for digitization.

"Setting up a project this big that spans several years isn't easy. Each partner plays an important role in the project's success. I'm proud of how flexible the departments have been and how well we're working together towards our common goal of digitizing these historical records and preserving part of Canadian heritage," comments Raphaël. Thinesh agrees and adds, "The DISC team has been fantastic in helping us so far."

Overcoming challenges

During the project, there have been multiple challenges, ranging from logistical issues (preparing, shipping and handling large amounts of fragile historical paper records) to quality issues (deciphering documents that are old and hard to read as well as understanding evolving styles of cursive writing and complex meteorological symbols recorded by generations of weather observers).

One of the biggest hurdles continues to be logistical coordination. Thinesh says, "Proper accounting is crucial. We needed to place ‘one in, one out' checks across the chain since several different groups are involved, and we're dealing with hundreds of boxes of paper records and hard drives of digital images."

Quality assurance is another issue. Since the teams are going to so much effort to digitize the records, they want to make sure that the images are clear enough for anyone to read and understand them. This is also important because the images will eventually be made available to the public. Thinesh adds, "Luckily, DISC has been very receptive to exploring new and better ways to improve both the accounting and the quality of the images."

Raphaël's team had the challenge of learning how to handle fragile paper records, some of which are over 200 years old. They had to develop special procedures to scan the documents without damage, to be able to return them in the best possible condition. "We've received training from LAC on how to handle historical documents, such as using gloves to protect the documents from touching skin, which could make them crumble," says Raphaël. Additionally, learning how to read cursive has, surprisingly, been a challenge for younger members of the team, particularly since writing on older documents is not always clear and crisp.

More than 1 million records digitized

Since 2017, more than 1.1 million paper records of varying sizes and types of forms, including 35,305 books, as well as maps, microfilms and microfiches dating back to the 1800s, have been digitized. The documents contain information on temperature, precipitation, pressure, relative humidity, and wind speed and direction.

Planning for the future

ECCC continues to receive paper weather records, particularly from remote areas in Canada that have limited Internet access. The goal is to transition to a paperless system in the next 10 years, and the project aims to create a self-serve model where people can easily search for and download the images from the web. The public can currently search for historical weather data, but for now, images from the historical weather collection are only available on request.

Both Raphaël and Thinesh find it rewarding to preserve the historical records. "I think it's a privilege to see those documents in person as they are part of Canadian history. There's also the ‘cool' factor of knowing that the public is going to be able to see them virtually."

To learn more about PSPC employees, projects and services making a difference for Canadians, read other articles on Our stories.