Medalta Museum Digital Preservation Plan

Organizational Structure

Medalta Museum, a living museum celebrating the history of Medicine Hat’s brick, clay and pottery industry, is one of many operations which function as part of the Medicine Hat Clay Industries National Historic District. Approximately 17 full-time and 4 part-time casual employees work and move freely within the various operations under this organization. An additional 16 part-time casual staff are dedicated to special events. Operational decisions are made by staff, who are ultimately accountable to the Executive Director and General Manager of the Historic District. In turn, the Executive Director and General Manager is accountable to the organization’s Board of Directors.

Current Practices and Obligations

• The museum is open year-round. Staff also support the operation of the Shaw International Centre for Contemporary Ceramics and its resident artists.
• The museum receives some of its funding from the Medicine Hat municipality and has strong ties with the city by serving as a cultural anchor. The expansive buildings and grounds are often host to various cultural events attended by the town’s residents and are also made available for private functions.
• Salaried staff run the museum on a daily basis — all of whom have training or experience in their respective fields of expertise.
• Museum operations have been expanding over the past decade, and planning must take into account both current needs and future requirements. For instance, very little of the museum’s holdings have been digitized. However, funding is in place to digitize a significant amount, both through photography and 3D scanning.
• Likewise, the museum has been using MS Access to manage collections records, but it is migrating this data to PastPerfect Online. This will likely increase demand on digital content, as it can be made available online.

Organizational Readiness

Museum staff and management are taking a leadership role in working with external organizations to prepare a digital preservation plan and policy where few currently exist in the Canadian museum environment. Both staff and management recognize the need for a digital preservation plan and are prepared to implement one that respects available resources.

Financial Constraints

Medalta receives funding from the municipality and from various private and public sector sources. Funding is currently in place for digitization projects, and as part of the sustainability of this work, a digital preservation plan needs to be put in place.

Simple plans that amount to backups with some preservation data will not be subject to financial constraints. More complex plans (such as the use of a trusted digital repository) will require additional funds to be earmarked as core (not project) funding.

Human Resources

The museum has salaried staff possessing various competencies in museum management and collections management. However, there are no onsite staff that are strictly dedicated to IT or technical support; instead, a contractor comes in as needed.

Technical Constraints

The majority of computing tasks for the Historic District are performed in the office at Medalta Museum; however, the buildings are spread across a very wide area. A vehicle is required to reach some locations. There is no private computer network between these buildings. The need for a network may be mitigated somewhat by migration to PastPerfect Online (allowing all machines access to the collections management database), but should this not be sufficient, management might consider the installation of a virtual private network (VPN), for instance, to allow printer and disk sharing across all building locations. For the present, implementation of a network is considered out-of-scope.

The upper limit of what could be digitized is extremely variable. Medalta’s management is adept at acquiring new assets, developing new business models and raising the funds to achieve these goals. There is also a very large existing inventory of artefacts and documents that have not been accessioned, and it is not yet clear how much of this merits digitization. Any digital preservation plan will have to be flexible to this unknown amount of digital content, without costing an undue amount (should the amount of content be minimal to begin with).

Office machines run on MS Windows, and all have high-speed Internet access.

The museum and all buildings associated with it are located in a floodplain.

Option 1 – Multiple Backups of Working Files (No Archival Software): Make Preservation Copies Using a Checksum Generator

This option uses external hard drives to make regular backups of digital assets as they exist (backup software is used, but not digital archiving software). The option also uses an MD5 checksum generator to produce annual preservation copies. All content is stored on hard drives, and drives are refreshed (replaced) every five years.

Option 2 – Use Disk Pools

For this solution, individual drives described in option 1 are replaced with expandable pools of hard drives across which multiple copies of the content is written, in the event of disk failure. Windows 10 Storage Spaces virtual drives are considered, as these were tested by CHIN to be a robust, easy-to-use system. Moreover, Windows 10 is free to upgrade for existing Windows 7 and 8 users.

The first disks to require expansion are likely to be external drives 1 and 2 (described in option 1), as each of these will require up to six times the storage space of the working (shared) drive. This solution can be applied to the working drive as well (not discussed further), as long as the working drive does not also hold operating system files.

Each external drive described in option 1 should be replaced with a single Storage Space (i.e. two Storage Spaces are required). Each Storage Space consists of a pool of hard drives that are connected to the working computer via a USB 3.0 hub with its own power source. Use one hub for each Storage Space.

Windows Storage Spaces offers various ways of ensuring content is not lost (i.e. “resiliency”) in the event of a drive failure. The most appropriate for archival purposes is using disk parity. In short, this form of resiliency will require longer write time to the drives (backups will be a bit slower than with other options), but it will maximize useable drive space. Three drives are required to start a parity Storage Spaces pool. If a single drive fails, the content remains on the other two drives, the system will inform users and the drive can easily be replaced. As additional space is required, Windows will inform users of the need to add additional disk drives, and this can be done quickly and easily without compromising existing data.

Option 3 – Same as Option 1 or 2 but use a Cloud Service in Lieu of the Second External Drive

This solution further increases the robustness of the system by ensuring that the second preservation copy is never in the same location as the first and by reducing the likelihood that preservation copies will be destroyed (the cloud service will perform its own backups).

Google Drive and Dropbox are examples of viable cloud service options. Dropbox offers 1 TB of cloud storage for $17/month, which will be sufficient initially. The service promises to provide as much space as required for that fee but has not yet replied to CHIN’s queries for further information.

Bandwidth and data limits with Medalta’s Internet service provider need to be reviewed before this option is selected. A speed test on the Internet connection can be established by performing an Internet search on [Internet speed test], selecting one of the test services and following instructions.

If upload speed is 10 Mbps (typical for basic high speed), then a 100 GB upload (a reasonable estimate of content size after one year of digitizing) will take just over 22 hours, making the process workable in the foreseeable future. Should the amount of data eventually become too cumbersome, either a higher speed Internet connection might be procured or the museum might revert back to options 1 or 2 (using a hard drive or a pool of drives for preservation copies) without any penalty.

Option 3 Pros:

• All pro considerations for options 1 and 2 with the exception of cost (option 1 is most affordable).
• Increases robustness of system by moving preservation copies offsite and by ensuring a third party is responsible for backups of this copy.
• Easily implemented.

Option 3 Cons:

• All cons considerations for option 1 except for the limit to storage capacity.
• More expensive than option 1.
• Bandwidth and data limits may be an issue with the Internet service provider.
• Cloud storage located in the United States is subject to the Patriot Act.

Managing new files

Newly created files on the working machine (the “source”) will be copied to a similar directory on the target drive (i.e. External Drive 1).

Managing deleted files

Deleted files are never lost. Any file that is deleted on the working drive is copied to an “Archive” folder on the backup drives.

Starting with Option 1

Image description (Diagram 1): This diagram depicts a hardware configuration of an office personal computer (working drive) linked via USB to a 5 TB hard drive, labelled “External HD 1.” A second USB link from the office machine runs to a second external 5 TB hard drive (labelled “External HD 2”), which is generally not left connected to the personal computer but is kept offsite. An Internet connection also connects the PC to the museum’s PastPerfect Online collections management system.

Migrating to Option 2

Image description (Diagram 2): This diagram depicts a hardware configuration which is identical to that shown in diagram 1 (above) with the following changes: the first external hard drive has been replaced with a pool of three 5 TB hard drives. Likewise, the second external hard drive has also been replaced with a similar pool of drives. The resiliency setting used for both these pools is “parity,” which is the recommended setting for archival use. Both pools can be expanded as required.

Detailed procedures of this recommended solution can be found in Annex A of this document.

Option 1 Procedures

The proposed solution recommends starting with procedures for option 1, then migrating to option 2 if the need for storage space outstrips the size of commercially available external hard drives.

To do immediately:

• Complete the migration to PastPerfect Online (done).
• Ensure that accountability to preserve collections management records rests with PastPerfect. If assurances of accountability are not sufficient, establish a protocol to obtain a regular copy of the records (in tab-delimited or a similar format) and store this copy in a secure location, such as the museum’s office.
• Designate an office machine (to be called the “working machine”) to host a shareable hard drive (“working drive”), and ensure all digital assets reside on this drive.
• Clean up the physical media – move content on any existing CDs to the working drive.
• For all digitized images, audio and film: use a file naming system that corresponds to the accession number of any accessioned artefact the content represents. For each artefact that is born-digital (i.e. digital content that is an artefact in its own right and does not represent a physical object being held by the museum), create a new record in PastPerfect.
• Ensure password protection at the operating system level (user sign-on screen when booting or logging onto a computer) for all machines.
• Download MD5summer checksum software (free) and Bvckup 2 backup software ($50) and install both on the working office machine.
• Acquire two additional external (USB) hard drives (5 TB each) for backup and preservation copies. Label the first of these drives as “Preservation Copies 1 Purchased MMM YYYY [Abbreviation: alphanumeric date notation with 3 letters for the month, followed by 4 digits for the year; e.g. Mar 2016] - Replace MMM YYYY [Abbreviation: alphanumeric date notation with 3 letters for the month, followed by 4 digits for the year; e.g. Mar 2016]” (where replace date is five years from the purchase date), and keep it connected to the working machine in the main office. Label the second of these drives as “Preservation Copies 2 Purchased MMM YYYY [Abbreviation: alphanumeric date notation with 3 letters for the month, followed by 4 digits for the year; e.g. Mar 2016] - Replace MMM YYYY [Abbreviation: alphanumeric date notation with 3 letters for the month, followed by 4 digits for the year; e.g. Mar 2016]” and keep it offsite.

• Use Bvckup 2 software to schedule a regular weekly backup of all non-databasedigital assets to the “Preservation Copies 1” external drive. The backup of all this content should be stored in a directory on the “Preservation Copies 1” drive entitled <Current Backup – Non Database Content>. Important settings for this backup include:
  • Archive any files that were deleted from the working drive.
  • Rename or move any files that were renamed or moved on the working drive.

• For any active database that resides on the working drive (if for instance asset group 5’s building #13 database is written to), additional monthly and weekly backups should be scheduled. This is necessary because renaming database files (i.e. version control) is often difficult, depending on how the database is used; anytime a database is written to without version control, the previous version is permanently lost.

Four separate weekly backups should be scheduled to write to the following directories on the “Preservation Copies 1” external hard drive:

<Database backup week 1>

<Database backup week 2>

<Database backup week 3>

<Database backup week 4>

Each of these backups should be scheduled to run once every four weeks. This creates a weekly history of the database files that is up to four weeks old at any given time. In addition to running at the scheduled time, the first of these backups should be run immediately.

• Likewise, for any active database files residing on the working drive, twelve monthly backups should be scheduled in Bvckup 2 to the “Preservation Copies 1” hard drive. Each of these backup jobs runs only once per year (at the end of the month corresponding to that backup job). Directories on the “Preservation Copies 1” external hard drive should appear as:

<Database backup 01 January>

<Database backup 02 February>

.

.

.

<Database backup 12 December>

In addition to running at the scheduled time, the current month’s backup should be run immediately.

• Once the first weekly and monthly backups have been run for the first time, create an annual preservation copy on the “Preservation Copies 1” drive, as per the “To Be Performed on an Annual Basis” section below.
• Using Bvckup2, schedule a job that runs only when manually executed. This job will be to copy the “Current Backup Non-Database Content” folder from “Preservation Copies 1” drive to “Preservation Copies 2” drive. Use this instead of the Windows copy feature, as it will be much quicker in future months.
• On the “Preservation Copies 2” drive, maintain copies of the monthly backups, as well as copies of the annual preservation content. Keep this drive offsite, out of the floodplain.
• Ideally, all three drives (working, “Preservation Copies 1” and “Preservation Copies 2”) should not be connected to the same computer at the same time, and copies from “Preservation Copies 1” to “Preservation Copies 2” should be carried out on a separate (virus-free) computer in the office or nearby building. Barring this, care should be taken to minimize the amount of time that all three drives (or sets of drives) are connected to the same machine.

• Depending on the operating system and brand of external hard drive used, a security measure may be available to password protect the drive. If so, it should be used on both external drives to prevent unauthorized access. Ensure that the password is available to select staff.

Perform the following as time permits:

• Digitize any analog media (cassette, film,  reel-to-reel, etc.) as time and budget permits. Keep working copies on the working drive in the office.
• For digitized images, audio and film, create a naming system that ensures each resource has a unique file name. An ideal file name convention is one that contains the record ID for any database with which it is associated (an accession code for instance). This allows digitized content to be quickly located through database searches and database content to be quickly identified for a given file.

To be performed routinely

• Migrate to new file and software formats as required. See the “File Formats and Data Migration” section of this document for details.
• Any time that a migration to new software takes place, ensure the migrated files with new formats (for the newer software) are in directories on the working drive that will be backed up and preserved.
• Maintain existing password protection on the working office machine and any machine that accesses it through a network.
• Keep the first external hard drive (“Preservation Copies 1”) connected to the working machine in the museum office.
• Keep the second external hard drive (“Preservation Copies 2”) offsite, out of the floodplain.
• Use version control (i.e. name files according to the sequence in which they were edited either with a version number or a modification date) on all documents that will be edited.
• Immediately rename any file that mistakenly overwrote another, so that the original file will be archived on the backup drives.

To be performed on a weekly basis

• Review and verify that all files in the identified asset groups are within the scope of the weekly backup (i.e. that all the relevant directories are listed in the weekly backup). Add any that are not to all four weekly backups as well as all twelve monthly backups.
• If active databases exist, ensure that the most recent weekly backup to “Preservation Copies 1” was properly run (check the Bvckup 2 log, and check the “modified” date of files in the target directories).

To be performed on a monthly basis

• If active databases exist, ensure that the most recent monthly backup to “Preservation Copies 1” was properly run (check the Bvckup 2 log, and check the “modified” date of files in the target directories).
• Connect both external “Preservation Copies” hard drives to a new virus free computer (this is ideal, but if a second computer is not available, using the working machine for the following procedures is acceptable).
  • Use Bvckup 2 to copy the <Current Backup – Non Database Content> folder from the “Preservation Copies 1” drive to the “Preservation Copies 2” drive. This will take a long time the first time it is done, but if the same Bvckup 2 job is used in the future, it will be much quicker.
  • Copy any new annual < Preservation copy backed up yyyymmdd> directories (as described in the “Managing Preservation Copies” section of this document) from the “Preservation Copies 1” drive to the “Preservation Copies 2” drive (using the copy or drag and drop functions in Windows Explorer).
  • If active databases exist, make a copy of the most recent monthly backup on “Preservation Copies 1” to “Preservation Copies 2” (overwriting any backup for the same month in the previous year).
• Reconnect the “Preservation Copies 1” drive to the working machine, and return the “Preservation Copies 2” drive to its offsite location.

To be performed on an annual basis

Update the Digital Asset Inventory

Identify any new digital assets, as well as any changes to existing ones. Make particular note of any changes to software used. If there are changes, identify what files (if any) need to be migrated to new formats. Migrations should first be performed on copies found in the working drive. Next, any files found in “Preservation Copies” (and which are no longer on the working drive) can be migrated.

Update the Digital Asset Inventory annually

• Starting immediately after the first backup on the working machine, copy (using Windows Explorer) the “Current Backup” folder and all its subfolders located on the external hard drive labelled “Preservation Copies 1” to a new folder on the same hard drive named “Preservation copy backed up YYYYMMDD” (where YYYYMMDD is the date the backup was made).
• If any active database files exist, starting immediately after the first monthly backup on the working machine, copy (using Windows Explorer) this backup (e.g. <Database backup nn Month>) to the newly created “Preservation copy backed up YYYYMMDD” directory (described above).
• Run MD5summer on the new “Preservation copy backed up YYYYMMDD” folder to store checksum metadata for it, and store this checksum data in the new directory.
• The monthly update of the offsite hard drive (“Preservation Copies 1”) should be carried out shortly after the above steps are completed.
• Test fixity of all preservation folders on both external hard drives, as per the “Testing Fixity” section of this document.

To be performed once every five years

• Acquire new external hard drives.
• Test fixity of files in preservation copies of the current hard drives.
• Copy all folders to the new hard drives, and dispose of old external hard drives, as per the “Refreshing Media” section of this document.

Create Disk Pools

• Upgrade to Windows 10 on all machines that will access the shared drive.
• For each pool to be created:

• Acquire a USB 3.0 hub that has its own power source and a minimum of four USB ports (six or even eight ports is ideal to allow for future expansion).
• Acquire three new external hard drives that are equal in size to that used in option 1 (i.e. six new drives in total). Label all new drives with the date purchased (e.g. “Preservation Copies Pool 1, disk 1, date purchased YYYY-MMM-DD” [Abbreviation: alphanumeric date notation with four digits for the year, followed by three letters for the month, followed by two letters for the day; e.g. 2016-Mar-01]).
• Follow Windows 10 instructions to create a Storage Space (set the resiliency type to “parity”) for three disks only. Do not use the drive in option 1 if it currently contains content. State a “Maximum Storage Space Capacity” that is sufficient to hold all digital assets in the foreseeable future (e.g. 40 TB). You do not currently need 40 TB worth of drive space to state this capacity; it is simply the maximum capacity to which the drive can be expanded in the future.
• Copy all content from the older drive (described in option 1) to the new pool.
• Follow Windows instructions to add the older drive (i.e. the drive already being used in option 1) and a fourth drive to the pool. This will reformat both drives, and copy existing information from the pool onto them.

• Repeat the above process for a second pool (and copy the contents of the second external drive described in option 1 to this pool).

Using Disk Pools

Pools should be treated in the same manner as individual drives described in option 1.

Replacing Drives

Drives within a pool that are frequently used (i.e. frequently read and written to) do not need to be automatically replaced every five years; instead the pool will inform users as these physically degrade. If drives within a pool are stored for several months without read or write access (this should not be the case under recommendations for options 1 and 2) then drives should be replaced within five years of the date purchased.