File Integrity Solutions for File Collections in Mac OS X

by rfshall

Summary: There are a variety of sophisticated tools for managing the completeness and file integrity of file collections on Mac OS X. These Mac programs don't seem to be as well known as their Windows counterparts. This page attempts to correct that.

How to Watch over Your Files

People who maintain collections of computer files need to be assured that all the files in the collection are perfect, complete, and organized. These concerns are met by three processes:

File Verification: checking that a file collection is complete and that its files are undamaged.
File Recovery: fixing damaged files or re-creating missing files of a file set.
File Management: verifying, organizing, and properly naming of files within (or into) a collection.

Performing these jobs together results in a high guarantee of individual file integrity. This assures a perfect, complete, and organized file collection that should survive against nominal data corruption.

File Verification

A good file verification program checks a list made of the original files against your current files--indicating which of your files are perfect, damaged, or missing. It works by matching individual file "fingerprints," called checksums, against the fingerprints of your unknown files. Most file verification programs allow you to create file "fingerprint" lists for your own collections.

checkSum+ 1.5.1 FREE GUI
Creates and tests md5, sfv, & csv files.
A drag-and-drop "explorer" window can calculate up to fourteen different file checksums on a file-by-file basis.
Uses cksum and other Unix utilities.

cfv (aka Command-line File Verify) 1.18.1 FREE CLI
Creates and tests sfv, csv, crc, md5sum, & md5 files.
Test-only support for PAR, PAR2, and torrent.
Filename correction supported.
Python script; requires simple 'make' and installation.

File Recovery

Parity files (both PAR and PAR2 varieties) contain superfluous data that can be applied to repair damaged files or recreate missing files in a series. Parity files are uniquely redundant, so you need as many unique PAR files or PAR2 blocks as you have missing files or damaged blocks.

MacPAR deLuxe 3.7 SHARE GUI
PAR and PAR2 file creation, testing, and recovery.
Also extracts RAR files.
- Works with Split&Concat 2.5 [SHARE/REMIND] to join segmented files automatically.
- Original MacPAR 2.0 [FREE] is still available, but it has no PAR2 features.

par2cmdline 0.4 FREE CLI
PAR2 file creation, testing, and recovery.
Requires simple 'make' and installation.

par 1.1 FREE CLI
PAR file creation, testing, and recovery.
Requires simple 'make' and installation; or use older Par OSX 1.0 on the same page.

File Management

When you need to manage your files: A file collection manager integrates file verification with file organization. It allows file and folder structures to be maintained across different media, and it includes file verification for managing and renaming purposes.

PicTracker 2.0.1 DEMO GUI
Uses tsv master files for verifying, moving, and renaming files.
Imports sfv and csv files; exports csv files.
- although originally intended an image-collection manager, the homepage notes that it can be used to organize any types of files.

The programs above include freeware ("FREE"), uncrippled shareware ("SHARE"), uncrippled shareware that reminds you to pay ("SHARE/REMIND"), and trial software ("DEMO").

Most of the solutions are available as OS X "clickable" applications (" GUI "). Some unix utilities (" CLI ") may need to be built and run from Terminal (located in "/Applications/Utilities/"), but are generally much faster than their "clickable" counterparts.

this draft of 9 june 2007; original version 2004