Bad RAID Controller Can Corrupt Data

SUMMARY

Do not rely on a RAID mirror for Perforce backups. A faulty RAID controller can produce corrupted data, and render useless redundant data on the mirror.

DETAILS

Perforce files (checkpoints, versioned files, and current and rotated journals) always need to be placed on a hard drive separate from the Perforce server data. This separation avoids losing backup files if your primary storage device is corrupted or otherwise fails.

Users of RAID systems might believe that, when using "mirrored" drives (such as  RAID 1 or RAID 10 drives), separate drives or backups are no longer necessary. This is not correct -- a RAID produces redundant data, but the RAID has no way of determining if the data is valid. Even faulty filesystem info is copied exactly as it was written originally.

In at least one customer's case, all Perforce-related data was stored on a RAID 10 filesystem. The RAID controller failed, corrupting all files on the primary image. The system copied all of the corrupted files to the mirror. The customer ended up with only corrupted files, and  no chance for recovery.

It is highly recommend that you place Perforce backup files (checkpoints, versioned files, and current and rotated journal files) on a hard drive that is independent of the Perforce database (db.*) files. Doing so maximizes your chances for completely recovering your Perforce server in the event of a hardware failure.