An End-to-End Architecture for Storage Reliability and Data Integrity

For over a decade now RAID has been used to improve the performance and reliability of storage systems. In the time since RAID became widely available the density of disk drives has increased by a factor of over 250 times. This remarkable increase in disk density has created new challenges for reliability and data integrity. Panasas Tiered Parity is the only comprehensive architecture available today that effectively addresses these new challenges in a comprehensive and scalable manner.

1. Increasing numbers of media errors – causing reconstruction activity and RAID reconstruction failures
2. Longer RAID reconstruction times and much longer recovery times in the event of a RAID reconstruction failure
3. Increased possibility of data corruption introduced outside of the storage system as orders of magnitude more data is being passed through memory, switches and other network infrastructure.

Panasas Tiered Parity is a comprehensive architecture that addresses individual reliability challenges with appropriate families of error detection and correction codes. Each tier operates independently of the others allowing the error detection and correction codes to be individually tailored for that tier. The three tiers are complimentary to each other and collectively provide the most comprehensive and scalable reliability architecture available for high performance storage today.

• Vertical Parity maintains the reliability of the individual disk drive. It addresses the challenge of ever increasing numbers of media errors by isolating and repairing them at the disk level before they are seen by the RAID array.
• Horizontal Parity maintains the reliability of the RAID group across multiple drives. It addresses the challenges associated with reconstruction times by using ObjectRAID to more quickly and efficiently complete reconstructions.
• Network Parity maintains the integrity of the data path between the storage system and the clients. It addresses the challenge of silent data corruption introduced by the network infrastructure by performing data integrity verification at the client node itself.

The challenge of increasing numbers of media errors becomes most apparent and severe during a RAID reconstruction. We have already seen that Panasas limits the volume of data that needs to be read during a reconstruction with ObjectRAID in the Horizontal Parity Tier. The addition of Vertical Parity corrects media errors at the disk level before they are exposed to the RAID array effectively eliminating the need for double or triple parity schemes. Vertical Parity provides a consistent level of reliability independent of constantly increasing disk density and media error occurrence.

The Horizontal Parity Tier is where traditional RAID is applied across an array of disks to improve both performance and reliability. Panasas ObjectRAID incorporates an advanced RAID level selection capability and innovative data reconstruction techniques to optimize performance and reliability. ObjectRAID reconstructions are typically up to ten times faster than those of traditional RAID schemes due to their ability to employ all drives in parallel and their ability to know exactly which objects need reconstructing. This eliminates the need to reconstruct free space and saves considerable time.

As disk drives continue to grow in size, the volume of data being stored, manipulated and transmitted across network infrastructure continues to rise. Network Parity delivers the capability to verify that the data received by the client or server node is precisely what was read from the disk.

Tiered Parity delivers the most comprehensive software approach to data reliability and integrity available in any storage system today. In addition, the unique and innovative approach to solving for media errors means that the uncompromised reliability that Tiered Parity delivers will be maintained independent of future increases in disk density.