Getting Clear about Object Storage
May 1, 2013 - 4:35pm
Panasas has a long history with object storage, starting with Dr. Garth Gibson’s groundbreaking work in the early 1990’s at Carnegie Mellon. Garth’s work on NASD (Network Attached Secure Disk) became the foundation for the OSD (Object Storage Device) standard and Panasas file system. He is also credited with being the key inspiration for the Google File System (GFS). Both file systems are object based storage, but should not be confused with web object storage systems such as Amazon Web Services or Microsoft Windows Azure. In fact there are two classes of object storage: 1) object storage devices which Panasas and Google use and 2) web object services which Amazon and others use. The differences are subtle, but the capabilities of the two systems are significant and the ultimate implementation models are not compatible or interchangeable.
Object based devices (OSD’s) and web objects are similar in that they both implement flexible storage containers that contain both data and metadata (i.e. attributes about that data). However the access protocols are different and the semantics of the objects are different. The OSD command set is a standard command set for SCSI. Data created on a Panasas system can be accessed as standard POSIX compliant file system via NFS, CIFS, pNFS or Panasas PanFS protocols, giving the files universal accessibility. By contrast, web object storage systems provide simplified, low performance data storage and cannot be easily shared with other file systems. In fact, there are several web object storage services and they are incompatible with each other. For example, if you write an application that uses OpenStack or Amazon Web Services, your application is specific to that interface.
What they both share in common is an intelligence layer that makes them significantly more powerful than traditional file systems, just how that intelligence is used is where they differ. Object-based file systems are designed to create very large, very high performance file systems to support high performance applications within a data center. Object-based web storage systems are designed to create large scale, low cost, low performance data repositories.
Dr. Brent Welch, Panasas CTO, gave an excellent overview of Object Storage at the Spring 2013 SNIA Tutorial. Attached is the link to Brent’s technical tutorial overview: