Expanding HPC Capabilities? It's Not 'All' About Performance
March 26, 2013 - 5:01pm
Panasas Director of Channel Sales for EMEA, Derek Burke, points out that pure performance isn’t the only important aspect of an HPC storage solution, especially when it’s tasked with processing multiple technical computing application workloads. Here is what he says…
Whether Panasas® ActiveStor® is the storage backbone of a departmental HPC system for a specific application domain or for a centralized HPC service for a diverse range of users and applications – ‘performance’ is generally given the highest consideration.
From a compute performance perspective, HPC providers have become highly proficient at providing heterogeneous systems with fat node, thin node, accelerators, (virtual) SMP systems and the management software to direct jobs to the fastest available resources.
From a storage performance perspective, it’s easy to select a storage solution for a single application. However, the complexity begins when we consider multiple application workloads. I recently heard an HPC director stating that his staff instructs users not to run certain (commercial and in-house) applications against their open-source based parallel storage solution. He explained that these specific applications generated a random I/O workload with many small files, yet the storage system was constrained by a single metadata server for the entire scratch file system. Those applications, therefore, negatively affected the performance for all users.
He was impressed that the Panasas® PanFS® file system scales metadata by associating specific volumes to specific blades. Therefore spreading the metadata workload across the hardware resources and protecting other users against IOPS hungry applications.
He was further impressed with the new capabilities contained in PanFS 5.0. Namely, intelligently targeting metadata and small files to the SSD’s, with up to a 10x increase in performance for metadata, small files and file system services.
Performance is typically the first thing we think about in HPC, but what happens to the data when it’s moved off of the high performance scratch? When it becomes project data or home directories, it’s moved to a more robust storage system that is less about performance and more about features like built-in failover, snapshots, quotas, multiple RAID levels, multi-protocol support (CIFS & NFS) and easy integration with data back-up solutions.
But, what if one storage system could satisfy ‘most’ (‘all’ would be overly optimistic) of your requirements? What if one system had adequate performance for your diverse range of applications? What if that system had enterprise-class reliability features as well as performance? Could you consolidate your storage systems and expand your capabilities?
Panasas solutions help consolidate storage systems and increase service capabilities to include the management of project and home directories data. I recently stumbled on the web site of a typical Panasas customer, e-Infrastructure South (UK), explaining how they use Panasas for /scratch, /work and /home directories, utilizing our snapshot features for reliability and reducing back-up data volumes.
e-Infrastructure South is providing an HPC and data management service for multiple users with various application workloads. It’s a great example of how a highly capable storage solution can help to expand your HPC services without adding multiple file systems and the personnel required to manage them.
This is an opportunity for organizations with centralized HPC services to expand and provide an improved service! It’s not just about performance; it’s also about the data!