Contact Sales

Contact Sales

How to Choose the Right Storage for AI and HPC

Praneetha Manthravadi | June 3, 2019

More and more, companies are using high-performance computing applications, such as large-scale simulations, discovery, and deep learning, to stay competitive, support research innovation, and deliver the best results to customers.

But, if your company is like most, it’s also struggling to pick the correct storage system that can support this important work.

The problem with traditional HPC storage
While traditional HPC storage such as Lustre and Spectrum Scale are powerful, they can also be extremely complex and expensive to manage. They introduce significant new administrative overhead for tuning, optimizing, and maintaining storage performance for different HPC workloads, driving higher TCO. They can also introduce reliability problems as systems scale and produce performance bottlenecks.

The problem with traditional enterprise storage
Traditional enterprise storage systems are more manageable, but their performance is often insufficient to handle data-intensive, demanding applications .
It’s no wonder companies like yours are struggling to make the correct storage choice. The wrong option will slow your ability to deliver products to market, compromising your overall competitiveness. The right choice will help you deliver insights quickly and improve time-to-results because your storage system will keep up with intensive computational processing and time-sensitive data delivery.

What kind of storage do you need for HPC and AI?
Fundamentally, you need storage that’s fast and simple and has these six key characteristics:

No performance limitation
Your HPC storage solution should have no performance limitations as it scales, so you can quickly address your changing storage needs and receive the full performance value for every node you add to the system.

Consistent high performance
Look for storage that is consistently fast regardless of the complexity of the data, application, users, and workloads. As these factors change and increase over time, you need to be able to trust the system to perform consistently and to meet expectations. No matter the changing requirements, you must be able to accurately predict system performance.

Intelligent data placement
You need intelligent data placement with separate, parallel, bottleneck-free data paths to metadata and data. It should leverage the unique performance characteristics of SSDs and HDDs to create the highest performance at the lowest cost.

Easy to deploy, manage, and scale
Your storage should be easy to operate and not require deep technical skills to manage. System administrators should be able to add capacity and performance in seconds, and a single IT administrator should be able to handle the storage system no matter the scale.

Reliable
The reliability of your HPC storage should increase with scale. It should automatically recover from failures and have no single point of failure. With plug-and-play storage, intelligent software orchestrates the recovery and repair process automatically. Errors are bypassed and work continues.

Self-tuning
Tuning requires deep knowledge of how the storage system works. It’s time-consuming, complex, and error prone, which decreases productivity. You need a self-managing system that has been tuned to optimize the performance of a vast majority of HPC applications and doesn’t require re-tuning as workloads change.


With the right storage, you can forget about it and focus on what’s important: turning your next great idea into real-world reality.


Related Articles