Intel's Whamcloud Acquisition
Intel’s recent acquisition of Whamcloud occurred with little media fanfare. Whamcloud is the main development arm for the open source Lustre file system, and provides support contracts for existing Lustre installations. On the surface this sounds like an unusual move for Intel, given that it has traditionally maintained a neutral position on file systems and is not in the support business. However, it should be noted that Intel has been keenly interested in parallel file systems for a long time and has published several papers on the use of high performance computing (HPC) and parallel file systems.
The acquisition helps to further strengthen Intel’s position in the HPC market where it currently commands over 74% share in compute processing (http://www.top500.org/lists/2012/06/highlights). We believe the acquisition of Whamcloud is a strategic move to ensure Intel maintains influence in driving the exascale computing program in conjunction with the national laboratories. In addition, it ensures that Lustre development will keep pace with Intel’s internal exascale development – removing a potential obstacle to the adoption of exascale computing for Intel’s customers.
An early sign of Intel’s strategic commitment to the exascale program was when Mark Seager, a major proponent of Lustre and a board member of the OpenSFS board, resigned from Lawrence Livermore National Labs (LLNL) to become Intel’s CTO for the HPC ecosystem (http://insidehpc.com/2011/02/15/inside-track-mark-seager-resigns-from-livermore-on-to-intel/).
Whamcloud shared a common heritage with the national laboratories as its founder and CEO, Brent Gorda, had previously held the position of deputy for advanced technology projects at LLNL. Brent will assume the role of GM, managing the “High Performance Data Division” at Intel. To quote Brent directly: “Working as one company [Intel and Whamcloud], we are now in a stronger position to advance our mutual goals and continue providing vendor-neutral solutions, delivering greater value to our customers, and moving the industry to exascale performance.”
Could this acquisition be explained by acknowledging the conflicting directions that Whamcloud has found itself being pulled in by the OpenSFS? The Commercial members of OpenSFS desire a file system that is more enterprise friendly (The Register, June 2012: “Whamcloud Flogs Wild Lustre Pig into Obedience” (http://www.theregister.co.uk/2012/06/21/whamcloud_chroma/), while the national laboratories are focused on building the next generation architecture for exascale compute. By taking a major ownership stake of the Lustre development effort, Intel has secured the ability to focus Whamcloud development on the program that it cares most about – exascale.
The Whamcloud acquisition highlights one of the major challenges with the open source Lustre file system. Without sufficient contributors to the Lustre code base - such as we see with pNFS and Linux - the progression of the file system is highly influenced by who controls the engineering contribution. With Intel’s focus on exascale it is unlikely that major effort will be spent building in the ease-of-use, enterprise availability and ongoing maintenance and support that is essential for commercial HPC.