Curious how the marketplace for distributed file methods, interconnects, and high-end storage is taking part in out in 2024? You then is perhaps out there evaluation that Hyperion Analysis is planning on rolling out over the following a number of months.
Hyperion Analysis is the HPC-focused analyst group that separated from IDC earlier than IDC’s guardian firm was The group, which relies in St. Paul, Minnesota, conducts periodic complete world HPC web site surveys to get a greater concept of the compute, storage, and networking investments that a number of the largest HPC websites world wide.
Mark Nossokoff, the Hyperion analyst who tracks storage, lately sat down with Datanami to talk concerning the agency’s analysis and the 2024 world HPC web site survey that’s at the moment underway. The corporate plans to begin releasing outcomes from the 2024 web site survey across the Worldwide Supercomputing Convention (ISC), which takes place subsequent month in Hamburg, Germany.
“We now have over 1,000 methods at over 100 websites who we ask about the entire gamut of the HPC, storage, and interconnect realm,” Nossokoff mentioned. “We ask about file system utilization utilization, capacities, on prem storage versus capability and storage within the cloud, most popular cloud storage suppliers, most popular on-prem storage distributors. So its has numerous statistics.”
As in earlier years, the 2024 report will gauge who’s profitable within the battle amongst HPC storage file methods. General and inside the NAS file system area, NFS was the clear winner in 2021, in response to 2021 HPC web site survey report shared by Hyperion, with a couple of 53% share throughout 2,006 HPC methods operating at 141 supercomputer websites spanning authorities, trade, and academia. That was adopted general by a number of parallel file methods led by Lustre at about 35%, and GPFS/Spectrum Scale at about 25%. , and HDFS at about 20%. ZFS, XFS, and different.
Hyperion will likely be seeking to see how the file system panorama has modified. NFS figures to proceed to honest properly, significantly amongst trade websites, whereas Lustre will seemingly keep its lead at authorities websites. Nossokoff may also be seeking to see whether or not relative newcomers, corresponding to Distributed Asynchronous Object Storage (DAOS), the open supply object retailer spearheaded by Intel in 2012, have gained any traction.
“It is going to be attention-grabbing to see if DAOS has gained any traction past Argonne and Google, the place it’s the basis for its Parallelstore storage service providing,” Nossokoff mentioned.
Hyperion may also be exploring different conventional NFS-based methods and rising knowledge platform architectures, together with these from DDN, Dell, Hammerspace, HPE, IBM, NetApp, Qumulo, VAST, Weka, and others.
(Pavel Ignatov/Shutterstock)
Hyperion will likely be seeking to see how the surge of funding in AI is impacting storage. Are they utilizing completely different storage varieties for coaching and inference workloads, or is it the identical? Which distributors are transferring forward within the race, and is there anyone new or surprising gaining traction?
“We’re requested loads of about interconnect adoption,” Nossokoff mentioned. “What structure is it? Is it a single [converged] interconnect that handles each storage and the MPI visitors?, Or are they unbiased interconnects, one devoted to server-to-server and the opposite devoted to system-to-storage?”
Lastly, Hyperion will analyze the community choice, which is usually a horserace between InfiniBand and Ethernet. For the 2021 web site survey, Ethernet was forward with a forty five% share, with 100Gb being the commonest pace, whereas InfiniBand had a 36% share of the general community market, with 100Gb and 200Gb sharing an equivalent 31% share inside the InfiniBand cohort. And the way will OmniPath fare, and can it climb above the 4.5% share it had within the final web site survey?
The location survey additionally delves into the compute facet of the home, which permits Hyperion to assemble very particular knowledge about what number of methods are in use, what number of processors, and many others. The corporate additionally examines person’s spending on HPC sources within the cloud, offering perception into cloud spending patterns on compute, storage, networking, and software program functions.
Hyperion doesn’t sometimes launch the complete outcomes from the location survey all of sudden. The examine is funded by a handful of purchasers, who get first dibs on the good knowledge. However finally the analysis trickles right down to the trade at giant, giving us an in depth peak behind the HPC curtain.
Associated Objects:
Object and File Storage Have Merged, However Product Variations Stay, Gartner Says
The Previous and Way forward for HPDA: A Q&A with Steve Conway
Hammerspace Hits the Market with International Parallel File System
DAOS, ethernet, file system, file methods, GPFS, HDFS, hpc, Hyperion Analysis, infiniband, Lustre, Mark Nossokoff, nas, parallel file system, storage, XFS, ZFS
