AI/ML

Huawei’s AI-focussed, all-flash storage

Published

Huawei’s storage product range is centered around two product families; the OceanStor Dorado SAN, NAS and S3 core business arrays, and the OceanStor Pacific unstructured data systems, along with a controller-less OceanDisk JBOD line. At its Huawei Connect Europe event in Madrid it updates to all three product lines, with AI-focused, all-flash, A Series Dorados, all-flash entry-level Dorados, an all-flash Pacific disk drive replacement system, and two OceanDisk smart disk enclosures.

As Huawei effectively does not sell in the USA, it might seem a non-existent storage supplier competitor there. However its storage products have been growing revenues in the double digits range, with Huawei Europe, for example, growing 20 percent last year. The company is the number 2 storage vendor worldwide, according to IDC. Bereft of USA sales it is still second globally, behind Dell and ahead of every other storage supplier. Think of the sales win rates needed to achieve that, and note that it is a strong competitor outside the USA.

Yuan Yuan.

Yuan Yuan, Huawei’s VP of its storage product line, said: “I summarise our storage portfolio in two words. One is complete. We provide whole portfolios from online transaction storage to branch office, … from SMEs to big enterprises who require the complete portfolio. … Second is unified. All these bundle hardware boxes and software were developed by one team. One team.”

The existing OceanStor Dorado v6 product line competes with Pure Storage and NetApp storage arrays. It features Huawei’s own Kunpeng CPUs, Ascend DPUs, and 128 TB QLC SSDs, and there are five products; Dorado 3000, 5000, 6000, 8000 and 18000. The new generation (v7) Dorado range features up to 100 million IOPS, a 10x increase in reliability, and supports S3 as well as file and block protocols. A table summarizes the two generations;

The V7 Dorados are called New-Gen Dorados. Max raw capacity based on single controller enclosure. Models 6000 and below have 2 controllers in one enclosure. Models 8000 and above have 4 controllers in single enclosure. The / symbol in the performance columns indicates Huawei has not released any data. Note the OceanStor A Series is dedicated branding. It’s not a Dorado model, even though the hardware architecture is same with Dorado

We understand that the OceanStor Dorado 3000 has typical capacities in the >200Tb to <500 TB area, and up to 5:1 (SAN) or 2.5:1 (NAS) DRR, while the Dorado 5310 model has typical capacities in the >500TB and <1000 TB area, and up to 3:1 (SAN) or 2:1 (NAS) DRR. It has capacity-optimized NVMe SSDs with 15.36, 30.72 and 61.44T TB capacities.

The 18000 has a full mesh architecture tolerating controller, drive enclosure or cabinet failures, with a 99.99999 percent reliability rating.

New-generation Dorados have already been deployed in Europe, in Germany for example.

The A Series is a faster, more powerful Dorado, though not sharing the ‘Dorado’ branding, for AI training and inference workloads, separate from standard Dorado arrays. It has a KV Cache offload capability, with fast data delivery to GPU servers, using open-source UCM (Unified Cache Manager sometimes called Unified Computing Memory) software.

UCM manages the KV Cache, automatically moving hot (frequently used) data to the GPU’s HBM (high bandwidth memory), warm data to DRAM, and cold data to SSD, based on real-time access patterns. Bearing in mind US tech export restrictions on HBM3 and HBM4 chips, the UCM technology lets Chinese data centers run large AI models efficiently on cheaper, older, or domestic memory, reducing HBM dependency by up to 80 percent in some workloads.

There are A600 and A800 products in the A Series range, and these, being distributed file/object systems rather than supporting block protocol like the other Dorados, are focussed on bandwidth rather than sheer IOPS or latency. Huawei claims their latency is 5× faster than any NFS/ Lustre system, and keeps 255 H100s at 99 percent GPU-utilization during checkpointing.

OceanStor Dorado A800.

The A800 is, Huawei says, the number 1 in the MLPerf storage benchmark. Each 8 RU controller enclosure supports 64 NVMe SSDs and provides raw capacity ranging from 245.76 TB to 983.04 TB. It has great scalability, with a single cluster able to scale out to 512 controllers and EB-level capacity.

The OceanStor Pacific product line is for storing data lakes and supports SMB, NFS, S3, HDFS, POSIX, MPI-IO and Iceberg data formats. It underlies Huawei’s DME (Data Management Engine), which provides a natural language, ChatGPT-like interface to the arrays for storage admins for management and troubleshooting. You can tell it do things like: “Add 100 LUNs for Oracle.” DME also supports the OceanStor A series and the OceanProtect data protection products, and you can get a DME datasheet here.

DME is part of Huawei’s Omni Dataverse AI Data Lake software, which includes a data warehouse, a vector database, data catalog, and data lineage functions.

Image taken from display screen with reflections.

The Pacific product line looks like this;

The OceanStor Pacific 9926 model has 8PB capacity in a 2RU enclosure from 36 x 122.88 TB SSDs. These are Huawei-made SSDs using third-party supplied NAND. These systems provide a claimed 95% space saving over equivalent capacity HDD systems and have an 0.25 W/TB rating, a 79 percent power-saving over disk drive storage. Get an OceanStor Pacific data sheet here.

There are two new OceanDisk systems; the 1610 and the 1800 smart SSD enclosures or JBOFs. They provide shared pools of NVMe-oF storage with RDMA RoCE data access.

The 1610 comes in a 2 RU chassis containing 36 x 161.44 TB QLC NVMe SSDs with a 2 PB maximum raw capacity. Its dual controllers can deliver up to 175 GBps bandwidth and 5.2 million IOPS. It can be integrated with mainstream third-party file systems such as Lustre, GPFS, and BeeGFS, allowing customers to upgrade their systems without changing the storage system architecture.

The 1800 is a 2 RU box with DPUs and 24 drives. It decouples storage and compute and supports NDS 3.0/GPU Direct KV passthrough and KV Cache offload. The 1800 delivers 16 million IOPS, 160 GBps, and has 25 µs latency.

Tencent Cloud and ByteDance use these systems.

Comment

Overall, view Huawei as having a full AI stack data storage offering, from hardware – its own CPUs and SSDs upwards, through structured (Dorado) and unstructured (Pacific) data arrays, intelligent JBOFs (OceanDisk) and AI pipeline software (Omni Dataverse). Although it does not have direct Nvidia certifications it has work-arounds, such as the UCM KV Cache offering, and more than competent channel partners.

Huawei has rebounded from the Trumpian assault in 2018 and built its storage product line revenues to overtake NetApp, Pure Storage, HPE and others, now being second only to Dell. View it as a formidable competitor with excellent products, territorial coverage and channel partners.