StreamFast: Stream arbitrary length data to SSDs with device-assigned addresses and no FTL

Hammerspace founder and EEO David Flynn wants to collapse the data plane into silicon, with zero-DRAM and FTL-free SSDS, and enable sealed, submerged and orbiting data centers in space.

He’s set up a StreamFast business to promote this concept and is working on it with the Open Flash Platform (OFP) group to replace all-flash arrays with directly accessed SSDs in JBOFs that have a controller DPU, Linux, its parallel NFS (pNFS) software, and a network connection.

“StreamFast” as in streaming writes fast, and the reason for that will become clear in a moment. StreamFast is tightly affiliated with Hammerspace, but it’s working down at the controller level and at the local file system level, whereas Hammerspace is at the system and the global file system level.

The basic pitch here is that having SSD controllers use a Flash Translation Layer, means they use DRAM to access it and this adds latency, write amplification, increased electricity draw and less physical space for flash chips in the SSD

Flynn explained in a briefing that a core technology component uses 2011-2014 era Fusion-IO’s DRAM-free SSD concept, which had no Flash Translation Layer (FTL) or a CPU in the SSD controllers. They had fixed logic and no firmware.

Set that aside. Consider that, Flynn says, “It takes one byte of RAM for every kilobyte of flash on the SSD;” a 1:1,000 ratio. Thus a 256TB SSD will need 256GB of DRAM in its CPU-based controller. Flynn added: “Think about that. If you’re going to have a petabyte flash on an SSD, that means you have to have a terabyte of DRAM with it.” And we are facing a DRAM shortage and price climb because it’s in short supply as DRAM fabricators make more profitable (and expensive) high-bandwidth memory (HBM) with their DRAM wafers to meet the sustained high demand from Nvidia and AMD for their GPU memory.

The FTL uses the DRAM to keep track of where everything is on the SSD.

Flynn says: “Nvidia wants people to build devices that can do a hundred million IOPS. … There’s no way you’re going to get those kind of IOPS if you have to look things up in RAM before you can go look them up in the flash.” He concludes: “The two main vectors of innovation, with flash getting ultra high capacity or getting ultra high performance, both of those are hindered immensely by the legacy architecture,” of SSD controllers needing DRAM for their FTL data.

We can avoid the need for an FTL by incorporating the filesystem into the SSD. With parallel NFS now part of Linux we can achieve this. Flynn again: “The SSD guys are facing a huge opportunity to go up the value chain by incorporating file system technologies into their devices and therefore they can get rid of block … it’s about the elimination of the block abstraction notion entirely and having filesystems that interact in a much more efficient way with the flash controller.”

Efficient? “If you do this, you can eliminate write amplification. With the block abstraction, you end up having to write a lot more data than what’s really necessary, because things have to be rounded up to whole blocks, and you end up doing read, modify, writes and all kinds of mess. … You can eliminate write amplification on initial write, entirely zero write amplification for data as it gets ingested.”

“Whether it’s to get ultra high capacity, ultra high performance, or to build something up the value chain of more value, as in embed the file system, we need to get rid of the block abstraction and move to something which is more native to flash. … And this happens to be what I already did at Fusion-IO back in the day.”

If you get rid of write amplification then you can extend QLC flash’s endurance: “We’re talking here about there being the difference to make QLC have sufficient performance to do what we do today with TLC.”

There’s more: “This simplifies the construction of the SSD to the point where it’s much more reliable. And this is why Meta and ByteDance and all of the hyperscalers, the clouds, have wanted an architecture that helps eliminate the FTL on the device because that FTL is a reliability liability.”

“This adds up to us being able to get time to market and bring new nodes, new flash devices, to market sooner, and use newer, less reliable types of flash, more consumer grade.”

More benefits are reclaimed SSD space and reduced power and cooling: “That DRAM is why the SSD runs hot. If you have to put a terabyte of DRAM in it, it has to be refreshed, it’s dynamic RAM. Especially when you look at idle power. And if you’re talking super high capacity and using flash as your capacity tier for storage, it’s the idle power that matters most.”

“Going to a DRAM-free, an FTL-free architecture, eliminates a huge amount of surface area on the SSD, which lets you put more flash there. It eliminates a huge amount of pin consumption on the controller because to put DRAM around the controller, you use just as many pins for the DRAM as you do for the flash.”

From the cooling point of view: “DRAM is the hotspot … if it’s DRAM-free and it’s just flash, your power dissipation is very even across the array. You don’t have hotspots.” So the cooling burden is less.

Ah, the space angle; because a cooler-running SSD needs less power and less cooling effort from the radiators that would have to be used in an orbiting space data center.

Okay, having a DRAM-free, FTL-free SSD architecture would have benefits but how exactly do we do this?

With device-assigned sequential addresses. Flynn: “The magic is that the device assigns sequential addresses to arbitrary strings of data that are streamed to the device, strings of data of arbitrary length. They don’t have to be whole bocks at all. They can be single bytes.” Each incoming string gets written to the flash after the previous string.

“The file system on the host can benefit from knowing that, when it sends data, there’s going to be adjacency in the data. That adjacency allows the file systems to better remember things with less memory.”

“It means you can build a file system that takes less RAM than a regular file system. So not only are we eliminating all of the RAM on the controller, but we’re allowing the file system on the host to be more efficient than a block-based file system.”

It’s akin to treating the SSD as a sequential blob store. The SSD device assigns the address, and it returns that back to the host’s file system.

Flynn: ”And here’s where the magic happens. The file system can more easily remember because the addresses coming back are largely sequential. More importantly, because they’re sequential, it can replay the blob stream, the stream of strings, if there’s a failure. So the host doesn’t have to remember the address synchronously because it can always just replay the sequential stream … because the data is written sequentially, it is a journal and you can replay it.” And: “So now I don’t have to synchronously record it, which means I can acknowledge writes immediately.”

He added: “We’re talking here about building something of more utility than the current generation of file systems because it remembers down to the very byte what was or wasn’t written. And it does that with less DRAM than normal file systems do with blocks.” He means that existing file systems, writing to block-based devices will pad write data out with zeroes to round up a string to fit a block’s length.

How much less memory is required, compared to the FTL-using SSD needing one byte of RAM for every kilobyte of flash? Flynn: “With the StreamFast file system, it’s a byte of ram for every megabyte of flash;” a thousand times better. Our math says a 1 PB SSD would then need 1 GB of memory on the host, instead of 1TB on an FTL-using SSD.

StreamFast would need to work with partners in the SSD ecosystem (NAND fabricators, controller suppliers, SSD builders, major buyers, etc.) to have this happen.

We asked if there was any link to SK Hynix and its setting up of an AI Co. in the USA? Flynn answered: “Can’t talk about specifics of our partnerships yet, but stay tuned. And I was out in Korea just a couple of weeks ago.”

What about timescales?

“Things are moving quickly now. … The team at one of these hyperscalers, when they saw the roadmap with one of these controller SSD memory partners, they’re like, that’s not soon enough. We need it right away. … I’ve already been talking with everybody and the interest has been extremely positive.”

Looking back: “Fusion IO used this approach not to build the file system, but to build the translation layer on the host. And now ,what we’re saying is, let’s use the technique to build the whole file system, something of more value, and just bypass the translation layer altogether.”

Space Data Centers

We’re told that the constraints of space data centers are pretty brutal, focused on power-per-compute, heat rejection, and mass. StreamFast helps in providing more compute per watt, because CPUs do less “data plumbing”. StreamFast’s whole point is collapsing the I/O software stack overhead so fewer general-purpose cycles are burned on storage/network chatter. In any power-limited environment (orbital solar, remote terrestrial “spaceport” campuses), that’s directly convertible into more AI throughput per kW. So, space datacenters need fewer watts and less heat rejection.

Elon Musk’s vision leans toward industrialized, containerized, mass-manufactured compute units (in orbit and eventually via lunar factories). Systems that minimize moving parts, reduce dependence on bulky host stacks, and simplify failure domains fit that packaging style. StreamFast, and all of the work with OFP, are reducing the number of components, to not just reduce power use but also limit the infrastructure size, package size, and overall mass.

Bootnote

1. Back in October last year Hammerspace selected the Xsight Labs E1 800G DPU to eliminate legacy storage servers from AI data storage.
2. If a host has to remember that a data string A has address X then that looks very much like a key:value storage item.
3. There is no StreamFast website.
4. Kioxia tried to build an FTL-free SSD wih its Software-Enabled Flash concept in 2022.
5. There is a parallel Open Flash Platform initiative to build 1 PB rack-scale storage units with StreamFast SSDs and we’ll cover this in a follow-up article.