Software-Defined Storage – Defined!

By on 03/16/2015.

hatersgonnahateRecently, it has felt like my entire professional existence has been co-opted by buzzwords and marketing.   This is due to a push by seemingly every vendor to establish themselves as leaders implementing emerging technologies. As an industry, we run around spinning … rephrasing … claiming … in an attempt to be named a leader in a given quadrant, and other such nonsense.  While at first these activities are fair and needed to highlight innovation, over time they move from the productive discussion of emerging trends to pure unadulterated marketing.

In my opinion this is where we find ourselves today with SDS.

For the uninitiated, “Software-Defined Storage” emerged shortly after Software-Defined Networking, establishing itself as a real thing.  Given the success and momentum of SDN, a new movement emerged.  Whether you call it the Software-Defined Datacenter (SDDC), Software-Defined Infrastructure (SDI), or the Datacenter Abstraction Layer (DAL), ultimately we’re talking about the same thing.  Handily, you can currently identify which ecosystem someone is most familiar with or supports by which phrase is used.  At its heart, all this Software-Defined stuff really comes down to is … management.

In order to lower the TCO of running a modern infrastructure, the infrastructure must be inherently more manageable. This means rich API support and preferably standards-based interfaces and platforms to improve the admin-to-infrastructure ratio.

The idea is simple …

If a given product can only be administered through a GUI or CLI, then it is difficult to integrate that component with the rest of the datacenter without either producing one-off integration in-house, or purchasing specialized tooling.  In a Software-Defined world, it is easy to integrate disparate systems into a common management platform.

With me so far? Let’s stay on this track …

Manageability is the engine powering the entire Software-Defined movement.  However, like any other emerging technology, the vendor “land grab” has left the majority of our user base confused and fractured.  We are attempting to have a meaningful discussion around the value of these new methodologies whilst not properly defining what the terms being used actually mean.  This is to be expected. Just glance at any Flash conversation and you’ll inevitably see someone bring up Phase-Change Memory… even though it’s not flash. (Hint: It’s non-volative RAM.)

The point here is to reiterate that words matter. Until that is true, it is impossible to fully advance our industry and really move the needle for our collective customers.

This, my friend, is why standards bodies exist! 

This is the problem they solve! They provide the necessary abstraction between vendors to allow new terms to be defined, and thus wholly understood and accepted. They also act as a sort of beacon as to when a concept or idea moves from quaint to primetime, and are called in when innovation and progress begin to loose out to buzzwords and FUD.

Recently the Storage Networking Industry Association (SNIA) assembled a working group of industry veterans from EMC, Evaluator Group, HP, Huawei, IBM, NetApp, and Toshiba to finally sit down and define what SDS is and is not.   The result of this effort was a Draft definition that went out for review in December, and was recently ratified!

Why does this matter?

Well now when someone starts to declare that Solution XYZ from Vendor A isn’t really SDS, we have a common definition to work from.  Meaning … we can establish common language and make actual progress, instead of bickering amongst ourselves.

Per the SNIA definition, SDS is:

“Virtualized storage with a service management interface. SDS includes pools of storage with data service characteristics that may be applied to meet the requirements specified through the service management interface.”

SNIA goes on to list the attributes of SDS:

Required:

  • Automation – Simplified management that reduces the cost of maintaining the storage infrastructure.
  • Standard Interfaces – APIs for the management, provisioning and maintenance of storage devices and services.
  • Virtualized Data Path – Block, File and Object interfaces that support applications written to these interfaces.
  • Scalability – Seamless ability to scale the storage infrastructure without disruption to the specified availability or performance (e.g. QoS and SLA settings).
  • Transparency (self-service) – The ability for storage consumers to monitor and manage their own storage consumption against available resources and costs.

Optional:

  • Scale-Out – Ability to seamlessly distribute the storage services horizontally. May allow for the building of the storage and data services “solution” incrementally.
  • Resource-Pooling – Should include the pooling of storage and other resources.
  • Commodity Hardware – Allow customers to “build it themselves,” providing their own commodity hardware to create a solution with the provided software.
  • Specialized Hardware – May work with either arbitrary hardware or may also enhance the existing functions of specialized hardware.
  • Resource Metering – May allow storage and data service owners to do cost recuperation via a chargeback model based on the authenticated storage consumer.

I’m almost certain that as you read that list you’re thinking to yourself …

“Wait! That’s not right! It’s missing <insert-feature-vendor-has-told-you-is-mandatory>!!!”

Given the current landscape of our industry, I would be surprised if that did not occur.  However in this case I think they darn near nailed it.  It’s not perfect, but it get’s all the important stuff right.

For instance, they call out that …

A SDS solution must have the ability to fully automate the complete storage stack.

They would prefer the automation be completed using a standard management interface (SMI-S/Cinder). However, they will recognize legacy methods so long as the solution implements APIs for all aspects of storage management.

This is the heart of SDS.

They also acknowledge the need to operate in a self-service, cloud-first world, mandating non-disruptive scalability, in addition to a fully virtualized or abstracted data path.

What they do NOT mandate, but agree are common elements of a SDS solution, are all the things we’re fighting over these days.

Commodity hardware vs specialized hardware.
Scale-Out.
Resource pooling.
Chargeback resource metering.

I think this was wise.  These are still hotly contested and thus not proven elements.  For instance, there are valid SDS configurations using both commodity hardware and specialized hardware, in some cases it’s even the same storage solution.  Given the fact that we have valid examples of both, these elements cannot be made mandatory features.  That is not to say they are not important.  It’s just to say they are not required to operate the storage system in a SDS model.

Personally, I don’t think the definition is perfect. It is a good compromise though, and I believe they got all the big stuff right. I’m not going to drill too much further into the definition, as it’s publicly available here.  But, before I let you go, I wanted to address one more potential issue…

There are already definitions of SDS, right?!  I mean … who died and made SNIA God?!  No one. There are multiple definitions, and I’m sure most vendors will still continue to define SDS however they feel like, or more likely how it would most benefit their specific product/solution/implementation.  However, I would point out that SNIA is as legit as they come, and this is totally within both their charter and area of expertise.  If you’re reading this and thinking …

“Pft! Yea but those storage guys don’t know anything about the new SDS solutions like VSAN!”

On the contrary, these guys eat, sleep, and breathe storage! They understand what elements actually matter and which do not.  The purpose of SDS is to make the storage infrastructure more agile, thereby allowing it to keep up with its SDN and SDDC siblings.

While there is a long list of features that most vendors will try and glop onto the list defined by SNIA, I believe SNIA did the right thing. And guess what… Almost EVERYTHING is SDS these days!  It’s not unique. The manageability of traditional Frame Arrays, Unified Storage, Commodity Storage, and Converged Storage is the same from the perspective of the Software-Defined Datacenter.  There are major differences in Power, Space, Cooling, Opex, Capex, etc, etc, Ad nauseam.  However, I’m hard pressed to think of something that isn’t capable of supporting SDS.

Therefore, I would like for all of us to just stop talking about it.  Seriously, everyone has it.  Rebranding your product line to “Software-Defined” is just silly, and trying to declare that your platform is the only way to support this operations model is a red herring.  Whether you use an existing interface, a standards-based approach, or custom/proprietary integration, as long as the storage can be managed as an element of the larger infrastructure, it is then indeed …

Software-Defined.

Glenn Sizemore
Glenn is an industry veteran who specializes in cloud and automation at NetApp. He has authored countless reference architectures and best practices for using Microsoft software with NetApp storage environments, and is a seasoned speaker at industry events around the world.

3 Comments

  • Hi Glenn,

    This is an appalling definition, it is totally ambiguous and the Required section is mostly about storage management rather than services.

    Maybe we need two versions:

    1. Software-Defined Storage Management (i.e. VVOLs)
    2. Software-Defined Storage Services (i.e. VSAN)

    As you say “Almost EVERYTHING is SDS these days!” therefore it is a pointless definition which has been done for marketing reasons.

    For me, and I am sure for most customers, Software-Defined Storage is about providing storage services that run on the hardware of the customers choice which is purchased and upgraded independently of the software (probably from a different vendor).

    Now that is a nice simple definition that everyone can understand rather than the vendor driven nonsense that the SNIA has come up with – very disappointing.

    Do you really agree with their definition as it does not seem to address the points you make in your introduction?

    More thoughts at http://blog.snsltd.co.uk/are-vmware-vsan-vvols-and-evorail-software-defined-storage-and-does-it-really-matter/

    Best regards
    Mark

    • Glenn Sizemore

      Hi Mark,

      I understand you’re point of view, but believe you missed mine. I do not believe the storage engine (data plane) matters in the SDS conversation. More importantly it would appear that SNIA agree’s. It’s fair for any reader to call out the inherent conflict there, but in this case I believe it is the only sane thing to do.

      If we really want to get into it, what you’re referring to is more properly categorized using the wikibon definition of server san… I hate the term, but it does a better job isolating the solutions I believe you’re referring to. Those solutions put the storage engine itself on the same hardware as the hypervisor and gain a level of hardware commoditization from that act. They also all almost universaly suck, and are only financially viable at small scale… but I really try to just stay out of that dog fight.

      The SDS conversation does matter though. It’s ALL about manageability; for instance take VSAN. The same policy based management that make the virt admins job easier applies to VVOLs. They are both software defined, the admin specifies the what in a policy, and the policy engine handles the where, how, and who. it’s just a superior management model and muddying that conversations with the inbreed cock fight that is HCI and server-san is just poisonous and unproductive. This is probably a pipe dream on my part, but I would very much like to keep them separate. One impacts everyone the other is mired in the technical implementation details and offers very specific advantages and disadvantages as one changes vendors. SDS is common and speaks to how the storage is managed, not how the bits are stored.

      thanks for reading, and sharing you’re views. FWIW I am in violent agreement with you’re post, I just want to split the two is all.

      ~Glenn

      • Hi Glen/Nick,

        Thanks for coming back to me.

        The problem with the SNIA definition, as you stated, is it is too loose therefore everything can be classified as SDS.

        Surely we can make an argument that every OS (server and storage) and hypervisor is SDS (i.e. VMFS is SDS) which is therefore a pointless definition.

        I would be interested in your thoughts on examples of technologies that are SDS and others that are not, and the reason for each of the classifications.

        Certainly I do not think that FAS with cDOT (or any other storage array), EVO:RAIL or Nutanix are SDS, but VSAN, Cloud ONTAP, ONTAP Edge and ScaleIO are.

        I think most customers would also agree with this definition.

        Keep up the good work – the datacenterdude.com site is an excellent read.

        Many thanks

        Mark