When thinking of NAS in relation to SAN, it’s just hard to say one is better than the other any more. How does one quantify something like that as “better” in today’s modern abstracted datacenter?
As a Technical Marketing Engineer for NFS, I could certainly think of reasons NAS is better, but even I could make a valid argument for SAN in some instances. Datacenters are not one-size-fits-all and not every nail uses the same hammer.
It’s another classic case of “it depends.”
For instance, if your workload uses a single server/cluster and doesn’t need to be accessed by multiple clients at the same time, doesn’t need to worry about ACLs and doesn’t need to be client-agnostic, then SAN might make more sense. But what about NAS?
Wait, aren’t SAN and NAS the same thing?
SAN and NAS are often confused with each other, given they use the same subset of letters in their acronyms and that the general concept is the same – accessing data over a network. In fact, when I’ve asked the question in interviews, people often stumble over what the difference between the two is, and instead, offer the acronym definitions.
SAN = Storage Area Network
NAS = Network Attached Storage
At the surface, they certainly don’t sound very different. However, there are many differences between the two.
- SAN is block-level and leverages SCSI commands. NAS is file-level and leverages standard NFS and SMB protocols.
- SAN can use iSCSI or FCP. NAS uses standard ethernet TCP/IP.
- iSCSI operates at a lower layer (session layer) of the OSI model than NAS does (all the way to application layer).
- FCP uses its own separate OSI-like model.
- NAS is older than SAN. Both had the goal to eliminate Direct Attached Storage (DAS). Both have succeeded.
So, why do I prefer NAS over SAN?
NAS is Client-Agnostic
When you provision a LUN for a client to serve data, it’s essentially a blank hard drive. That means you have to format it, create a volume, partition, etc. That filesystem has to match what your client supports. So, when you attach a LUN to a Windows box, you use a NTFS file system. If you want to mount that LUN to ESXi later… well…
It’s not impossible to mount a Windows-style filesystem to other clients. But it’s not easy and probably not the best idea.
With NAS, the filesystem is that of the underlying storage system. Depending on the vendor, you could potentially access the same data sets from any type of client with the desired permissions. For instance, NetApp’s Data ONTAP has been doing this for years. Try doing that with SAN…
When you set up a LUN on a storage device, regardless of the vendor, you have to define the OS type that will reside on the LUN. If you specify the wrong one, then data will be written across the LUN with an offset that can cause performance headaches for years that get worse as you add more data. There are best practices and tools to help storage admins achieve success with SAN, but with NAS, you don’t even have to think about it.
That said, while NFS exports themselves cannot be misaligned by nature, the virtual machines inside of them with further virtualized vDisks certainly can be. This is an issue that transcends storage arrays, even though they often take the brunt of the blame, even though misaligned virtual disks on an NFS Datastore are often the result of P2V migrations. One of the cool features of the NetApp Virtual Storage Console (VSC) is the scanning of all of your datastores with a report of which virtual machines are misaligned.
With NAS, you simply set up the servers and external dependencies (LDAP, NIS, DNS, AD, etc), create/permission the shares and you’re done. Your clients access the storage as they always have accessed shares over a network. Sure, there are exceptions and one-offs, but if the clients are using the standard protocol specs and have an ethernet card, then everything is gravy.
But SAN? Yikes.
- First, you have to set up the SAN storage and take note of the WWNs/WWPNs and create groupings for those.
- Then, you have to check the compatibility matrices to ensure the client drivers, storage OS version, client OS versions, HBA types, etc are all in line.
- Then you install utility kits/SAN tools on the clients.
- Configure the fabric on the FC switch/zones or set up the iSCSI initiator.
The storage vendors have to test each one of the above thoroughly and ensure it qualifies. Then they have to add it to the list. NAS? Not nearly as much needed. Things stay pretty vanilla in NAS and only change when a new OS or NAS protocol version is released.
In addition to the steps mentioned above, the following is required to attach a LUN to a server.
- Remember what an initiator and target is.
- Do this for EVERY SINGLE SERVER you want attached to the SAN.
- Oh, did you want to attach that LUN to multiple servers at the same time? Sorry.
- Oh, did you forget to format the LUN properly/use the right OS type? Oops. Start over.
- Oh great… where is the disk? Did I rescan for it? Did I use the exact correct configuration?
What does it take to set up a NAS share?
- Ensure the client has access.
- Mount the share.
- Go enjoy the rest of your evening.
Yes, there are some extra things involved if you want to get really fancy with NAS, but at its core, NAS wins in simplicity.
If you are trying to back up SAN, you have an extra piece of the puzzle to consider that you do not have to worry about in NAS: The filesystem.
When backing up SAN, you either do it at the file-level or block-level. To do it at the file-level, your backup application is likely going to use… NAS. Backing up the LUN as a file (such as through NDMP) can be tricky, as the LUN itself is a file with special descriptors that are specific to the storage system that inform the storage OS the file is a LUN. So you are left with the same problem mentioned before regarding the fact that SAN is not client-agnostic. Once you format the LUN, the backup application will need to play nicely with the filesystem you formatted it with.
To back up a LUN at the block-level, the storage system OS would need to support that. Data ONTAP (and many other storage vendors) do that by way of snapshots. But there’s a catch – the storage does not know anything about what the server that has the LUN attached is doing at the time of a snapshot. The server could be doing in-flight reads or writes at the filesystem level that could get truncated in the backup copy, rendering that snapshot fairly useless. So the server would need a way to tell the storage it has flushed whatever caches it needed to and that nothing is happening; go ahead and snapshot. This is done via 3rd party backups and APIs. Sometimes they’re cheap/free. Often times, they’re not.
When I worked in support at NetApp, I worked my fair share of NAS and SAN cases. I learned a lot over the course of those 5 years, but one thing stood out the most – troubleshooting SAN was a pain in the butt.
With SAN, you have more moving parts, plus the fact that the first thing you have to do when you get a case is verify if everything in the stack is supported. Then you have to verify configuration, ensure the utilities are installed, review logs… and if the case wasn’t resolved at that point, you had to dig deeper. Perhaps a perfstat. Maybe some debug logging. Finally, when you get to the point where nothing makes sense and you need a trace, guess what? You have to get a specialized device to sniff packets. Even when you get one, reading the traces aren’t real straightforward.
With NAS, you get past the “is it supported” portion pretty fast. And if you want (and I often do), you could jump right into a packet capture to see where your failure is occurring.
Granted, it’s been a while since I’ve had to work a SAN case, so things may be easier now. And honestly, because I didn’t *love* SAN and was more familiar with NAS to begin with, I never made a huge effort to get better at it.
This is especially true when dealing with FCP. HBAs are not cheap (though they are getting cheaper). And you need one in every server you hope to connect to a LUN. Add in the cables, switches, licenses… and it gets really pricey.
This is less of an issue with iSCSI and FCoE, as it’s all ethernet-based. But then you have to factor in the cost of overhead/troubleshooting, etc.
NAS costs less because IT departments have been doing it for years and have the infrastructure already in place.
Naturally, the notion of having a dedicated path to storage that operates at layers that require much less processing and overhead would suggest that SAN has a significant advantage over NAS for performance. And you’d be mostly right.
This is especially true with 1GB networks and chattier protocols. As NAS protocols get more advanced, they require more to be done over the wire. However, you rarely see 1GB networks in enterprises anymore. Everyone these days is using 10GB links, usually aggregated via LACP to provide mega-trunked pipes to serve up their production data. And with 40GB ethernet on the horizon with RDMA for NAS potentially following it, you might start to see less and less SAN.
In fact, NAS performance has become an afterthought – major application vendors such as Oracle, Microsoft and VMWare are touting NAS for their products in enterprise environments.
Why am I using SAN, again?
As I mentioned previously, SAN is definitely better in some instances, but there are limitations. And as far as data resiliency, you can’t beat storage failovers for SAN, as there isn’t any locking involved on LUNs that cause disruptions when you have a storage outage that requires disk ownership to change like you’d see in stateful NAS. In those cases, your outages would come from client failures. There are many good reasons to use SAN, but as the NFS TME for NetApp, I obviously am biased. I mean, I kind of have to prefer NAS, don’t I?
What if I want to use both?
You’re in luck! Several storage vendors support doing both NAS and SAN on the same storage system, including NetApp. The most important part of the decision process comes down to what is best for your specific use case.