Minding the Store

As SAN and NAS storage drops in price, more companies are implementing them. What's your role in managing these devices?

• By Bill Heldman
• October 01, 2002

• Users need a place to back up their critical data—preferably on some server disk that you currently back up with your formal server backup program.
• The e-mail system is so heavily utilized that your e-mail database is growing by leaps and bounds. You just can't get your boss (and others) to give up their multi-megabyte inboxes!
• The Home and Shared directories on your file servers are gargantuan.
• You've introduced a geographic information system and the spatial images take up massive quantities of disk space.
• Your Oracle and SQL Server databases are colossal.

Where are you going to put all this data? You could keep buying servers with a few hard drives in them, but at some point your servers begins to procreate like rabbits and your farm overrunneth. The biggest question you need to ask yourself in this situation: "Am I maximizing the usage of all this disk space?" Some studies that show that only about 30 to 50 percent of disk space is utilized on all these servers. And that unused disk is expensive!

Also, even though RAID array controller cards have brought a tremendous measure of fault tolerance to servers, there are two problems associated with RAID systems:

• If more than one drive in a RAID 5 array fails, you're toast—you've got to rebuild the array and restore from tape.
• Admins often don't get it when it comes to RAID. They don't have hot or cold spares available and at the ready or they incorrectly rebuild the mirror or stripe set with parity.
• Additionally, even though most Admins routinely back up their systems, they don't often validate the backups by performing routine restoration tests to make sure the data on the tape can actually be restored. (I'm raising my hand as I write this paragraph—been there, done that, bought the soundtrack.)

On top of all that, a $1,700 hardware RAID array controller card, while better than nothing at all, is still a single point of failure (SPOF). While your users' data is certainly safer when on a drive hooked to a RAID card, it still isn't completely safe.

So, is there a solution that can help offset some of these issues? Is there a way that we can provide ample, inexpensive disk space for our users, a place where all of our clients—Linux, Unix, Windows, even mainframe—can go to get their data?

Storage Area Network (SAN)
A SAN is actually a collection of equipment that makes possible the consolidation of disk storage. Following are some of the critical elements in a SAN:

• Storage array—This is a device that has a lot of disks in it. Typically these devices are rated in the number of supported Terabytes (TB). Most high quality (expensive) storage arrays provide some sort of fault-tolerance software in them that continuously monitors the disks, proactively moves data off of bad sectors and even "phones home" if a disk is going to go bad so that the vendor's service department can bring a new one out.
• Network Attached Storage (NAS)—A NAS device (discussed below) is essentially a server with a lot of disks in it using a highly scaled down OS.
• Fiber-channel—The fibre-channel standard is used to connect devices to the storage array. Typically servers that run applications will connect to the storage array to access their data. Most connections are done through some form fibre-channel.
• Host Bus Adapters (HBAs)—Think of these as fibre-channel network cards, HBAs go into your servers so that you can connect them to the storage array.
• Fibre-channel switches—There are two categories of fibre-channel (FC) switch: normal and "director-class". The director-class switch, invented and patented by McData (www.mcdata.com) allows for more intelligent monitoring and traffic management of the FC switch fabric.
• Software—The value-add and the most important feature of a high-quality storage array installation is the software that's used to run, manage and report on the array. Big companies like EMC really shine in this area, but their software is pricey. Other big players such as Legato and Veritas have gotten into the storage array software game. There are a variety of needs: reporting, representing NetBIOS-type share information to Windows clients, monitoring disk health, managing disk space, real-time copying to a remote device backup software, etc.

The combination of all of the above constitutes what most people call a SAN. The idea is that you have users who connect to the SAN via software running on the storage device that allows them to do so, even though they're not fibre-channel-attached and then to access data that's managed by servers that are fibre-channel-attached. This is a SAN in its idealized form—there are other variations such as servers that connect to a FC switch, which, in turn, talks to the storage array; NAS devices that connect to the storage array and so forth. The reason we call a SAN what we do is that with a little bit of architectural expertise you can set up what are called "islands of storage" that provide highly available and fault-tolerant data space and that can act as backups to one another.

Network Attached Storage (NAS)
There are simple solutions out there that allow you to snap on a piece of disk space for your users without having to get into the whole SAN-box (ha-ha). The solution is called Network Attached Storage (NAS), and a wide variety of vendors today offer NAS boxes. NAS solutions typically utilize a mini-OS with enough intelligence to get the data to users (it ain't Windows folks) although big players such as EMC provide very robust NAS solutions. Admins use a set of tools (which vary from vendor to vendor) to manage the NAS.

NAS devices support a wide variety of file systems and generally support all "open systems" clients. When we say "open systems" we mean Linux, all flavors of Unix, Windows, Netware and Macintosh along with the various file systems that these OSes require. We do not mean mainframe clients. If you want a storage device that can host both mainframe and open systems clients you need a larger, much more expensive class of device such as one that IBM or EMC could provide.

Some NAS devices use the Network Data Management Protocol (NDMP) to allow administrators to back up the device. NAS devices do not support niceties such as e-mail, file management or user authentication services.

We're talking basic, inexpensive, reliable easy-to-manage storage. Basic NAS boxes are extremely plug-and-play. (See www.networkappliance.com as an example of a company heavily involved in this type of NAS device).

Admins interested in getting started with SAN/NAS technology and who have minor data consolidation issues to worry about should investigate NAS architectures.

Storage Arrays
A larger storage array differentiates from its NAS cousin in the software that it runs. If you purchase an EMC Symmetrix or Clariion or a Hitachi Data Systems (HDS) 9900, for example, you are paying largely in part for the software that makes that bad boy so highly fault-tolerant. There are expensive up-front software costs, as well as equally expensive ongoing software maintenance fees. However, if any component of the array goes south, it is the vendor that makes sure that you continue operation. EMC and IBM guarantee five-nines uptime (99.999%) while HDS (and its certified reseller—Sun Microsystems) guarantee 100% uptime. How do they do it? Redundancy, redundancy, redundancy. Plus good software. If there appears to be a problem with a section of disk, the operating system will proactively move the data to a good sector and mark the bad place as bad. It will continuously monitor the array. If the bad sector gets worse—that is, it spreads—the OS will actually phone home to tell the vendor that it has a problem and to please bring out a new disk.

One night, I was a server administrator on call in a shop that used EMC Symmetrix. The night operator called me at about 2:00 a.m. to tell me that there was an EMC technician at the door—our Symm had a bad disk and the guy wanted to replace it. The operator asked me what I thought she should do. "Let the guy in!" I said. We had no clue the disk was failing—all of the data was accessible and just fine.

Fibre-Channel Switches
A storage array device typically comes with a certain amount of fibre-channel ports (fibre-channel in the mainframe world is provided through the FICON standard). If the number of hosts you plan to connect to the array exceeds the amount of ports in your array, then you've got to augment with a FC switch.

A fibre-channel switch is used to connect a lot of devices to the SAN, or to connect devices a long way away from the SAN to the SAN, or both. FC switches aren't any different than IDF/MDF switches in the sense that you buy them by the number and by the kind of ports they have in them. There is one noticeable difference however: For what you pay for one FC switch you could probably equip an entire IDF or MDF. FC switches are priced in the tens of thousands of dollars.

Host Bus Adapters
HBAs are also quite expensive. Plan on spending in excess of $1,000 for a typical HBA. Note that you won't need HBAs for a little NAS device that you purchase and hang on the network. You get into HBAs when you have a sophisticated storage array that comes with some FC ports, but not enough for you to hang all your hosts onto the SAN.

Generally your mission-critical data will live on the storage array and your servers that talk to the data will need to be attached to the SAN via FC. We call such servers "channel-attached." As a rule of thumb, you should plan on putting two HBAs in any such mission-critical server.

Fiber-optic cable is generally utilized as the connection between the SAN and the HBA, though copper is also supported in fibre-channel technology.

Note that tape back-up devices such as tape libraries are completely capable of connecting to a SAN via HBAs and are thus equipped to provide tape backup of the data on the SAN.

Backin Up My Words
SAN technology is complicated, high-tech and extremely expensive. But it's also the best way to go if you're considering protecting your user's mission critical data. Another reason to consider SAN/NAS solutions revolves around the idea behind data consolidation. You want to get all that file, home-directory and shared-directory off of your servers and put it onto a central location. These are the main reasons you'd consider a SAN/NAS solution.

There are major players that support both mainframe and open systems on the same storage array. If you don't have a requirement for mainframe storage, you may be able to save significant dollars with a downscaled storage array device (such as the EMC Clariion or offerings from Compaq, HP and others).

When you build a SAN, you're building a little storage network that is comprised of several different hardware components that are managed with software specially designed for the SAN.

If you're considering designing a more complex SAN/NAS solution that a simple 1U NAS box, you'd be wise to see experienced counsel before proceeding.