In-Depth

Storage Gets Slim

Forget about low-carb diets. Storage consolidation is the new craze hitting the fat-conscious world.

• By Chris Wolf
• December 01, 2004

Getting trimmer is a national craze these days; South Beach and Atkins dieters, fat-blocking pills and workout machines are everywhere. The idea is that less weight means more energy, a stronger, more fit body and fewer health problems now and down the road.

The same philosophy applies to storage consolidation, for many of the same reasons. Networks on the Consolidation Diet are leaner than their counterparts, often while increasing their data storage (or caloric) intake. As with humans, trimming down increases efficiency and eases management, making for a healthier storage environment.

Unfortunately, you can't go to the bean counters with talk of storage diets, so let's take a look at the technical and financial benefits of storage consolidation.

Exponential data growth makes tracking data an arduous task. When asked about a specific file's location, many administrators have to first determine where the file should be, then determine where it physically resides. For example, suppose the CEO needs his "Christmas Present List" back. He remembers saving it to his mapped network drive about three weeks ago and believes he may have deleted it around a week ago. In this case, the file should be in the CEO's home folder on one of your trusted file servers, but it isn't.

Now suppose user home folders are subject to nightly backup, volume shadow copies every four hours, and weekly data archiving. When you need a file, where do you look?

While keeping up with data is certainly a problem, there are many more contributors to an out-of-shape network. Adding servers to cope with your storage problem means more operating systems to track, software updates, permissions, driver and firmware updates for each box. This could mean extra nights or weekends spent at the office to perform the updates. Your storage requirements may become so great that you can't back up everything in one night over a 100Mbps LAN. For some organizations, this means altering data protection strategies in order to stay within the backup window. Others may need additional local backup hardware for each new server.

Throwing additional resources at data growth often results in significant increases in financial overhead. Additional storage resources and servers equate to more administrative overhead, higher software and hardware maintenance costs, and increased operational costs. For nearly all of us, storage growth results in a significant increase in total cost of ownership (TCO).

Assess Your Storage Environment
Microsoft clearly sees the trend of consolidating both storage and server resources. Naturally, it sees Windows Storage Server 2003 as a means to that end. To help with these efforts, Microsoft teamed up with Quest to offer a free Storage Assessment Tool (available here: http://wm.quest.com/products/FreeTools/).

The Storage Assessment Tool can inventory storage resources on up to 20 servers. The tool will inventory your servers' storage resources and allow you to generate two reports. The first report, the Enterprise Storage Statistics Report (see Figure 1), quantifies your total storage landscape. The second, Enterprise Storage Recovery, shows how much storage space is being wasted across your enterprise. The report includes information on:

• Presence of duplicate files across servers
• Inappropriate stored data, such as MP3s and MPEGs
• Old or inactive data
• Potential consolidation savings

Raw numbers can help you formulate a consolidation plan, but probably won't aid in your sales pitch to the CFO. To put a price tag on the savings of storage and server consolidation, take a look at the ROI calculator at www.storageoptimization.com/roicalculator. (As always, you should be wary of taking vendor ROI calculations at face value. Read Doug Barney's feature story on this topic, "10 Ways to Debunk ROI Claims.")

Figure 1. This report provides statistics on used and unused storage. (Click image to view larger version.)

Planning the Migration
To determine technical requirements for your consolidated network, spend a good deal of time documenting your data access needs. Only your organization knows what data needs to be continually available and what data can be offline for up to several hours. Regardless of your specific environment, you'll likely have to determine if each of the following technologies are appropriate for your consolidation project:

• Storage Area Network (SAN)
• Network Attached Storage (NAS)
• Clustering
• Virtual Machine (VM) technology

The SAN Decision
If you're consolidating and pooling storage resources, a SAN is a good bet. When you set up a SAN, you're building a separate network, usually based on Fibre Channel, to handle your storage resources. This gives you the ability to both move and protect your data without having to impact your LAN. With backup and disk storage devices on the same storage network running at 1Gbps or higher, it's much easier to complete your backup within your window than with a traditional 100Mbps LAN backup.

On the other hand, if you can run backups over a 1Gbps Ethernet, why go with a Fibre Channel SAN? There are alternatives to SANs for high-speed backups, but SAN vendors such as Brocade have been busy the last few years expanding the capabilities of the SAN. For example, you can perform block-level copies of data from one SAN volume to another, without consuming CPU cycles on your network servers. SANs can also dynamically allocate drives in a tape library to whatever SAN-attached server needs them at the moment (current backup vendors that support this capability include CommVault, Veritas and Legato). In pre-SAN storage configurations, a library or tape drive would have to be SCSI-attached to a single server, meaning any access to the drive would have to be through one server. With a SAN, all attached servers have the potential to access any storage device on the SAN.

Needing NAS?
NAS is an appliance optimized for file serving. Some NAS solutions, like the HP StorageWorks NAS 1200 and 2000, support applications such as Exchange Server 2003. The key point to remember with NAS is that you're buying an appliance geared toward a specific purpose, such as file and print serving and/or e-mail serving. You have little flexibility in what you can install on the NAS, with the exception of antivirus and backup software.

The great NAS advantage is the devices come pre-configured and can usually be deployed in minutes. In a market traditionally dominated by Network Appliance and EMC, countless NAS vendors are adding to the competition by offering NAS appliances running the Windows Storage Server 2003 OS. EMC is offering its own NAS running Windows Storage Server 2003, a diversion from its previous proprietary EMC OS.

Many NAS appliances can integrate with a SAN via a Fibre Channel host bus adapter. While this is a nice advantage, remember that the lack of ability to install applications on a NAS may steer you toward a solution involving traditional application servers. However, for dedicated file serving, NAS is usually a good choice.

Clustering Considerations
If you have applications or services that can't be unavailable for more than a few minutes, consider clustering in your consolidation plan. Windows 2003 supports server clusters up to eight nodes. With server clustering, you can configure your high-availability servers as virtual servers in a cluster. Application data will exist on a shared storage bus accessible to all nodes in the cluster. In the event of a hardware or software failure, your virtual application server could failover (move) from one node to another in the cluster with minimal impact on client access. It's also easy to scale a cluster as your server loading increases.

Three-node server clusters can only share disk storage configured in a SAN, while two-node clusters support shared storage located on either an external SCSI array or SAN. If there's a possibility of your organization's clustering needs expanding to three or more nodes, implementing a SAN at the very beginning will save you money in the long run.

Figure 2. Use the ROI calculator to determine total cost of ownership (TCO). (Click image to view larger version.)

Remember, clustering means availability. If your sizing estimates dictate that three Windows 2003 servers will be able to drive your consolidated network, then server clustering may be a real possibility. For example, you could run your file and print servers in a cluster, configuring each to run as its own virtual server. This setup would give you immediate fault tolerance for both your file and print services.

If you were already planning to add a SAN, the additional cost of two Windows 2003 Enterprise Edition licenses (required for server clustering) would be minimal compared to the benefit of high availability. You can run as many virtual application servers as the hardware on each cluster node can handle. With this in mind, server clustering can give you the benefit of both consolidation and high availability.

Virtualization Value
Besides their server consolidation and recovery benefits, virtual machines also provide a new alternative to storage consolidation. With several virtual machines running on a single physical host system, less hardware is required to connect the virtual machines to storage resources. For example, you could connect a tape library to a physical system hosting four virtual machines. All four virtual machines would have local access to the tape library. Without the use of VMs, your alternative would be to use four physical servers and perform LAN-based backups to a single media server, or to invest in a SAN to facilitate backups. This type of functionality is supported by both VMware (GSX or ESX) and Microsoft (Virtual Server 2005). Also, these products allow you to connect virtual machines to shared storage resources in a SAN.

Beyond Virtual Machines
Virtualization of storage resources has also become a hot trend in recent years. With administrators juggling backups, snapshots and archives, they don't always know where the most recent copy of a lost file is located. Storage virtualization gives you a logical view of the storage resources, allowing you to browse backup files by server without having to know what physical medium the backup files reside on. This means you don't need to know anything about how a file was backed up (or snapped or archived) in order to recover it. Two storage vendors, Veritas and CommVault, have done significant research in this area.

Parts of the Plan
Storage consolidation projects can be more difficult than traditional network reshuffling because administrators involved in the project often have little experience with today's storage networking technologies.

When you decide to take the plunge, follow these general steps:

1. Document the current environment. It's hard to know where you're going unless you know what you have. When you start a diet, you step on a scale. When you look to consolidate storage and server resources, you need to know how much data you have, as well as estimate growth. Products like the Storage Assessment Tool will help you inventory your current storage resources.
   
2. Identify tools and technologies. Once you know what you have, it's time to make decisions on the direction of your consolidation efforts. Looking for a quick deployment, lots of storage and built-in redundancy in a box? Maybe NAS is right for you. If you want multiple systems to access and share a pool of storage resources, consider a SAN. If redundancy is required, clustering may be warranted. Virtual machines can give you the benefit of high availability as well as additional flexibility in connecting servers to shared storage resources, but that may depend on your comfort level.
   
3. Design the new infrastructure. After deciding on the best option, it's time to work on design. If this is your first crack at a storage consolidation project, try and get your vendors involved. Most vendors trying to push their products have no problem evaluating your proposed design. How aggressive the vendor is at securing your business will likely dictate how much it will charge (if anything) for this service.
   
4. Implement the infrastructure. Implementation is when theory meets reality. At this point, you should be putting your new servers, OSes, applications and management tools in place and testing access to storage resources. It's likely that implementation will be an incremental process, starting with the most critical servers and storage resources. While it may prolong the process, testing each new server or storage resource will almost certainly save some of your sanity.
5. Migrate data. After implementation comes data transfer. Many backup applications can assist with this part of the process. If you plan to incorporate data archiving products, ensure that the original data's modification time isn't altered by the move. You don't want all of your migrated data to appear as brand new files on the new storage media.
   
6. Test. With the data in place, it's now time to test the new infrastructure prior to it going live in production. Testing must include data access, security, and also data protection and recovery. Now is the time to also ensure that your backup solution is successfully recovering data by testing restores—a task that should continue on a periodic basis.
   
7. Add new resources to your production environment. After testing, you can incrementally bring your new environment online. You shouldn't even have to hold your breath the first morning your consolidated infrastructure is live.

Best Practices
Now that you're ready to join the consolidation craze, keep these practices in mind to better your chances for a successful consolidation project:

• Use Mature Products. I prefer to use hardware and software that's been around the block. Regardless of how great the sales pitch, bringing an unknown product into your project is always a risk. Products that have the backing of OS vendors like Microsoft are more likely to do what they promise, and their vendors are more likely to still be in business at the end of the project.
• Migrate in stages. Roll out your design in phases, addressing the most critical needs first. Following the infrastructure design, I prefer to build the new storage infrastructure and migrate the data for a couple of servers at a time. Next, test data access and bring it online. Once I'm confident that everything is working as expected, I move to migrating data and consolidating additional servers.
• Avoid proprietary hardware solutions. An easy trap to fall into is going with a proprietary solution from a single hardware vendor. In storage networking, you'll have little problem integrating solutions from vendors such as Brocade, McData or Nishan Systems. However, you'll encounter some disk vendors that may try and push their storage and backup solutions on you. The problem is that their proprietary solutions will likely only work with their storage devices, and if you decided to purchase an end-to-end solution from the vendor, you're married to them. This will make a divorce costly. Many vendors have support matrixes on their Web sites, so it should be easy for you to check and verify the hardware and software compatibilities of your planned new storage infrastructure.
• Design for scalability. Don't skimp on scalability. If your current requirements warrant the purchase of an eight-port Fibre Channel switch, consider purchasing a 16-port switch. If you're considering a storage infrastructure based on Ethernet, think about investing in a switch that supports Internet SCSI (iSCSI), Internet Fibre Channel Protocol (iFCP) or both.
• Design for availability. Include redundancy in any storage design. This could start at storing data on a hardware RAID 5 array, for example, with an extra physical disk as a spare. While RAID can protect the data itself, you also need to be concerned with protecting data access. This may involve implementing a clustering solution that will eliminate a single point of failure, such as a failed NIC. Another alternative is to maintain standby virtual machines on an alternate server. This way, if your primary VM goes offline, you can bring up a spare VM as a replacement. Products such as VMware's VirtualCenter and VMotion allow you to synchronize an online VM with a standby VM on an alternate server.
• Invest in management software. While consolidating to a more complex storage network will result in increased overall efficiency and storage resource utilization, products designed to manage data in these complex environments will make your life considerably easier. Many of the backup storage vendors already mentioned in this article have both data and storage resource management solutions available.

Trimming Down
Like any slim-down plan, getting started on the storage consolidation diet isn't easy and requires quite a bit of work at the beginning. However, the long-term performance and health benefits for your network are immeasurable.

As nearly every organization's data continues to grow at an exponential rate, the old process of simply throwing more storage at a problem is no longer effective. The more independent "data islands" you have, the more difficult and costly your network will be to manage. To keep pace with storage growth, as opposed to being overwhelmed by it, storage consolidation is the clear choice.