Getting the most out of an iSCSI SAN

One best practice overrides all others when deploying an iSCSI storage area network (SAN). An IT organization must separate the storage traffic, either physically or logically, from the ordinary LAN traffic to run the network efficiently.

The SAN shouldn't have to compete for bandwidth. Isolating the storage traffic will help to improve response times, prevent bottlenecks, nip potential performance issues in the bud and build in security. Segregating SAN traffic also helps to address the TCP/IP overhead and flow control issues inherent in an Ethernet network.
"A lot of people assume, 'I've got a couple of free Ethernet ports on existing servers. I'll just use those for SAN and throw my data traffic over them,' " said Rick Villars, a storage industry analyst at International Data Corp. "That's not going to work very well. You're going to find that the iSCSI traffic is much more intense than the data traffic."

Logical separation means setting up a virtual LAN to segregate the SAN traffic from the LAN traffic. The physical route involves dedicating separate Ethernet switches to the iSCSI network – not hooking into corporate switches that were already in use. That may necessitate the addition of a couple of new Ethernet ports.

Beyond the best practice of separating the storage traffic, the rules of good storage management apply. Some other best practices for operating iSCSI SANs include:

Buy, build and configure all parts of the iSCSI SAN as fully redundant.
Robert Passmore, an analyst at Gartner Inc., emphasized that any SAN or storage outage is disruptive, so 100% uptime is the goal. That means an organization must pay attention to even "the tiniest details," he noted.

His advice: Build two independent fabrics and select switches that support nondisruptive firmware upgrades. Choose a storage array that is fully redundant with no single points of failure; it should support multiple spare disks and automatic rebuilds and nondisruptive upgrades of hardware and software. Each controller board should have at least two iSCSI ports, each going to a separate Ethernet fabric. If you buy a 16-port switch, buy another one and connect one network interface card (NIC) card to one switch and another NIC card to another switch. Take regular snapshots of the data synchronized with the application, the SAN switch configurations and the array configurations. Consider setting up a second site with fully redundant hardware, and create a disaster recovery plan.

On the staffing side, establish a trained and dedicated storage team to monitor the environment and enforce rigid change control procedures. "I was finding a lot of users who thought that high availability was restricted to buying hardware with no single point of failure and building redundant networks," Passmore said. "They were ignoring human error."

Ensure that the management tools are secure.
Deploying software tools that let IT shops set administrative rolls and limit the scope of actions each individual can take will help keep the iSCSI SAN environment under control. Ensure that only authorized employees can access the tools and physical assets. Administrators need to make sure that only the desired servers can access the designated target storage.

"The good news with regular Ethernet is that any server should be able to get to the storage," said Greg Schulz, founder and analyst at consulting firm The StorageIO Group. "That's also the bad thing. It's a potential security risk. So take adequate steps to make sure that only the servers that are intended or authorized are the ones that are in fact able to access the storage."

Use performance analytics tools.
Examining server workloads and measuring the IOPS going in and out of the storage array can help to determine which network architecture is the best fit, advised Andrew Reichman, an analyst at Forrester Research Inc. Although some organizations do that by trial and error, diagnostic tools afford the more scientific approach.

Reichman recommends building a cost model to assess the benefits of iSCSI and facilitate intelligent decision making on which network is right for each application and workload.

One option is a hybrid SAN in which mission-critical or performance-oriented servers go on the Fibre Channel network and smaller and less critical servers go onto iSCSI. Choosing a multiprotocol storage array that supports both Fibre Channel and iSCSI will help; some enable a primary path on Fibre Channel and a secondary path on iSCSI to build in redundancy at a lower cost point.

When using iSCSI SANs with virtual server environments, devise a plan to effectively balance the load between the servers, the storage and the network.
One of the drivers for iSCSI SAN growth is the popularity of server virtualization technology from vendors such as VMware Inc., Microsoft Corp. and Citrix Systems Inc. The special characteristics of those environments need to be factored into the equation.

"If you don't have discipline in setting up virtual servers and adding additional performance and capacity, you can reach a point where suddenly you've got 400 virtual machines all trying to access the same storage array," said Villars. "There, it doesn't matter if you have Fibre Channel or iSCSI. If you put that kind of load on the system, somewhere in the network or in the array, it will break."

Because server workloads can be shifted to different physical boxes with ease in virtual environments, check to see if the storage array comes with advanced virtualization functionality of its own. Pointing a server to a virtual volume, rather than assigning a disk to a specific server, will make it easier to shift data between different disks or systems without disrupting the server application.

Also useful in virtual server environments is thin provisioning, whereby disk storage space is allocated on demand to boost efficient use of storage capacity in SANs. "In a virtual environment, if you don't have thin provisioning, you can very quickly eat up all your storage," said Villars.