ESX Storage Journey Takes II – iSCSI Storage Limitation

30th May 2008

Preparations for file server migration
Deadline is near to virtualize one of our File Print+ Symantec AV server.  I’ll need to virtualize and hook up a LUN from a Dell MD3000i via iSCSI which is connected to ESX and map it as a raw partition to the file server.

This is one hell of a work considering that we’ll need to p2v the server running on Win2k and upgrade it to Win2k3 at the same time ensuring clients will be able to access to the services after upgrading. But what the heck, as far as the management is concerned, we are the ones who makes all things possible. 

I’ll need to test out the environment for this new virtualized system before we put it to production.  I would also need to test out if the 2TB storage limit is in effect if we mount it as a raw partition on guest via the host iSCSI initiator.

Equipment
The MD3000i unit is still pending arrival so I’ll be running some test on our existing Clarion 3-20, flexing out its iSCSI capabilities which we’ve never even thought of using.  Test is conducted on a Dell 2900 with 16Gb of ram attached with 2 FC HBA and 6 NICs running ESXi.

Setup
I started off by firing off the Navisphere console and setup a 2.4TB LUN to test out the 2TB barrier of ESXi.  On the ESXi, I’ve created a new VMkernel port and have assigned to it a dedicated NIC.

With CHAP disabled, the LUN, 2.4TB appears on the VI client after a click on the rescan; I was trilled.  That is just part one, we’ll need to test it on the guest to be sure. 

I mounted the raw partition as "virtual compatibility mode" into the guest and proceed to power up the Win2k3 Ent R2.  Only 367.28GB was detected out of the 2.4TB.  Thinking it might be due to "virtual compatibilty mode" I proceed to delete the drive from the guest and created a new raw partition as "Physical mode".  This time its worse; it states there unknown under storage device.  I’ve gotten the latest vmware-tools installed but issue still persist.  (*refer Note)

After several more attempts, I proceed to remove the virtual hard drive from the guest and did a iSCSI rescan from ESXi and poof goes the drive; it went missing.  Could it be EMC?  I don’t really trust that unit; its like a total black box with a few click-me buttons.

Note: It was the next day after all these test has been done that I’ve realized that I’ve got the FC HBAs connected to the SAN as well.  Navisphere clearly shows the two connections as separate entities during testing but yet in this case, the LUN was connected via FC instead. 

I’ve tried unplugging the HBA  and found that LUN size higher then 2TB is not detectable via ESXi’s iSCSI initiator – below.  Further investigation shows only FC is able to detect LUNs higher then 2TB but the 367Gb persist in Win2k3 Ent.  Could that be some kind of limitation on the iSCSI initiator?

 

iSCSI Storage limitation
It took me a lot more tries to remove and adding in the partition from Navisphere, making sure that I’ve added in the correct LUN into the appropriate Storage Group. No good, the LUN is not showing up anymore in ESXi.  And finally it came up right after I recreated and attached it to a 2TB LUN. (I’ve unplugged both the HBAs at this point)

Now, I’ll need to mount it in guest to make sure it works.   With the LUN mounted as a raw partition in the guest, Win2k3 Ent R2, disk manager shows a solid 2TB drive ready to be used.

It was an awfully long, tedious and frustrating day spent testing on the solution but least I now know that its going to work.  Will be spending more time on VMware’s Community forum on this issue while preparing for the next phase of our file server migration.

till then…..