Try config'ing vaai on the vmware side. Should be able to get perf on par with nfs.

also if you are looking for a vsa for view, you can get the beta..That vsa is pretuned for view deployements. go to our main site and search for beta.. it will take you right to it.

Hi Linda

We're running VAAI. It's most apparent when we clone a cold machine that VAAI is functional. As to the VSA we're not quite interested in BETA products as of yet as this is a production appliance.

/Denis

Were you getting 60k IOPs w/ nexenta nfs or was that your NetApp machine? How are you testing for these numbers (I'm happy my setup pushes >30k IOPs @8k)? What network layout are you using? I'm inexperienced w/ NFS so I can't make direct comparisons and defiantly need more info about the iSCSI setup.

Hi Jeff

The 60k is on the Nexenta box. We are testing within the vm same testing methology we are using to test NFS. There are quite a few advantages to iSCSI so we have decided to keep using that protocol and optimize if we are able. On the physical side we are running a Nexus 5000 series 10G switch, with an Emulex 10g card on the nexenta appliance and a Intel x520 on our ESXi machines.

On the ESXi we are using round robin on the 10g nics on a dedicated storage vlan.

As to throughput on the vms themselves we see 200mb/s+ writes(as expected with our ibm ssd's) with writeback disabled. But the reads are terrible topping up at 120mb/s and usual staying at around 60mb/s

As I said we are fine pushing 20-30000 iops but curious as to why there is such a difference between iscsi and nfs.

Regards

Denis

Ok now we're getting somewhere. On the esxi hosts can you run esxtop and press u to get to the lun page and then press P (capitol or you'll switch to power) and type in the naa. number for the lun. Two things to look at here are is traffic roughly flowing between the two (or more) paths? Second, what is the DAVG/cmd field operating at?

On nexenta how did you setup iscsi? Did you have a single IP address, multiple addresses, or multiple interfaces?

Have you turned on Jumbo frames on either esxi or nexenta? What version (build) of esxi are you running?

Did a test right now, standard IOMeter on a Win7 machine residing on the LUN.

Here is what i'm seeing:

The traffic appears to be flowing through all of the paths configured. You can see a screenshot of the output. This is at an IOMeter setting of QD=32.

The nexenta appliance is configured with multiple addresses residing on two different interfaces(dual 10G Nic, one ip address per NIC).

Jumbo frames are not enabled(they weren't enabled for NFS either) and the ESXi is version 5.0 Update 1 fully patched.

That is weird to have 5 (an uneven number of) paths. I see you aren't having high latency on your paths (1.32ms on path3 is your worst offender). What is odd is to have a 1400IO difference between your slowest and fastest links. Also are you testing with 4k byte packets? On that test I see 11.3k write IO and 44.42MB/s Was your IO meter reporting something similar? Are you testing directly against the disk or do you have a partition on it? If you have a partition, what is the cluster size you used?

The standard test I've used (after taking it from someone else) is to use the following IOmeter test to simulate real world usage: 8k with 65% read 60% random (and I set it to 8k alignment since I use 8k block sizes for all my NTFS partitions so that I match the underlying block sizes). Can you include that in your test results as well? If you watch the IO Test, does it start high for any period of time then fall off fast or is it constant around 45MB?

Can you provide a screenshot of your VMKernels for the iscsi connectivity? If this isn't production (or can be taken out for this test) can you do the following: Disable all other paths (either disable the paths or set RR back to Fixed). Can you also try setting sync disabled so we can see how much traffic the system could potentially push through the single (as well as all 5) paths (This is purely to see if we are able to get a performance bump).

I'm testing 4k byte packets yes, as I was under the impression that this was the average workload when using a desktop machine. And iometer is consistent between what it reports and what ESXi says.

I'll have a go with 8k 65% read, 60% random and see what numbers i get, also that sounds intruiging about setting NTFS partitons to 8k, logical that would produce better results. I'll post a screenshot of the vmk's handling iSCSI traffic to you when i get back on monday but it is a production machine and so i would have to take it offline to test further :) - but nothing is impossible :)

Hi Jeff

I'm curious as what QD you are testing with in IOMeter?

Regards

Denis

Hi denis, sorry for the delay. Yes I use QD32 for most testing. If I'm testing a "light" workload I'll use 3, but I've also used 64 and 128 to see the limits of the system.

For a simple comparisons things i've gotten to test on here are some figures:

Device                         IOPs    MB/s
MD3000i (14 7.2k Raid6)         1000    8.3
QNAP (4 7.2k Raid10)            130     1.0
Nexenta (5x2 Intel 320 RaidZ2)  27200   222.7