sawtooth like data transmission

Added by Martin Forster over 3 years ago

Hello all,

im just filling my nexenta box with data. Im using cifs shares with deduplication. As copy tool i used 3 instanced of richcopy 4.0 from 2 differenz machines. i get a funny behaviour.

And this are my Questions 1.) Expected behaviour ? 2.) further Troubleshooting steps ? 3.) How to fix it ?

In the files you find a screenshot of analytics, a iostat -x 1 a show network stat a show appliance sysinfo

Many Thanks


Replies

RE: sawtooth like data transmission - Added by Angel Vidal over 3 years ago

It looks like TCP congestion protocols at work. It is somewhat expected. There are a few different algorithms for controlling TCP congestion depending on the server and client. I dont have experience with tweaking this under opensolaris, but hopefully this will take you in the right direction.

Here's a link to Cisco's page on this: https://www.cisco.com/web/about/ac123/ac147/archivedissues/ipj9-2/gigabit_tcp.html

RE: sawtooth like data transmission - Added by Martin Forster over 3 years ago

Network Congestion ? when i look at the numbers, i see that i am far away from filling the 1Gbit link. So no i dont think that it is network congestion. Usually even with a network congestion proglem i get something around of 70% of the whole bandwidth.

I think its the ZFS Caching and IO Congestion Algorithm described here: http://blogs.sun.com/roch/entry/thenewzfswritethrottle

There are also some forum and mailing list discussions in the solaris sites which talks about that.

Im sure that there is a way to tweak it a little, but i needed a usable thing fast. So i already switched back to openfiler. Which is far behind the features which i would get with zfs, but at least it works as expected.

I tried to switch a folder from the latency setting to throughput, but this caused my my nmc to crash. Even reboot hung for a long time. This was the last thing which made switching back to openfiler.

a minor Problem which i also experienced was how to get proper permission for a file share. i described that in a other forum thread. All the hints were not usable for me because the extended ACL version of the tools was missing on my box.

I think if this flaws get fixed, and the webgui polished to proviede a easy way to setup cifs with proper ACL's, this distribution would be really a star. If i had time, i would thry out the evalution version of the .com box. Maybe its different from the community box ?

Kind Regards

Martin

RE: sawtooth like data transmission - Added by Tony Mayse over 3 years ago

I am experiencing the same problem from two different Windows 7 machines. Up to this point I have been connecting mostly from OS X & Linux machines & have seen no such limit in the connectivity. I haven't yet tested XP to see if it's affected by this problem or not.

Perhaps this is more of a Windows issue than a NexentaStor issue, but really I need the two to play well together.

Has anyone else had such an issue? I really need some guidance on solving this issue.

RE: sawtooth like data transmission - Added by Ryan W over 3 years ago

Martin Forster wrote:

a minor Problem which i also experienced was how to get proper permission for a file share. i described that in a other forum thread. All the hints were not usable for me because the extended ACL version of the tools was missing on my box.

I think if this flaws get fixed, and the webgui polished to proviede a easy way to setup cifs with proper ACL's, this distribution would be really a star. If i had time, i would thry out the evalution version of the .com box. Maybe its different from the community box ?

Kind Regards

Martin

Nope. You can do all the exact same ACL stuff with the CE as the "Enterprise" box. I run both. the /usr/sun/bin/chmod and /usr/sun/bin/ls are the appropriate solaris tools for chmodding from a CLI. You can also do all your ACL work from the GUI too very easily.

RE: sawtooth like data transmission - Added by Martin Forster over 3 years ago

Hi again,

@ Tony, i dont think its a network problem. A network problem does not explain the heavy reads on my system. But try this one http://thesystemadministrator.com/TheSystemAdministrator/Tips&Tricks/DisableTCPAuto-TuningtoSolveSlowNetwork_%28Vista%29/

@Ryan, sorry i used the wrong path :-(. Hoever whith the webgui i was able to enable access, only the renaming operation failed. A simple reapply of the same ace in Windows Explorer fixed that. And this resulted in no visible change in the webgui and the windows Explorer. I dont understand what is going on there ...

Regards

RE: sawtooth like data transmission - Added by Tony Mayse over 3 years ago

Thank you, Martin. That worked. The answer was to turn off tcp autotuning by running a cmd prompt as administrator & entering netsh tcp interface set global autotuning=disabled. As soon as I did that I had a brief hesitation on my first transfer to the NexentaStor but then everything was rosy after that.

RE: sawtooth like data transmission - Added by Martin Forster over 2 years ago

hiho

status update.

i just wanted to to say: at the moment i have a virtual nexental 3.0.3 runnin on a vSphere4.0U1 Host provideing NFS dedupe Storage to my hosts.

the Nexenta vm has 1vCPU (5130 Intel) 4 GB RAM, 6 10k Drives per Raw device mapping. configured as raidz1 128kb Blocksize. When i write large files to the appliance (over the esx with veaam fastscp) i archive around 30MB/sec. During this times i have nearly 100% cpu. The volume has 800 GB, 80 GB deduped 380 GB RAW, no clue how large my ddt table is at the moment. but it seems to fit into my memory at the moment.

So i have a good network performance in this case. i havent tested cifs right now.

Regards

RE: sawtooth like data transmission - Added by Alex Destiny over 2 years ago

Hello, I've got the same problem (sawtooth network utilization) on a W2K3 Server client, connected through 1Gb iscsi. How to fix it in my case? Best Regards, Alexey

RE: sawtooth like data transmission - Added by Cactus Data over 2 years ago

Updated URL: http://www.curuit.com/disable-tcp-auto-tuning-to-solve-slow-network-(vista-%10-windows-7-%10-windows-2008)-20070621145/

RE: sawtooth like data transmission - Added by joseph schafer over 2 years ago

Hello, Thank you for the hint

C:\Windows\system32>netsh interface tcp show global C:\Windows\system32>netsh interface tcp set global autotuning=disabled C:\Windows\system32>netsh interface tcp show global

However, it did not seem to have an impact.

Attached are views of the sawtooth paramaters prior to disabling autotuning (before/idle, during, & after) and also three views of the same after setting autotuing=disabled. I tested immediately after setting and saw no change either - and then I rebooted the window box, def, verified that the autotuing remainde disabled and tried again. thoughts welcome. thanks, joseph