sawtooth like data transmission
Added by Martin Forster over 3 years ago
Hello all,
im just filling my nexenta box with data. Im using cifs shares with deduplication. As copy tool i used 3 instanced of richcopy 4.0 from 2 differenz machines. i get a funny behaviour.
And this are my Questions 1.) Expected behaviour ? 2.) further Troubleshooting steps ? 3.) How to fix it ?
In the files you find a screenshot of analytics, a iostat -x 1 a show network stat a show appliance sysinfo
Many Thanks
greenshot_2010-05-05_14-44-34.jpg (155.7 KB)
sysinfo.txt (1.2 KB)
network_stat.txt (1 KB)
iostat-x.txt (4.8 KB)
Replies
RE: sawtooth like data transmission - Added by Angel Vidal over 3 years ago
It looks like TCP congestion protocols at work. It is somewhat expected. There are a few different algorithms for controlling TCP congestion depending on the server and client. I dont have experience with tweaking this under opensolaris, but hopefully this will take you in the right direction.
Here's a link to Cisco's page on this: https://www.cisco.com/web/about/ac123/ac147/archivedissues/ipj9-2/gigabit_tcp.html
RE: sawtooth like data transmission - Added by Martin Forster over 3 years ago
Network Congestion ? when i look at the numbers, i see that i am far away from filling the 1Gbit link. So no i dont think that it is network congestion. Usually even with a network congestion proglem i get something around of 70% of the whole bandwidth.
I think its the ZFS Caching and IO Congestion Algorithm described here: http://blogs.sun.com/roch/entry/thenewzfswritethrottle
There are also some forum and mailing list discussions in the solaris sites which talks about that.
Im sure that there is a way to tweak it a little, but i needed a usable thing fast. So i already switched back to openfiler. Which is far behind the features which i would get with zfs, but at least it works as expected.
I tried to switch a folder from the latency setting to throughput, but this caused my my nmc to crash. Even reboot hung for a long time. This was the last thing which made switching back to openfiler.
a minor Problem which i also experienced was how to get proper permission for a file share. i described that in a other forum thread. All the hints were not usable for me because the extended ACL version of the tools was missing on my box.
I think if this flaws get fixed, and the webgui polished to proviede a easy way to setup cifs with proper ACL's, this distribution would be really a star. If i had time, i would thry out the evalution version of the .com box. Maybe its different from the community box ?
Kind Regards
Martin
RE: sawtooth like data transmission - Added by Tony Mayse over 3 years ago
I am experiencing the same problem from two different Windows 7 machines. Up to this point I have been connecting mostly from OS X & Linux machines & have seen no such limit in the connectivity. I haven't yet tested XP to see if it's affected by this problem or not.
Perhaps this is more of a Windows issue than a NexentaStor issue, but really I need the two to play well together.
Has anyone else had such an issue? I really need some guidance on solving this issue.
RE: sawtooth like data transmission - Added by Ryan W over 3 years ago
Martin Forster wrote:
a minor Problem which i also experienced was how to get proper permission for a file share. i described that in a other forum thread. All the hints were not usable for me because the extended ACL version of the tools was missing on my box.
I think if this flaws get fixed, and the webgui polished to proviede a easy way to setup cifs with proper ACL's, this distribution would be really a star. If i had time, i would thry out the evalution version of the .com box. Maybe its different from the community box ?
Kind Regards
Martin
Nope. You can do all the exact same ACL stuff with the CE as the "Enterprise" box. I run both. the /usr/sun/bin/chmod and /usr/sun/bin/ls are the appropriate solaris tools for chmodding from a CLI. You can also do all your ACL work from the GUI too very easily.
RE: sawtooth like data transmission - Added by Martin Forster over 3 years ago
Hi again,
@ Tony, i dont think its a network problem. A network problem does not explain the heavy reads on my system. But try this one http://thesystemadministrator.com/TheSystemAdministrator/Tips&Tricks/DisableTCPAuto-TuningtoSolveSlowNetwork_%28Vista%29/
@Ryan, sorry i used the wrong path :-(. Hoever whith the webgui i was able to enable access, only the renaming operation failed. A simple reapply of the same ace in Windows Explorer fixed that. And this resulted in no visible change in the webgui and the windows Explorer. I dont understand what is going on there ...
Regards
RE: sawtooth like data transmission - Added by Tony Mayse over 3 years ago
Thank you, Martin. That worked. The answer was to turn off tcp autotuning by running a cmd prompt as administrator & entering netsh tcp interface set global autotuning=disabled. As soon as I did that I had a brief hesitation on my first transfer to the NexentaStor but then everything was rosy after that.
RE: sawtooth like data transmission - Added by Martin Forster over 2 years ago
hiho
status update.
i just wanted to to say: at the moment i have a virtual nexental 3.0.3 runnin on a vSphere4.0U1 Host provideing NFS dedupe Storage to my hosts.
the Nexenta vm has 1vCPU (5130 Intel) 4 GB RAM, 6 10k Drives per Raw device mapping. configured as raidz1 128kb Blocksize. When i write large files to the appliance (over the esx with veaam fastscp) i archive around 30MB/sec. During this times i have nearly 100% cpu. The volume has 800 GB, 80 GB deduped 380 GB RAW, no clue how large my ddt table is at the moment. but it seems to fit into my memory at the moment.
So i have a good network performance in this case. i havent tested cifs right now.
Regards
RE: sawtooth like data transmission - Added by Alex Destiny over 2 years ago
Hello, I've got the same problem (sawtooth network utilization) on a W2K3 Server client, connected through 1Gb iscsi. How to fix it in my case? Best Regards, Alexey
RE: sawtooth like data transmission - Added by Cactus Data over 2 years ago
Updated URL: http://www.curuit.com/disable-tcp-auto-tuning-to-solve-slow-network-(vista-%10-windows-7-%10-windows-2008)-20070621145/
RE: sawtooth like data transmission - Added by joseph schafer over 2 years ago
Hello, Thank you for the hint
C:\Windows\system32>netsh interface tcp show global C:\Windows\system32>netsh interface tcp set global autotuning=disabled C:\Windows\system32>netsh interface tcp show global
However, it did not seem to have an impact.
Attached are views of the sawtooth paramaters prior to disabling autotuning (before/idle, during, & after) and also three views of the same after setting autotuing=disabled. I tested immediately after setting and saw no change either - and then I rebooted the window box, def, verified that the autotuing remainde disabled and tried again. thoughts welcome. thanks, joseph
RE: sawtooth like data transmission - Added by joseph schafer over 2 years ago
whoops, here are the next three shots from autotuning=disabled