Hi folks, We have a performance problem with one of our servers that reads thousands of small XML files from a SMB share and then stitches them together in to a collection of slightly bigger files on the local drive. The problem server can only manage to process about 1 XML file every 2 secs when it used to be doing 10s of files every second. Running a fixed test and wiresharking the SMB conversation going on the server that is performing very slowly I can see according to SMB service response times the following:
On another server that performs many times faster:
Given that the same test (using the exact same XML files) has been run on both machines, I'd have expected roughly similar number of SMB commands/packets being sent but you can see on my poorly performing server there are several times more of every command and there are two extra commands which are appearing: NT Trans and also under the Trans2 sub-commands I'm seeing "QUERY FILE INFO" which I don't see on the server that performs well. Is there an obvious reason why I should see so much more traffic going on for the same test and why am I seeing lots of "NT QUERY SECURITY DESC" commands? I should add that my Wireshark cap filter is limited to just the server I'm pulling the files from and I get the same results over and over. Access to the share from both machines is the same too. I have tried the same test on a variety of servers and all of the others I have tested show the same results as the server that performs well. These particular servers are both Windows 2003. I have checked SMB signing is off for both and other lanmanworkstation settings are the same from what I can see in the registry. I don't believe network performance is the issue because I've carried out a test using FTP instead. Any help would be greatly appreciated. Cheers Sam asked 03 Oct '13, 08:01  billbofagends edited 03 Oct '13, 08:03 |
3 Answers:
Long time since I asked this question I know but we have finally resolved the issue. The server in question is VMWare virtual machine on ESXi 5.0 and has VMWare tools installed. Part of VMware tools includes shared folders. After removing VMWare tools and then reinstalling without shared folders included, the issue was fixed and performance was restored. Pretty obscure but have proved this on quite a few other servers now. Hope this helps someone else out! answered 08 Jul '15, 04:38 billbofagends |
Does the bad-performing system use a (different) virus scanner or the same virus scanner with a different configuration? I have seen virus scanners introduce a lot of extra SMB calls. answered 03 Oct '13, 09:56  SYN-bit ♦♦ |
On the poorly performing server check if it is doing additional lookups to other servers in your environment, while your main transactions occurring. For instance it might be checking credentials with your domain controller or doing DNS lookups (that the other servers aren't). This might point to a configuration issues, inadequate authentication credentials or somee such that is putting additional overhead and latency on your transaction. The best may to measure network-oriented performance is looking at the SYN to SYN-ACK response time from the client. Normal this is done at the kernel level in the server and hence isn't very load dependent. And again measure close to the server to eliminate link latency. answered 03 Oct '13, 16:30  martyvis |
Good shout. We have Symantec installed on all our machines now and I have tried disabling it but it didn't help. After further poking around with it, however, you can config it to ignore certain folders etc which would be worth trying. I don't have privileges to do this by the look of it but I shall get it checked out. Cheers :-)