This is our old Q&A Site. Please post any new questions and answers at ask.wireshark.org.

I am a newbee using Wireshark, so hope for a little help :-) I have an issue with an SQL DB application which craches. I my tracefile I see very high Delta Time on SMB2 and a max SRT(s) of 625.347497 on Notify on SMB2 Service Response Time.

Where should I start to look? Is it the network, server or Client?

===========================================================================
SMB2 Service Response Time Statistics - sql-db-issue_00001_20170103094009_00006_20170103145622_00001_20170104090747:
Index  Procedure  Calls  Min SRT (s)  Max SRT (s)  Avg SRT (s)  Sum SRT (s)


SMB2
Close 6 361 0.000291 0.146918 0.001722 0.621558 Create 5 371 0.000331 1.787312 0.009587 3.556769 Find 14 271 0.000600 0.194669 0.005546 1.502843 Flush 7 1 0.165415 0.165415 0.165415 0.165415 GetInfo 16 58 0.000330 0.009854 0.002310 0.133973 Ioctl 11 51 0.000420 0.182926 0.010477 0.534342 Negotiate Protocol 0 3 0.000743 0.000900 0.000814 0.002442 Notify 15 18 0.000661 625.347497 71.007926 1278.142673 Read 8 601 0.000328 0.302919 0.028557 17.163042 Session Logoff 2 3 0.000797 0.004081 0.001950 0.005849 Session Setup 1 4 0.000784 0.007014 0.002552 0.010209 SetInfo 17 9 0.000598 0.000858 0.000745 0.006704 Tree Connect 3 5 0.000346 0.001530 0.000663 0.003313 Tree Disconnect 4 5 0.000816 0.009827 0.004345 0.021723 Write 9 13 0.000331 0.008499 0.002117 0.027518


13686 4.777025 10.34.80.11 10.34.80.165 SMB2 162 29.962738000 256 Notify Response

Thanks in advance.

asked 04 Jan '17, 23:55

bachsec's gravatar image

bachsec
6112
accept rate: 0%

edited 07 Jan '17, 11:04

grahamb's gravatar image

grahamb ♦
19.8k330206

Could you explain the topology of the system a little? Is it:

Client PC ---- Database Server ---- SMB File Server

or

Client PC / Access ---- SMB File Server

or

????

(07 Jan '17, 10:49) PaulOfford

Hi PaulOfford

It is Client(with developed frontend) ----> SMB Server with SQL Express as DB for the frontend Application

The application connects to the SQL express instance in the SMB server.

(08 Jan '17, 00:11) bachsec
1

There are two ways to connect to a Microsoft SQL database; named pipes or TDS over TCP. You could be using named pipes but Find and Notify are SMB2 commands that I wouldn't expect to see.

By the way, Notify response times might be long as Notify is used to notify a client if a change has been made to a file or directory. See https://wiki.wireshark.org/SMB2/Notify

Can you confirm that the client application connects to the database using named pipes? If you are not sure, check to see if there is any traffic in your trace between the client and port 1433 on your server with the filter expression:

ip.addr==the_client_ip_address and tcp.port=1433

TCP port 1433 is the default port number for Microsoft SQL Server. Although it can be set to a different value and so you might want to check with whoever supports the database.

Sorry to keep asking dumb questions but I don't want to send you off on a wild goose chase.

(08 Jan '17, 01:57) PaulOfford

Hi PaulOfford

In the tracefile I see at lot of traffic between the client and the serve on port 1433. 80% of the packages in the trace is that, when I filter with the filet you wrote.

(08 Jan '17, 12:04) bachsec

Does this graph say anything.

(08 Jan '17, 12:21) bachsec

no. sorry.

(08 Jan '17, 13:16) packethunter

It has answered the question I had. So, although your client application may use SMB file services, the connection to the database is via TDS/TCP.

You say in the title that the application crashes. Do you literally mean that? Does it throw an exception, dump and then exit? Or do you get an error message? If so, what's the message and have you determined what it means?

(08 Jan '17, 14:29) PaulOfford
showing 5 of 7 show 2 more comments

Hello bachsec, and welcome to ask.wireshark

Application errors like missing access rights or lock conflicts are easily spotted with this display filter: smb2.flags.response == 1 and not smb2.nt_status == 0

Looking at the Service Response Time Statistics can be helpful. Alas, in this case we only get marginal info.

The most remarkable part in the statistics from your trace file is one instance of a long time for the Create command. The Create function is called to obtain a file handle. This can be used to either create a file or to open an existing file. 1.7 seconds is considered "long" for most cases. However, a a virus scanner checking a 4 GB ISO image in 1.7 seconds would be impressive. So we have a number, but no unit to measure against. The delay could also be caused a TCP retransmissions or other network issues. The long response times for the notify request are usually not a problem, as this an asynchronous message from the server like "Hey client, a time stamp on a file has changed".

A trace file would help a lot. Depending on the configuration (DFS, Branch Cache etc.) a full network trace from the client view would be the best option. Certain problems might require two traces taken simultaneously with client and server.

Please bear in mind, that SMB traffic usually reveals the file contents and (depending on the configuration of workstation and server) could also reveal users passwords. To investigate that problem we need at least the SMB or SMB2 headers with the return codes. I am not aware of a tool that sanitizes SMB traffic as one might desire.

If you can not share the trace file with the general public a network consultant might be your best option.

Good hunting

permanent link

answered 05 Jan '17, 10:32

packethunter's gravatar image

packethunter
2.1k71548
accept rate: 8%

Hi Packethunter

With the filter you gave me, I can see something is that errors E.G. Error: STATUS_BUFFER_TOO_SMALL

(08 Jan '17, 00:14) bachsec

Hmmmm. "Buffer too small" is a regular message for certain protocols, that use named pipes. The client sends a request and a default buffer size, say 1kByte. If the server's response exceeds 1 kB the response will have the return code "Buffer too small". Another field will hold the required buffer size for the response.

Here are a few root causes for performance problems, that I have encountered during various analysis projects:

  • Missing index or too many indices (like, 300+ fields for a single row)
  • Disk errors or other hardware problems -> Check the System event log
  • A virus scanner interfering with the application -> Try to put the DB file on an exclusion list. Remove the DB-file(s) from the list if the virus scanner is not involved
  • Bad application design no 1: Too many greedy locks (like exclusive write lock, if a shared read would do)
  • Bad application design no 2: Keep on reading the same data within milliseconds (in the magnitude of 5'000 unnecessary reads per transaction)
  • Bad application design no 3: Using a file share, when a database would be preferable
  • Bad system design no 1: Use RAID-5 storage for files or databases that are written frequently.
  • Bad system design no 2: Too little memory, CPU too weak, (time-) critical data on a USB-2 attached device, other hardware bottleneck
  • Poor system maintenance: Backup application has consumed 12 days of CPU time (guess, when the problem started; then guess who got blamed)
  • Wrong video adapter for the client (on-board Graphics adapter work no so well for CAD applications)
  • Poor network design (round trip times exceeding 10 msec when 100 microseconds are easy to achieve without investment)
  • Half-duplex / full-duplex mismatch (manifests itself in serious retransmissions, easy to spot with wireshark)

After a day at the Sharkfest conference you will certainly have a list that is five times this long. The majority of these issues is best found with diagnostic tools on the server, like the Performance Monitor. Wireshark will help you in many situations to identify the culprit (server, client, network ...). Getting to the root cause requires other tools, once you have arrived at the server.

I am not sure, if I can help you further without a trace file.

Good hunting

(08 Jan '17, 09:16) packethunter
Your answer
toggle preview

Follow this question

By Email:

Once you sign in you will be able to subscribe for any updates here

By RSS:

Answers

Answers and Comments

Markdown Basics

  • *italic* or _italic_
  • **bold** or __bold__
  • link:[text](http://url.com/ "title")
  • image?![alt text](/path/img.jpg "title")
  • numbered list: 1. Foo 2. Bar
  • to add a line break simply add two spaces to where you would like the new line to be.
  • basic HTML tags are also supported

Question tags:

×27
×10

question asked: 04 Jan '17, 23:55

question was seen: 3,713 times

last updated: 08 Jan '17, 15:01

p​o​w​e​r​e​d by O​S​Q​A