This is our old Q&A Site. Please post any new questions and answers at ask.wireshark.org.

I have developed an application which reads data over a high volume network (2 sensors, both 1khz).

The application reads the packets for a random time, and then crashes due to - what I believe to be - a native buffer overflow.

The following image depicts the exacts packet sequence each time the crash occurs - a large Dup ACK block followed by 2 RST flags.

Am I correct in thinking that this crash occurs due to the RWIN buffer being too small, and this in turn leads to the sending sockets shutting down. Any advice or suggestions is most appreiciated.

Edit - Updated with image of sequence pre DUP-ACK

alt text

alt text

asked 04 Sep '13, 04:40

Jnanathan's gravatar image

Jnanathan
6114
accept rate: 0%

edited 04 Sep '13, 05:20


I have developed an application which reads data over a high volume network (2 sensors, both 1khz).

Is that just an application on top of a standard OS or are those 'embedded devices' with a their own TCP/IP stack?

That many DUP-ACK looks more than strange. I don't think that any recent OS (Windows, Linux) would send that many DUP-ACK that fast. So, I guess these are either duplicate packets (possibly a routing loop) or another problem in the TCP/IP stack.

Can you please add the following information:

  • Check the IP ID of the packets marked with DUP-ACK. Is the ID IP increasing/changing or is it the same in every packet?
  • What OS are you using
  • Where did you capture the traffic

BTW: is it possible to post the capture file instead of a screenshot? You can post the capture file on google docs, dropbox or cloudshark.

UPDATE

To me it looks like the Win7 system (your application) is not able to process the data (fast enough). In both TCP streams, the TCP window size reported by Win7 drops constantly until it reaches 0 (see tcp.stream eq 1 - it shows it pretty nicely).

IO-Graph

The embedded system keeps sending ACKs and the Win7 finally (after a long time) closes the connection with a RESET.

So, the key to your problem is: Why does the Win7 system lower its TCP window size, i.e. why is it incapable of handling the data (fast enough).

As that is your own application, the only way to understand that behavior is by debugging the application. Maybe the application does not receive the data it is expecting and your program logic waits for the correct data, until the TCP receive buffers are full.

Regards
Kurt

permanent link

answered 04 Sep '13, 06:14

Kurt%20Knochner's gravatar image

Kurt Knochner ♦
24.8k1039237
accept rate: 15%

edited 06 Sep '13, 02:49

Hi Kurt, to answer your questions:

  1. Yes the data is coming from embedded devices which have implemented their own TCP IP stack
  2. The IP.ID does increase with each DUP ACK in the sequence
  3. The operating system that the program is running is Win 7
  4. I have uploaded the capture file to mediafire, as it was too large for cloudshark. The link is found below
(05 Sep '13, 08:57) Jnanathan

I think the link got lost ;-)

(05 Sep '13, 09:18) Jasper ♦♦

Cheers Jasper, it can be found below. I had to break the link into seperate parts to prevent it from being taken as spam.

https://mega.co.nz/#!8M90VKaQ!RnvEnmIMWhm312QLrRy3R0CM6SsLaORN9x6K8rBLLb0

(06 Sep '13, 00:46) Jnanathan

Yes the data is coming from embedded devices which have implemented their own TCP IP stack The operating system that the program is running is Win 7

O.K. please tell us the IP address of the Win7 system and the address of the embedded device.

BTW: I 'fixed' the link in your comment.

(06 Sep '13, 00:52) Kurt Knochner ♦

Thanks Kurt. The address of the Win7 system is 109.168.0.2, and those of the two sensors are .106 & .108.

(06 Sep '13, 01:15) Jnanathan

see the UPDATE in my answer.

(06 Sep '13, 02:22) Kurt Knochner ♦

Cheers for the break down Kurt. I believe this to be an issue with the hardware that the application is running on. The sensors are sending at a rate of 1khz each, and the host machine gets overloaded after a period of time - In that the CPU cannot process packets from the receive buffer, and this in turn causes the backlog. Thank you for your assistance.

(06 Sep '13, 05:30) Jnanathan

Thank you for your assistance.

You're welcome.

Hint: If a supplied answer resolves your question can you please "accept" it by clicking the checkmark icon next to it. This highlights good answers for the benefit of subsequent users with the same or similar questions.

(06 Sep '13, 05:46) Kurt Knochner ♦
showing 5 of 8 show 3 more comments

I see only one RST; the others belong to a different connection (with the IP .108 instead of .106). But from that long long long list of Duplicate ACKs I can tell that something is going very wrong, and I guess your application has trouble with packet loss and recovery. There is no way to tell what stopped the sending process because it is not in the screenshot - the screenshot only tells the end of the story. It would be helpful to see how the Duplicate ACK series got started.

permanent link

answered 04 Sep '13, 04:57

Jasper's gravatar image

Jasper ♦♦
23.8k551284
accept rate: 18%

Hi Jasper.

Fist of all, I should have noted the 2 IPs of both sensors - being .106 & .108

Secondly, I have re-edited the original with an image of the sequence before the DUP-ACK segment. As you will see there is a "TCP Window full" alert. Any idea what is causing this - might it be the buffer window on the receiving end?

(04 Sep '13, 05:10) Jnanathan

Window full means that the sender has sent as many bytes as the receiver can buffer, so if you see that message it means that the sender has to pause and wait for the next acknowledge with a new window size to come in. In your case I'm pretty sure that this situation is the problem, because it doesn't seem to recover from that at all. I would expect a "Window Update" message from .2 to .108, but apparently it never arrives, so .108 cannot continue. The stack of .108 is also not behaving like most stacks, because it should do Window Probing instead of ACKing the last segment.

(05 Sep '13, 09:27) Jasper ♦♦
Your answer
toggle preview

Follow this question

By Email:

Once you sign in you will be able to subscribe for any updates here

By RSS:

Answers

Answers and Comments

Markdown Basics

  • *italic* or _italic_
  • **bold** or __bold__
  • link:[text](http://url.com/ "title")
  • image?![alt text](/path/img.jpg "title")
  • numbered list: 1. Foo 2. Bar
  • to add a line break simply add two spaces to where you would like the new line to be.
  • basic HTML tags are also supported

Question tags:

×81
×42

question asked: 04 Sep '13, 04:40

question was seen: 5,743 times

last updated: 06 Sep '13, 05:46

p​o​w​e​r​e​d by O​S​Q​A