I am troubleshooting a conversation between 2 servers. There are 2 processes, one process goes through clean each time, the other conversation keeps showing me TCP ACKed unseen within the conversation. I start the capture long before either transmission begins, so we are not missing any packets. I have confirmed there are no ports being blocked. The first transmission does the 3-way handshake, then data begins transmission, ACKs are sent and session closes. The second transmission I get 3-way handshake, data begins transmission and ACKS are sent. However, every now and then, wireshark will give me a "TCP ACKed unseen". Eventually the session closes normally. At the end however, the application sending the data isn't marking the transaction as complete. I understand "ACKed unseen" is telling me wireshark is seeing an ACK for a transmission that it cannot find in the capture. I can "Follow TCP Stream" and see the data just fine. Searching the internet about "TCP ACKed unseen" shows the error usually gets chalked up to either a bug or a wireshark problem, but in these cases, I think they are showing correctly. The only thing is I have done several of these kinds of captures to troubleshoot this type of transmission a lot. One thing that is fairly consistent is when the apps talk properly, I see no "ACKed unseen" and when the apps are not set properly, I see these sprinkled within its conversation. My assumption is an upper layer problem gets manifested into a lower level confusion for wireshark. Then picture this... I am on a conference call with app people, firewall people and network people (me). Once I say, "Wireshark shows an ACK on an unseen packet, but I see no missing packets", everyone shuts down and stops looking at their end because they believe it proves a network problem and I end up searching for phantom networking issues. This server and many more handle a lot (thousands) of transactions daily, the ones that work always work, but now and then new setups don't and I am brought in because everyone blames the network. We go round and round wasting a lot of time searching for network issues until someone who sets up this process fixes something. Then behold the traffic goes clean! I get the feeling that wireshark is interpreting upper layer problems as lower level missing packets sometimes. I need to improve my understanding of why is possible for wireshark to believe its missing a packet, when perhaps there is upper layer trouble. If I can better explain this error, maybe I can speed up the process of what to communicate. asked 11 Jul '14, 09:06  njcwotx |
This is a static archive of our old Q&A Site.
Please post any new questions and answers at ask.wireshark.org.
after a deeper look, I discovered that wireshark is showing packets out of order.
The ACKs are showing up nanoseconds before the datapacket...Looking at text below the ACK number in the 'ACKed unseen' packet matches the next packet or the one just after. How can this be?
:Start 3-way handshake |2014-07-11 09:55:59.262612000| 56231 > 6651 [SYN] |TCP: 56231 > 6651 [SYN] Seq=0 Win=8192 Len=0 MSS=1380 WS=256 SACK_PERM=1 | |(56231) ------------------> (6651) | |2014-07-11 09:55:59.262978000| 6651 > 56231 [SYN, |TCP: 6651 > 56231 [SYN, ACK] Seq=0 Ack=1 Win=8192 Len=0 MSS=1460 WS=256 SACK_PERM=1 | |(56231) <------------------ (6651) | |2014-07-11 09:55:59.305316000| 56231 > 6651 [ACK] |TCP: 56231 > 6651 [ACK] Seq=1 Ack=1 Win=66048 Len=0 | |(56231) ------------------> (6651) | :End 3-way handshake|2014-07-11 09:55:59.308956000| 56231 > 6651 [ACK] |TCP: 56231 > 6651 [ACK] Seq=1 Ack=1 Win=66048 Len=1380 | |(56231) ------------------> (6651) | |2014-07-11 09:55:59.309459000| [TCP ACKed unseen s |TCP: [TCP ACKed unseen segment] 6651 > 56231 [ACK] Seq=1 Ack=2761 Win=66048 Len=0 | |(56231) <------------------ (6651) | |2014-07-11 09:55:59.309469000| 56231 > 6651 [ACK] |TCP: 56231 > 6651 [ACK] Seq=1381 Ack=1 Win=66048 Len=1380 | |(56231) ------------------> (6651) | |2014-07-11 09:55:59.353147000| 56231 > 6651 [ACK] |TCP: 56231 > 6651 [ACK] Seq=2761 Ack=1 Win=66048 Len=1380 | |(56231) ------------------> (6651) | |2014-07-11 09:55:59.353706000| [TCP ACKed unseen s |TCP: [TCP ACKed unseen segment] 6651 > 56231 [ACK] Seq=1 Ack=5521 Win=66048 Len=0 | |(56231) <------------------ (6651) | |2014-07-11 09:55:59.353714000| 56231 > 6651 [ACK] |TCP: 56231 > 6651 [ACK] Seq=4141 Ack=1 Win=66048 Len=1380 | |(56231) ------------------> (6651) | |2014-07-11 09:55:59.353882000| 56231 > 6651 [ACK] |TCP: 56231 > 6651 [ACK] Seq=5521 Ack=1 Win=66048 Len=1380 | |(56231) ------------------> (6651) |I have come to the idea that when you consider the port being captured is very busy and the transactions I am looking at are of remarkably different sizes, tshark ended up saving the packets ever so slightly out of order in a few places on the capture. All of the packets are there. So I am chalking this issue up to being out of order during the capture process. With the packets coming in so fast and tshark having to process each one, it has to save them quickly and its possible it sometimes has to save them out of order to keep up.
Can you provide a trace file of this? Reading ASCII dumps is no fun at all. An upload to www.cloudshark.org would be nice. If you have to sanitize your trace first use TraceWrangler (www.tracewrangler.com). You can randomize all IPs and have TraceWrangler remove all payload by telling it to cut after layer 4, because for this issue we only need to see the TCP layer.
Generally the TCP expert of Wireshark is usually correct, except in some cases were it marks an out-of-order as retransmission (which is going to be fixed in 1.12)