According to the documentation, there should be a 'Packet Pane' that assembles all the packet data into a single hex dump https://www.wireshark.org/docs/wsug_html_chunked/ChAdvReassemblySection.html However, whenever I try to capture the data I'm sending, basically 52 ascii characters (repeated every second), all I see is [TCP Retransmission] and [TCP segment of a reassembled PDU] but neither show the raw hex data. I was hoping to see Wireshark assemble the packets and display each 52 ascii character burst. http://imgur.com/jOfDowV My setup is similar to the "Switch + Tap" setup described in the documentation, https://wiki.wireshark.org/CaptureSetup/Ethernet Except, that my device who's traffic I'm capturing is connected to this tap, https://greatscottgadgets.com/throwingstar/ where one port goes to the router, and the other port goes to the pc running wireshark. I understand the last port should also connect to my pc/wireshark, except at the moment, I'm not interested in the data going to my device, I just want the data my device sends. Also, the complexity of using a 2nd nic for that port and having wireshark merge those 2 streams seems like it would be very difficult for a newcomer like myself. How can I configure Wireshark to reassemble the data it sees coming from the device? asked 21 May '16, 21:47  benn n |
2 Answers:
The first set of TCP packets from the Arduino does get reassembled; look at frame 492. All the segments before it are marked as "TCP segment of a reassembled PDU" because they're TCP segments that are part of a reassembled HTTP request. There are a lot of them because your Arduino is, as @sindy says, sending really tiny TCP segments. After that, there are two TCP segments that are somehow not getting handled correctly by Wireshark - the TCP dissector isn't showing the CR in the first segment or the LF in the second segment and isn't handing it to a dissector, either. Then there's a ACK and a FIN+ACK; presumably both sides closed the connection. Then Weird Stuff Happens, as @sindy indicates. As Wireshark isn't seeing both sides of the conversation, it might not realize that the connection is closed, and thinks there are retransmissions. I'm not seeing any 52-byte packets there; I'm just seeing the two HTTP connections, a bunch of broadcast/multicast traffic, and a few other packets. Maybe your 52-byte packets aren't even getting sent; what protocol are they using, and, if it's something like TCP or UDP, what port are they going to? Or are they being sent as HTTP packets, in a way that expands them to a lot more than 52 characters? Are you sending them by directly writing to a TCP connection, or are you passing 52 bytes to some "send this as plot data over HTTP" library? I'm not seeing any actual data points in the HTTP POST request, however. None of the reassembly issue has anything whatsoever with the low-level details of how it's being captured - not with the tap, not with wired vs. wireless, none of that. Changing the tap would make no difference; doing a wireless capture would make no difference. As for the retransmissions, there's nothing that can be done to determine what's going on there without capturing the traffic going to your device, as well as the traffic coming from your device, in a single capture. It might be that retransmissions are really happening, or it might be that Wireshark's analysis of the TCP traffic is confused by the absence of the traffic going to your device. If I take the capture and throw away all packets before the last SYN, the retransmission errors go away, and what I see is another HTTP POST request; this one uses chunked encoding - but it's incomplete. Is it just sending an endless stream of points? If I tell Wireshark to reassemble HTTP headers but not HTTP bodies, it still looks as if it's reassembling the body of the request, which is arguably a bug, but I'd need to look at that some more. If it's an endless stream of points, Wireshark won't be able to reassemble it, as there's no end. At best, it'd be able to show you each chunk. answered 22 May '16, 17:15  Guy Harris ♦♦ edited 23 May '16, 17:02 Not sure if there's underlying errors in the transmission, or if there's re-transmissions, but the data does seem to get accepted since it plots correctly... I can disable http processing since this is not a standard port 80 usage. I have a usb NIC that I can add to the laptop, but not sure how easy it is to bond or trunk the 2 so that Wireshark captures merged data. Maybe the better way to do this, will be to use an active tap (ex.dual-comm DCSW-1005)? (23 May '16, 23:00) benn n
The correct answer is "there's an underlying error somewhere in Wireshark". It's properly reporting frame 521 as a retransmission of frame 514, as they're on the same connection and have the same sequence number,, but it shouldn't be reporting frame 522 as a retransmission, as there's no other TCP transmission from 192.168.1.148 port 1026 to 52.1.214.92 port 80, on any connection, with that sequence number.
There's nothing non-standard about it; the second connection is unusual, in that it just keeps sending points to the plot.ly server indefinitely, and presumably expects the server to render the points as they arrive rather than waiting for the entire body to be sent, but that's not non-standard. Given that this is HTTP, disabling HTTP processing would mean that Wireshark would do a worse job of displaying the traffic. The only real problem here is that, if I strip out everything before packet 513, so that the first connection, and the initial SYN and client-side RST, are removed, so that Wireshark doesn't get confused and think it has a ton of retransmitted TCP segments, and can properly process the HTTP request in the second connection, it treats all the TCP segments containing the (un-ending) body of the request as data to be reassembled, even though I've turned off all of HTTP's reassembly preferences other than "reassemble the HTTP headers". (24 May '16, 00:55) Guy Harris ♦♦ With both sides of the communication captured and in-sync, https://www.cloudshark.org/captures/a08f1590cc7c how can I format the display to show the few lines of ascii text that is being sent and received? For example, I'm trying to find out what is being sent immediately after the connection closes.. since the streaming data does get plotted, I imagine the connection must re-open, but I can't find it within all the packet fragments! (16 Jul '16, 19:55) benn n |
Oh man... the code sending the data does many things wrong. In fact, re-assembly in Wireshark does work, and as you actually do use HTTP, there is no need that you'd write your own dissector. Take the capture you've posted, and look at frame 492. You'll see that the Protocol column of the packet list doesn't say just TCP like for all the previous tcp packets but HTTP, and the INFO column shows the So the first trouble is that you're effectively using TCP almost like a serial line. Wireshark shows the PSH bit to be set in all data packets, which, unless there is something wrong with the TCP stack you use, means that you write the bytes to the socket with a PUSH flag set. Doing so causes the tcp stack to send the data collected so far packet immediately, rather than to wait for the whole message to be put together and send it in a single packet. There are exceptions to this rule, like frame 661 where the payload has 16 bytes rather than a single one like in the other cases. The second trouble is the missing reverse channel as @Guy Harris has stated in his Answer. The point is that unless the Wireshark dissector can see both directions of the session termination, it doesn't recognize the session to be closed and new one to be open. Look at frames 510, 511, 512. Arduino sends a SYN packet from port 1026, followed by a RST packet, and then another SYN packet from port 1026. This indicates that it attempted to open a new tcp session, then changed its mind and did an emergency closure (RST), and immediately after it has sent a new SYN from the same local port, attempting to open a new session. Without the RST, this would definitely be illegal as some timeouts have to be followed before reusing the same local socket to open a new session towards the same server; with the RST, I am not sure. However, from that point on, Wireshark considers all packets coming from 1026 to be retransmissions, as it considers the session to have been emergency-closed in the meantime, and doesn't bother to dissect them deeper. Look into the TCP RFC whether it is legal to reuse the local port so quickly after sending a RST packet from it; if yes, the receiving server may have dealt with it properly and only Wireshark may be confused as the opposite direction is missing. answered 23 May '16, 03:33  sindy Also, it is not a good idea to close a TCP session right after sending the first packet through it if you know that during a couple of seconds you'll need to send another data to the same server. It is better to keep it open for future use and only close it if you know in advance that you won't send anything else soon, or if the remote side (the server in your case) closes it on its own initiative. If you don't know how long it is going to take before you'll need to send another piece of data (not your case currently), it is wise to start a timer each time you send a piece of data, and close the session upon that timer's expiration. (23 May '16, 03:38) sindy I can't say for sure about how the TCP sessions are controlled, because the project example I'm following abstracts all the low-level information out : From what I see in the Arduino code however, it does check to see if the connection is still alive before sending the data: http://pastebin.com/4bmTuLsa It sounds like the only way to reconcile the data I think I'm sending, with the actual network traffic is to re-capture , but in both directions. Then will Wireshark's Re-Assembly of port 80 be something readable? (23 May '16, 22:56) benn n
Probably not. It's impossible for Wireshark to reassemble the body of the second POST request, as you don't have all the packets including the Arduino saying "I have no more points to plot" and closing the connection, and, for some reason, telling Wireshark to reassemble the HTTP headers but not the body, so that you'd see each individual segment of the body as HTTP data, doesn't seem to be working. (24 May '16, 00:58) Guy Harris ♦♦ Depending on what is your actual goal, it might help you reach it to export the packets belonging to that second tcp session (i.e. everything from the last SYN packet, 512, in the original capture) into a new file, using As @Guy Harris wrote, the reassembly works on PDUs (in this case, HTTP ones) which are split among several adjacent packets and complete (finished). The plotly thing seems to never stop sending, so the PDU is never complete (at least until you stop it manually if it is possible at all). So on one hand, it is slightly saving bandwidth by sending all the values as a body of a single common HTTP POST, on the other hand, it is terribly wasting it by adding 59 bytes of overhead to almost every data byte it sends. (Sending each {x,y,streamtoken} tuple using a dedicated POST could still be done inside a single TCP session). (24 May '16, 02:49) sindy It's been a while, but I got it to work! Actually, unlike before, now I captured both sides of the network tap. https://www.cloudshark.org/captures/a08f1590cc7c So now in theory all the data that is sent/received is there and synchronized correctly, but Wireshark still doesn't show it as nice re-assembled packet! For example, I know for sure that plotly sends "All Streams Go" yet I can't find it in the capture. Since there's not much data being sent, is there a way to reconstruct the packets to view all data at once? (16 Jul '16, 19:50) benn n What you can do is to use In your case, there is stream 0, which is the initial POST which plot.ly has responded with a 200 OK, and stream 1 which contains only the request. There is nothing in the request body itself which could tell the Wireshark dissector that it is complete. Plus the last packet of that request is the last packet in the capture, so the http dissector in Wireshark cannot use a recognized beginning of another request to conclude that the previous one has been completed. So it is waiting for an event which never comes, and so it marks all packets as "TCP segment of a re-assembled PDU". But I suspect there is no closing event expected to ever come, i.e. that it is the principle of the service that you feed it with data continuously, within a single HTTP POST's payload, until you manually stop the sending application. So I've written before that I'd like to see the reverse direction in the capture in order to see whether Wireshark wouldn't eventually recognize the end of a PDU by seeing a proper closure of the TCP session carrying it; in this capture, you've provided both directions but the capture is cut while the TCP session is still in progress, so the idea got ruined. Worse than that, you seem to have cut the capture in the middle of transmission of a point - the last packet carries the "35" string expressing the length of the (17 Jul '16, 00:41) sindy showing 5 of 6 show 1 more comments |
So did you write a dissector for this "52 ASCII characters per message" protocol?
Can you share the capture rather than the screenshot? The screenshot suggests that
you have completely suppressed the display of the packet bytes pane in Wireshark settings (make sure that there is a checkmark next to
View -> Packet Bytes)either there are real retransmissions, or you've somehow somehow managed to interleave two sessions with identical source and destination socket tuples, which causes not only Wireshark but also the receiving application understand the incoming packets as retransmissions.
Re-assembly means that if a higher layer (application) PDU is so large that more a single transport layer packet is not sufficient to transfer it, the last packet carrying that PDU contains the dissection of it in the dissection pane and a hex dump of the complete PDU as a separate tab in the packet bytes pane; dissection of all the previous transport packets shows the part of PDU as "data", and the packet bytes pane has no tabs in it.
If the whole PDU of the application protocol fits into a single transport packet (or even if more PDUs fit), you do not need any re-assembly to have them dissected and displayed. However, in case of an unsupported application protocol, you won't get a proper (e.g. ASCII) representation of the PDU unless you create a dissector for your protocol, which is a piece of code which instructs Wireshark what to do with the data of your protocol.
Thanks, confirmed the View->Packet Bytes is checked. The re-transmissions does seem strange, especially since everything works correctly (i.e. the device, an arduino, is sending the data to the plot.ly server and it charts correctly).
Because it sends the plotting data as ascii character strings, I didn't setup any kind of dissector/interpreter for the data.. I was hoping that as long as I can see the full data sent at every burst (52 characters every second), that would be enough to see how I can port this to a non-arduino device. For some reason, when I try it on an MSP430, it doesn't work.
I don't see where to attach a capture file here, so I placed it online: https://www.cloudshark.org/captures/9563f9aef707
Not sure if it would help, but I was thinking of removing the Throwing Star, and trying to monitor this wirelessly. I initially wanted to avoid this approach since I'm using Win7/8 and apparently Wireshark/WinPCap doesn't support monitor mode... also there's tons of non-related wifi traffic so filtering becomes key. https://wiki.wireshark.org/CaptureSetup/WLAN
I did look into Riverbed's AIRPCAP but it's cost prohibitive.. even though I have wifi hardware that would seem to support monitor mode : Netgear A6200 / ZyXEL AC240 / TP Link TL-WN722N https://www.acrylicwifi.com/en/blog/how-to-capture-wifi-traffic-using-wireshark-on-windows/ http://www.tamos.com/products/commwifi/adapterlist.php