Packet with HTTP 302 Redirect decoded as TCP not HTTP

Hi Kurt - sorry to disturb: have you had a chance to look into the root cause why it's not being decoded with that option enabled? If so any clues? Just curious now.

Many thanks,

D.

sorry, not time atm. Maybe next week.

I think the reassembly code is not working properly; it should not consider the TCP segment in frame 10 as part of a larger payload, so it should be displayed as HTTP decoded content. Somehow the HTTP dissector seemt to tell the TCP dissector that there be should more content to reassemble while there isn't. Looks like a case for http://bugs.wireshark.org to me ;-)

I believe there is a 0x0a and/or 0x0d missing at the end of the data, but I did not really check, as I mentioned: no time atm.

Hi Jasper, Hi Kurt,

Thanks for looking into it. I've just uploaded the screenshots with packets 10 and 162 in the same trace resp for comparison (both carry HTTP 302 in a single packet):

https://drive.google.com/folderview?id=0B31e47Ucqt4BSEE5ci1HdlByQ00&usp=sharing

Packet 162 is decoded correctly no matter if that option is ON or OFF; packet 10 however is only decoded when that option is OFF. Perhaps it'll help one of you or someone else notice the critical difference between them as the ones I see look fine; \r\n is there in both of them. I'll file a bug on it then in the meantime - hope that's OK.

Thanks & Regards,

D.

Packet 162 has the PSH flag set, which signals "this is the last packet of the payload".

Oops, hence packet #10 encoding issue then? Well spotted )

I just filed it before I saw your comment:

https://bugs.wireshark.org/bugzilla/show_bug.cgi?id=9848

No problem, Wireshark should still decode packet #10 correctly even though there is no PSH flag.

Ah so PSH isn't mandatory then... Please excuse my ignorance of such TCP basics but what's the right criteria to use when deciding whether to look for any more packets when reassembling? (either before or after the one in question)

Packet reassembly is dictated by the protocol on top of TCP. So the HTTP dissector has to tell the TCP dissector "there will be xyz payload bytes", and then the TCP dissector gathers them together (as far as my understanding goes).

In case of packet #10 there is no reassembly required, but somehow the TCP dissector thinks there is. A push flag is a typical flag found in the last packet of a larger payload, signaling "process this right away". So there can't be more payload bytes after that and the TCP dissector decodes it.

Thank you - clear now. Just tried setting PSH - no difference, so PSH doesn't seem to be the criteria being used...

As http is "line-based" I think the http dissector is looking for a blank line with \r\n to determine the end of the http PDU. The packet in frame 10 doesn't have such a terminator so the http dissector is waiting for more data to find the line terminator. As the tcp stream is subsequently closed it never sees that final terminator so can't reassemble the PDU's.

In fact, I think it won't be called again for that stream as there is no further data so it never gets a chance to complete the reassembly. For whatever reason the capture doesn't show the FIN from the server to close the stream off, I suspect the \r\n and the FIN have been lost somewhere.

Update As @Jasper pointed out the FIN is in there with the data response.

The FIN is piggy-packed in frame #10 ;-)

If I disable Reassembly frame #10 shows this:

Hypertext Transfer Protocol
    HTTP/1.1 302 Found\r\n
    Location: https://hotspot.inmarsat.com/index?origUrl=http%3A%2F%2Fwww.bbc.co.uk%2F\r\n
    Content-Type: text/html; charset=iso-8859-1\r\n
    Proxy-Connection: close\r\n
    \r\n
    [HTTP response 1/1]
    [Time since request: 0.803409000 seconds]
    [Request in frame: 7]

Looks like there are two \r\n with the second on a blank line.

Oh yep, didn't look at the actual flags breakdown, I don't often see a data bearing packet with a FIN in my usual traffic.

Possibly a non-conforming server then forgetting to add the line terminator, although I can't be bothered to go and check the http RFC's to confirm that.

Same here, no fun looking thru RFCs for MUST, SHOULD, etc,. Let's have @Kurt do this, he's our resident PEP ("pinpoint the exact problem") specialist :-)))

Let's have @Kurt do this,

??? :-))

• packet #162 has \r\n right after Location
• packet #10 has \r\n further down, not right after Location

Could that be why? Will try to move Location closer to \r\n now )

That's the separator between the header and the body. Look at the values in the hex bytes pane, no line terminator.

It appears that disabling either HTTP header or body reassembly allows it to display, but that probably just makes the HTTP dissector work with whatever's in the segment and if it reaches the end of the tvb it says job done.

@Kurt: your answers are almost always going to extremes with lots of details and stuff... I'm too lazy for that kind of thing :-)))

@Jasper, you're busy. Not lazy :-)

@grahamb, true, working on TraceWrangler every free minute... Silly me offered a Sharkfest talk about new features, so I only have 3 month left to put cool stuff in to have something to show for... yikes :-)

@dmitriy: try and let us know if it changes anything. If it does, add the info to the bug as well please.

@Jasper:

Just tried moving Location to the position just above \r\n - didn't help:

Content-Type: text/html; charset=iso-8859-1\r\n

Proxy-Connection: close\r\n

Location: https://hotspot.inmarsat.com/index?origUrl=http%3A%2F%2Fwww.bbc.co.uk%2F\r\n

\r\n

@Dmitriy,

Edit:

OK, so I read the RFC. The body isn't terminated with a blank line, either the body length should be indicated in some way e.g. Content-Length header, or by the server closing the connection.

In this case the server closes the connection, but I think that currently in Wireshark there are no means for the tcp dissector to indicate to the http dissector that there is no more data to come to allow the reassembly to terminate.

@Graham:

Ah that one... just tried moving \r\n just before the body to the end of it to keep the packet length same: no luck I'm afraid...

Location: https://hotspot.inmarsat.com/index?origUrl=http%3A%2F%2Fwww.bbc.co.uk%2F\r\n

Content-Type: text/html; charset=iso-8859-1\r\n

Proxy-Connection: close\r\n

The document has moved\r\n

@Dmitriy, I've edited my comment after reading the RFC.

@Graham:

So if we remove the body completely thus making packet #10 similar to packet #162 it should work? Also what if the appropriate parsing logic is enhanced with checking for the FIN flag?

@Dmitriy

Yep, removing the body should work.

The latter case is more difficult, as the http dissector doesn't know, or need to know about the FIN flag. A normal application, such as a browser will see the stream being closed and then use whatever body has been received, Wireshark isn't really like that as it doesn't "understand" the protocols, just how they should be formatted on their layer.

@Graham:

Removing the body alone isn't good enough: looks like Wireshark doesn't like something about Content-Type field (it parses the packet as HTTP after I removed that field completely)

In this case the server closes the connection, but I think that currently in Wireshark there are no means for the tcp dissector to indicate to the http dissector that there is no more data to come to allow the reassembly to terminate.

Based on this comment from earlier I thought it's TCP dissector that signals "that's it" to the http dissector? Is it the other way round then?

Update: can further confirm that removing the body isn't necessary: removing Content-Type field is enough (after which the body is not decoded as ASCII text of course).

Each dissector for a layer, e.g. tcp, calls dissectors for layers above it when it has data for them. It doesn't call them when it has no data, so the http dissector isn't called again for the tcp stream 0 after the data in packet 10. There would need to be a mechanism on that last call in packet 10 to say this is it, no more. I don't think that exists at the moment.

I thought tcp dissector could do that (i.e. signal to http dissector "no more data for you" by looking at the FIN flag in this tcp stream... looks like it's impossible/complicated.

Meanwhile as per my update above removing Content-Type field alone is sufficient. Not sure why though: perhaps because it just makes the body invisible w/o actually removing it?

Removing the body means the http dissector isn't waiting for more data.

going to extremes with lots of details and stuff...

@Jasper: extremes? Well, I guess that depends on the perspective ;-) what you see here is my regular mode of operation. It would look differently if I were to go into my own definition of 'extremes' ;-)) I like 'details and stuff' ...

@Kurt: it's extreme compared to my mode of operation. But I'm lazy ;-)

@Graham:

Removing the body means the http dissector isn't waiting for more data.

Interesting why the same doesn't apply to the header as well then: can, say, a URL in Location field not be as long as to exceed one packet? (at least when taken together with the rest of the packet contents)

Anyway... would be great to see it fixed one day as this TCP segment reassembly option is apparently ON by default for a reason. The fix would save others time we lost trying to understand what's going on since our browser was successfully executing that redirection despite receiving no HTTP 302 packets whatsoever as it seemed initially )

Hi All,

The original problem reported in this thread is now resolved using FIN flag: https://bugs.wireshark.org/bugzilla/show_bug.cgi?id=9848

I've just come across the same symptom in a different scenario where neither of the 2 successive TCP packets making up an HTTP POST request carry FIN flag - they both carry only PSH and ACK flags (packets ##4-5): https://drive.google.com/file/d/0B31e47Ucqt4BdmttaGRscjE3QkU/view?usp=sharing

HTTP 200 OK response (packets ##9-10) is likewise affected.

Wondering if anything can be done about these too by chance? Disabling TCP Reassembly every time is a bit of a nuisance.

Many thanks in anticipation.

The symptom I see with that comment is that frame 4 is marked as TCP ("TCP segment of a reassembled PDU") and frame 5 is marked as HTTP (because it's the second of the two segments containing the HTTP POST request in question).

Disabling TCP reassembly causes frame 4 to be marked as HTTP but not to show the full POST request and frame 5 to be marked as TCP, because it's not the first part of an HTTP request.

The underlying problem is that, when reassembly is enabled, all but the final segment of a message that spans multiple segments aren't marked with the protocol being reassembled. Fixing that might be possible, but it'd require more work. (I.e., mark frame 4 as HTTP, with the Info column saying it's part of a reassembled PDU, and frame 5 as HTTP, with the Info column saying it's a POST request.)

Thank you: makes sense. Do I file a new enhancement request or reopening 9848 makes more sense assuming it's possible?

This isn't really the same issue as 9848, this is a separate one. The current situation has been around for a long time and hasn't been discussed much if at all, but don't let that stop you raising an enhancement request.

Done: https://bugs.wireshark.org/bugzilla/show_bug.cgi?id=12411