TCP Retransmissions, reassembled PDU, TCP Out-of-order, issue with slow connectivity

Question

Hello everyone, I am troubleshooting an issue with slow and sometimes no connectivity, when a end user tries to open a file (PDF) on a file server over the internet. The users has internet connection, the problem only happens when accessing a specific site to view files. I've confirmed the issue is not on the end users workstation as the issue happens from my PC as well.

Network is as shown : User-> switch -> (in)firewall(out) -> switch -> gateway router

Here are some of the packets: The packets shown are from a trace on the switchport connected to the outside port of the firewall.

No. 1093 | time (Since prev pkt) 0.010808000 | src addr 23.200.30.6 | desti addr 10.10.1.45 | Strm indx 4 | src prt 443 | protocol TCP | info **[TCP Spurious Retransmission] [TCP segment of a reassembled PDU]**
No. 1838 | time (Since prev pkt) 0.011681000 | src addr 23.200.30.6 | desti addr 10.10.1.45 | Strm indx 4 |src prt 443 | protocol TCP | info : [TCP Spurious Retransmission] 443>56886 [ACK] Seq=990774 Ack=1358 Win=31872 Len=1380[reassembly error, protocol TCP: New Fragment overpaps old data (retransmission?)]
No. 1782 | time (Since prev pkt) 0.038045000 | src addr 23.200.30.6 | desti addr 10.10.1.45 | Strm indx 4 | src prt 443 | protocol TCP | info : [TCP Retransmission] [TCP segment of a reassembled PDU]
No.283 | time  (Since prev pkt) 0.001030000 | src addr 23.200.30.6 | desti addr 10.10.1.45 |Strm indx 4 |   src prt 443 | procotol TLSv1.2 | info : [TCP Retransmission] Application Data
No.358 | time (Since prev pkt) 0.000533000 | src addr 23.200.30.6 | desti addr 10.10.1.45 | Strm indx 4 | src prt 443 | procotol TCP | info : [TCP Out-Of-Order] [TCP segment reassembled PDU]
No. 317 | time (Since prev pkt) 0.000001000 | src addr 23.200.30.6 | desti addr 10.10.1.45 | Strm indx 4 | src prt 443 | procotol TLSv1.2 | info : [TCP Out-Of-Order] Application Data

Here is the location of the PCAPS and network diagram: https://drive.google.com/folderview?id=0B41R9G9kL5L-S1JWZ0Z2bTFKN3c&usp=sharing

Answer 1

I feel a bit weird answering a Question based on a capture file provided by a 3rd party, but the world isn't black and white.

The first question would be what kind "firewall" do you use (and @Lewis Travieso, if you are still interested, the same question applies to you)? Looking at the captures, I can see that the device is as if filtering out the turbulences happening at the WAN side so that they wouldn't affect the LAN side. But obviously the primary goal is not the filtering but either caching or, even more likely, analysing the files as they are downloaded for possible malicious contents.

If you apply a display filter tcp.stream == 9 at both files and have a closer look at the packets which Wireshark's TCP analysis marks as unusual ones in the WAN-side capture, you'll see the following:
while at WAN side, the incoming packet with seq 21240, next seq 22620 (in frame 344) is immediately followed by an incoming packet with seq 44700, next seq 46080 (in frame 346), at LAN side the "same" packet with seq 21240, next seq 22620 (in frame 348) is immediately followed by a packet with seq 22620, next seq 24000 (in frame 350) after that packet has arrived to the WAN side (in frame 350).
~~The WAN-side packet in frame 346 has arrived out of sequence most likely because between the firewall and the server, there are multiple paths with significantly different propagation time.~~
EDIT: as @mrEEde has noted for the capture provided by the OP, the ip.id field suggests that the weird order of WAN-side packets mentioned earlier is not caused by multiple routes between the server ad client ends. As the same behaviour can be observed in the capture provided by @zareefaqmar, and lacking any knowledge about the network architecture at the server end, I hereby guess that there is some NAT/firewall/load balancing device there which assigns its own values of ip.id to packets it forwards from the actual server.

The "firewall" device at the client end is also responsible for the crash of the TCP session. The last incoming packet from WAN side to be properly retransmitted to LAN side is the one in frame 462 (WAN)/frame 464 (LAN); after that, the firewall sends (inevitably impersonating the remote server) a locally generated RST, although at the WAN side, subsequent packets (frames 463 and 464) have arrived from the remote server in the meantime. But the firewall stops talking to the server completely, so after a short while, the server starts retransmitting the last packet it didn't get acknowledged, and it sends a RST itself after still getting no acknowledge for almost 20 seconds.

So my conclusion is that the "firewall" is actually a more sophisticated security device, which analyses the HTTP responses as it forwards them, and it either does not like or merely does not understand the contents of the pdf file, so it causes the TCP session to fail as a consequence, either intentionally or due to a bug. Notably, Wireshark's File->Export Objects->HTTP didn't recognize the pdf payload either.

The url (gltopdf) suggests that the pdf file is generated, so maybe its contents is really malformed?