This is a static archive of our old Q&A Site. Please post any new questions and answers at ask.wireshark.org.

Why am I getting RSTACK packet after SYN ?

0

I am running a scanner against my server. I am trying to understand why TCP conversations (12-17) below have a SYN (sent by the scanner) with a RSTACK response sent by my server. Does anyone know why? Thanks in Advance.

I am sorry that the data below seems to get reformatted. It should be 4 values per line where the value is null for the column representing the receiver.

Packet Scanner Sends Service Sends tcp.stream 1 SYN 0 2 SYNACK 0 3 RST 0 4 SYN 1 5 RSTACK 1 6 SYN 2 7 SYNACK 2 8 ACK 2 9 PSHACK 2 10 FINACK 2 11 FINACK 2 12 ACK 2 13 SYN 3 14 SYNACK 3 15 ACK 3 16 PSHACK 3 17 ack 3 18 FINACK 3 19 FINACK 3 20 ack 3 21 SYN 4 22 SYNACK 4 23 ACK 4 24 PSHACK 4 25 ACK 4 26 FINACK 4 27 FINACK 4 28 ACK 4 29 SYN 5 30 SYNACK 5 31 ACK 5 32 PSHACK 5 33 ACK 5 34 RSTACK 5 35 SYN 6 36 SYNACK 6 37 ACK 6 38 PSHACK 6 39 ACK 6 40 RSTACK 6 41 SYN 7 42 SYNACK 7 43 ACK 7 60 FINACK 7 65 ACK 7 78 FINACK 7 79 ACK 7 44 SYN 8 45 SYNACK 8 46 ACK 8 61 FINACK 8 66 ACK 8 85 FINACK 8 86 ACK 8 47 SYN 9 48 SYNACK 9 49 ACK 9 62 FINACK 9 67 ACK 9 87 FINACK 9 88 ACK 9 50 SYN 10 51 SYNACK 10 52 ACK 10 63 FINACK 10 68 ACK 10 89 FINACK 10 90 ACK 10 53 SYN 11 54 SYNACK 11 55 ACK 11 64 FINACK 11 69 ACK 11 93 FINACK 11 94 ACK 11 56 SYN 12 57 RSTACK 12 58 SYN 13 59 RSTACK 13 70 SYN 14 71 RSTACK 14 72 SYN 15 73 RSTACK 15 74 SYN 16 75 RSTACK 16 76 SYN 17 77 RSTACK 17 80 SYN 18 81 SYNACK 18 82 ACK 18 83 PSHACK 18 84 ACK 18 91 FINACK 18 92 ACK 18 95 FINACK 18 96 ACK 18

asked 07 Nov '13, 08:16

malhenry's gravatar image

malhenry
21559
accept rate: 0%

edited 07 Nov '13, 09:54

Jasper's gravatar image

Jasper ♦♦
23.8k551284

I do not believe the socket is closed during the time of RSTACK. Also TCP conversation 18 seems to be successful after 6 failed connections.

(07 Nov '13, 08:21) malhenry

Thanks for the edit Jasper!

(07 Nov '13, 10:13) malhenry

with a RSTACK response sent by my server. Does anyone know why?

impossible to tell without further information.

  • what is the OS and version of the server
  • is there a firewall involved
  • how do you scan (which tool)
  • what do you scan (same port over and over again, or different ports)
  • what are the time deltas between the frames
  • can you post a full capture file, instead of just the text output
(07 Nov '13, 15:38) Kurt Knochner ♦

The server is Windows Server 2008 R2 as a VMware VM. The firewall is disabled. There is no AV running on the server. The scanner is QualysGuard it is doing a PCI scan which involves many ports and tests (which Qualys does not describe in detail). Every time this scan is run, Qualys says there was 7 good connections followed by 3 bad. Well there are more than 10 TCP streams in this capture. Qualys says there are two phases to the scan: discovery and vuln testing and that maybe the discovery packets are not included in their logging. I will see if I can at least post the deltas....not sure I can post the capture file.

Thanks.

(08 Nov '13, 08:27) malhenry

not sure I can post the capture file.

O.K. as an alternative, can you please post the output of the following command:

tshark -nr tcpinfo.dump.pcap -T fields -e frame.number -e frame.time_relative -e ip.src -e ip.dst -e ip.id -e tcp.srcport -e tcp.dstport -e tcp.flags -e tcp.seq -e tcp.ack

If you want, you can replace the real IP addresses with dummy values (search - replace in a text editor).

(08 Nov '13, 15:42) Kurt Knochner ♦

2 Answers:

2

When you sort the packets chronologically it is obvious that your server only allows 5 concurrent connections from the same client.

This seems to be caused by the SOMAXCONN limit as a forum thread in stackoverflow.com suggests.

When streams 12-17 send their SYNs we still have 7-11 in an established state.

As soon as the server closes stream 7 the next SYN (stream 18) gets in.

alt text

answered 08 Nov '13, 22:22

mrEEde's gravatar image

mrEEde
3.9k152270
accept rate: 20%

edited 13 Nov '13, 12:36

Just see that the state is CLOSEWAIT (FINACK arrived) when the SYN12 comes in...

(08 Nov '13, 22:27) mrEEde

Looks like a good possibility. I will check the application software. It would be awesome if you are right.! Thanks!! Here is the packet capture - http://cloudshark.org/captures/28fee41588d1

(12 Nov '13, 14:52) malhenry

The trace shows the server is being scanned by Linux and the Windows server is rejecting SYN packets whenever there are more than 5 active connections to the client.

(13 Nov '13, 07:49) mrEEde

This may not be the best forum for the following post. If there is one more appropriate, please inform. Ultimately I have to have development change our server software in order to pass the PCI scan. One bandaid is to increase the number of open connections in the listen call to the number that the scanner will request on this port, but this seems like a kludge. So I am trying to figure out a better solution. I wonder if timing is an issue. For example I notice in tcp.stream 7 that: our server responds to syn with synack in .00004 sec our server responds to finack with ack in .00003 s our server responds to ack with finack in 3.5 s Is this because the overhead of tearing down a connection? Or is something wrong with our server. Thanks...

(13 Nov '13, 11:23) malhenry

Your problem was discussed in http://stackoverflow.com/questions/4709756/listen-maximum-queue-size-per-windows-version

YOur server is actively closing idle connections after exactly 5 seconds, this is why you see the FIN_ACK going out that late in stream 7.

(13 Nov '13, 12:38) mrEEde

Great link! Can I assume that you are saying to examine my keepalive tcp parameter in light of the needs of my application? http://tldp.org/HOWTO/TCP-Keepalive-HOWTO/overview.html Thanks.

(13 Nov '13, 14:32) malhenry

No, TCP Keepalive is not involved here. What I was trying to say is that when a client connects to your server and does not send the expected data (or no data at all), your server will close the connection 5 seconds after the accept(). This is a timer in your server application

(13 Nov '13, 22:44) mrEEde

In looking at stream 7, the only 5 second delta that I see is between the 3 way handshake and FinAck sent by my windows 2012 server in response to the FinAck sent by the scanner. So the original finack is sent by the scanner. So unless I am mistaken, I think my server is responding to a finack packet, rather than a timer expiring. Comments?

In any event, are you referring to the Fin_wait_2 timer http://msdn.microsoft.com/en-us/library/windows/hardware/ff550023(v=vs.85).aspx ? My earlier analysis 4 posts ago was slightly wrong. The ack and finack (packets 65 and 78) sent by my server were in response to the finack sent by the scanner.

The largest delta between any consecutive packets in stream 7 is between packets 65 ack and 78 finack which are both sent by my server. Is a 3.5 sec delay between an ack and finack normal for a windows 2008 R2 server on a Hyper-V vm? My server is not under heavy load. Lastly doesn't the accept() call map to the action of pulling a connection out of the socket receive buffer? If so, I don't think that there is a packet that corresponds to this action and therefore there is no way to tell in wireshark when an accept() occurred. Am I mistaken? Thanks.

(14 Nov '13, 07:52) malhenry

"I think my server is responding to a finack packet, rather than a timer expiring. Comments?" While it is true that the finack packet acks the scanner's FIN packet I would think it is too much of a coincidence that the delta time between frame 41 and frame 78 http://cloudshark.org/captures/28fee41588d1?filter=frame.number%3D%3D41%20or%20frame.number%3D%3D78 is almost exactly 5 seconds. So my feeling is, the server didn't get notified that a fin has arrived and the server wakes up after its own 5 secs idle timer popped

(16 Nov '13, 06:46) mrEEde

"In any event, are you referring to the Fin_wait_2 timer" No, the FINWAIT2 timer starts when your server closes the connection and TCP is waiting for the scanner's ACK to your FIN.

(16 Nov '13, 06:49) mrEEde

"Lastly doesn't the accept() call map to the action of pulling a connection out of the socket receive buffer?" Not from the TCP receive buffer but from the listener's backlog queue

(16 Nov '13, 06:51) mrEEde
showing 5 of 11 show 6 more comments

0

Ultimately I have to have development change our server software in order to pass the PCI scan.

That's a rather bad idea and unfortunately a (common) result of totally useless and dumb PCI scans, just to pass that scan.

First: The tool scans the same port over and over again. What kind of security problem do they (Qualys) think to find by that kind of scan?

Secondly: The scanner of Qualys is kind of broken, as it scans the sever with source port 9150 (seems to be the destination port on the server), which results in a TCP Reset from either the firewall or the server itself. See Frame#4 in http://cloudshark.org/captures/28fee41588d1 How much sense does that make?

{RANT: So, instead of 'fixing' your server software go back to Qualys and ask them to fix their scanner ;-)) Maybe your server software is not broken, and it has good reasons to allow only a certain number of concurrent connections from one client. That could be a security measure in itself. So, by 'fixing' that behavior, you might pass a dumb PCI scan, but actually you might also weaken the security of your server software.

But let's be honest: PCI scans are not about improving security. It's all about passing a bunch of dumb tests and getting a certificate ;-) }

BTW: Is there a firewall between the client and the server? There should/must be one, if you want to pass a PCI audit ;-)).

Did you check what the firewall does to those scanning tests. Maybe it's the firewall and not the server software that limits the number of concurrent connections.

Regards
Kurt

answered 14 Nov '13, 08:18

Kurt%20Knochner's gravatar image

Kurt Knochner ♦
24.8k1039237
accept rate: 15%

edited 14 Nov '13, 08:20

Just curious: How did you know it was Qualys? Any indication in the trace ?

(16 Nov '13, 06:54) mrEEde

@malhenry mentioned it in a comment.

(16 Nov '13, 14:29) Kurt Knochner ♦