This is our old Q&A Site. Please post any new questions and answers at ask.wireshark.org.

Are there any good write ups on the effects on performance/throughput on normalizing the S-ACK option out at a firewall or IPS? I was trouble shooting some slow SMB2 file copy issues and saw that the SYN had the flag at the sender but by the time it gets to the destination the S-ACK options has been NOOP'd by a sourcefire IPS. I'd like to take something back to the security guys and try and convince them how stupid this is.

asked 28 Feb '14, 11:28

RTJ10's gravatar image

RTJ10
16449
accept rate: 0%


There is no such write up that I am aware of, but I could write one in my blog at a later time. Facts are

  1. S-ACK is the best way to deal with packet loss. If you remove S-ACK from the options you'll revert to Fast Retransmissions or even RTO (worst case), which is not as efficient.
  2. S-ACK has no known relevance when it comes to network security, so stripping it does not improve security but decreases throughput when recovering from packet loss. So it does not make sense to remove it, unless you want to keep your TCP behavior consistent with ancient TCP stacks.

If your security guys do not believe me have them contact me and I'll convince them :-)

permanent link

answered 28 Feb '14, 11:48

Jasper's gravatar image

Jasper ♦♦
23.8k551284
accept rate: 18%

I know I saw something by Laura Chappell about ASA's normalizing the S-ACK option out but I can't find the video on youtube, perhaps because she mentions it in passing and never did a presentation or writeup. :(

(28 Feb '14, 11:57) RTJ10

She presented that one two years ago on Sharkfest I think. I don't have a youtube link either, but it still doesn't make any sense to remove S-ACK options from packets. I think it is a result of the security devices not recognizing the option at all and dropping it for that reason. The vendors should get their TCP normalization engines up to speed and let those options pass.

(28 Feb '14, 12:00) Jasper ♦♦

No, they are doing it on purpose, here's a quote from my security guy:

"After reviewing your captures can see what you're referring to. Yes, we are normalizing TCP traffic across the Sourcefire sensors -- Unfortunately, turning Normalization off will severely impact inspection policies."

(28 Feb '14, 12:22) RTJ10

tell them to open a support case. Sourcefire needs to fix their "Normalization", which seems to be on the level of the last century.

(28 Feb '14, 12:33) Jasper ♦♦

Well, I passed on your comments to the security folks, I don't think I am very well loved ATM. :)

(28 Feb '14, 18:56) RTJ10

I bet they need to remove the S-ACK option to be able to search through the stream properly for matching policies, without buffering too much. After all, it's not a whole message but just segments, so they need to see the segments sequentially in order for match rules to work right. I doubt they want to have a large TCP buffer per connection/stream in their IDS to reassemble, and if they don't buffer, then they can't reassemble the stream properly if S-ACK is used; and even if they did buffer, it might be too late by then because the offending/bad segments already went by and will be accepted by the side they're trying to protect.

(28 Feb '14, 19:11) Hadriel

Wellll, not being an IPS/IDS guy...wouldn't that make sense on outside IPS/IDSs, but safe to turn off on inside devices so as to not negatively affect network performance in or across datacenters?

(28 Feb '14, 19:23) RTJ10

I'm not sure I understand what you mean... if it's crossing an IPS/IDS, presumably you want the IDS/IPS to apply its policies - otherwise why have it involved to begin with? If you're just talking about traffic between hosts on the private/internal side only, put their internal traffic on a separate network/vlan that the IDS/IPS ignores or isn't even in the path of. No?

(28 Feb '14, 20:58) Hadriel

Sorry @Hadriel, but S-ACK has no negative effect on buffering segments - on the contrary, it helps getting the complete payload reconstructed in case of packet loss as fast as possible for content inspection.

(01 Mar '14, 12:39) Jasper ♦♦

@Jasper: imagine you're an IDS/IPS receiving TCP segments that you need to reconstruct into a stream to run match policies against (regex or whatever). You currently have last processed TCP seq #3 and are waiting on #4. You see #5+6+7 come through. Do you (a) buffer those and wait for #4 some time later, or (b) drop them and force the far-end to resend them all again in sequence? If you choose (a), you're buffering for some time X. If you choose (b), you have to force S-ACK off, but you can avoid buffering quite a bit depending on the match policies logic. (but as I said, this is only a guess - I don't work for an IDS/IPS vendor)

(01 Mar '14, 13:00) Hadriel

Really, "force the far end to resend them all in sequence?" That is not a good strategy (to say it in a nice way), because it will delay the transmission a lot, plus nobody can guarantee that the "in sequence" will work the second time - there may be more packet loss or just out-of-orders, and you'll never complete the full transaction.

No, if an IDS/IPS is doing policy matching against TCP payloads it needs to gather (buffer) all segments until the payload is complete.

(01 Mar '14, 13:57) Jasper ♦♦

If it doesn't work the second (or third or whatever) time, your internet connection between host+server is so bad it doesn't matter much what happens. But anyway I didn't say it was a great strategy - I just said I bet that's why they do it. I could be wrong - maybe there's some other reason (aside from just not having implemented it yet type thing).

(02 Mar '14, 02:26) Hadriel
showing 5 of 12 show 7 more comments

S-ACK has no known relevance when it comes to network security,

hm.. While trying not the sound like a smartass, I 'sometimes' fail to do so ;-)) So, here we go...

Software is full of bugs, and a TCP/IP implementation suffers from the same problem. So, people could come up with all kind of sick ideas to misuse the SACK feature, by sending wrong (forged) SACKs or whatever. If there is a bug in the TCP/IP stack, it could lead to almost anything, from a DoS, to a crash, to whatever you can think of.

Actually, that's exactly what happened some (long) time ago.

http://www.iss.net/security_center/reference/vuln/TCP_Invalid_SACK.htm
http://www.iss.net/security_center/reference/vuln/TCP_Malformed_SACK_DoS.htm

Even Snort failed to handle SACKs in certain environments.

http://archive.cert.uni-stuttgart.de/vuln-dev/2005/09/msg00006.html

So, if you look at it this way, SACK did have a known relevance (years ago) when it comes to network security.

Nevertheless, the problems of the year 2005 are not a valid/good reason to strip the SACK options today.

I also believe it's somehow related to the TCP reassembling/buffering problem within the IDS scanner engine. However, I don't know snort/Sourcefire well enough to know why they might do that.

Yes, we are normalizing TCP traffic across the Sourcefire sensors

There is a normalizer preprocessor available for snort (and Sourcefire, I guess).

http://manual.snort.org/node17.html#SECTION003217000000000000000

They say, it's helpful to enable it in inline mode, to minimize the chances of evasion.

See also here: http://vrt-blog.snort.org/2010/11/inline-normalization-with-snort-290.html

@RTJ10: Maybe the same holds true for your Sourcefire configuration.

Regards
Kurt

permanent link

answered 02 Mar '14, 08:41

Kurt%20Knochner's gravatar image

Kurt Knochner ♦
24.8k1039237
accept rate: 15%

edited 02 Mar '14, 08:49

@Kurt Knochner - Thank you very much for the comments and links!

@Jasper Thank you as well! I will need to track down a public email for Laura and see if she'd be willing to share the PPS or presentation.

(29 Apr '14, 06:42) RTJ10
Your answer
toggle preview

Follow this question

By Email:

Once you sign in you will be able to subscribe for any updates here

By RSS:

Answers

Answers and Comments

Markdown Basics

  • *italic* or _italic_
  • **bold** or __bold__
  • link:[text](http://url.com/ "title")
  • image?![alt text](/path/img.jpg "title")
  • numbered list: 1. Foo 2. Bar
  • to add a line break simply add two spaces to where you would like the new line to be.
  • basic HTML tags are also supported

Question tags:

×55
×8
×3
×1

question asked: 28 Feb '14, 11:28

question was seen: 1,771 times

last updated: 29 Apr '14, 06:42

p​o​w​e​r​e​d by O​S​Q​A