This is our old Q&A Site. Please post any new questions and answers at ask.wireshark.org.

When having one Diameter message spanned in two different TCP packets (TCP is used as Diameter transport protocol), wireshark sometimes is not able to reassemble properly the Diameter messages.

We have seen the following situations: 1) The payload in the TCP message is not starting as a Diamter message (probably wireshark does not understand the Diameter version and the Diameter message length are coming). In this situation, wireshark shows the Diameter message as a CONTINUATION of the previous ones. 2) The payload in the TCP message seems to be starting as a Diameter message (probably wireshark understands a Diameter version and a valid message length is coming), but the truth is it is the continuation of a Diameter message which was sent in the previous TCP packet. In this situation, wireshark shows the Diameter message is containing a unknwon command code containing a malformed Diameter message.

This is usually seen when a packet is lost or at the beginning of the trace. Once the TCP payload is starting with a valid Diameter message (with its Diameter version, length, and valid command code) wireshark is decoding the messages properly, until a new packet is lost.

In the end, it seems a system is sending invalid Diameter messages when it is not.

Have you already seen this problem? Do you know if that might be related to how wireshark reassembles Diameter messages?

By the way, both TCP (Allow subdisectors to reassemble TCP streams) and Diameter (Reassemble Diameter messages spanning multiple TCP segments) options to reassemble Diameter/TCP messages are set.

asked 27 Mar '14, 11:29

Juan_ma's gravatar image

Juan_ma
1112
accept rate: 0%


From what I understand of your question (hint: a publicly posted capture somewhere helps tremendously) you are expecting the Diameter dissector to be able to correctly reassemble and dissect messages where some fragment of the message is missing?

I'm not at all familiar with Diameter but in general this isn't going to happen for any protocol as each message fragment will contain vital information for the reassembly and if any is missing then reassembly will be at best broken, and at worst the dissector will complain about a malformed (incomplete) message.

This situation will especially apply for protocols running over stream based (i.e. no message boundaries) transport protocols such as tcp as the application protocol doesn't have any idea of what transport fragmentation may be taking place so can't ensure a viable application pdu is in each transport layer fragment.

permanent link

answered 28 Mar '14, 03:25

grahamb's gravatar image

grahamb ♦
19.8k330206
accept rate: 22%

1

If you have a sample capture showing the 2nd case (Wireshark thinks it's Diameter but it's not really) it's possible someone could improve the Diameter dissector's heuristics to remove these false positives (i.e., make it claim that it's just continuation data).

(30 Mar '14, 14:48) JeffMorriss ♦
1

It sounds like Wireshark is reporting this correctly Juan. From it's perspective, at the beginning or end of your trace file if only one of the two segments that make up the Diameter message exist in the trace file, it's not going to call it a correctly-formed Diameter message.

Diameter over TCP, so I'm assuming this is a credit control interface? If you can share it you can upload it here and post a link: https://appliance.cloudshark.org/upload/

(30 Mar '14, 19:24) Quadratic

Finally I managed to get a file without sensitive information. I also manipulated all ip addresses, mac addresses and Diameter addresses, so take into account checksums will not be valid.

Here there is a link to the pcap file: https://www.cloudshark.org/captures/6cf577bd1721

From frame 1 to 14, packets are decoded as DIAMETER CONTINUATION messages since it is the beginning of the trace and the payload is not beginning with the starting of a Diameter message.

From frame 15 to 683, packets are correctly decoded, main reason is starting of payload in packet 15 coincides with the beginning of a Diameter message.

687 frame was received out of order (684 is really the continuation of 687). At this moment wireshark stops reassembling Diameter messages (don't know if somehow it could realize 684 is the continuation of 687)

From frame 691 to 693, packets are decoded as DIAMETER CONTINUATION messages since the payload is not beginning with the starting of a Diameter message.

Payload in 694 is starting with 01, which is how Diameter messages are starting (Diameter version) and next three bytes are considered the length of the Diameter message, those three bytes are including value 07 40 00 in hex which basically is understood by wireshark as a length of 475136 bytes!!! The truth is payload of frame 694 is the continuation of frame 693, but its starting is similar to a Diameter message (I wonder if Diameter dissector's is just chaecking Diameter version is coming...).

All frames from 694 to 1398 are reassembled as a Diameter message of 475136 bytes, this huge Diameter message is not understood by wireshark and considers it a malformed Diameter message.

Frame 1400 is not starting as a Diameter message (no 01 at the beginning) and hence wireshark considers it a Diameter continuation

However frame 1401 is starting with 01 and the next three bytes read are 2c 01 00, which is understood by Diameter dissector as a Diameter message of 2883840 bytes!!! That would mean messages from 1401 to 5602 are understood as one Diameter message.

Same behaviour is repeated until frame number 11375 where the starting of the TCP payload coincides again with the beginning of a real Diameter message (Diameter version/ Diameter length/Diameter Flags/Diameter Command Code...)

I guess it is a problem on the Diameter dissector's heuristics as Jeff pointed out.

(03 Apr '14, 05:50) Juan_ma
1

You were exactly right that the Diameter dissector was only looking at the version and not the length. I submitted a change which strengthens those heuristics so the length is checked (only works for messages under 8192 bytes--if you know messages might be bigger than that in the real world, let me know and I'll increase the number) and that none of the reserved Flags bits are set.

(Really should have moved this over to a bug report but oh well...)

(03 Apr '14, 11:01) JeffMorriss ♦

Sorry, I'm quite new on this fora and didn't know I could create a bug report. I guess you'll take the point of requesting to solve this issue, right?

In relation to the Diameter message size, I know cases where 64 Kbytes are sent in the real world, and the truth is Diameter RFC is theoretically allowing 16777215 bytes.

I think it would be better to check command flags fields coming after the length (see text copied from RFC 6733):

Command Flags The Command Flags field is eight bits. The following bits are assigned:

   0 1 2 3 4 5 6 7
  +-+-+-+-+-+-+-+-+
  |R P E T r r r r|
  +-+-+-+-+-+-+-+-+

and taking into account "r" bits must be zero

  r(eserved)

     These flag bits are reserved for future use; they MUST be set to zero and ignored by the receiver.
(04 Apr '14, 01:01) Juan_ma
1

64 kbytes, really? Wow, I saw someone screaming over 1800 bytes. But that's in the Telco world. I can push a change to increase the limit to 64kbytes. I know that theoretically the messages can be up to 2^24-1 bytes, the question is just what will ever actually happen in the real world.

The change I pushed yesterday already checks that the r-bits must be 0. But that's still a pretty weak heuristic--I think we really need a length check.

Do you think 64kbytes is enough or maybe we need some more room like 80kbytes?

(04 Apr '14, 07:03) JeffMorriss ♦

In the Telco world some Diameter applications like 3GPP Sh interface can contain XML documents or binary data of whatever size. I think sending more than 80 kbytes would imply a very inefective use of Diameter protocol, but you cannot prevent from that.

I still think it would be good to check command code flags after the length to guarantee it is a valid Diameter message. Byte after length must always contain X0 where X (most significant 4 bits) can take the following hex values: 0, 2, 4, 6, 8, 9, C and D; and least 4 significant bits must always be 0. That is, commmand code flags byte can only contain 00, 20, 40, 60, 80, 90, C0 and D0 values following Diameter RFC, and any other command code flags is invalid.

(05 Apr '14, 04:11) Juan_ma

Jeff - Just to give you some insight on what's happening right now in the mobile industry as it relates to this:

The IMS network architecture is the defined method of deploying Voice over LTE (VoLTE), which is something that operators have been delaying for years with workarounds (CSFB, etc., kind of like how ISPs have used NAT workaround for the move to IPv6), but IMS and VoLTE is going to be in full swing this year. Since that architecture calls for all "services" including voice to be served by SIP application servers, where those servers store all provisioning data for all applications on a single database (the HSS or "Home Subscriber Server"), and since the communication between those systems calls for entire XML documents to be passed as content within an AVP in a Diameter message, I think the need to deal with very large individual Diameter messages is likely to increase significantly.

So in short, I would agree with Juan that the need to support possibly very large Diameter message is going to increase as we go. I'd further add that while Diameter is already central to LTE/EPC networks, since VoLTE will see it replace virutally all SS7/Sigtran signaling for applications like gsm_map, inap and camel, Diameter in general is going to become one of the most important protocols for mobile protocol analysis. That and SIP, DNS, GTPv2, and the radio/s1ap pieces.

(05 Apr '14, 15:41) Quadratic
1

Trust me--I know. My employer makes a product which sits in the middle of all this Diameter traffic.

But think about it for a minute: a 64kbyte subscriber profile (let's round it out and say it's a 64kbyte Update Location Answer) means that you'd need a 1 Gbit link (coming out of your HSS all the way to your MME) for every ~2000 ULAs you want to send per second: (1,000,000,000 bits/sec) / (8 bits/byte) / (64,000 bytes/ULA) == 1953. Sure you can have faster networks and more than 1 HSS but you've also probably got lots of MMEs. And let's not even start talking about roaming (I pity the roaming hub who connects to a network which dishes out 64kbyte profiles!).

To put some more perspective on that: I've seen traffic forecasts of 0.5 to 1.0 ULR/subscriber/hour. Taking the lower number (for argument's sake) that works out to needing that 1 Gbit/sec link (doing only ULR/ULA) for every ~14M subscribers. Taking the higher number means you can only support 7M subscribers.

As I said earlier, I heard screaming (well, more like "great displeasure") from an operator who discovered one of their partners was sending them ~1800 byte ULAs.

So I really have a hard time imagining anyone getting away with 64kbyte subscriber profiles. I imagine other interfaces will be similar. But if you guys are really sure, I can always take Michael Mann's suggestion (in the change quoted above) and put in a preference for the max Diameter message size, maybe with a default closer to 64k than the 8k it is now.

@Juan_ma: as mentioned my change already checks for the lower 4 flags bits being 0. But you do raise a good point that it could also ensure the E & R bits aren't both set.

(06 Apr '14, 11:07) JeffMorriss ♦
1

They do scream about DIAMETER message size - but at the same time they want more data pushed over it. :)

For example, some people are asking my employer to stuff entire SIP messages inside DIAMETER, for logging/troubleshooting purposes... yes I know, DIAMETER isn't a good logging transport protocol, but tell that to some folks who see it as the do-everything protocol between the call control systems and the management/back-office systems.

So... since SIP messages sometime grow quite large (for example NOTIFY messages with buddy lists in them), I'd expect DIAMETER to grow quite large too someday.

(06 Apr '14, 11:38) Hadriel

So how large would you consider "large?"

(I'm not saying that my choice of 8kbytes is the right one but I also find it highly unlikely we'll ever see a message of size 2^24-1.)

(06 Apr '14, 12:42) JeffMorriss ♦
1

Probably 64KB for the default.

I know that makes the odds of a false-positive higher, but maybe not too bad? You could make it a max of 65534 instead of 65535, just to avoid having a 0x00FFFF successfully match it incorrectly (because that's probably a common sequence of bytes inside DIAMETER payload).

A lot of normal users don't know about preference settings, and it would be hard to figure out the reason why it's not decoding something it should. :(

(06 Apr '14, 14:52) Hadriel
1

@JeffMorriss, we seem to be talking about two different things here - The large messages aren't going to come from S6a/d, they're going to come from Sh. ULAs will have limited subscriber data, and EPS-specific subscriber data. Sh interface will have entire XML documents of profile data that can be about virtually anything an application would want to provision.

The operators you're talking about sound like they're just doing traditional LTE roaming, but the concern is the home network's Sh interface between the HSS and the AS's (eg: MMTel, RCS, etc.). I can only imagine what third party developers will do to those XML documents for their own custom applications.

(06 Apr '14, 15:14) Quadratic
1

OK, I pushed another change which checks for the E- and R-bits being set and raised the size limit to 65534 (@Hadriel, not a bad idea about the commonality of 65535).

@Quadratic, sure, my example's from S6a but the principle's the same, right? How much bandwidth are the operators going to be willing to allocate for signaling?

(06 Apr '14, 17:04) JeffMorriss ♦
1

I think time will tell, but I'd be surprised if operators push back much on the size of those XML documents. Reasons being:

  • Sh XML pushes aren't really in-line with user signaling in the traditional sense and aren't as delay-sensitive as something like S6a.

  • Everyone wants to follow the standards (which call for all that provisioning info being passed over Diameter/XML).

  • If there's ever a choice between giving more granular features for an application to end users and reducing the size of the XML documents for the sake of bandwidth for the provisioning file transfer, I believe the needs of the end user will always win.

Like I said, time will tell just how crazy we get with provisioning records over Sh but I won't be surprised to see very large packets on that interface over the next couple years.

(06 Apr '14, 20:34) Quadratic
Everyone wants to follow the standards (which call for all that provisioning info being passed over Diameter/XML).

Yeah, that's the real reason. Otherwise they'd use HTTP for that provisioning-related stuff like normal people. But the 3gpp standards process is dominated by vendors/manufacturers who would feel threatened if HTTP were used; they'd rather keep it as specialized as possible for a vertical market. And their customer telco's have a deep-seated religion about following 3gpp standards no matter what - for very good reasons in some cases, but not all.

(06 Apr '14, 20:58) Hadriel
showing 5 of 16 show 11 more comments
Your answer
toggle preview

Follow this question

By Email:

Once you sign in you will be able to subscribe for any updates here

By RSS:

Answers

Answers and Comments

Markdown Basics

  • *italic* or _italic_
  • **bold** or __bold__
  • link:[text](http://url.com/ "title")
  • image?![alt text](/path/img.jpg "title")
  • numbered list: 1. Foo 2. Bar
  • to add a line break simply add two spaces to where you would like the new line to be.
  • basic HTML tags are also supported

Question tags:

×752
×78
×58
×47

question asked: 27 Mar '14, 11:29

question was seen: 6,319 times

last updated: 06 Apr '14, 20:58

p​o​w​e​r​e​d by O​S​Q​A