This is our old Q&A Site. Please post any new questions and answers at ask.wireshark.org.

For manually parsing .pcap files, I have found header files. But, they don't seem to have the IPv6 structures. I actually did find a place that had the definition of the IPv6 header. But, I'm not sure if that is all I need. I'm not sure if, when using IPv6, the TCP and UDP headers are the same as IPv4. It didn't look like it, but, it is conceivable that I was in error. So, I'd like some confirmation.

Also, Wireshark has the ability to save files in some pcap variations. For example, there is the variation that supports nanosecond times. Not sure that I need that, but, it might be interesting to have the docs for that as well.

Thanks.

asked 18 Feb '14, 09:00

Justin%20Crapola's gravatar image

Justin Crapola
1112
accept rate: 0%


I think you're mixing PCAP file format structures with frame format structures. A pcap file has a file header, and multiple frame headers, one for each frame. The frame data itself is just a bunch of bytes. To parse those bytes you need the dissectors which have nothing to do with the file format.

The (old) pcap format ist documented here: http://wiki.wireshark.org/Development/LibpcapFileFormat

The newer and much more powerful pcap-ng format is found here: https://www.winpcap.org/ntar/draft/PCAP-DumpFileFormat.html

Regarding nanoseconds: if you do not have or want to support specialized high speed capture hardware you can forget about nanoseconds, because a PC/MAC/normal computer cannot capture timestamps that precise.

permanent link

answered 18 Feb '14, 09:29

Jasper's gravatar image

Jasper ♦♦
23.8k551284
accept rate: 18%

edited 18 Feb '14, 09:30

Hi, thanks for the response. I will endeavor to explain better. First off, I have a working tool that I use for decoding a specific protocol. It is actually DNP, which, Wireshark does have DNP decoding. But my tool does a lot more, including stitching together DNP messages if they are fragmented across multiple IP packets.

So far, so good. When I first wrote my tool, I looked at .pcapng and .pcap. I decided to go with the old .pcap files. At least for the time being. I could possibly support .pcapng someday. But, I decided to start with just .pcap files.

So, my tool can, if using IPv4 TCP, open a .pcap file, step though the headers, find DNP packets, parse them. DNP is at least usually TCP, but, it is sometimes done with UDP. But, I was only initially interested in IP4 TCP.

Well, I was recently given a DNP project using IPv6 UDP. So, I wanted to update my tool to support UPD and IPv6.

As it so happens, my tool is written in Delphi. (Pascal) So, I found definitions of the headers in C, and converted them to Pascal.

In any case, the page you linked to for the .pcap file format ( http://wiki.wireshark.org/Development/LibpcapFileFormat) does NOT have IPv6 information. It doesn't have the header definitions.

I did happen to find somewhere the definition of the IPv6 header, and added it to my Pascal source. But, when I look inside the UPD packet itself, I don't find what I'm expecting, I don't find DNP packets. Thus, it seems that maybe in IPv6, it uses different headers for UDP and/or TCP when using IPv6 instead of IPv4. And so I'm not stepping into the packet at the right place. Or, possibly, I have a bug and I'm not looking where I think I am. This is possible.

But, I figured that there should be somewhere a header file that has all the structures defined, and so I could double check my structures verses the structures I created.

I was going to post in this comment my Pascal structures, just for reference, but, I'm running out of characters allowed. So, I will post that in a separate comment.

(18 Feb '14, 11:38) Justin Crapola

Well, I tried to post my Pascal structures, but, the wordwrap worked weird and it came out unreadable. I can try again if someone is interested....

(18 Feb '14, 11:47) Justin Crapola

The format of DNS packets is the same over IPv6 as it is over IPv4. With IPv6 you need to take care of extension headers, which can be inserted between the IP header and its payload (but they're pretty rare).

Nice to see someone else is using Delphi to decode network packets :)

(18 Feb '14, 11:48) Jasper ♦♦

In any case, the page you linked to for the .pcap file format (http://wiki.wireshark.org/Development/LibpcapFileFormat) does NOT have IPv6 information.

It also doesn't have IPv4 information, because there is no IPv4 or IPv6 information in the pcap file format. There's time stamp information, packet length information, and raw packet data; the raw packet might, or might not, be an IP packet.

(18 Feb '14, 14:44) Guy Harris ♦♦

Thus, it seems that maybe in IPv6, it uses different headers for UDP and/or TCP when using IPv6 instead of IPv4.

The same headers are used for UDP and TCP atop both IPv4 and IPv6. They're not in the same place relative to the beginning of the IP header in IPv4 and IPv6, but that's because the IPv4 and IPv6 headers are different, not because the {UDP,TCP}-atop-IP headers are different for IPv4 and IPv6.

(18 Feb '14, 14:47) Guy Harris ♦♦

Well, I tried to post my Pascal structures, but, the wordwrap worked weird

If you're posting code, indent every line of the code with 4 extra spaces. Ask.wireshark.org uses Markdown syntax.

(18 Feb '14, 14:50) Guy Harris ♦♦

Wireshark does have DNP decoding. But my tool does a lot more, including stitching together DNP messages if they are fragmented across multiple IP packets.

For DNP-over-UDP, Wireshark can be told to reassemble fragmented IP packets, so it can also stitch together the IP fragments of a DNP-over-UDP message fragmented at the IP layer.

For DNP-over-TCP, Wireshark can be told to reassemble the TCP segments of a DNP message that crosses TCP segment boundaries, so it can also stitch together the TCP segments of a DNP-over-TCP message fragmented at the TCP layer (or at the IP layer, but IP fragmentation is rare for TCP).

(18 Feb '14, 14:53) Guy Harris ♦♦

I'd be very interested in looking at a DNP3 capture that isn't correctly re-assembled in Wireshark. Please create a bug in the Wireshark Bugzilla and attach the capture to it.

In addition to that, if there's a part of the protocol that aren't yet covered and you would like Wireshark to do so, please create an entry for that in Bugzilla and mark it as an enhancement, also attaching a capture that illustrates the issue.

(19 Feb '14, 01:42) grahamb ♦

Guy Harris:

As far as the .pcap file docs not including IP4 or IP6 information, well, since IP4 and IP6 packets can be in .pcap files, I basically expected those to be documented in the .pcap file documentation. So, if that isn't where I need to find the documentation for the various headers, then, please, point me the way.

I am by no means a TCP/IP guru. I know the concepts of the 7 layer model. A number of years ago, I read a book on TCP/IP where it discusses how TCP/IP and UDP, and the user protocols (FTP, HTTP, etc.) map to the 7-layer model. But, I by no means remember it all. I've managed to parse the .pcap files I had seen, when dealing with IP4 DNP packets, but, that does not really mean I know what I'm doing... LOL.

I did not know that Wireshark can be told to reassemble packets. I'm by no means an expert Wireshark user. Thanks for that information. You say that IP fragmentation is rare for TCP. Well, it is common for some of the devices I talk with -- terminal servers. They are transporting DNP to serial devices. The device isn't DNP aware, and just sends how many bytes it wants to. Thus the DNP packet gets spread over 2 or more packets, however the terminal server feels like it. Thus, at least as I was using Wireshark, it would just show what seemed to be two (or more) garbage DNP packet instead of one good one. So, how would I tell Wireshark, in that instance, to reassemble the DNP packets?

grahamb:

See my response to Guy as to how I am even getting packets not properly reassembled. I wouldn't have considered this a bug. As far as enhancements to Wireshark's DNP parsing, well, lets just say that I'm more familiar with and comfortable with my own tool. I've seen other people prefer to use Wireshark's parsing because they are more used to it. I developed my own for situations where I can't use Wireshark, such as a RS-232 capture. In any case, my tool does things that I wouldn't expect Wireshark to do. Like, I can request it to extract, for instance, object 30 (analog input) index 0, and it will search though the file and find all occurrences of that specific data being sent and display it in a spreadsheet.

(20 Feb '14, 12:39) Justin Crapola

PCAP is a file format for storing packet data. It is not a packet data dissection format. The information you are looking for is found here: http://tools.ietf.org/rfc/index

Basically all relevant protocols are documented in RFCs, unless proprietary. That's where I read about how to dissect them, but you could also take a look at how Wireshark does it by looking at the source code here: https://www.wireshark.org/docs/wsdg_html_chunked/ChSrcObtain.html

(20 Feb '14, 13:01) Jasper ♦♦

As far as the .pcap file docs not including IP4 or IP6 information, well, since IP4 and IP6 packets can be in .pcap files, I basically expected those to be documented in the .pcap file documentation.

Nope, no more than the pcap file documentation documents Ethernet, 802.11, ATM, PPP, etc., etc., etc.. A pcap or pcap-ng file contains raw octets for the packet; the file specifies only the format of the first-layer headers (with the link-layer header type field in the pcap header and the pcap-ng Interface Definition Block), and the tcpdump.org link-layer header types page documents the values, mainly using links to the documentation for those header types (for example, it links to the IEEE Get page for 802.3, rather than explicitly documenting Ethernet itself).

Those headers generally either have fields specifying the next protocol layer or have the next protocol layer hardcoded; those fields are documented elsewhere.

So, if that isn't where I need to find the documentation for the various headers, then, please, point me the way.

IPv4 - see RFC 793. IPv6 - see RFC 2460.

(20 Feb '14, 13:23) Guy Harris ♦♦

You say that IP fragmentation is rare for TCP. Well, it is common for some of the devices I talk with -- terminal servers. They are transporting DNP to serial devices. The device isn't DNP aware, and just sends how many bytes it wants to. Thus the DNP packet gets spread over 2 or more packets, however the terminal server feels like it.

That doesn't require IP fragmentation. If a 4000-byte DNP packet is sent over TCP on an Ethernet network, that could be sent as 3 un-fragmented IP packets:

  • a 1514-byte Ethernet packet, with a 14-byte Ethernet header, 20-byte (minimum length, so no options) IPv4 header, 20-byte (minimum length, so no options) TCP header, and the first 1460 bytes of the DNP packet;
  • a 1514-byte Ethernet packet, with a 14-byte Ethernet header, 20-byte (minimum length, so no options) IPv4 header, 20-byte (minimum length, so no options) TCP header, and the next 1460 bytes of the DNP packet;
  • a 1134-byte Ethernet packet, with a 14-byte Ethernet header, 20-byte (minimum length, so no options) IPv4 header, 20-byte (minimum length, so no options) TCP header, and the last 1080 bytes of the DNP packet.

If one of those Ethernet packets failed to reach the receiver, it would not be acknowledged by the receiver's TCP implementation, and the sender's TCP implementation would eventually attempt to send it again (and, if "selective acknowledgment" isn't used) the packets after it - but the packets before it wouldn't need to be resent. If, however, the sender tried to send the DNP packet in one TCP segment, and relied on IP to do the fragmentation, if one fragment didn't arrive, the entire segment, all three fragments, would have to be retransmitted.

(20 Feb '14, 13:38) Guy Harris ♦♦

So, how would I tell Wireshark, in that instance, to reassemble the DNP packets?

If they're fragmented at the TCP layer, you need to make sure the "Reassemble DNP3 messages spanning multiple TCP segments" preference for DNP is on (Edit -> Preferences, open up Protocols, and select DNP 3.0) and make sure the "Allow subdissector to reassemble TCP streams" preference for TCP is on (same instructions, but select TCP). You might also have to make sure the "Validate the TCP checksum if possible" is off, if reassembly still doesn't occur, because that might be due to TCP checksum or segmentation/desegmentation offloading resulting in the dissector finding a checksum error and refusing to reassemble.

If they're fragmented at the IP layer (likely with UDP, unlikely with TCP), you need to make sure the "Reassemble fragmented IPv4 datagrams" preference for IPv4 is on (same instructions, but select IPv4) and the "Reassemble fragmented IPv6 datagrams" preference for IPv6 is on (same instructions, but select IPv6).

(20 Feb '14, 13:52) Guy Harris ♦♦

(I'm the OP, using a different account)

This particular project got put on the back burner, so, I haven't been back. Thanks for the information on the preferences.

And, to the person that send the RFC docs, thanks. That will help. I was hoping to find if somebody had a C++ header file with those headers defined, something that I could easily take and run with. (As I said, I'm actually using Delphi, but, I can take a C++ header file and make it Delphi easily enough.) I had found with Google searching something that looked like it had the IPv6 header in it, but, it doesn't quite match the RFC document. So, I guess I got my hands on an errant header file which probably explains part of my problem.

If someone knows where I might can find a good C++ header file with all the headers defined, that would help, but, if I have to just go by the RFC and make my own, that will do. Thanks.

(14 Jul '14, 17:20) Paul Doland
showing 5 of 14 show 9 more comments
Your answer
toggle preview

Follow this question

By Email:

Once you sign in you will be able to subscribe for any updates here

By RSS:

Answers

Answers and Comments

Markdown Basics

  • *italic* or _italic_
  • **bold** or __bold__
  • link:[text](http://url.com/ "title")
  • image?![alt text](/path/img.jpg "title")
  • numbered list: 1. Foo 2. Bar
  • to add a line break simply add two spaces to where you would like the new line to be.
  • basic HTML tags are also supported

Question tags:

×51
×3

question asked: 18 Feb '14, 09:00

question was seen: 3,323 times

last updated: 15 Jul '14, 01:47

p​o​w​e​r​e​d by O​S​Q​A