This is our old Q&A Site. Please post any new questions and answers at ask.wireshark.org.

Hello, I'm trying to strip off the GTP headers of a Gn trace and be left with the TCP/IP stream, which I can then feed into tcptrace for analysis. Any idea how this can be done? What layer 2 protocol is then used for the IP packets, which won't have the GTP headers anymore? Thanks, Dan

asked 23 Feb '12, 00:44

Dan%20Eman's gravatar image

Dan Eman
16112
accept rate: 0%

edited 26 Jul '12, 23:36

Kurt%20Knochner's gravatar image

Kurt Knochner ♦
24.8k1039237


You can use bittwiste (Linux and Windows version available).

bittwiste -I gtp-u.pcap -O gtp-stripped.pcap -D 15-54

This removes (-D) the frame IP-, UDP- and GTP-Header. Result: The encapsulated IP header will be the new frame IP header ;-) Maybe you'll have to adjust the number of stripped bytes for your environment (IP Options).

It's easier to look at the sample data:

GTP: http://cloudshark.org/captures/374cf36574b6
GTP Stripped: http://cloudshark.org/captures/69de59cb48d6

Regards
Kurt

permanent link

answered 26 Jul '12, 09:35

Kurt%20Knochner's gravatar image

Kurt Knochner ♦
24.8k1039237
accept rate: 15%

edited 26 Jul '12, 09:36

@Kurt, I guess the range depends on the version of GTP? I looked at another capture file from the Wireshark menagerie (3503-rdp_2_packets.trc) where the version was indicated as "GTP release 97/98" and the GTP header was 20 bytes. It looks like your capture file has "GTP release 99", which apparently only has a 12 byte GTP header, thus I assume the range for a 97/98 version would be -D 15-62.

(26 Jul '12, 18:29) cmaynard ♦♦

good hint. Thanks!

Yes, the byte range may vary, depending on IP Options and apparently also GTP version. But it's easy to find the byte range in Wireshark by looking at the packet bytes pane. BTW: Is that trace file from your personal archive or somewhere on the internet?

(26 Jul '12, 22:50) Kurt Knochner ♦

The capture file is from Wireshark's own collection of capture files that are submitted in bug reports, on the wiki, through the mailing lists, etc. It's a really useful collection, but unfortunately, I don't know a way for everyone to access them - well, those not marked as "private" that is. If there's another developer who knows how to do it, please post the solution. If there is no way currently, maybe someone (Gerald?) could make it possible for the non-private capture files to be made available for download by the general public?

(27 Jul '12, 05:53) cmaynard ♦♦

If there is no way currently, maybe someone (Gerald?) could make it possible for the non-private capture files to be made available for download by the general public?

That would really help! Sometimes it's easier to understand a problem with a capture file and the Wireshark Sample Captures (wiki) are kind of limited to the well known protocols (more or less).

(27 Jul '12, 05:59) Kurt Knochner ♦

And I think what would make it really nice would be not only having the ability to download the capture files, but if there was a way to search for capture files containing specific protocols (or possibly other search criteria?), and have only those capture files of interest displayed ... basically similar to pcapr.net.

Well, since this isn't a discussion forum, maybe we should move this to wireshark-dev?

(27 Jul '12, 07:01) cmaynard ♦♦

maybe we should move this to wireshark-dev

good idea.

(27 Jul '12, 09:01) Kurt Knochner ♦

Well, I tried using gmane to post to wireshark-dev, but for some reason, the post never showed up. This is an annoying thing w/gmane that seems to happen quite a lot, but I still prefer to use gmane instead of my company e-mail, since my company always sticks their annoying confidentiality disclaimer at the end of my e-mail, which I (and many others) despise. Anyway, feel free to try posting something if you want to see if Wireshark capture files can be made available for download ... and be protocol-searchable as well.

(29 Jul '12, 07:33) cmaynard ♦♦

We'd be interested in providing a CloudShark system to the Wireshark dev team. This would give you lots of control over your capture files and allow a capture to be public or require authentication. There are several models that could be deployed. Captures can be organized and searched using the tagging system. If anyone wants to take the lead on this, I'd be happy to talk with you and explore this possibility. You can contact us at http://appliance.cloudshark.org/contact.html

(30 Jul '12, 17:52) joemc

Thank you for this offer! Can you please post it on the wireshark-dev list?

(31 Jul '12, 00:29) Kurt Knochner ♦
showing 5 of 9 show 4 more comments

The problem with the above methods is that it blindly strips a number of bytes off the packet while the pcap file may also contain other content than GTP-User.

I use the following python script (which is not perfect as I'm in no way a developer). It does the job fairly quickly even on large files and has proven to be a great tool.

#!/usr/bin/env python
'''Remove GTP layer from PCAP file'''
import dpkt, struct, time, re, socket
import platform
import sys

# Check for arguments
if len(sys.argv) < 3 or len(sys.argv) > 3:
    print "Usage:\n", sys.argv[0], "input.pcap", "output.pcap"
    sys.exit()

# Open files for input and output
try:
    fi = open(sys.argv[1],'r')
    fo = open(sys.argv[2],'w')

    # Prepare PCAP reader and writter
    pcapin = dpkt.pcap.Reader(fi)
    pcapout = dpkt.pcap.Writer(fo)

    for ts, buf in pcapin:
        # make sure we are dealing with IP traffic
        # ref: http://www.iana.org/assignments/ethernet-numbers
        try: eth = dpkt.ethernet.Ethernet(buf)
        except: continue
        if eth.type != 2048: continue

        # make sure we are dealing with UDP
        # ref: http://www.iana.org/assignments/protocol-numbers/
        try: ip = eth.data
        except: continue
        if ip.p != 17: continue

        # filter on UDP assigned ports for GTP User
        # ref: http://www.iana.org/assignments/port-numbers
        try: udp = ip.data
        except: continue
        try:
            if udp.dport != 2152: continue
        except: continue

        # extract GTP flags to detect header length
        gtpflags = udp.data[:1]
        try:
            if gtpflags == '\x30': payload = udp.data[8:]
            elif gtpflags == '\x32': payload = udp.data[12:]
            else: continue
        except: continue

        # at this point we have a confirmed ETH/IP/UDP/GTP packet structure
        # UDP payload is GTP header + real user payload
        try:
            # append real user payload to ethernet layer and writeout
            eth.data = payload
            pcapout.writepkt(eth, ts)
        except: continue

    fi.close()
    fo.close()

except IOError as (errno, strerror):
    print "I/O error({0}): {1}".format(errno, strerror)
permanent link

answered 27 Sep '13, 06:22

Kiloohm's gravatar image

Kiloohm
412
accept rate: 0%

Thanks for providing another option. I think tracewrangler might be the best overall solution though.

(27 Sep '13, 08:42) cmaynard ♦♦

If this is desired functionality open a bug requesting the feature, it should be easy to implement as part of "Export PDUs". Preferably also attach a sample file to test with.

(27 Sep '13, 09:02) Anders ♦

I just tried tracewrangler on one of my files and it resulted in an access violation error. Perhaps the file is too large (750 Mbytes).

As for adding it as a feature in Wireshark, it could be great but it would likely be needed as well through command line to be able to parse large files.

(27 Sep '13, 09:09) Kiloohm

The file size limit for TraceWrangler is 2GB at the moment, so 750MBytes should not be a problem. I don't have any files that size with GTP headers, so I can't really reproduce the issue, but I'll see what I can do.

(27 Sep '13, 12:09) Jasper ♦♦

I just uploaded a new TraceWrangler build that has a couple of fixes regardings memory leaks, some especially for Edit tasks. I tried with a 900 MB file and it worked fine (while it crashed before due to mem problems), but since I don't know what frames you had in your file it might still run into trouble.

(05 Oct '13, 16:53) Jasper ♦♦

Hi Kiloohm,

I've used your script successfully but it also removes the dot1.q vlan tag ... can you help ? tks

P

(20 Feb '14, 06:14) Trigas
showing 5 of 6 show 1 more comments

Hey!!

This can be done using perl. Let me know if you still required this. I may have some scripts to achieve the same.

permanent link

answered 26 Jul '12, 05:34

Vineeth's gravatar image

Vineeth
1111
accept rate: 0%

Not to resurrect a dead thread, but I noticed this one in a search and I think I have a better answer than the bittwiste example, since it keeps the top IP header when the desire is to create a raw IP capture starting with the inside IP header.

To do that, export the trace into a hex dump form (via Wireshark or Tshark), and execute this. This script assumes no IP options fields are present in the top IP header (would be a very unusual case), and it expects the input file in the same directory called hex_input as it's written there. It also is assuming that you've got an Ethernet header to start with, and GTPv1, but the number of bytes to subtract is controlled by the one line 'substr($packet,0,84)= "" ' so it's easily modified to do what you want. I quickly made it a bit generic but in my normal version of this I have it set to use perl's system commands, first to have tshark read the GTP-encapsulated capture file to get the hex printout, then at the end here I call text2pcap to rewrite the new capture file. Since the script starts by building a packet array out of text and follows by rebuilding it, it's actually a very useful script to tinker with, to quickly modify a few packet bits while they're nicely stored in an array.

Simply put, this script will take a hex dump file, puts it into an array, cut X number bytes, and rebuild a new hex dump file. It's written below to pull the internal IP packets out of GTP-U, and from there you can easily call the text2pcap utility to build it into a .pcap file as the script's output conforms to the expected text2pcap input format.

#!/usr/bin/perl

# Used to build an array of packets out of a hex dump and reassemble
# them into a new hex dump. Allows for easy manipulation of packets
# in hex while cleanly stored into an array prior to reconstruction.
# In this case, we're stripping 42 bytes to account for Ethernet, IP and GTP headers.
# Written by Russell DeLong ([email protected]).

use strict;

# Declarations
my $input_file = "hex_input";
my $n;
my @line;
my $line_count;
my @packets;
my $packet;
my $packet_length;
my $offset;
my $lead_zeros;
my $bytes;

# Take capture file input.
open (INPUT, $input_file) or die "couldn't open $input_file\n";

# Build a packet array.
while (<INPUT>){
        @line = split('  ', $_);
        if (@line[0] =~ m/0000/) {
            $n = $n + 1;
        }
        @packets[$n] = @packets[$n] . @line[1];
        @packets[$n] =~ s/ //g;
}

# Rebuild the text file from @packets array.
foreach (@packets) {
        $packet = $_;
        substr($packet,0,84)= ""; # Strip's 42 bytes from the top of the packet.
        $packet_length = $_ =~ tr/[0-9a-zA-Z]//;
        # The +0.999... is a cheap way to round up for the last line.
        $line_count = int(($packet_length/32) + 0.9999999999);
        for ($n=0; $n < $line_count; $n++){
            $offset = sprintf("%x",($n*16));
            # Assumes no offset greater than 4 hex characters.
            $lead_zeros = 4 - ($offset =~ tr/[0-9a-zA-Z]//);
            $lead_zeros = '0' x $lead_zeros;
            $bytes = substr($packet,$n*32,32);
            # Adds a space character after every byte.
            $bytes =~ s/([0-9a-zA-Z]{2})/$1 /g;
            print "$lead_zeros$offset  $bytes\n";
        }
}
permanent link

answered 06 Jul '13, 15:29

Quadratic's gravatar image

Quadratic
1.9k6928
accept rate: 13%

edited 06 Jul '13, 16:27

The new export PDU functionality in trunk could be expanded to export the GTP payload.

(07 Jul '13, 04:29) Anders ♦
Your answer
toggle preview

Follow this question

By Email:

Once you sign in you will be able to subscribe for any updates here

By RSS:

Answers

Answers and Comments

Markdown Basics

  • *italic* or _italic_
  • **bold** or __bold__
  • link:[text](http://url.com/ "title")
  • image?![alt text](/path/img.jpg "title")
  • numbered list: 1. Foo 2. Bar
  • to add a line break simply add two spaces to where you would like the new line to be.
  • basic HTML tags are also supported

Question tags:

×40
×13
×8
×1

question asked: 23 Feb '12, 00:44

question was seen: 15,440 times

last updated: 20 Feb '14, 18:18

p​o​w​e​r​e​d by O​S​Q​A