Wireshark with customized Radius dissector crashing on RHEL 6.1 64 bit OS

Question

Hi,

We are using Wireshark 1.0.1, customized RADIUS dissector to suit our requirements. We developed this on RHEL4.7, few years back for one of our clients.

Currently we are moving on to RHEL 6.1 64 bit OS with the same source code and when we try to run our wireshark, it opens up good and when we try to load a capture file, it crashes.

As per our requirements, we cant directly move on to latest version of Wireshark before solving this issue with v1.0.1.

The same source code runs perfectly on Windows XP and RHEL 4.7 32 bit without any issues.
The source code compiles good on RHEL 6.1 64 bit OS.
The problem is there only when we try to run compiled wireshark on RHEL 6.1 64 bit OS.
- The Wireshark GUI opens good, But when I load a RADIUS capture file (the dissector for which is enhanced) wireshark crashes.
- Capture file containing other protocols (example SNMP, TCP) loads good.

When I try to debug using gdb, these are the logs that i see,

*Program received signal SIGSEGV, Segmentation fault.
except_pop () at except.c:258
258     set_top(top->except_down);
Missing separate debuginfos, use: debuginfo-install GConf2-2.28.0-6.el6.x86_64 ORBit2-2.14.17-3.1.el6.x86_64 PackageKit-gtk-module-0.5.8-19.el6.x86_64 atk-1.28.0-2.el6.x86_64*

and the list of Missing separate debuginfos goes on.

When i back traced the stack, here is what i see,

(gdb) bt
0  except_pop () at except.c:258
1  0x00007ffff6165e41 in dissect_packet (edt=0x16c7430,  pseudo_header=<value optimized out>, pd=0x167c700 "", fd=0x1538ba0, cinfo=0x16c7440) at packet.c:349
2 0x0000000000430f02 in add_packet_to_packet_list (fdata=0x1538ba0, cf=0x7932c0, dfcode=<value optimized out>, pseudo_header=0x8b1928, buf= 0x167c700 "", refilter=<value> optimized out>) at file.c:1004
3 0x00000000004329c8 in cf_read (cf=0x7932c0) at file.c:532
4 0x000000000044973a in menu_open_recent_file_cmd (w=0x1539cd0) at menu.c:1709
5 0x00000036d240bb3e in g_closure_invoke () from /lib64/libgobject-2.0.so.0
6 0x00000036d2420e23 in ?? () from /lib64/libgobject-2.0.so.0
7 0x00000036d24220af in g_signal_emit_valist () from /lib64/libgobject-2.0.so.0
8 0x00000036d24225f3 in g_signal_emit () from /lib64/libgobject-2.0.so.0
9 0x00000036d8a7dcce in gtk_widget_activate () from /usr/lib64/libgtk-x11-2.0.so.0
10 0x00000036d8964bdd in gtk_menu_shell_activate_item () from /usr/lib64/libgtk-x11-2.0.so.0
11 0x00000036d896688a in ?? () from /usr/lib64/libgtk-x11-2.0.so.0
12 0x00000036d8953ef3 in ?? () from /usr/lib64/libgtk-x11-2.0.so.0
13 0x00000036d240bb3e in g_closure_invoke () from /lib64/libgobject-2.0.so.0
14 0x00000036d24209ed in ?? () from /lib64/libgobject-2.0.so.0
15 0x00000036d2421f4a in g_signal_emit_valist () from /lib64/libgobject-2.0.so.0
16 0x00000036d24225f3 in g_signal_emit () from /lib64/libgobject-2.0.so.0
17 0x00000036d8a76b2f in ?? () from /usr/lib64/libgtk-x11-2.0.so.0
18 0x00000036d894ac6a in gtk_propagate_event () from /usr/lib64/libgtk-x11-2.0.so.0
19 0x00000036d894bddc in gtk_main_do_event () from /usr/lib64/libgtk-x11-2.0.so.0
20 0x00000036d7c5fffc in ?? () from /usr/lib64/libgdk-x11-2.0.so.0
21 0x00000036d1838f0e in g_main_context_dispatch () from /lib64/libglib-2.0.so.0
22 0x00000036d183c938 in ?? () from /lib64/libglib-2.0.so.0
23 0x00000036d183cd55 in g_main_loop_run () from /lib64/libglib-2.0.so.0
24 0x00000036d894c2c7 in gtk_main () from /usr/lib64/libgtk-x11-2.0.so.0
25 0x00000000004470ca in main (argc=0, argv=0x7fffffffe2f0) at main.c:3197

Linux Server Details:

[[email protected] ~]# uname -a Linux
rmonpa64 2.6.32-131.0.15.el6.x86_64 #1 SMP Tue May 10 15:42:40 EDT 2011 x86_64 x86_64 x86_64 GNU/Linux
[[email protected] ~]#

Wireshark Details:

[[email protected] ~]# wireshark --version wireshark 1.0.1 Copyright 1998-2008 Gerald Combs <[email protected]> and contributors. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. Compiled with GTK+ 2.18.9, with GLib 2.22.5, with libpcap 1.0.0, with libz 1.2.3, without POSIX capabilities, without libpcre, without SMI, without ADNS, without Lua, with GnuTLS 2.8.5, with Gcrypt 1.4.5, with MIT Kerberos, without PortAudio, without AirPcap. NOTE: this build doesn't support the "matches" operator for Wireshark filter syntax. Running on Linux 2.6.32-131.0.15.el6.x86_64, with libpcap version 1.0.0.

Built using gcc 4.4.5 20110214 (Red Hat 4.4.5-6). [[email protected] ~]#

Any information in this regard would be greatly helpful to us.

ldd which wireshark command output http://pastebin.com/iHPcPUEM

strace output: http://pastebin.com/jDbwwkf9

Thanks, Purandhar K

Answer 1

1

This looks similar to the crash reported in bug 7978. That was fixed in r46066 by taking out a 'return' from within a TRY/CATCH/ENTRY block. Is it possible your modifications have introduced such a return?

answered 08 Aug '13, 06:05

JeffMorriss ♦
6.2k●5●72
accept rate: 27%

Hi Jeff,

Thanks for your reply.

Tried debug the issue using eclipse and also with gdb, didnt give me any useful information on the crash. (may be i am missing something in them). I tried with Valgrid and it took me through to the issue directly.

The issues were,

I am trying to use 'pinfo' in my dissector and that crashes saying that I am trying to use an undeclared variable. Strange, pinfo is declared in wireshark's packet.c file and used in few other places in dissector.
in except.c, the except_pop() was throwing segmentation fault. Changing the following 'set_top(top->except_down);' to 'set_top(top);' worked. I havent changed anything in except.c.

Now that wireshark with my customized Radius dissector can load the capture files, i will dig in to see the actual reason for crash. I will keep your comments in mind while doing it.

thanks again!

-Purandhar

(08 Aug '13, 22:55) Purandhar K

I am trying to use 'pinfo' in my dissector and that crashes saying that I am trying to use an undeclared variable.

In C and C++, an "undeclared variable" is a compile-time error, not a crash (a crash is a run-time error); is your dissector written in C (or C++), or in Lua?

(08 Aug '13, 23:58) Guy Harris ♦♦

Hi Guy,

I have customized the default Radius dissector in C, extended it to dissect custom AVPs in Radius packets.

My bad (tired eyes), its 'Conditional jump or move depends on uninitialised value(s)' and not undeclared.

-Purandhar

(09 Aug '13, 01:55) Purandhar K

in except.c, the except_pop() was throwing segmentation fault. Changing the following 'set_top(top->except_down);' to 'set_top(top);' worked. I havent changed anything in except.c.

That change does not look correct to me. You may have band-aided the problem to avoid the crash but it's likely the functionality will not be correct with such a change.

But anyway if you can "fix" the crash like and thus be allowed to upgrade to an at least half-modern version, hopefully the crash will just be fixed in the newer version.

(09 Aug '13, 07:11) JeffMorriss ♦

Thats the plan, quick fix the issues and move on to the latest wireshark version, 1.10.1 as soon as possible :) .

However, i will keep looking for the root cause for the crash and will update here when i find it.

(09 Aug '13, 07:24) Purandhar K

Hi, I have been looking for the root cause for the issue mentioned in this ticket. The problem within the custom Radius dissector is resolved. I had to write up few lines of code as work around for 'Conditional jump or move depends on uninitialized value(s)'.

Wireshark still crashes when i load a Radius capture file. I dont have any returns from TRY CATCH Blocks.

Backtrace of the recent crash:

==27733== Invalid read of size 8
==27733==    at 0x59ADA37: except_pop (except.c:258)
==27733==    by 0x5B7CD61: dissect_frame (packet-frame.c:346)
==27733==    by 0x59B3D9F: call_dissector_through_handle (packet.c:396)
==27733==    by 0x59B44E5: call_dissector_work (packet.c:485)
==27733==    by 0x59B4630: call_dissector (packet.c:1787)
==27733==    by 0x59B5F51: dissect_packet (packet.c:332)
==27733==    by 0x430F01: add_packet_to_packet_list (file.c:1004)
==27733==    by 0x4329C7: cf_read (file.c:532)
==27733==    by 0x449739: menu_open_recent_file_cmd (menu.c:1709)
==27733==    by 0x36D240BB3D: g_closure_invoke (in /lib64/libgobject-2.0.so.0.2200.5)
==27733==    by 0x36D2420E22: ??? (in /lib64/libgobject-2.0.so.0.2200.5)
==27733==    by 0x36D24220AE: g_signal_emit_valist (in /lib64/libgobject-2.0.so.0.2200.5)
==27733==  Address 0x3604 is not stack'd, malloc'd or (recently) free'd
==27733== Process terminating with default action of signal 11 (SIGSEGV)
==27733==  Access not within mapped region at address 0x3604
==27733==    at 0x59ADA37: except_pop (except.c:258)
==27733==    by 0x5B7CD61: dissect_frame (packet-frame.c:346)
==27733==    by 0x59B3D9F: call_dissector_through_handle (packet.c:396)
==27733==    by 0x59B44E5: call_dissector_work (packet.c:485)
==27733==    by 0x59B4630: call_dissector (packet.c:1787)
==27733==    by 0x59B5F51: dissect_packet (packet.c:332)
==27733==    by 0x430F01: add_packet_to_packet_list (file.c:1004)
by 0x4329C7: cf_read (file.c:532)
by 0x449739: menu_open_recent_file_cmd (menu.c:1709)
by 0x36D240BB3D: g_closure_invoke (in /lib64/libgobject-2.0.so.0.2200.5)
by 0x36D2420E22: ??? (in /lib64/libgobject-2.0.so.0.2200.5)
by 0x36D24220AE: g_signal_emit_valist (in /lib64/libgobject-2.0.so.0.2200.5)

Your guidance would be of great help to me. I am still using wireshark-1.0.1 version, and can only move on to 1.10.1 after seeing the packets loaded on 1.0.1 :(

Thanks!

(27 Aug ‘13, 07:57) Purandhar K

showing 5 of 6 show 1 more comments

Answer 2

Issue has been resolved. problem was with one of the strcpy function, trying to copy a bigger string in to smaller one. Strange thing is same code runs well without crashing on RHEL4.7 32 bit.

Anyway, thanks for your time and guidance.

-Purandhar