Using Lua and Tshark I'm attempting to obtain the XML payload from SOAP messages exchanged with my web service. I went with a listener approach (see program below), but it doesn't appear to be working properly. The s_xml_cdata print statement simply prints "RB14". I don't receive the full XML. Admittedly I'm new to both Tshark and Lua so I may be making some rookie mistakes here. I scoured the Web for any examples, but I've yet to come up with anything helpful. My ultimate goal is to save the SOAP XML to a flat file and/or redirect to a named pipe.
asked 20 Jun ‘11, 11:51
edited 20 Jun ‘11, 12:41
Guy Harris ♦♦
Here's Lua that extracts the XML fields to a file (with a dotted line in between fields). I tested it against your pcap in tshark and Wireshark. For tshark, run:
Note that the file is written in append mode to:
answered 29 Jun ‘11, 20:10
edited 30 Sep ‘11, 12:40
Well we seem to be getting somewhere now -- the complete XML is printed but by bits and pieces over several packets. Here's the output - files.me.com/sethlwilson/vlrmhm.
answered 30 Jun '11, 11:04
For some reason, I had problems combining the proto variable function call along with the tostring() call So I broke them up and added a check in between.
answered 20 Jun '11, 15:56
XML CDATA is often encoded with
This last line is a call to my function
So this function will take any long line and break it by the sep character, which for XML happens to be the greater than symbol. It returns an array of all your lines. Use a simple For loop to print your lines.
Comments and improvements welcome!
answered 21 Jun '11, 11:28
I do appreciate your help. Here is my latest rendition incorporating your latest suggestion. The file:write(s_xml_cdata) prints only 4 bytes of the xml, and that data is the value contained in the first set of tags. I'm running this with tshark using the following command: tshark -f "tcp port 8280" -X lua_script:C:Userssetwilxml.lua. My service is going over that port specifically. I have the following Wireshark packaage installed:
Version 1.6.0 (SVN Rev 37592 from /trunk-1.6)
Copyright 1998-2011 Gerald Combs [email protected] and contributors. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
Compiled (64-bit) with GTK+ 2.22.1, with GLib 2.26.1, with WinPcap (version unknown), with libz 1.2.5, without POSIX capabilities, without libpcre, without SMI, with c-ares 1.7.1, with Lua 5.1, without Python, with GnuTLS 2.10.3, with Gcrypt 1.4.6, without Kerberos, with GeoIP, with PortAudio V19-devel (built Jun 7 2011), with AirPcap.
Running on 64-bit Windows 7, build 7600, with WinPcap version 4.1.2 (packet.dll version 184.108.40.2061), based on libpcap version 1.0 branch 1_0_rel0b (20091008), GnuTLS 2.10.3, Gcrypt 1.4.6, without AirPcap.
Built using Microsoft Visual C++ 9.0 build 21022
This answer is marked “community wiki”.
answered 21 Jun ‘11, 12:37
To dump XML documents from a tap/listener:
answered 21 Jun ‘11, 18:10
Thanks helloworld, now I'm getting somewhere! Funny thing though is that using the code below I only get a portion of the xml and the amount differs between the request and the response. I confirmed though that the offset given by the FieldInfo object is the same as that indicated in the Wireshark GUI. Do you think that there may be some stray non-printable characters in the tvb that are interfering with string conversion? Again I'm running this with tshark from cmd.exe.
answered 22 Jun ‘11, 09:05
Well, I finally managed to obtain the XML payload using the below approach which probably isn't too terribly efficient. During a live capture I capture a single request-response relay and print their soap envelope contents. As for the request, I get all of the xml, byte for byte; but as for the response, there are some bytes missing at the very end of the xml:
The missing bytes are ...
At first I thought I was unintentionally lopping off those bytes in my convoluted algorithm, but in fact those bytes are even missing in xml_fieldinfo.value ( xml_fieldinfo.len / 2 == string.length(xml_string) ).
Why would the value member be truncated? Do you think that it's a bug, or is it an imposed limit?
answered 24 Jun ‘11, 12:54
Here is the pcap file in question: http://files.me.com/sethlwilson/cd4tw7
answered 29 Jun '11, 12:22
I tried your Lua program, but I'm not getting any output (temp.xml is never created) when replaying my pcap. Tap.packet() never fires.
answered 30 Jun ‘11, 07:30