This is a static archive of our old Q&A Site. Please post any new questions and answers at ask.wireshark.org.

Can Wireshark capture USB data on Linux without truncation to 48 bytes?

0

As per the title: can Wireshark capture USB data without truncation to 48 bytes?

I understand that this is a limitation of usbmon - whether it's a hard limit that can never be breached, I don't know, hence the question. But truncating the data to 48 bytes renders the capture useless for my present purposes.

asked 26 Sep '16, 08:34

davehigton's gravatar image

davehigton
6114
accept rate: 0%

edited 26 Sep '16, 13:36

Guy%20Harris's gravatar image

Guy Harris ♦♦
17.4k335196

Can you be more specific regarding the kernel version, type of USB communication to be captured, the application used for capturing (tcpdump, Wireshark/tshark capturing via dumpcap, anything else) etc.? The thing is that I am routinely using usbmon to capture USB traffic and I haven't encountered any frame truncations. But as I mainly concentrate on isochronous transfers, there may be an issue which is related to a particular transfer type which I haven't run into. usbmon declares to truncate the text output, but not the binary one which is used by tcpdump and/or dumpcap from the Wireshark suite. Also, I am using tcpdump to capture to a file at the source machines and then use Wireshark to analyze the saved files on my laptop, so there may be an unnoticed dumpcap issue.

(26 Sep '16, 10:40) sindy

To quote the usbmon documentation:

These events can be received from a character device by reading with read(2), with an ioctl(2), or by accessing the buffer with mmap. However, read(2) only returns first 48 bytes for compatibility reasons.

If your kernel and libpcap support memory-mapped access for usbmon ("accessing the buffer with mmap"), there is no 48-byte truncation limit. That's why we need to know the kernel version and the libpcap version.

(26 Sep '16, 13:38) Guy Harris ♦♦

Kernel is 4.4.0-38-generic; it's a fully up to date Ubuntu 16.04 LTS 64 bit OS.

Type of USB communication to be captured: even the traffic from a keyboard is sometimes truncated, but the real killer is traffic with a Bluetooth dongle, where packets frequently are in the region of 256 bytes.

Capture is directly with Wireshark after selecting the usbmon2 interface. I'm not running separate capture and analysis sessions.

libpcap is 1.7.4, tcpdump is 4.7.4, but I am not aware of how what I'm doing interacts with either of them.

(27 Sep '16, 13:21) davehigton

libpcap is 1.7.4, tcpdump is 4.7.4, but I am not aware of how what I'm doing interacts with either of them.

As you start the capture from Wireshark, you use libpcap via dumpcap which is a process that Wireshark spawns for the actual capturing and reads its output. You do not use tcpdump at all.

even the traffic from a keyboard is sometimes truncated, but the real killer is traffic with a Bluetooth dongle, where packets frequently are in the region of 256 bytes.

Well, my question was related to the type of transfers (isochronous, interrupt, bulk, control) which are affected. But what you just wrote sounds to me as if only some of the URBs are truncated ("the traffic from a keyboard is sometimes truncated") and some are not even though larger than 48 bytes. Is this a correct understanding or it is just that some URBs from the keyboard are smaller than 48 bytes and thus are not truncated?

Just to be sure, can you capture some USB traffic using tcpdump -i usbmon -s 0 -w tcpdump_usb_test.pcap, press Ctrl-C when done, and then check whether the USBs in this file (tcpdump_usb_test.pcap) are also truncated if you open it using Wireshark?

(27 Sep '16, 13:40) sindy

Firstly, I apologise for taking so long to respond.

Well, my question was related to the type of transfers (isochronous, interrupt, bulk, control) which are affected.

Ah, I didn't understand, sorry.

Is this a correct understanding or it is just that some URBs from the keyboard are smaller than 48 bytes and thus are not truncated?

Nothing in the capture exceeds 48 bytes; anything that had more than 48 bytes on the wire is truncated. Some packets only have 48 bytes on the wire and therefore are not truncated. My apologies for being unclear.

tcpdump -i usbmon -s 0 -w tcpdump_usb_test.pcap

Slightly modified - needs sudo, also needs to be usbmon2 rather than just usbmon, but this capture is NOT truncated.

So thank you for at least giving me a work-round.

Is it possible to make live capture-and-display work the same way?

(01 Oct '16, 11:58) davehigton

needs to be usbmon2 rather than just usbmon

that's true if you need to reduce the amount of captured data and system load, as use of usbmon causes traffic from all root hubs to be captured, while use of usbmon2 causes only traffic from root hub 2 to be captured.

Is it possible to make live capture-and-display work the same way?

well, it is possible but not really convenient. You can tell tcpdump to write the output to a named pipe and let Wireshark read from the pipe, as described at the Capture Setup chapter of Wireshark's wiki:

mkfifo /tmp/my_pipe
wireshark -k -i /tmp/my_pipe &
sudo tcpdump -i usbmon2 -s 0 -w /tmp/my_pipe &

But the difference in behaviour between tcpdump and Wireshark makes me wonder what version of Wireshark you are running? Have you compiled it from source, or has it come with your Ubuntu distribution?

(01 Oct '16, 12:31) sindy

But the difference in behaviour between tcpdump and Wireshark makes me wonder what version of Wireshark you are running? Have you compiled it from source, or has it come with your Ubuntu distribution?

You may well ask what version. Dash suggested it was the current stable version (2.0.2 something - this is Ubuntu 16.04), but Wireshark's Help About Wireshark said 1.12.8 (Git Rev Unknown from unknown).

I didn't compile it myself, and it isn't part of the Ubuntu distribution. I must have installed it from a download from the Wireshark downloads site. I did have a folder named Wireshark 1.12.8, which I have deleted.

Anyway, within the last few minutes I've attempted to remove Wireshark via apt-get, then install the current dev one (2.2.0 something) via apt-get. However, Help About Wireshark still says version 1.12.8, and the truncation issue when capturing from usbmon remains.

So, somewhat bizarrely, I can't be certain what version I'm running, as the information I have is contradictory.

Does Help About correctly identify Wireshark's version? If so, then it looks to me as if the removal process didn't work properly (and I would appreciate some pointers about removing it completely).

(01 Oct '16, 14:45) davehigton

Sorry - I must correct myself here. I am looking at a capture from the mouse. All packets are 52 bytes on the wire. All packets from host to device are truncated to 48 bytes; all packets from device to host have the full 52 bytes.

(01 Oct '16, 15:03) davehigton

Wireshark's Help About identifies the version of Wireshark that is displaying Help About.

(01 Oct '16, 15:36) grahamb ♦

Wireshark captures network traffic by running the dumpcap program, which is part of the Wireshark package; dumpcap uses libpcap, just as tcpdump does, to capture traffic.

This is why it's strange that tcpdump isn't truncating any packet but Wireshark - which means dumpcap - is truncating packets going from the host to the device.

Perhaps they're using different versions of libpcap.

What do the commands which wireshark, which dumpcap, and which tcpdump print?

What does the command dumpcap -v print (all of what it prints, not just the version of dumpcap itself)?

When you say "install the current dev one (2.2.0 something) via apt-get." what do you mean by "dev" in "current dev one"?

(01 Oct '16, 23:25) Guy Harris ♦♦

Apologies again for the long absence.

I had a zombie copy of Wireshark 1.12.8 that refused to die - whatever I installed, or even when I uninstalled it, 1.12.8 was still there.

I did kill it in the end. Now I've got 2.0.2 installed. Properly.

OK, does it capture truncated now?

I just tried capturing from usbmon2 and simply hitting the NumLock key several times. There are some pairs of lines for control out transactions. For each pair, the first has source as host and destination as 2.3.0, length 49. The packet details show 49 bytes on wire, 49 bytes captured, and I can recognise a control transaction with bmRequestType = 0x21, bRequest = 0x09, wValue = 0x0200, wIndex = 0x0000, wLength = 1 and one data byte (contents 0x01 when NumLock goes on, 0 when off). Good. The second of the pair has "[Packet size limited during capture]" in the Info column. Source is 2.3.0 and destination is host. The packet details say 49 bytes on wire, 48 bytes captured. But this is just an ACK for the out transaction, isn't it? Have any data been truncated? Or is this just a spurious error message from the dissector?

(10 Oct '16, 03:25) davehigton

Discrepancy between "on the wire" and "captured" size is normally not an issue of dissector, at least it would be an issue of the frame dissector, not the usb one. Both sizes are stored in the capture file.

Given the description of your activity, you should see two interrupt transfers per keypress (because the press sends one message and the release sends another one) plus one SEND STATUS (or something similar) for the change of NumLock indicator LED status. What you describe seems to be only the exchange related to LED control.

However, to properly dissect what you now see as generic "USB CONTROL OUT" as a USBHID message SEND STATUS, you need to have the capture running already when connecting the keyboard, because Wireshark chooses the right dissector based on information from the USB descriptors which are only transferred over the bus right after the device gets connected. It should be safe to disconnect the keyboard after starting the capture and connect it back, but if you have another keyboard, better keep the main one connected and use the second one for the experiments.

Not to be misunderstood - this won't change the interpretation of the frame size, it will only improve the dissection of the contents.

Also, please export that single weird frame into a separate capture file, publish that file somewhere in the web and place here a link to it. Cloudshark is popular here, but any file sharing service or your own web site will do. To be sure whether it is a dissection issue or a capture file issue, it needs a hex view to see what is written in the pcap file.

But as you said the issue didn't happen when you used tcpdump, please do exactly the same activity while capturing using tcpdump, and then do the same activity once again while using dumpcap (with the same parameters like tcpdump) to capture.

It's simply weird that the (probably) same libpcap on the same (definitely) usbmon should yield different results.

(10 Oct '16, 04:25) sindy

The packet details say 49 bytes on wire, 48 bytes captured.

So whatever capture mechanism is being used, in that case, is, in fact, truncating the capture at 48 bytes.

Does this happen if you use tcpdump to capture on usbmon2 and save the packets a capture file with '-w', and then read the capture in Wireshark?

Does this happen if you capture on usbmon2 in Wireshark?

What does tcpdump -h print? (The entire output, not just the tcpdump version number.)

What does dumpcap -v print? (The entire output, not just the dumpcap version number.)

What does

ls -l `which tcpdump`

print?

What does

ls -l `which dumpcap`

print?

(10 Oct '16, 23:23) Guy Harris ♦♦

What does tcpdump -h print? (The entire output, not just the tcpdump version number.)

tcpdump version 4.7.4libpcap version 1.7.4
OpenSSL 1.0.2g  1 Mar 2016
Usage: tcpdump [-aAbdDefhHIJKlLnNOpqRStuUvxX#] [ -B size ] [ -c count ]
        [ -C file_size ] [ -E algo:secret ] [ -F file ] [ -G seconds ]
        [ -i interface ] [ -j tstamptype ] [ -M secret ] [ --number ]
        [ -Q in|out|inout ]
        [ -r file ] [ -s snaplen ] [ --time-stamp-precision precision ]
        [ --immediate-mode ] [ -T type ] [ --version ] [ -V file ]
        [ -w file ] [ -W filecount ] [ -y datalinktype ] [ -z command ]
        [ -Z user ] [ expression ]

What does dumpcap -v print? (The entire output, not just the dumpcap version number.)

Dumpcap (Wireshark) 2.0.2 (SVN Rev Unknown from unknown)

Copyright 1998-2016 Gerald Combs <[email protected]> and contributors. License GPLv2+: GNU GPL version 2 or later <http://www.gnu.org/licenses/old-licenses/gpl-2.0.html> This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

Compiled (64-bit) with GLib 2.48.0, with libpcap, with POSIX capabilities (Linux), with libnl 3, with libz 1.2.8.

Running on Linux 4.4.0-43-generic, with locale C, with libpcap version 1.7.4, with libz 1.2.8. Intel(R) Core(TM)2 CPU 6600 @ 2.40GHz

Built using gcc 5.3.1 20160407.

What does ls -l which tcpdump print?

-rwxr-x— 1 root pcap 1106360 Jun  2  2015 /usr/sbin/tcpdump

What does ls -l which dumpcap print?

-rwxr-xr-x 1 root root 88272 Apr  8  2016 /usr/bin/dumpcap

Does this suggest that I still have an out of date version of tcpdump?

(13 Oct ‘16, 13:21) davehigton

What does tcpdump -h print?

tcpdump version 4.7.4
libpcap version 1.7.4
OpenSSL 1.0.2g  1 Mar 2016
Usage: tcpdump [-aAbdDefhHIJKlLnNOpqRStuUvxX#] [ -B size ] [ -c count ]
[ -C file_size ] [ -E algo:secret ] [ -F file ] [ -G seconds ]
[ -i interface ] [ -j tstamptype ] [ -M secret ] [ –number ]
[ -Q in|out|inout ]
[ -r file ] [ -s snaplen ] [ –time-stamp-precision precision ]
[ –immediate-mode ] [ -T type ] [ –version ] [ -V file ]
[ -w file ] [ -W filecount ] [ -y datalinktype ] [ -z command ]
[ -Z user ] [ expression ]

(13 Oct ‘16, 13:29) davehigton

What does dumpcap -v print?

Dumpcap (Wireshark) 2.0.2 (SVN Rev Unknown from unknown)

Copyright 1998-2016 Gerald Combs [email protected] and contributors. License GPLv2+: GNU GPL version 2 or later http://www.gnu.org/licenses/old-licenses/gpl-2.0.html This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

Compiled (64-bit) with GLib 2.48.0, with libpcap, with POSIX capabilities (Linux), with libnl 3, with libz 1.2.8.

Running on Linux 4.4.0-43-generic, with locale C, with libpcap version 1.7.4, with libz 1.2.8. Intel(R) Core(TM)2 CPU 6600 @ 2.40GHz

Built using gcc 5.3.1 20160407.

(13 Oct ‘16, 13:29) davehigton

What does ls -l which tcpdump print?

-rwxr-x— 1 root pcap 1106360 Jun 2 2015 /usr/sbin/tcpdump

(13 Oct ‘16, 13:31) davehigton

What does ls -l which dumpcap print?

-rwxr-xr-x 1 root root 88272 Apr 8 2016 /usr/bin/dumpcap

(13 Oct ‘16, 13:32) davehigton

Does this indicate that I still have an out of date version of tcpdump?

(13 Oct ‘16, 13:34) davehigton

What you describe seems to be only the exchange related to LED control.

Absolutely. The interrupt in message from the keyboard wasn’t in question, so I didn’t mention it. I believe that I’m seeing /all/ messages on the USB, including the control transfer to command the LED status, and a USB ACK (a level that we don’t normally care about) to signal back from device to host that the control transfer succeeded.

(13 Oct ‘16, 13:40) davehigton

It indicates that you have a reasonably recent version of libpcap, and a reasonably recent version of tcpdump, which is using that version of libpcap, and that dumpcap is using the same version of libpcap.

It’s libpcap that matters here.

(13 Oct ‘16, 13:41) Guy Harris ♦♦

Does this suggest that I still have an out of date version of tcpdump?

Your tcpdump is older than your dumpcap but it doesn’t matter as tcpdump is not part of the Wireshark project, so unlike your Wireshark 1.12.x was, your tcpdump is not a zombie as reinstallation of Wireshark does not affect it.

What really bothers me is that, as now confirmed, both tcpdump and dumpcap use the same libpcap (1.7.4) and still give different results. As said, can you please capture the same usb activity into a file once using tcpdump and once using dumpcap with the same parameters (-i usbmon -s 0 -w /some/file/name and compare the results? Doing so should highlight whether the bug is between libpcap and the rest of dumpcap code or, less likely, between dumpcap and Wireshark during live capture.

(13 Oct ‘16, 13:43) sindy

Does this indicate that I still have an out of date version of tcpdump?

On second thoughts: no, according to www.tcpdump.org.

(13 Oct ‘16, 13:43) davehigton

So:

Does tcpdump truncate any USB packets when you capture a particular sequence of traffic, such as the “repeatedly hitting Num Lock” sequence?

Does a capture done with Wireshark do so for the same sequence of traffic?

Does a capture done with dumpcap on the command line do so for the same sequence of traffic?

(13 Oct ‘16, 13:55) Guy Harris ♦♦
showing 5 of 24 show 19 more comments


One Answer:

0

Wireshark on Linux can capture USB traffic without truncating it to 48 bytes, kernel 4.4 is new enough such that you do not have to worry about the historical limitation. Are you sure that your device is not sending less? While comparing libpcap versions is an option, to be absolutely sure you can first whether the kernel gives actually gives you data.

So, check whether the kernel (USB device) really returns more than 48 bytes. To do so, either access the device nodes in debugfs (cat /sys/kernel/debug/usb/usbmon/*u) or invoke ioctls on the /dev/usbmon* devices. (Here, * is the bus number or 0 for a catch-all.)

Pay especially attention to the number before the equals sign. This is the actual data length. For the textual output in debugfs, the data after = is truncated to 32 bytes (see DATA_MAX in the kernel sources, drivers/usb/mon/mon_text.c).

You can run the above experiment while capturing with tcpdump -i usbmon* (or Wireshark), the results should match up. If not, you could try to run strace against the capture process (tcpdump or dumpcap), but if this happens, then something is seriously wrong. Maybe you are capturing from the wrong usbmon device?

Related material:

answered 14 Oct '16, 14:29

Lekensteyn's gravatar image

Lekensteyn
2.2k3724
accept rate: 30%