remote capture with tcpdump and named pipe

Question

I set a remote capture to a host using tcpdump and a named pipe like so

ssh -t [email protected] "tcpdump -s 0 -n -w - -U -i eth0 not port 22" > /tmp/remotecapture.fifo
wireshark -k -i /tmp/remotecapture.fifo

and tried some variations that resulted in

· no packets being read by Wireshark
· Frame 1 too long (1936288800 bytes)
· No packets captured! As no data was captured, closing the temporary capture file!
· wrong libpcap format errors

Redirecting to a simple file does work but not in a format Wireshark can read later.

When Wireshark threw an error regarding libpcap format, I also tried clipping the first line that's passed before the raw packets, to no avail though.

tcpdump: listening on eth0, link-type EN10MB (Ethernet), capture size 65535 bytes

EDIT TL;DR
check Guy Harris' answer below and my comment on 14 Feb
· root account
ssh [email protected] "tcpdump -s 0 -n -w - -U -i eth0 not port 22 2>/dev/null" > /tmp/remotecapture
· non-root account
ssh [email protected] "echo thePassword | sudo -S tcpdump -s 0 -n -w - -U -i eth0 not port 22 2>/dev/null" > /tmp/remotecapture

Answer 1

You are probably writing to a regular file /tmp/remotecapture.fifo. So, did you create a named pipe with?

rm /tmp/remotecapture.fifo
mkfifo /tmp/remotecapture.fifo

If no, please do so before you write to it with tcpdump, plus: let Wirshark read from the fifo first.

rm /tmp/remotecapture.fifo
mkfifo /tmp/remotecapture.fifo
wireshark -kni /tmp/remotecapture.fifo&
ssh -t [email protected] "tcpdump -s 0 -n -w - -U -i eth0 not port 22" > /tmp/remotecapture.fifo

BTW: make sure, that the SSH command does not write STDOUT messages into the fifo, as Wireshark would not be able to read that!

See also:

https://ask.wireshark.org/questions/23609/remote-capture-via-ssh-and-pipe
https://ask.wireshark.org/questions/26523/wireshark-named-pipes-disconnect
https://ask.wireshark.org/questions/13059/capturing-from-multiple-pipes

Regards
Kurt

Answer 2

0

ssh sucks.

Truly, ssh sucks. If you run a command on a remote machine that writes to the standard output and error, both the standard output and the standard error of the command get sent to the local ssh command's standard output!

This means that tcpdump's "hi, I'm capturing" message, which it sends to the standard error, gets sent to ssh's standard output, which means it gets written into the middle of your capture file.

So try discarding the extra "hi, I'm capturing" message:

ssh -t [email protected] "tcpdump -s 0 -n -w - -U -i eth0 not port 22 2>/dev/null" > /tmp/remotecapture.fifo

answered 26 Jan '16, 16:53

Guy Harris ♦♦
17.4k●3●35●196
accept rate: 19%

Thanks for pointing that out. I did try clipping the first line, but redirecting to null on the remote host is the right way. That said, I only see a couple of arp and icmp packets, although there's an active ping process to the interface being dumped. Next I see a spurious frame that seems to be two hex digits off, 08:0:c:29:80:e0 instead of 0:c:29:80:e0:25, and not long after that a message like Frame 4 too long (6422539 bytes). If I leave not port 22 out, it fails almost immediately with a similar size error. If I don't redirect to named pipe I see the raw output in stdout, as expected.

(27 Jan '16, 02:57) wsk

Did you actually try this with tcpdump?

(29 Jan '16, 18:19) wsk

Did you actually try this with tcpdump?

Yes.

(29 Jan '16, 18:36) Guy Harris ♦♦

It still doesn't work for me. In case it's something to factor in, I ssh from a host running OS X to a guest vm running Ubuntu, and between Ubuntu vm guests. Any ideas on how could I further troubleshoot this?

(02 Feb '16, 01:57) wsk

I have just tried the following:

ssh [email protected] "tcpdump -i eth0 -w - 2> /dev/null" > /tmp/test.cap

After logging in, waiting for some traffic to be captured, and then pressing ^C, I've open the test.cap file in Wireshark, and its only complaint was that the file was cut short in the middle of a packet. Can you try the same (and, just for the case that it was not clear, leave a space between the - and the 2) to check whether the source of your issue is the ssh or the named fifo?

(02 Feb '16, 02:29) sindy

Any ideas on how could I further troubleshoot this?

Try running the command ssh -t [email protected] ": 2>/dev/null" >/tmp/hello and then see what's in the file /tmp/hello.

(02 Feb '16, 02:35) Guy Harris ♦♦

@sindy, I get the same message, it's still not clear to me where's the problem though.

@Guy Harris, with that command the session is immediately closed, /tmp/hello is 0B.

The fifo is written to, but this seems to happen not as expected (see post from 27 Jan, 02:57 above).

Any ideas?

(09 Feb '16, 18:41) wsk

@sindy, I get the same message, it's still not clear to me where's the problem though.

I.e., you get the same "Frame 1 too long" error if you follow @sindy's instructions and then try to open /tmp/test.cap in Wireshark?

If you do

ssh [email protected] "tcpdump -i eth0 -w /dev/null 2> /dev/null" > /tmp/test.cap

what are the contents of /tmp/test.cap?

(09 Feb '16, 18:52) Guy Harris ♦♦

I tested this on Ubuntu Server 14.04.3 LTS.

@sindy, if I run the following command

ssh [email protected] "tcpdump -i eth0 -w - 2> /dev/null" > /tmp/test.cap

I can open test.cap with Wireshark and get this message

The capture file appears to have been cut short in the middle of a packet.

@Guy Harris, if I run

ssh [email protected] "tcpdump -i eth0 -w /dev/null 2> /dev/null" > /tmp/test.cap

test.cap's size remains 0B.

(11 Feb '16, 15:06) wsk

OK. Both tests seem to indicate that there must have been some typo in your original attempts, because:

2> /dev/null prevents tcpdump's stderr from being grabbed by the ssh session at the server and thus intermixed with the contents of the capture file at the client (as @Guy Harris wrote, ssh mixes stdout and stderr together at server end so you cannot split them back at client end)

-w /dev/null 2> /dev/null indicate that stderr should go to trash and the binary pcap data should also be written to trash instead of stdout (while -w - means to use stdout for the binary pcap data). So if tcpdump would be printing something to stdout, it would be the only output remaining, but the resulting file size is 0 bytes.

But there is still some space left for doubt.

The capture file appears to have been cut short in the middle of a packet is a consequence of the fact that you've stopped the ssh and tcpdump together by pressing ^C, so both processes were killed abruptly, not gracefully. You need to be extremely lucky that only part of the last packet would not be written to the output file in this case.

So please repeat both tests above with additional parameter of tcpdump, -c 50. This will make tcpdump stop the capture after first 50 packets.

If the result of "my" test is a file of 50 packets about which Wireshark does not complain, and the result of @Guy Harris' test is still a 0 B file, you know for sure that both tcpdump and ssh are OK if you use tcpdump with -w - and 2>/dev/null.

Finally, try again with the named pipe, first also with the -c 50. I never tested with a named pipe on linux, only on Windows, where the capturing process waits for the receiver to connect to the pipe, then starts writing to it, and when the receiver disconnects again (which is a consequence of pressing "stop" in Wireshark), the capturing process terminates. Here, with ssh between the two, it may behave differently.

(11 Feb '16, 22:31) sindy

If I run

ssh [email protected] "tcpdump -c 50 -i eth0 -w - 2> /dev/null" > /tmp/test.cap

the resulting file can be opened in Wireshark without the 'packet cut short' warning.

If I run

ssh [email protected] "tcpdump -c 50 -i eth0 -w /dev/null 2> /dev/null" > /tmp/test.cap

the resulting file has size 0B.

If I leave out option -t it works

wireshark -k -i /tmp/remotecapture &
ssh [email protected] "tcpdump -s 0 -n -w - -U -i eth0 not port 22 2>/dev/null" > /tmp/remotecapture

And with a non-root account use this

ssh [email protected] "echo thePassword | sudo -S tcpdump -s 0 -n -w - -U -i eth0 not port 22 2>/dev/null" > /tmp/remotecapture

I couldn't find a better alternative for the latter, please let me know your suggestions.

Instead of option -U you can prefix tcpdump with stdbuf -o 0.

For the sake of readability I added a tl;dr to my question.

(14 Feb '16, 06:23) wsk

showing 5 of 11 show 6 more comments

Answer 3

0

There is also the 2.1.x extcap plugin sshdump that runs tcpdump or dumpcap remotely at the end of an ssh tunnel, although that only seems to work on linux at the moment.

answered 02 Feb '16, 05:32

grahamb ♦
19.8k●3●30●206
accept rate: 22%

More detailed info about this plugin, or extcap in general for that matter, seems hard to come by. Will this ever be available for other systems?

(09 Feb '16, 18:48) wsk

Unfortunately user visible documentation for extcap is still lacking, despite requests. There is README.extcap in the sources\docs directory.

Basically extcap allows plugin processes external to Wireshark to provide "capture" interfaces. The sshdump plugin makes an ssh connection to the remote machine, runs tcpdump\dumpcap and provides the returned packets to Wireshark. To make it run on OSX would require some work on the OSX build machine to add libssh (presuming there is a version of the library that runs on OSX), and the Windows version will compile but I can't persuade it to actually capture successfully, see change 12117.

(11 Feb '16, 07:43) grahamb ♦

Thanks for the reference. libssh can be easily installed via MacPorts. I'm actually looking for a solution with a generally installed command like tcpdump.

(11 Feb '16, 15:15) wsk

libssh is need to build Wireshark (and sshdump), not on the remote machine. The remote machine only needs tcpdump or dumpcap to use sshdump.

(12 Feb '16, 02:24) grahamb ♦

Answer 4

Seeing as this is actually fairly recent... For what it's worth, I'll leave here my preferred one-liner solution for live remote capture on non-standard ssh ports using Windows' version of wireshark and standard tcpdump on whatever flavor of Linux the remote machine has. It doesn't use the questionable plink (whose main disadvantage is that its flags don't map over to the far more widespread ssh's...), but instead relies on cygwin/standard ssh command. It doesn't strictly require you to use a key to authorize, but if you don't, it stops being a one-liner.

Also, it isn't hugely efficient in that some of the packets get lost to whatever perils there exist in ssh tunnels. Yet it allows you to benefit from the convenience of Windows Wireshark while utilizing the power of Linux's tcpdump.

I experimented with opening remote/local tunnels seeking to capture and display all the packets that came, even if they would be displayed with a delay, and so far this so the most accurate representation. Still, not ALL the packets show up on the capture, compared to a local file tcpdump dump. For various reasons of convenience, I can't / don't want to use named pipes unless they can be incorporated into a one-liner. The quest continues.

(Unconvinced about gzip|gunzip, but so far my tests show that fewer packets get lost this way, at the cost of speed and latency of course. Looking into stdbuf at the moment.)

Anyway, using cygwin with standard packages and either replacing or defining anything in caps, you'd run:

ssh [email protected]_HOST -p REMOTE_PORT -i "LOCAL_CYGWIN_PATH_TO_PRIVATE_KEY" -o CheckHostIP=no -o StrictHostKeyChecking=no "tcpdump -ni REMOTE_INTERFACE -s 0 -w - not port 22 and not port REMOTE_PORT" | "LOCAL_CYGWIN_PATH_TO_WIRESHARKEXE" -k -i -

If you use the standard port 22, it gets simpler:

ssh [email protected]_HOST -i "LOCAL_CYGWIN_PATH_TO_PRIVATE_KEY" -o CheckHostIP=no -o StrictHostKeyChecking=no "tcpdump -ni REMOTE_INTERFACE -s 0 -w - not port 22" | "LOCAL_CYGWIN_PATH_TO_WIRESHARKEXE" -k -i -

If you encounter a problem with some packets getting lost, inexplicably, I've found that gzip" | gunzip helps prevent some of it.