Track streaming from Netflix in near real time

Question

Hello,

I have a question that might be a bit odd, so I will try to explain the application first. I have an object that I would like to change state depending on whether Netflix is being streamed or not. So when Netflix is not being streamed the system is in state A, when streaming starts it transitions to state B and when streaming ends i returns to state A again. I would like it to occur in near real time (20-30 seconds after streaming has started/stopped).

I already have the system setup using Wireshark to monitor other network traffic, so I thought that perhaps it could be possible to utilize Wireshark for Netflix as well. But I am not a network wizard, which means that so far I have concluded that http requests to Netflix.com only occurs when loading the page initially and does not monitor streaming. A bit of research has made me understand that this is because Netflix actually get the content from multiple providers.

Does anyone have a good idea how to do this, or can tell me that it is in no way doable.

Accepted Answer

1

It would mean using Wireshark to analyze the NetFlix traffic to see what would be distinctive for NetFlix traffic (which I have not done myself). Then when you do have discovered some display filter that will match the streaming packets (or just the requests for streams), you can use a while loop to detect the state like this:

while true
do
    if [[ `-i <int> -a:5 -Y <netflix-filter> | wc -l | tr -d " "` == 0 ]]
    then
        echo "State A"
    else
        echo "State B"
    fi
done

answered 27 Oct '16, 16:08

SYN-bit ♦♦
17.1k●9●57●245
accept rate: 20%

Thanks for the answer, I get the way to do it. I am just unsure how to configure the filter, I have identified the package requests and the return answers from the Netflix CDN. They look like this, but I am unsure how to filter them.

alt text

(28 Oct '16, 00:23) Nerq

OK, I fired up NetFlix here in NL and made a trace. As the video streams are SSL encrypted, the only information you can go on are the IP addresses and the SSL handshake. In the SSL handshake, there is a SNI extension that you could filter on:

$ tshark -r netflix.pcapng -Y "ssl.handshake.extensions_server_name contains nflxvideo.net"
10193  39.751633 0.000000 192.168.0.133 → ipv4_1-cxl0-c076.1.ams001.ix.nflxvideo.net SSL 296  64 Client Hello
10196  39.755777 0.004144 192.168.0.133 → ipv4_1-cxl0-c108.1.ams001.ix.nflxvideo.net SSL 296  64 Client Hello
10201  39.764074 0.008297 192.168.0.133 → ipv4_1-cxl0-c108.1.ams001.ix.nflxvideo.net SSL 296  64 Client Hello
10559  39.930507 0.166433 192.168.0.133 → ipv4_1-cxl0-c076.1.ams001.ix.nflxvideo.net SSL 583  64 Client Hello
11454  40.247803 0.317296 192.168.0.133 → ipv4_1-cxl0-c076.1.ams001.ix.nflxvideo.net SSL 583  64 Client Hello
11787  40.328245 0.080442 192.168.0.133 → ipv4_1-cxl0-c076.1.ams001.ix.nflxvideo.net SSL 583  64 Client Hello
11925  40.387909 0.059664 192.168.0.133 → ipv4_1-lagg0-c007.1.ams001.ix.nflxvideo.net SSL 297  64 Client Hello
11926  40.387987 0.000078 192.168.0.133 → ipv4_1-lagg0-c007.1.ams001.ix.nflxvideo.net SSL 297  64 Client Hello
20793  43.554383 3.166396 192.168.0.133 → ipv4_1-cxl0-c108.1.ams001.ix.nflxvideo.net SSL 583  64 Client Hello
$

So you could use that, however, there are sessions that have very little data and sessions that do have data:

$ for stream in `tshark -r netflix.pcapng -Y "ssl.handshake.extensions_server_name contains nflxvideo.net" -T fields -e tcp.stream`
do
    tshark -r netflix.pcapng -Y tcp.stream==$stream | wc -l
done
    2245
   26409
   28133
    1740
     165
      18
     251
      26
   20844
$

Also, I only captured for 1-2 minutes, so I have no idea how often a new SLL session will be set up during the watching of a video stream. I will leave that to you to find out. Then you might need to do a two-step filter process. One to collect the destination IP addresses of the ClientHello messages and put them in a list. Then a second run on the capture to measure the amount of traffic towards these IP addresses.

Please note that I converted your "answer" to a "comment" as that is how this site works best, please see the FAQ for more details.

(28 Oct '16, 01:06) SYN-bit ♦♦

Thanks for correcting my comment to an answer, and thanks for the reply.

I looked at the SSL and thought that it might not happen often enough, so instead I tried with a simple filter looking at the host of the source traffic. It looks like this:

tshark -Y "ip.src_host contains "nflx""

And that actually works fairly well on my laptop giving me a lot of packages while streaming. But for some reason it does not work on the Raspberry Pi that I have setup to monitor traffic between my modem and router. It seems that the Raspberry is resolving the name into an IP for some reason (like I said I am not a networking wizard, so I have no idea why). I am looking into it, but so far I have only seen that the -n flag should not be set, and it is not.

(28 Oct '16, 01:57) Nerq

Could you add -o nameres.network_name:TRUE -o nameres.use_external_name_resolver:TRUE to your tshark command to make sure name resolving is enabled?

You can look at the nameresolving settings that it is using without those two options with:

$ tshark -G currentprefs | egrep "^#?nameres"
#nameres.mac_name: TRUE
#nameres.transport_name: FALSE
nameres.network_name: TRUE
#nameres.dns_pkt_addr_resolution: TRUE
#nameres.use_external_name_resolver: TRUE
#nameres.name_resolve_concurrency: 500
#nameres.hosts_file_handling: FALSE
#nameres.vlan_name: FALSE
#nameres.load_smi_modules: FALSE
#nameres.suppress_smi_errors: FALSE
$

(28 Oct '16, 02:09) SYN-bit ♦♦

I tried doing that, and I still get nothing showing up. The tshark command now looks like this

sudo tshark -o nameres.network_name:TRUE -o nameres.use_external_name_resolver:TRUE -Y "ip.src_host contains "nflx""

Running the other tshark command yields me the following:

#nameres.mac_name: TRUE
#nameres.transport_name: FALSE
#nameres.network_name: FALSE
#nameres.use_external_name_resolver: TRUE
#nameres.concurrent_dns: TRUE
#nameres.name_resolve_concurrency: 500
#nameres.hosts_file_handling: FALSE
#nameres.load_smi_modules: FALSE
#nameres.suppress_smi_errors: FALSE

I talked with a colleague and he said that it probably likely that it is because the Raspberry is not the one doing the actual request, although I do not know if this is true.

(28 Oct '16, 03:47) Nerq

Maybe the Pi version of Tshark is not compiled with a name resolver? What is the output of tshark -v? And do you see name resolving for other IP's if you run sudo tshark -o nameres.network_name:TRUE -o nameres.use_external_name_resolver:TRUE? If you do not filter, do you see traffic to the NetFlix IP addresses coming by the Pi?

(28 Oct '16, 04:16) SYN-bit ♦♦

I found that the command will work

sudo tshark -o nameres.network_name:TRUE -o nameres.use_external_name_resolver:TRUE -o nameres.transport_name:TRUE -Y "ip.src_host contains "nflx""

Thanks for guiding me in the right direction.

(28 Oct '16, 23:44) Nerq

showing 5 of 7 show 2 more comments