NSA sniffing and encryption

These simply wont work or will see dramatically reduced functionality on the subset of traffic that is encrypted.

ACK

I think the impact on Wireshark will be far milder.

Well, I'm not quite sure. Yes, Wireshark (and similar tools) are very useful even if you cannot read the payload. However, based on personal experience, a lot of problems are within the client/server communication (at the application layer) and you won't be able to diagnose those problems with more encryption in place (if that happens anyway).

Those who control end points can still use Wireshark the same way by instrumenting their applications to save either the keying material or plaintext.

Besides some browsers, which client application do you know that dumps session keys? I know virtually none.

So, this will be a bigger problem than it is already today. Sure, some applications can be configured/tricked to accept a intermediate certificate of an SSL proxy. But, it's a hassle today and it will get worse with more encryption in place.

I think the future for network tools are traffic and flow analysis.

That's also what I think. So, there should be good tools available for that special purpose. I know there are good commercial tools already available, but I don't know (yet) a good open source tool for it. Should that kind of functionality be part of Wireshark?

Network zones containing the most important assets will probably be under "Network Security Monitoring" tools such as those included in the Security Onion distro.

ACK

I've already seen sort of this happen when customers decided against link aggregation and implemented RSTP instead, because it is easier to capture a single active link than to deal with multiple aggregated links.

Interesting. My experience is rather the opposite. Nobody thinks about troubleshooting (and/or security) until the first incident, just to realize that the current infrastructure does not help to analyze a problem.

Even if a lot of communication is encrypted we will still troubleshoot.

I'm sure we will, but the way troubleshooting is done will have to change.

There is an old rule I do network analysis when it comes to encryption: "if the encrytpion is the problem, things usually go wrong when the encryption is being negotiated. Once the encrypted tunnel is up, it just works."

Regarding encryption you are right. However, the problem it's not the encryption itself, the problem is failing client/server communication that gets tunneled over a working encrypted communication.

And if you really need to look into an encrypted tunnel you'll need to man-in-the-middle it in some way, or decrypt it with the master keys (which will be a problem when it comes to deploying forward secrecy, I know).

See my comment in the answer of @VIVEKRJG for this. Yes, it's possible and it's hassle today (due to client software restrictions) and it will get worse with more encryption in place.

Sounds like you don't expect any challenges (for troubleshooting tools) coming from encryption, cloud services and network speeds, right?

No, actually I don't expect more than the usual challenge we already face in today's networks. I think that most internal traffic will stay unencrypted - in many cases not because the company wants to support troubleshooting, but simply because implementing stuff like SSL often requires additional steps when setting up servers/applications and admin's are lazy :-)

Or the admins have way too much work to do and just keeping the major stuff running takes all available time :-)

•the ever increasing network speeds (100 Gbit/s ethernet available). (Problem: hard to record data at full speed)

Unless some of the technices used by NTOP like Direct Nic Access PF_RING or similar is GPL:ed and incorporated into libpcap we might see a move to use specialised HW to do high speed capturing. I think the problem may be more of high speed filtering actually as saving 100Gbit/s to disk is a challenge in it self.

is GPL:ed and incorporated into libpcap

libpcap is BSD-licensed, so presumably you mean "un-GPLed" - GPLed code can't be in libpcap. It might be possible to add PF_RING support to libpcap, although I'm curious whether PF_PACKET sockets with TPACKET_V3 would work as well (that also needs non-GPLed code to support it; it's on the list of things for me to look at in my Copious Free Time(TM)).

I share most of your views regarding encryption, yet I still believe that there will be a broader shift to encryption, even on the LAN, as the default mode of future applications will be encrypted, also as a lesson learned from the NSA 'scandal'. If that creates a problem for network troubleshooting depends on the protocols used, the applications and many more factors. However, I tend to believe that it will make quite a few thing more complicated. Even today I often have problems to get the server keys for TLS decryption, due to incompetence, security issues, etc. With DHE ciphers it's even harder, unless you have a client that reveals the sessions keys (not that often in the wild). With more encryption in place (which is good!) it will get even harder.

Regarding network usage and bandwidth: Yes, there are already good commercial (hardware) solutions that are able to filter and/or store capture data at high speeds. But what about open source tools like Wireshark? Will they be able to compete in that field, or do they need new, unique features (like more automatic 'expert analysis modules', automatic flow analaysis, etc.) to be able to still deliver a good value for users in the future?

I view Wireshark as a receiver, or user of those kinds of network capturing technologies rather than as a competitor. Those tools specialize in filtering or sorting flows out of high-capacity links and forwarding the right flows to the right analyzers. They do filtering at very low layers, typically no higher than L4, relying on analyzers (Wireshark, Snort, etc.) to receive the smaller flows to do deeper dives on them.

One area that the open source community still needs to fill, whether as an evolution of Wireshark or as something else, is flow analysis and performance management counters built at that deeper level. Right now we have people digging into Lua or perl/tshark scripting, calling Wireshark dissectors to build application-level counters for all sorts of metrics that no open-source tool other than Wireshark can dissect. When I see the same or similar task being tackled independantly by all sorts of people in the industry (the task of automating packet captures and scanning through time intervals for a count or averaging of a given value of a given field of a given protocol that only Wireshark can decode or count, for the purpose of reporting), it speaks to a need for a new tool or a revisioning of Wireshark's scope as an ad-hoc, session-level analyzer.

So in short, Wireshark is the microscope for a session, dedicated capture/filter/aggregator tools on high capacity links are the birds-eye view that can feed the microscope something to start looking at, but there's something missing in between for analysing application-level flows for the purpose of performance management and reporting.

One area that the open source community still needs to fill, whether as an evolution of Wireshark or as something else, is flow analysis and performance management counters built at that deeper level.

Yep, that's what I'm missing as well.

it speaks to a need for a new tool or a revisioning of Wireshark's scope as an ad-hoc, session-level analyzer.

I agree, that (real-time) monitoring features within Wireshark would be interesting for some people.

swarm my Wide Are Network after I ran a port scan on a suspicious IP (just with the intention of scaring them away)

To scare away the NSA with a port scan sounds like the second most clever idea to me. A better idea would be to steal some secrets documents and publish them. That would really pi.. them off. I hope nobody plans to do such a thing in the near future!!

I honestly think encryption will only make them think you have something to hide.

Well, the only use of encryption is to hide something and if the wish to hide something is suspicious in itself, we should all have glass walls in our homes. After all: If you don't have to hide something .... ;-)

but an encrypted connection that stands out might very well make the analysts hope to catch someone important to get promoted.

How does one encrypted connection stand out more than another, if it is not terminating in the Al-Qaeda headquarters?

Make yourself as boring as possible.

That's not a very clever advice, because that's the tactics of sleeper agents. Due to that they will head for the boring people first!

I wish a good night. And please remember, there is always a big brother to care for you. :-))

I wasn't trying to scare away THEM. Jesus. Just some other suspicious IP. It wasn't one of theirs. They showed up because of that scan. Hopefully that makes more sense.

I think you really took the rest of what I said too far. But the truth is that I notice unusual encrypted connections. I've been doing this for 10 years and I know when someone is waving around more security than normal. You're better off hiding it in ways that are not as obvious. There's a reason book ciphers are becoming popular again: they don't stand out at all.

So those are my thoughts. I guess I was a bit more scatter-brained before. Sorry. =) I'm at least a little tiny bit clever after all these years, I swear! (only a little)

I think you really took the rest of what I said too far.

That was just a bit fun. I should have marked it with <fun></fun> tags ;-)

But the truth is that I notice unusual encrypted connections.

O.K. how do you define unusual encrypted connections?

I've been doing this for 10 years and I know when someone is waving around more security than normal.

Interesting. How do you identify that, based only on the encrypted communication? I mean what is the criteria within the encrypted data?

There's a reason book ciphers are becoming popular again: they don't stand out at all.

book ciphers in electronic communication? Where (in practice) have you seen that lately?

BTW: Regarding the AISRC addresses on you WAN. You said:

Just some other suspicious IP. It wasn't one of theirs.

That could have been faked addresses to scare you away ;-) I do that sometimes to frighten script kiddies if they scan my network :-))

Regards
Kurt

Ahh, okay. I define all communications through what I see normally. It's like seeing a stranger. Google, Akamai, and FaceBook all have predictable protocols that they "like" using. They respond to new webpages the same, and when they change, it's usually something I'd have honestly expected sooner. (like the transfer to https by facebook)

All I can say about book ciphers is that I've met a lot of crypto-nerds over the years, and I know a few people who'd rather not have their vices tracked, or used against them later in court. The idea isn't to be so secure as unnoticed. They work.

The AISRC stuff could have been to scare me away, but I hadn't heard about the Iran nuclear hack yet though, and it was the w3af that I read later was also used there.

Who cares? My point was that there is security in being unnoticed. You can't hack, crack, or spoof being invisible in a crowd because it seems like you're talking about food on twitter.

I accept that as a final statement for this answer ;-)