This is our old Q&A Site. Please post any new questions and answers at ask.wireshark.org.

I have been plagued the last few days with terrible performance with one encrypted website that is critical for my business. I have spoken to the website engineers and they see no problems and blame the ISP. The ISP sees no problems and blames the website provider.

What I have found is when using a different ISP I see no problems with performance. I have tried to different locations with the original ISP and both have performance problems. Two different locations with a different ISP have no problems.

I know enough about Wireshark to find it, install it and get a trace. I filtered down to just the client and website and in a 1019 packet https trace I see a few sections with quite a few retransmites and also what seems like quite few resets. There are some big gaps in time while the browser is just "spinning" and the last packet seems to always be a RST,ACK like the following:

1016    98.420253   192.168.1.100   xxx.xxx.138.177 TCP     54  55459 > https [RST, ACK] Seq=1107 Ack=9871 Win=0 [TCP CHECKSUM INCORRECT] Len=0
1367    162.943692  xxx.xxx.138.177 192.168.1.100   TLSv1   1434    [TCP Retransmission] Ignored Unknown Record
1400    163.197565  192.168.1.100   xxx.xxx.138.177 TCP     54  55461 > https [RST, ACK] Seq=1064 Ack=461 Win=0 [TCP CHECKSUM INCORRECT] Len=0
4201    447.512103  192.168.1.100   xxx.xxx.138.177 TLSv1   91  Encrypted Alert

I have been dealing with the problem for 5 days now and everyone is saying all the lights are on it's just that no one is home. I really feel like there is some bizarre issue with my ISP and it is but I do not know how to prove it to myself much less the ISP.

Is this a place to start? Any ideas? Anything I can capture that would be meaningful to present to the ISP?

Thanks!

asked 11 Jul '11, 18:17

3500PT's gravatar image

3500PT
16112
accept rate: 0%

edited 11 Jul '11, 19:45

Guy%20Harris's gravatar image

Guy Harris ♦♦
17.4k335196


If I understand you correctly, with ISP-1 you have performance problems to all locations? And with ISP-2 you have no performance problems at all. First question would be "How are you connected to ISP-1 and ISP-2, any routers, firewalls, other devices in the path?

Then for troubleshooting, you would want to make tracefiles for a couple of sites over ISP-1 and also the same sites over ISP-2. That way, you can show them it's not just one site (that they want to blame). You'd still have to pinpoint the problem in the tracefile before handing the tracefiles over to them.

You could do the analysis yourself, hire someone to do it for you or maybe find someone here to do a basic analysis for you. If you can make the tracefile available somewhere, I can have a quick look for you, but I'm limited in my free time.

permanent link

answered 12 Jul '11, 00:40

SYN-bit's gravatar image

SYN-bit ♦♦
17.1k957245
accept rate: 20%

SYNbit - I am not sure we are saying the same thing. With ISP-1 I only have problems with one particular web site. Everything else is fine including other secured [https] sites. Only this one site gives problems and it is erratic. We are in it all day long. For short periods - a few minutes to maybe 1/2 hour - performance will be acceptable but then the wheels just fall off. It might take 10 minutes to load a screen that normally takes 3 or 4 seconds. This can last a couple of hours.

I did get a Wireshark capture file last night while performance was good but I'm not really capable of looking at the two and figuring out any differences. Plus, it may not be useful since I only captured traffic between the workstation and the website. That may not be useful. I don't know.

Yes, there is a Linksys firewall between the internal network and the cable connection but I have tested by connecting directly to the cable modem and still see the performance issue so I do not think it is a router issue.

I use the same ISP for my residential connection and see the same performance symptoms at home with or without my home router.

I have been to two other locations with a different ISP and worked for hours without seeing the same symptoms while people at the office were having problems. I had them on the phone and we were hitting the same screens simultaneously. Interestingly to me, at the office the website is 9 hops away according to tracert and at the main location I have tested at on ISP-2, it is 15 hops aways.

Only the last two hops are common between the tracerts. The website appears to be hosted with Rackspace. ISP-1 looks to control all the routers up to the handoff to Rackspace.net. I can't really tell if that is true for ISP-2. There is one hop that times out, the next is *.pnap.net and then rackspace with ISP-2.

Could there be some funky peering problem, which seems to be what packethunter is suggesting? What would explain the erratic behavior? BGP I suppose or something else that is causing traffice to traverse multiple routes. Would any of this even show up in a Wireshark capture from my location?

If you did actually look at a trace or two, what would I need to capture to get started? Also, what delivery mechanism is typically used? I don't have an ftp server or anything like that.

I the meantime, I have already ordered a second Internet connection from the ISP that has been working. I'm hoping to have redundancy this way and then direct all the traffic for this one site down the ISP-2 path. It is not really an expense I planned on but it makes sense from a DR or continuity perspective. Any thoughts on a router/firewall for a small business that can handle two Internet connections? I did find a Fortinet 60C that doesn't seem outrageously priced but I don't know much about the company or product.

packethunter - "friendly ISP" - isn't this like a unicorn?

Thanks for the feedback, though!

side note: I really don't want to see what all crap is flowing on my LAN, do I? There sure is a lot of background chatting going on. My first look at a capture showed my machine constantly arping and a ton of other noise. I turned off UPnP on the router and killed some network discovery junk on my PC, leftover I think from the initial setup of the Linksys, but it did not seem to make any difference. One should never look under the hood unless one is prepared to get dirty, eh?

(12 Jul '11, 06:44) 3500PT

Fortunately looking glass servers are easier to find than unicorns: Try http://www.bgp4.as/looking-glasses for a directory. On second thought, intermittent link failure are usually found and fixed by the ISP.

Your workstation spitting out ARP requests sounds like bad news. Did you check for signs of botnet activity, like SPAM transmission?

Keep on hunting!

(12 Jul '11, 08:15) packethunter

I have not done indepth debugging of the arp activity. It is not the biggest item on my todo list right now. I did run malwarebytes and one online a/v with no problems found. I know I have installed several things over time related to networking and there is probably one of those still running I have not yet found. One for sure was the linksys client software which had a util like nmap that does network discovery. I thought killing it would stop the arps but it appears there is something else lurking. It might even be something like the network printer/scanner software trying to find new printers. Hard to say right now.

Is there a util that will tell me exactly what piece of software is doing the talking? I know netstat will do some diag but I'm not terribly familiar with the options on it.

I thought I walked away from all this network analysis well over a decade ago. Sigh.

(12 Jul '11, 11:08) 3500PT

(I changed your "answer" to a "comment", as that's the way this site works best, see the FAQ)

OK, if this is the only site you have problems with and the problems are intermittent. I think having a tracefile of a slow session and one of a normal session will be fine to work with as a start.

(12 Jul '11, 14:03) SYN-bit ♦♦

Unfortunately there is no file upload feature (yet) on ask.wireshark.org. You could use www.cloudshark.org or www.pcapr.net or maybe any other public file sharing site. However, since you masked the ip-addresses in your post, it might not be possible for you to post it publicly. If you want you can mail me the tracefiles and I will have a (short) look at them. My address can be found in my profile...

(12 Jul '11, 14:03) SYN-bit ♦♦

Thanks. I received a call back from a supervisor at ISP-1 saying they have discovered a problem and I am not alone in experiencing the problem. I was not able to get any clues as to what they found. Unfortunately, they also have given me no estimate for time to repair. I'm glad the finger pointing seems to have stopped after six days of trouble but hope they figure out a solution quickly.

(12 Jul '11, 15:27) 3500PT

Good to hear they acknowledged the problem and I hope they will get it fixed soon!

(just for interests sake, you're still welcome to mail the tracefiles if you want some more info on what's visible in them)

(12 Jul '11, 15:34) SYN-bit ♦♦
showing 5 of 7 show 2 more comments

After checking out the tips provided by SYNbit you might want to check your providers stability. A flapping link can cause a certain pain.

Link failures (or more precisely: route tables in the Internet) can be analyzed with a looking glass server provided by a friendly ISP.

Historical views for the last few days can be visualized with the wonderful tool BGPlay, available at bgplay.routeviews.org

permanent link

answered 12 Jul '11, 01:44

packethunter's gravatar image

packethunter
2.1k71548
accept rate: 8%

Your answer
toggle preview

Follow this question

By Email:

Once you sign in you will be able to subscribe for any updates here

By RSS:

Answers

Answers and Comments

Markdown Basics

  • *italic* or _italic_
  • **bold** or __bold__
  • link:[text](http://url.com/ "title")
  • image?![alt text](/path/img.jpg "title")
  • numbered list: 1. Foo 2. Bar
  • to add a line break simply add two spaces to where you would like the new line to be.
  • basic HTML tags are also supported

Question tags:

×69
×55
×53
×5

question asked: 11 Jul '11, 18:17

question was seen: 5,639 times

last updated: 12 Jul '11, 15:34

p​o​w​e​r​e​d by O​S​Q​A