router retransmitting packets

Question

I would like help figuring out the cause & solution to a packet retransmission issue. I am getting lots of "TCP out-of order", "TCP DUP-ACK", & "TCP Retransmission". This occurs mostly (90%) between two devices communicating within the same VLAN. So client A (192.168.12.151) sends message to client B (192.168.12.100), through a Sonicwall router (192.168.11.200). Sonicwall router set up 4 VLANS, trunk to Layer 3 switch. Cisco switch has 4 VLANs.

You can see packet 8 send the packet from 151 to 100. But, packet 9, the router replaces the mac address, the source is now the router

Any ideas?

Packet 8
Internet Protocol, Src: 192.168.12.151 (192.168.12.151), Dst: 192.168.12.100 (192.168.12.100)
Ethernet II, Src: Crestron_2a:0c:c6 (00:10:7f:2a:0c:c6), Dst: Crestron_1f:3f:42 (00:10:7f:1f:3f:42)
8   1.049321    192.168.12.151  192.168.12.100  TCP 49157 > crestron-cip [PSH, ACK] Seq=1 Ack=1 Win=62 Len=5
Packet 9
Internet Protocol, Src: 192.168.12.151 (192.168.12.151), Dst: 192.168.12.100 (192.168.12.100)
Ethernet II, Src: Sonicwal_90:ee:c2 (00:17:c5:90:ee:c2), Dst: Crestron_1f:3f:42 (00:10:7f:1f:3f:42)
9   1.049394    192.168.12.151  192.168.12.100  TCP [TCP Out-Of-Order] 49157 > crestron-cip [PSH, ACK] Seq=1 Ack=1 Win=62 Len=5

Answer 1

0

I'm assuming here that you have a pretty small subnet defined to force the traffic through the SonicWall box. What you're seeing is 100% normal. L3 routers replace the MAC as the IP packet traverses through it. They also decrement the TTL in the IP header, which I'm sure you'll see as well.

Chances are, you are capturing the same packet twice (as evidenced by the different mac address but same packet) and Wireshark is interpreting it as a retransmission (because it saw it twice).

So try capturing from just one subnet and see where that takes you. You didn't mention the original problem, though. Are you trying to troubleshoot a specific problem?

answered 22 Jan '13, 08:10

hansangb
791●2●6●19
accept rate: 12%

I am seeing the comms between two devices drop out, then come back anywhere from 20 minutes to 1.5 hours later.

The main device in question is an automation controller. it uses IP tables to establish rules for "allowed" connected clients. The controller has about 20 devices that all but 5 stay connected. The controller stays online, but comms between these 2/3 devices only keep dropping. Here is a error log from controller exhibiting the same exact issue.

PAC2>ipt
IP Table:
CIP_ID  Type    Status     DevID  Port   IP Address/SiteName
    E0  CIP     UNKNOWN           41794  192.168.012.102
    F9  CIP     ONLINE            41794  172.031.001.140
    FA  CIP     ONLINE            41794  172.031.001.140
    FB  CIP     ONLINE            41794  172.031.001.140
    FC  CIP     ONLINE            41794  172.031.001.140
    FD  CIP     ONLINE            41794  172.031.001.140
PAC2>

Warning: Ethernet Device Slot-03.IP-ID-FA Not Responding (IP = 172.31.1.140). Device Offline
TimeStamp: 11:18:53  1-17-13   UpTime: 5 days 19:50:23.78  Task: CIPretx
Warning: Ethernet Device Slot-03.IP-ID-FB Not Responding (IP = 172.31.1.140). Device Offline
TimeStamp: 11:18:56  1-17-13   UpTime: 5 days 19:50:27.18  Task: CIPretx
Warning: Ethernet Device Slot-03.IP-ID-FC Not Responding (IP = 172.31.1.140). Device Offline
TimeStamp: 11:18:56  1-17-13   UpTime: 5 days 19:50:27.38  Task: CIPretx
Warning: Ethernet Device Slot-03.IP-ID-FD Not Responding (IP = 172.31.1.140). Device Offline
TimeStamp: 11:18:57  1-17-13   UpTime: 5 days 19:50:27.58  Task: CIPretx
Notice: CIP device fa: back ONLINE
TimeStamp: 11:19:52  1-17-13   UpTime: 5 days 19:51:22.47  Task: UDP_Serv
Notice: CIP device fb: back ONLINE
TimeStamp: 11:19:52  1-17-13   UpTime: 5 days 19:51:22.48  Task: UDP_Serv
Notice: CIP device fc: back ONLINE
TimeStamp: 11:19:52  1-17-13   UpTime: 5 days 19:51:22.48  Task: UDP_Serv
Notice: CIP device fd: back ONLINE
TimeStamp: 11:19:52  1-17-13   UpTime: 5 days 19:51:22.49  Task: UDP_Serv
Warning: Lost Ethernet link on LAN A. (flags = 1)
TimeStamp: 11:44:12  1-17-13   UpTime: 5 days 20:15:43.21  Task: EnetLISR
Warning: IP Address released/expired from server 192.168.202.20
TimeStamp: 11:44:13  1-17-13   UpTime: 5 days 20:15:44.22  Task: CIP Ping
Warning: Ethernet Device Slot-03.IP-ID-FA Not Responding (IP = 172.31.1.140). DHCP Address was Released.
TimeStamp: 11:44:13  1-17-13   UpTime: 5 days 20:15:44.23  Task: CIP Ping
Warning: Ethernet Device Slot-03.IP-ID-FB Not Responding (IP = 172.31.1.140). DHCP Address was Released.
TimeStamp: 11:44:13  1-17-13   UpTime: 5 days 20:15:44.24  Task: CIP Ping
Warning: Ethernet Device Slot-03.IP-ID-FC Not Responding (IP = 172.31.1.140). DHCP Address was Released.
TimeStamp: 11:44:13  1-17-13   UpTime: 5 days 20:15:44.25  Task: CIP Ping
Warning: Ethernet Device Slot-03.IP-ID-FD Not Responding (IP = 172.31.1.140). DHCP Address was Released.
TimeStamp: 11:44:13  1-17-13   UpTime: 5 days 20:15:44.26  Task: CIP Ping
Notice: Link established on LAN A
TimeStamp: 11:45:37  1-17-13   UpTime: 5 days 20:17:07.57  Task: PhyInitA
Notice: New Address obtained from 192.168.202.20
TimeStamp: 11:45:38  1-17-13   UpTime: 5 days 20:17:08.59  Task: DHCPRedo
Notice: Primary Hostname Resolution Configured To Use DNS
TimeStamp: 11:45:40  1-17-13   UpTime: 5 days 20:17:11.33  Task: CIP Ping
Total Errors Logged = 2110

(22 Jan ‘13, 09:01) Scubagreg67

The ip addresses in your capture file, are nowhere in the logs !?! So, how are the two pieces of information related to each other?

(22 Jan ‘13, 15:07) Kurt Knochner ♦

Did this ever get resolved I am having similar errors with a crestron device.

(24 Feb ‘15, 13:14) yert