Bizarre switching issues...

[mhoppes]

I've got a bizarre issue I have been chasing for months and can't figure out.

Ubiquiti Rocket AC Gen2s. Netonix switch.

The Ubiquiti Rockets at the first site have a management VLAN. The customer traffic goes through untagged. Throughout the day I can watch pings to the radio drop, but customer traffic keeps flowing. When this happens I also can't get into the GUI of the radio. Usually it's 5-6 pings... but sometimes it's long enough to set off our monitoring (which would require around 30-60 seconds of no pings).

It seems to randomly pick on the radio thats going to have the issue. Sometimes it will be 2 at the same time, other times it will be one then another shortly after. To further add confusion - I have a Ubiquiti router at this site that is handling routing for the customers and management traffic. When I lose pings to the radio, I can still ping it from the router (directly connected on the same subnet). At no time do I lose pings to the switch. I seem to remember Chris mentioning something about this that he saw in his network way back at the beginning of the Netonix era.


I have another site where we are trying to trace down an issue with a Rocket and Netonix. On this one the Rocket and customers are on the same untagged VLAN in the same IP range. When I first power on the Rocket it comes up, and customers attach and it will start passing traffic, but I can't ping it or talk to it for a good 30 minutes. After 30 minutes it can be talked to and is completely happy. During this time the Rocket does not show up in discovery or MAC Table. I have swapped the Rocket with a new one and the same issue happens.

The rocket is on the same subnet as the switch, customers, and other APs on the same switch. In all cases I have restarted the switches and routers with no change. Does anyone have any idea what in the wild world of radio is going on here?

[mike99]

Can you still ping the router IP address use as gateway for the radio (if whole subnet if affected or only the rockets) ?

[mhoppes]

I can ping the router... and when this happens it's only one or two Rockets in the subnet, but the entire subnet.

[mike99]

Strange. Have you look at STP tab to check if the port is forwarding ?

[mhoppes]

STP is turned off.

[mike99]

If it can ping local but not remote, it's normally a routing issue. Gw lost ? Are the rocket configured as dhcp client ?

Any dynamic routing protocol ? If so, is it possible that the're /32 routes from time to time that redirect somewhere else ? Have you done traceroute when it happen to see if the trace go up to the router of the good tower ?

[mhoppes]

Static routing and the Rocket shows:
10.200.56.0/24 dev br1 proto kernel scope link src 10.200.56.23
default via 10.200.56.1 dev br1

[mike99]

First thing I would check is, when it's happen.

if 10.200.56.1 still ping (specifically this IP address of the router).
If yes:
- on 10.200.56.1, the ARP table entry for 10.200.56.23.
- if traceroute up to 10.200.56.23 change through MTR when it's happen.
- ssh to router and, from router, ssh to radio. Then, make a traceroute from radio to anywhere.