Another Cold reboot Issue

[fiberpipe]

I have seen a few issues on cold reboot, we have one switch that reboots ever so often. It is running firmware version 1.3.8 , we have another one with no issues. Any suggestions, this switch is located in a remote area and not accessible during the winter. Please see the log below, model number is ws-12-250-ac with Ubiquiti wireless devices. The switch is almost 2 years old.



Dec 31 17:00:08 kernel: vtss_port: Loaded port module on board Luton26, type 5
Dec 31 17:00:09 system: Setting MAC address from flash configuration: EC:13:B2:61:16:18
Dec 31 17:00:10 system: starting ntpclient
Dec 31 17:00:11 admin: adding lan (eth0) to firewall zone lan
Dec 31 17:00:22 dropbear[741]: Running in background
Dec 31 17:00:27 system: starting ntpclient
Sep 27 15:00:41 switch[787]: Detected cold boot
Sep 27 15:00:42 switch[787]: PoE enabled on port 1, PoE Smart is starting cable check
Sep 27 15:00:42 switch[787]: PoE enabled on port 3, PoE Smart is starting cable check
Sep 27 15:00:42 switch[787]: PoE enabled on port 4, PoE Smart is starting cable check
Sep 27 15:00:42 switch[787]: PoE enabled on port 5, PoE Smart is starting cable check
Sep 27 15:00:42 switch[787]: PoE enabled on port 6, PoE Smart is starting cable check
Sep 27 15:00:42 switch[787]: PoE enabled on port 7, PoE Smart is starting cable check
Sep 27 15:00:43 switch[787]: PoE enabled on port 8, PoE Smart is starting cable check
Sep 27 15:00:43 switch[787]: PoE enabled on port 9, PoE Smart is starting cable check
Sep 27 15:00:43 switch[787]: PoE enabled on port 10, PoE Smart is starting cable check
Sep 27 15:00:47 switch[787]: Port 1 cable check results: Open, Open, Abnormal termination, Open
Sep 27 15:00:47 switch[787]: Port 1 passed PoE Smart cable check, turning on power
Sep 27 15:00:47 switch[787]: Port 3 cable check results: Ok, Ok, Short, Short
Sep 27 15:00:47 switch[787]: Port 3 passed PoE Smart cable check, turning on power
Sep 27 15:00:47 switch[787]: Port 4 cable check results: Open, Open, Short, Short
Sep 27 15:00:47 switch[787]: Port 4 passed PoE Smart cable check, turning on power
Sep 27 15:00:47 switch[787]: Port 5 cable check results: Open, Open, Short, Short
Sep 27 15:00:47 switch[787]: Port 5 passed PoE Smart cable check, turning on power
Sep 27 15:00:47 switch[787]: Port 6 cable check results: Ok, Ok, Short, Short
Sep 27 15:00:47 switch[787]: Port 6 passed PoE Smart cable check, turning on power
Sep 27 15:00:47 switch[787]: Port 7 cable check results: Ok, Ok, Short, Open
Sep 27 15:00:47 switch[787]: Port 7 passed PoE Smart cable check, turning on power
Sep 27 15:00:47 switch[787]: Port 8 cable check results: Ok, Ok, Short, Short
Sep 27 15:00:47 switch[787]: Port 8 passed PoE Smart cable check, turning on power
Sep 27 15:00:47 switch[787]: Port 9 cable check results: Open, Open, Short, Short
Sep 27 15:00:47 switch[787]: Port 9 passed PoE Smart cable check, turning on power
Sep 27 15:00:48 switch[787]: Port 10 cable check results: Open, Open, Open, Open
Sep 27 15:00:48 switch[787]: Port 10 passed PoE Smart cable check, turning on power

[sirhc]

1) Upgrade to v1.4.5rc2

2) Post up screen grabs of Status Tab and Device/Status Tab

Describe the site for me.

1) Type of power
2) Type of tower
3) If you have AC grid power is the AC service ground bonded to the tower ground rods (Very Important)?

Read these posts on grounding:
viewtopic.php?f=30&t=1816
viewtopic.php?f=30&t=188
viewtopic.php?f=30&t=1429
viewtopic.php?f=17&t=1786&start=30#p13447
https://community.ubnt.com/t5/airFiber/ ... rue#M31070

If this tower is so hard to get too maybe put in a new switch before winter sets in and use this switch in a more accessible area if you're concerned about it. Or send it in via RMA and have us check it out. A check up is free

[fiberpipe]

We have not upgraded to the current RC yet, please see below the site information you requested.

We have ac power to the site with a back up power supply, all devices are grounded. We have 7 Ubiquiti Rockets as sectors running on POE from the switch, Air fiber PTP which is running on its own power supply. I have attached a site picture, the devices on the left side in the back ground are not part of out system, they have been removed since this picture has been taken.





We are waiting you response prior to upgrading firmware on the switch and sending screen shots .

THx

[sirhc]

We have not upgraded to the current RC yet, please see below the site information you requested.
We are waiting you response prior to upgrading firmware on the switch and sending screen shots .

I would upgrade as soon as possible.

We have ac power to the site with a back up power supply, all devices are grounded.

You say: "All devices are grounded" - You do realize that GROUND IS NOT GROUND. The question was is your Electrical service ground bonded to your tower ground system?

Your statement usually indicates that a person feels Earth Ground is Earth Ground and there is no difference nor a reason to bond all grounding systems on a site together.

IT IS VERY IMPORTANT TO BOND TOWER GROUNDS TO ELECTRICAL SERVICE GROUNDS WITH A HEAVY WIRE LIKE #2.

If your tower grounding system is not bonded to your Electrical service ground they will have 2 distinct different ground potentials and then the Ethernet cables are that bond and Ground Current will flow THROUGH your switch and cables between the 2 different ground potentials as excess voltage/current will seek the least resistance to ground, sometimes this ground current can be many amps. If your tower ground rods are better than your electrical service ground rods everytime there is a surge from the electrical service your surge protector will clamp that surge to Earth Ground which it now sees a better ground on your tower so that excess voltage and current will traverse your Ethernet cable to reach that ground and POP goes equipment. It also works the other way if the ground potentials are in reverses and there is a Static or ESD charge on the tower and the voltage/current will traverse your Ethernet cables trying to get to the service ground which is seen as less resistant and POP. Sometime this ground current can be not quite enough to damage anything but can cause weird thing s to happen such as a reboot of a device.

Did you read the suggested post on grounding:
viewtopic.php?f=30&t=1816
viewtopic.php?f=30&t=188
viewtopic.php?f=30&t=1429
viewtopic.php?f=17&t=1786&start=30#p13447
https://community.ubnt.com/t5/airFiber/ ... rue#M31070

We have 7 Ubiquiti Rockets as sectors running on POE from the switch, Airfiber PTP which is running on its own power supply.

If the airFIBER is running through the switch in any way you definitely want to upgrade as there is a KNOWN BUG in the airFIBER firmware with Flow Control and not just with our switches but any switch or router. Our newer firmware has Pause Frame Storm Protection which will detect a Pause Frame Storm and DIsable Flow Control on that port if a storm is detected. Or you can simply follow UBNT advice and disable Flow Control with airFIBER radios. Not sure if they have released a fix for this as of yet?

The problem with wireless links and Flow Control is complex as the link capacity can change drastically from environmental conditions and cause a sudden congestion of packets in a switch or router. UBNT was simply sending Pause Frames even unsolicited at the sending device basically as fast as they could generate them any time their buffers filled up which happens almost immediately when condition s change that reduces the capacity of the link such as interference or rain fade is your talking about the 24 GHz model.

I have suggested to them that in the event link capacity is greatly reduced that the radio ignore the fact that Flow Control is Active and simply drop those packets and let TCP/IP end to end protocol recover on its own instead of machine gunning unsolicited Pause Frames at the sending device.

We also mentioned if you have a damaged cable or end that intermittently shorts cable pairs this can cause the switch to reboot as it drops the voltage for a moment. Think about when you turn on a large electrical device in a room and the lights dim a moment. Well with a dead short for a moment the dip in power can result in a reboot.

It is also possible you have a damaged or failed devices which is why I suggested swapping it and see if the problem persists in which case you know to look for a site issue. If the issue stops you can assume the switch is the cause.

All of my suggestions are simple debugging and elimination steps any IT person can walk through to narrow the cause down to a single device.