v1.5.23 Bug Reports and Comments

[yahel]

Update.

For the switch which does not forward Multicast on P14 - we just replaced the fiber and SFP modules with an AOC cable.
No change - still no OSPF peering over that interface.


Yahel.

[sakita]

Did one more NTP test.

Disconnected the cable to the NTP server and powered up the switch (switch configured with NTP enabled). Let it run for about 15 minutes or so without the clock set.

Plugged the NTP server back in and then logged back into the switch and NTP disable / apply and NTP enable / apply. The countdown popup came up and started counting. After a few seconds I hit browser refresh to get back to the login page. Logged back in and got the change confirmed popup. NTP was still enabled and the clock was set.

Seeing RTGLW's comments from a few days ago about doing the disable / apply then enable / apply on SNMP and having it revert back after the timeout seems similar. Like the newer version acts different in that disable / apply / enable / apply when logged in didn't used require logging back in to verify the change (e.g. re-logging in was required for IP address change but not some other changes).

Edit Additional Info: NTP disable / apply just works. It is NTP enable / apply that requires logging back in to confirm the configuration change. I accidentally discovered that if the NTP server is still disconnected NTP disable / apply just works and so does NTP enable / apply. If the NTP server is accessible then configuration change confirmation is needed.

Also, I just noticed that there aren't any logon or logoff entries in the switch log (this is direct login using the default credentials as this is a lab setup).

[sirhc]

Update.

For the switch which does not forward Multicast on P14 - we just replaced the fiber and SFP modules with an AOC cable.
No change - still no OSPF peering over that interface.


Yahel.

This is where I am confused with your issue with OSPF and SFP as my entire OSPF ring uses SFP ports. I can share my screen while we talk about it and look into switches and routers as well.

Do you have skype?

[Stephen]

Update.

For the switch which does not forward Multicast on P14 - we just replaced the fiber and SFP modules with an AOC cable.
No change - still no OSPF peering over that interface.


Yahel.

Hi Yahel, so, since we haven't been able to identify the source of this trouble yet. Would you be able to swap out this switch with a different one to see if the issue is persistent across switch's in this place in the network?
I know this request is more disruptive than the others taken so far, if unable to do so right now that is OK.

[sirhc]

Update.

For the switch which does not forward Multicast on P14 - we just replaced the fiber and SFP modules with an AOC cable.
No change - still no OSPF peering over that interface.


Yahel.

Hi Yahel, so, since we haven't been able to identify the source of this trouble yet. Would you be able to swap out this switch with a different one to see if the issue is persistent across switch's in this place in the network?
I know this request is more disruptive than the others taken so far, if unable to do so right now that is OK.

We are under the impression that you have other identical switches with no issue, so replacing it with the same model to see if problem persists or goes away.
If problem goes away and under warranty send back to us or if out of warranty Julian can probably fix it.

And as we discussed you already tried different SFP modules, same as the ones in other WS-12 units that don't have the issue?

Or do I have this all wrong?

[oeyre]

Probably worthwhile everyone noting their WS model & SFP vendor/model to get some kind of good/bad list for the OSPF issue.

Bad:
WS-12-250-AC (Board Rev F)
1.5.22 & 1.5.23 (works with 1.5.16)
FibreStore SFP-10G-DAC

[yahel]

Chris,

I don't have Skype - I'm happy to jump on a Zoom call (I do have Zoom).

yes - we have hundreds of similar switches like the one giving us problems -- this our most popular model: WS-12-250-DC
So far, we have upgraded about 20 switches to 1.5.22 (or 1.5.23) - and they work fine.
HOWEVER - none of the ones upgraded uses SFP ports at all!!

The first one that we've upgraded that use SFP ports, is the one giving us problems - we have therefore stopped all other upgrades until this gets resolved.
We use SFP ports in most of our switches - so this would be a problem if not isolated to a single switch (which we don't know).
All other switches are running 1.5.14

We could, try to upgrade some more switches that use SFP -- perhaps we could identify some instances where the SFP port is non-critical or is a part of a LAG with a regular port -- I think it would be helpful to try this... See if the problem persists in other locations...


More tomorrow - we'll try upgrading a couple more switches tonight.

Yahel.

[sirhc]

@Yahel

I run OSPF ring in and out of each tower uses an SFP port bonded to a copper port. SFP Copper port goes to router (Cisco 2951) and copper port powers backhaul mostly AF24, AF24HD, or AF5XHD

I have no issues.

I assume you run VLANs?

Are you running PPPoE, I forget?

So here is my network (OSPF Ring) and 1 of my towers switch TABs that have any changes, see pic below:
Notice Port 1 is bonded (VLAN) to Port 25 - Bandwidth IN
Notice Port 2 is bonded (VLAN) to Port 26 - Bandwidth OUT

Ports 23 & 24 run a Static LAG to Cisco as 2951 will not support LACP. This just provides backup and load balance which is not really needed but hey.

Note: Whereas 2 towers (BV and VR - lightly loaded) still run original Rocket M5 most towers including DA screen shots are Rocket 5 AC Lite or AirPrism Gen 2, just never went back and renamed port descriptions.

[oeyre]

SFP port bonded to a copper port

So for your use case its acting like a media converter? Or are both sides copper?

You said you use Cisco genuine, that's Finisar, right? What models?

[sirhc]

SFP port bonded to a copper port

So for your use case its acting like a media converter? Or are both sides copper?

You said you use Cisco genuine, that's Finisar, right? What models?

Not a media converter as SFP Port 25 is copper SFP to Cisco 2951 and Port 1 offers 48VH to power AF24HD (LOOK AT MY SCREEN SHOOTS ABOVE)
SFP Port 26 is also copper to Cisco 2951.


Port 1 and Port 25 are in a private untagged VLAN, again see screen shoots. (Backhaul from previous tower in loop)

Port 2 and Port 26 are in a private untagged VLAN, again see screen shoots. (Backhaul to next tower in loop)

In the old days when my WISP was bigger, we had 2 loops or Areas and some towers had 3 or 4 backhauls in which case we would use:

Port 3 48VH to AF24 and port 5 to Cisco 2951 in a private untagged VLAN

Port 4 48VH to AF24 and port 6 to Cisco 2951 in a private untagged VLAN

I only lease a few poles and run fiber from my office to HC Tower, about 1 mile of fiber. But those fiber plug into Cisco 2951 fober SFP modules.

No I do not just use CIsco SFP modules, most of my modules are Cisco compatible from Amazon. Its a crap shoot, that's why I always bench test them first.
I put them in I get a link, I do a firmware upgrade or 2. I do a warm reboot or 2. I disable then enable port a couple times.

Now up until v1.5.22 I had many more SFP modules fail that test now I have only found 1 or 2 but I think they are just OLD and warn. Now Cisco brand SFP modules almost 99.9% worked fine.

Hope this clear this up but again slowly review my pictures of my TABs above to get a better idea what I'm doing.