Throughput Chart reading more than what's flowing through BH

[sirhc]

I am looking at your pictures, you showed me a screen grab of the upper right corner of the switch UI showing port 3 you did not show me a graph of port 3.

As I said we have been down this road several times where people questioned the graphs and in the end they discovered their error.

All we do is take a snapshot of the port statistics every second and compare it to the previous snapshot and calculate the throughput and store them in an array. There is very little chance to get that wrong.

I did go double check my switches last night and confirm they are reporting correctly. If you look below you can see I selected port 1 which is the port connected to the AF24 radio. The graph corresponds correctly to the traffic coming across that AF into and out of Port 1

CLICK IMAGE BELOW TO VIEW FULL SIZE

If you like later today I can call you if you setup Teamviewer on your computer so I can see if I can see what is happening but I have no doubt that our graphs are correct, I even just doubled checked.

[bhesterberg]

I have teamviewer already. Let me know when you want to take a look.

[bhesterberg]

As far as the graph goes, are you telling me that when you click on the port on the status page, that the reading at the top right isn't for that port? Even though it says it's for that port?
So here is the GRAPH from the throughput of port 3, and the GRAPH for the BH that's on that port.

Netonix PORT 3 GRAPH

BH that's on the Netonix PORT3 throughput

[bhesterberg]

So you can see, the BH is saying there's about 150-200mb of traffic, while the Netonix is showing 600-800mb of traffic. I don't even have 800mb of bandwidth available... LibreNMS shows the switch running 26GB at times.

LibreNMS Showing 26gb of throughput on switch

[bhesterberg]

The picture I posted earlier, in my second set of pictures, I captured the top right corner of the switch showing PORT3's throughput, you said that wasn't good enough and you needed the graph. Just so we are clear, when you click on a port on the status page, THAT port's throughput is shown at the top right in the screenshot I took for you. It's the same readout as the pretty graph at the bottom. So I'm not sure why you couldn't use that screenshot. So here is another one, of port 3, with the top right corner AND the graph. See, they are the same.

[sirhc]

As I said how the graphs work is they look at the port statistics every second and compare to the previous second readings to see how much data passed through a port in that 1 seconds. That telemetry is then stored in a simple multi-dimensional array and dumped into an off the shelf graph module used by hundreds of manufacturers including UBNT to generate the graphs.
The port statistics are part of the switch core and we do not write that as it is built in and if they were wrong millions of users from many different switch manufacturers would have reported this bug years ago since this switch core has been in production for almost 10 years now.
The Total Throughput Graph looks at all ports and adds them together to create the Total throughout graph and if you did not deselect any ports will almost always show Tx and Rx as almost identical as traffic in one port goes out another port. Rare occurrences that can affect this are Pause Frames, Errors (retries), packet drops, network broadcast storms, and multi-cast traffic.
If you think about it with multicast packets they can come in one port and out multiple ports and broadcast storms are also duplicated on all ports which can inflate the traffic.
I am telling you that since interface graphs are based on port statistics and they are handled internally and not by us there are few ways if any that it can be wrong.
The only way it could be wrong is if Eric messed up the math which would make all switches report wrong which is why I checked my switches last night and they are correct.
There is no way a hardware failure could cause this stated behavior, your missing something or something on your network is generating traffic you are unaware of?

This switch has been in for about 8 months. Been running on 1.4.7 until this morning.
So this morning I did a backhaul upgrade. Switched from the old UBNT 5gac BH to a Metrolinq ML2.5-60. I was nervous about using 60GHz so I wanted to leave the old 5gac BH in place as a failover. I hadn't done RSTP in the past, but needed to for both BH's to run at the same time. I'll paste the RSTP config page below. Everything worked fine before the upgrade, and it seemed to work fine after as well. It wasn't until about 4 hours after the switchover that I started seeing extremely high throughput on the switch looking at "total throughput". It was showing 500-700mb throughput. I checked the new BH and it was pushing about 120mb. Everything feeds through this new BH so I'm not sure where the extra 500mb is coming from or going to. What else do you want to know? Is this affecting my available throughput on the switch or is it a bug?

So my guess is this has something to do with the new link [Metrolinq ML2.5-60] and or some other misconfiguration with the backup link.

Curious if you tried:
Powering down the new link failing over to the original link and seeing if graph traffic returned to normal?
Then power down the old link only using the NEW link and see what the graph traffic does?

[bhesterberg]

Can we get past the graph thing? I get how graphs work. And if I didn't, your last three posts repeatedly explained it...over...and over.
Look you egomaniac, I'm not saying your switch is bugged. But I AM saying that it's showing traffic that's not being generated from my network and is flowing nowhere else but ON THIS SWITCH. All the backhauls connected to it show one amount of throughput while the switch is now showing between 2x and 3x more than what the backhaul shows.
As far as trying to power down the ML and use the old backup, I just tried it. Same thing. BH shows about 150mb throughput and the Netonix is showing 300-500mb of throughput.

[bhesterberg]

SOLVED:
Talked with Chris for about an hour on the phone and with TeamViewer. Ends up there is an issue, of sorts.
At some point I installed 2 Netonix managers, on different computers and on either side of the switch. Seems this is a bad idea.
We ended up thinking that each manager was fighting for control of the switch and causing lots of traffic. Chris got into the CLI and pulled up the running services and showed many SSH connections, also causing the switch to be extremely sluggish.
Disabled one of the Netonix managers and rebooted he switch. All is well now. Thank you Chris

[sirhc]

SOLVED:
Talked with Chris for about an hour on the phone and with TeamViewer. Ends up there is an issue, of sorts.
At some point I installed 2 Netonix managers, on different computers and on either side of the switch. Seems this is a bad idea.
We ended up thinking that each manager was fighting for control of the switch and causing lots of traffic. Chris got into the CLI and pulled up the running services and showed many SSH connections, also causing the switch to be extremely sluggish.
Disabled one of the Netonix managers and rebooted he switch. All is well now. Thank you Chris

You know this praise will only inflate my ego making me an even bigger egomaniac !
Only 4 more praises like that and I will be a SUPER EGOMANIAC and earn my cape like Al Gore.

Glad your working Brian and I also learned we need to warn people NOT to install 2 managers.