Switched VS Routed Backhaul

[adairw]

Lets talk about it.
Right now we are all routed everywhere, back haul and AP's and it's fine. But I've been thinking lately about moving to a switched setup just to make things that much faster.

Things I like about our current setup are, it's easy to troubleshoot because a trace route will show you whats up, generally. I can also run OSPF in PTP mode instead of broadcast and forgo a DR/BDR election. Moving to switched would force me back to broadcast or PTMP mode. Early on broadcast gave us some heartburn. moving to PTP fixed those because there is no election. I did learn that I can set a DR and force other routers NOT to participate in the election if I want, which helped, but I don't know if that's good from a design stand point.

Things I like about a switched setup, it's got to be a little faster, how much? I don't know. That's one things I'll have to do some testing on. Plus we run MPLS, never played with MPLS on a big switched network.

I intend to bring up four or five routers in a lab to do some testing. just want to see what others think. All routers are mikrotik and we use VPLS for customer traffic.

[sirhc]

Well we run a routed network with BFD
Each tower has it's own router
Each AP on each tower is in its own VLAN to the tower router
Each customer has their own sub net
Residential customers have their radios in NAT router mode which we lock out to them
Commercial customers radios are in Router not NAT so their radio is their sub-net gateway and their radio has a static route entry in the tower router

Now that being said here is me telneted in a router at VR tower which is SEVEN AF24 hops, seven routers, and seven towers away from my fiber pinging that router.

Pretty dang fast to me! And with BFD OSPF convergence "usually" occurs in 1 ping if a back haul drops out.

ping.png (10.17 KiB) Viewed 23228 times

I fought tooth and nail against going to a routed network but I am not sure if I would ever go back to a flat network if you paid me!

[adairw]

I don't want a flat network either. I will forever be routed, but I see some benefit to the back haul portion being switched. Tower routers would hang off the switched section and AP's still routed. this would make INTER router communication switched speed (basically)

[sirhc]

I think I get what your saying. I will draw up a diagram tomorrow and express my opinion on it.
I am just too burnt out tonight and need to decompress from the past months bringing the WISP Switch to market.....and answering dual threads

Only 4 days off since January!

[adairw]

Hey! at least I've been agreeable to helping work through problems! :)

I want to see these switches take over the wisp market and am glad to help. sadly I don't feel like I can put them in my network yet. Well....maybe now I feel better since vlans do work over the sfp's.

I own a WISP, what's a day off?????

[sirhc]

Hey! at least I've been agreeable to helping work through problems! :)

I want to see these switches take over the wisp market and am glad to help. sadly I don't feel like I can put them in my network yet. Well....maybe now I feel better since vlans do work over the sfp's.

I own a WISP, what's a day off?????

Well to be fair your VLAN SFP issue was fixed in v1.0.4 apparently you just needed to reboot it for the fix to take effect. I am curious as to why, and plan to discuss with Eric when I call him next time. No one else reported this issue since v1.0.4, maybe they rebooted theirs or maybe their SFP modules are different who knows for sure the point is it is fixed.

You found a bug about defaulting not clearing the POE settings and it was fixed within 24 hours.

Our response time to fix issues reported is faster than most all other companies I know, sadly your problem was fixed in the firmware awhile ago which is why we could not reproduce it in our lab mock up of your issue these past couple days as we did not have the issue.

I run a pretty good sized WISP and they have been in service for months with no issues.

At this point there are several hundred in service and I am pretty sure people would be reporting issues if they were having them.

I saw more reported issues with the ToughSwitch and EdgeMax switches than ours when they came out and software updates were not as fast to get fixed.

I get it, we are the small guy, we have to be twice as good and half the cost, being a WISP I am used to that mentality from my customers sadly.

Let's compare.
MUCH better operating temperature range
Passive POE monitoring
Simple UI with almost all the features asked for and those that are not there yet will be soon.
Ability to power high power links like airFIBER
Better graphing with up to 1 hour graphs instantly when you log in
SNMP on FIRST release
Serial Console Port
Removable rotatable ears to mount to wall
17 second reboot time and we leave the POE ports up so radios do not bounce

I mean we listened to everyone, we work all night to fix software bugs when reported, in an emergency you can call someone.
Made in North America (if this does not matter I might move manufacture-ring to China and lower costs substantially)

Well I am at home going through 6 towers and 6 switches and I have no problems at all!

Been that way for months.

I am a real WISP building WISP equipment the way WISPs wanted them to be. You people have an opportunity to support a company that listens or don't and hope the other one will listen. So far not even UBNT listens half the time and they think they know what you need more then you know what you want.

We wanted AF11/18/23 we got AF5
We wanted an AP like the CAP I am building we got the airGATEWAY
We wanted removable ears and serial ports
We wanted switches that powered airFIBER units
We wanted extended operating temperature ranges we got 40C

We listen which is why we have a small 8 and 12 port version in the works
People wanted DC versions, in the works out Q1 2015

But hey all we can do is try and build what you ask for and hope you support us.

[adairw]

it was probably fixed in 1.0.4 as you state. The fact I needed to reboot is good info for you to know. Maybe you can find out why?
Anyway, if I ever sound critical of the product, don't take it personally. I think it's great and I love that you are working to come up with a product that people really want. That's why I'm willing to spend so much time testing. I want it to work for myself and others.
You're doing all the good. go have a beer or ten. :)

[rebelwireless]

Adairw, so are you saying that you would have all of your backhaul/sidehaul links simply switched in a single layer2 using RSTP? what are your routers tower side that you think a switch is going to be measurably faster? How many hops?

[adairw]

Kinda of, but not with RSTP. (yuck).

I have my data center in the middle of 5 main tower sites. from those sites I feed other smaller tower sites. At this point nothing is more than 2 hops away from the data center, even with 20 towers.
So yes, all these primary backhauls would be just switched. We are starting to get more people who are interested in point to point connectivity across our network between locations. Lots of our links are either 11Ghz licensed or AF5/24, so fast, low latency links.
If I'm on the south end of my network and want to connect to location on the north end, I may have to hit 4-5 routers to get there. If it was switched, it would hit the same number of radios but only two routers (the ingress and the egress router).

My asking this question was more to get other peoples thoughts on the matter. I have my own opinion on both ways.

The WISPSwitch got me thinking about things from a reliability stand point. I'm not saying mikrotik routers are junk and unreliable, my whole network utilizes them. But at the same time I also like the idea of being able to plug all my high capacity backhauls in to a switch (the WS) and then hang tower routers off it. If a tik took a shit, at least the back haul would keep on working, etc.

I'm pretty sure that the switched portion would only be marginally "faster", but I think it would be a bit more direct.

I still plan on using the tower routers to build rings to other towers utilizing OSPF. This would allow me to achieve the same things I have now being all routed but also implement the switch setup... *shrug*

[rebelwireless]

MPLS! With Mikrotik you can do MPLS+OSPF+TE+BFD+VPLS to get a fairly fast failover, dynamically routed MPLS network, with layer2 VPN.

If you want to get something more reliable that the mikrotiks, then move up to a cisco, juniper MX, or brocade MLX. The down side is that Mikrotik only supports basic MPLS and TE tunnels, no FRR support :( And getting a Cisco and a Mikrotik to do anything other than OSPF and 'regular' MPLS together isn't going to be fun.

To me it's like do Mikrotik for your entire backhaul networks, or Cisco/Juniper for your entire backhaul network.

Sure would be nice to get MPLS-TE FRR and VPLS on Edgerouters.....