Major improvement by switching to Netonix

[bobwolcott]

I have been building what I call a campus-wide network at a very rugged industrial site. Originally, the network was created to support a few IP security cameras that the client had me install. At the time, I had no idea how the whole network would grow. I've now got 13-15 very high res (5 & 6MP) cameras for security, 4 or 5 cameras at a lesser 3MP used for process monitoring and I've got parts of the company's automation network running on VLANs across parts of it.

The security cameras are all Mobotix and 2 of the owners live adjacent to the site in houses on different sides of the property. I've set them up with monitoring stations in the main office and each home.

There has been a long term plan to run fiber throughout the site, and we're waiting until the time is right for them to do the underground work so we can finish it, but it's been almost 3 years at this point (Arrgh!). So, in order to get them going I used Ubiquiti 5Ghz link radios and TS-8s for much of the network infrastructure. Oh yeah, they recently asked me to set them up with WiFi to cover most of the site, so I turned to Ubiquiti again and have 12 or 13 Outdoor AC units going up.

One of the key features of the Mobotix system is the iPad app used to monitor the video and sound alarms when something is walking around the plant at night. One of the bosses has been telling me for months that his app would just shut down on him in the middle of the night and would therefore no longer alert him when things go bump in the night. I started a support ticket with Mobotix and their engineers had no answers. They said they had some similar reports and were looking into it, but I never got any resolution.

I found out about SmokePing and made up a Raspberry Pi in an NEMA 12 enclosure. I put it in a central point in the network and had it monitoring paths all over the place. I noticed a high (9%+) loss of packets on the path over to the house where the app was crashing and determined the switch had some part to play in the losses. There's a nearly constant 35Mb/s packet throughput on the link due to the monitoring station in the house.

I couldn't get a TS-8 to replace the existing one and thought I would try a Netonix. After emailing a question about the WISP switches I wound up in a phone conversation with Chris who explained some of the differences between his switches and the TS-8s. The main point that struck me was when he talked about Ubiquiti using a SOHO chipset vs the Netonix that utilizes an enterprise core having more memory for things like combining 100Mb port streams with 1Gb streams. Differing port speeds is part of the topology of my network, because the cameras have 100Mb ports and the link radios have 1Gb ports and I've got cameras out there on switches that are handling the link radios.

So, I ordered a couple WS-12-250-ACs and put them in place of TS-8s in the link. Just like that, my customer's app problems and the 9% packet loss disappeared! Chris was right! Evidently, the iPad app would crash because of the dropped packets. Most IP video is UDP, so the loss of packets didn't really resonate. In retrospect, the app was crashing when some of the TCP handshaking was getting lost.

I've worked on a lot of campus networks, but nothing with the kind of constant load this one has, so this was a real eye opener for me. I really am grateful that Chris took the time to explain some of the differences in switch architecture and how they affect performance.

Anyway, I thought my experience might be of some help to others out there.