Link Speed Aware QOS

[josh]

You have to keep latency *incredibly low* on the device to have accurate metrics through multiple devices, which basically screams for an ASIC or an FPGA. Your statistics will only be good as long as your device latency is low and stable(1). All of the devices will need layer 2 connectivity.

Basically, you'll just use a version of "mac ping" at a ridiculous data rate. This is similar to how an AirFiber floods the link with data, but gives "actual" traffic a much higher priority.

You can use this info to show latency, jitter, and calculate a BDP (Bandwidth Delay Product) for roughly determining capacity (see 1).

I would be very careful to avoid certain patents that exist for things like this though.

That said, I kinda think this idea is crap, no offense. QoS on modern networks is incredibly difficult, because so much traffic ends up using common ports, particularly http and https. There are ways to "kind of" determine "what" the traffic is by looking at certain parts of the SSL handshake, or by doing real-time traffic analysis via signature detection, but coding that takes a lot of time and dedication to keep right (as well as updated). QoS via source is also difficult, because so much traffic transits via CDN these days, and those IPs change all the time. You really need DPI to perform proper QoS in 2014, and keep in mind though that QoS only has much of an effect whenever the link is saturated, otherwise you're just looking at FIFO traffic patterns.

You could QoS common things via DSCP mapping tablets, but Ubiquiti radios (and others) are already doing that.
(I was the maintainer and corrector of the following table for quite some time: http://wiki.ubnt.com/AirMax_-_QoS_DSCP/TOS_Mappings )

[josh]

Side note:

The Metro-E folks (MEF and others) use ITU Y 1731 and then there is IEEE 802.1ag. Both are pretty similar. I'd look into those if you still want to investigate this.

[rebelwireless]

josh, I think the problem with those is that they (pretty much identical specs) are very much tuned for known link speeds and fault detection. The variable of changing link conditions I think is too much for them to handle.

I think a good dynamic shaper for ptp links plus something like MPLS-TE allowing for 'spillover' to the secondary path would go a long way......

again, it's the ever changing link quality and no real clear way to identify what is truly available without brute forcing it like AF. I like statistics though, just track it and adjust the shaper to known throughput for specific readings from the radio.

I think that this would work well. We know that 50Mbps aggregate is good at x signal, noise, etc. if anything changes look up the history of the new condition and adjust, or guess based on previous numbers.

[sirhc]

Well I love Josh and all but I have to disagree that what we want to do is not crap and will work as planned.

We have done some preliminary testing and if we did not feel we could it we would not try.

[josh]

If you're just using DSCP...fine, whatever, that's not difficult (assuming you don't strip DSCP at your provider edge). If you plan on Netonix writing and updating it's own signatures and behavior detection templates and algos... that may get pretty expensive.

[rebelwireless]

Nah, there's an open product that can do this that is actively maintained in nDPI, just use that.

[Rory]

Yeah, and on AirFiber links this would be simpler. It is my understanding that the AF products get their speed ratings by keeping the link saturated and measuring the actual throughput. Assuming that is true, it would be a simple matter to read those measurements and adjust the QOS accordingly.

Obviously units that do not perform measurements in that manner would require more thoughtful software engineering to generate the same results. But I would be willing to bet that some time spent in a lab running tests comparing actual throughput vs the wireless link speeds & ccq / noise levels / etc would allow us to come up with a workable algorithm that would function in *MOST* environments. Of course this would need to be tunable, but I really do think that this idea could allow for networks running with fluctuating interface speeds to operate at the same jitter and latency levels as if the interface speeds were static, and that would be leveling the playing field between the fixed wireless industry more traditional data services.

[josh]

Nah, there's an open product that can do this that is actively maintained in nDPI, just use that.

What are you talking about?

There's ntop, and ntop-ng.

ntop-ng is the commercial version of ntop, which is also much newer but can get spendy depending on what you want. That said, it's performance with PF_RING and some other neat tricks on modern xeons is pretty impressive (considering the lack of dedicated hardware).

nDPI was started due to the opendpi project, which was started by ipoque. We are currently using ipoque, but are migrating to procera as we speak. AFAIK, the OpenDPI project was killed awhile back. I don't know of anybody that maintains it.

What am I missing?

[rebelwireless]

nDPI is actively maintained and updated and in development. It's LGPL licensed and available as a stand alone library. Seems like a pretty good choice for packet analysis. There is already an nDPI-netfilter module that could be improved upon, meaning nDPI wouldn't require massive amounts of coding or a solo effort to use as a L7 matcher for netonix.

[josh]

nDPI is actively maintained and updated and in development. It's LGPL licensed and available as a stand alone library. Seems like a pretty good choice for packet analysis. There is already an nDPI-netfilter module that could be improved upon, meaning nDPI wouldn't require massive amounts of coding or a solo effort to use as a L7 matcher for netonix.

Sent to me from the NTOP-NG team on November 11, 2014
"Josh
I have spoken with my colleagues. We believe that what we can offer is to develop an extension for netfilter that allows you to mark traffic with L7 application protocols so you can shape. drop etc using netfulter.
We believe that we can develop it for 5000 Euro (~6200 USD).
Please let us know if this is an option for you
Regards Luca"