Link Speed Aware QOS

[josh]

An email prior to that:

"Hi Josh
thanks for contacting us. We have developed a linux kernel module for netfilter for a customer that is using ndpi to mark traffic and thus drop/shape application protocols. We’ve plans for adding inline traffic management policies on our ntopng product but this is scheduled for early next year.

What is the speed you’re targeting?

Regards Luca"

[rebelwireless]

Yeah, they'll do a commercial module against ntop-ng, but you can build this ndpi-netfilter module:
https://github.com/betolj/ndpi-netfilter
against ntop r8598 pretty easily and it matches protocols quite well. I've got this running in my lab right now and I'm going to do a live test with some select customers here in the next week. I'm actually installing the ubuntu 14.04 vm at the datacenter tonight that will be the testbed. It's catching bittorrent and netflix within a few kilbytes and I'm tagging DSCP with iptables mangle.

[josh]

I'm curious as to how much traffic you will be able to push through that setup without PF_RING while maintaining low latency and a high rate of accuracy.

[rebelwireless]

Same here. In the lab its pretty fast, sub ms, but I don't know yet what will happen with dozens of torrents and how much the latency will climb as CPU usage increases.

If it works well but just doesn't scale then a commercial module might be interesting. Depends on how the ~$6k works with ongoing licensing etc.

[rebelwireless]

I'll add that I plan on marking connections and then packets, so I don't need to constantly run DPI against a connection once its type has been established, only against connections with no marks. We'll see how that works out.

[keefe007]

Can't you estimate link capacity by calculating off the serialization delay?

[rebelwireless]

Unfortunately I cleared the iptables counters that were running for 4 hours to add a could definitions :/

anyway, here is about 5 minutes:

Code: Select all

 

Chain FORWARD (policy ACCEPT 234K packets, 231M bytes)
 pkts bytes target prot opt in out source destination
40520 60M DSCP all -- any any anywhere anywhere protocol BITTORRENT DSCP set 0x1b
 370 62753 DSCP all -- any any anywhere anywhere protocol DNS DSCP set 0x1b
 190 148K DSCP all -- any any anywhere anywhere protocol FACEBOOK DSCP set 0x1b
 15 1547 DSCP all -- any any anywhere anywhere protocol GMAIL DSCP set 0x1b
 2972 3261K DSCP all -- any any anywhere anywhere protocol GOOGLE DSCP set 0x1b
34261 50M DSCP all -- any any anywhere anywhere protocol HTTP DSCP set 0x1b
58590 3090K DSCP all -- any any anywhere anywhere protocol HTTP_CONNECT DSCP set 0x1b
15762 1037K DSCP all -- any any anywhere anywhere protocol HTTP_PROXY DSCP set 0x1b
 220 12509 DSCP all -- any any anywhere anywhere protocol ICMP DSCP set 0x1b
 2100 2739K DSCP all -- any any anywhere anywhere protocol NETFLIX DSCP set 0x1b
 767 903K DSCP all -- any any anywhere anywhere protocol YAHOO DSCP set 0x1b
71865 107M DSCP all -- any any anywhere anywhere protocol YOUTUBE DSCP set 0x1b


So this is nDPI + nDPI-netfilter on 1 core of a 2.26 Core2Duo 256MB ram KVM, ubuntu 14.04server. The only things installed in this vm are the build chain, squid3, nDPI, and nDPI-netfilter.

Some interesting things.
I see no noticeable increase in latency across the VM.
Matchers work very well, except netflix in a silverlight container must be encapsulated slightly differently. In the above, most of the 50M of http is netflix over silverlight. Netflix on my phone shows up as netflix. Bittorrent is found immediately.

CANNOT mark connections first because some protocols look like http initially, so have to just make DSCP on packets so nDPI can get fed more for analysis.

I made a quick'n'dirty setup script:

Code: Select all

 for x in `cat /root/protocols`; do for y in `cat /root/chains`; do iptables -t mangle -A $y -m ndpi --$x -j DSCP --set-dscp 27; done; done

and files /root/chains and /root/protocols with obvious contents. Just setting DSCP 27 for the test, will add a DSCP tag per protocol.

I also did a rather ugly 'hairpin' and a separate routing table for the test subs. Need both inbound and outbound to go through the mikrotik as that's where the queues currently are so I have a 'long way around' route allowing this. I may go a step further and build queues on a production box. queues are just massively simpler for my shaping model on 'tik (PCQ and high allowances)

it's matching FORWARD here, but also does INPUT and OUTPUT if I'm going through squid3.

It shows some promise for sure I think!.

[josh]

I think this may be a decent solution if you could get PF_RING and a few other things supported on the correct hardware.

That said, something like IPOQUE, Procera, etc will always be better, as they have entire teams dedicated to always making sure the current application detection signatures are as up to date as possible.

It's basically ntop-ng's cash flow vs a much larger enterprise/carrier company like procera's cash flow.

[rebelwireless]

I don't buy into the more money = better product malarkey. I also don't need perfect matching or sub ms device traversal times. Burning up 1ms for l7 detection is a small price to pay.

[josh]

It's not the "more money paid makes it a better product", it's the "they are used to dealing with enterprise and carrier customers and this is what they specialize in and are a much bigger company with more resources to devote" deal :)

My concern is not that it will work, it is "will it work at scale and will the definition updates be timely, and what do I do for timely support if something breaks" questions.