Flow Control

[sirhc]

Flow Control can only do so much on a flat network.

Picture this, you have 100 customers on a tower requesting data from thousands of different sources (think of all the adds per page) on the internet, all these sending servers have FAT pipes. These servers start off by sending the data AS FAST as they can but when the packets hit your connection to the internet they are shaped by your upstream provider and since this is a routed way point on the source destination path they are able to tell each sending stream to slow down as needed.

But now they enter the world of Layer 2 which was never designed to deal with anything other then the physical line speed which they do know such as 10/100/1000 and HD/FD so these packets start their journey on your flat layer 2 network. Now they come to their first switch which says 1000FD zip through onto the next and then peel off to a port that says 100FD but in reality it is much less then 100FD and in fact is only HD and the capacity fluctuates from moment to moment but there is no way to inform the senders that there will be a bottleneck here so packets start zipping in from all over the net at 1G speed and start slamming into the port that is labeled 100FD but in reality it is more like 60HD +/- 10-20% at any given moment in time but no one has the ability to tell any of the packets coming along that there will be a bottleneck here.

Now to make things worse the AP is stuck on retrys on a terrible link which further throws this whole timing issue off and now the port buffers are full so packets keep getting dropped but still no way to tell all these streams to slow down. They just keep sending and sending, and packets that are dropped will time out on the sender side and be re-sent which eats up your bandwidth on duplicate packets.

Now you interject Flow Control. FLow control has the ability all be it a TERRIBLE method of helping to a point by sending a PAUSE FRAME back the line and tells the interface to pause for 50 micro-seconds. Now this pause frame is indiscriminate in that it will STOP all packets even those bound to a totally different interface/destination that then slows everyone else feeding through that port.

People with Flat networks dealing with wireless links multiple hops out are going to have issues. Flow control is a great bandaid if it can get to a routed way point such as a router as the router then has the ability to slow down individual stream from specific sources and not affect other streams.

Also with flat networks errors from bad cabling become amplified.

With my WISP I have a router at every tower that way the Flow Control Pause Frames which there is no way around with HALF DUPLEX links that have VARIABLE capacity. I do not feed the back hauls through the same port that feeds AP's because I do not want my backhauls that carry packets for the next tower out to be affected by PAUSE FRAMES generated on this tower. That is why if you look at my tower setups CLICK HERE TO SEE MY TOWERS SETUP I have the back hauls configured in the switch as a mid-span going to the router and all the AP's at that tower feed through a LAG. Now you might ask why I use a LAG because a single 1G port is more then enough to handle the traffic here and you are correct but by using 2 ports I am dividing the PAUSE FRAMES up meaning there is only half the pause frames on each port.

Look, if we increased the buffers to a gazillion that would be worse as packets would get held up in the que so long that they would time out and the sender would send a duplicate packet and if you were one of the packets in the que line you might look back and see your replacement packet waiving at you.

Flow Control is a tool that is needed in WISP's current situation but if you expect to run one large flat network and your thought is to push pause frames all the way back across multiple hops in your network your going to have to issues.

Switches are TRANSPARENT BRIDGES with no authority to control packet flow and if you create too large of buffers and packets become stale waiting to go then the sender will issue a replacement packet and the original packet is still standing in line.

AP's that are in bridge mode are also transparent bridges that advertise false capacity, saying hey I am 100FD but in reality they are less than 60M on average and only half duplex and sometimes they stop and try repeatedly to send a packet through on a bad connection.

So anyway this is just some food for thought, just envision what is happening on your network from the perspective of the packet, the sending servers, half duplex links that advertise they are FD and much faster then they really are.

[rebelwireless]

Half Duplex radios is why I have always recommended routing for wireless links. I do quite a few PTP links to bridge campus networks or similar and MOST of my clients come to me via word of mouth because someone else has set them up and it doesn't work right.

It's easy enough to slap up a few nanostations as a layer2 link between buildings and have a network segment, but then have the link essentially collapse when someone tries to copy a large file from the file server gigabitFD>~60MbpsHD link. Flow control can only do so much. Worker complains that they can't copy the file. Another user's computer is pushing off it's backblaze exacerbating the link mismatch.

Split the two sites into separate subnets, route right on the far side radio and setup as dhcp-relay so there is no added equipment. a few routes. magically, transfers are smooth and everything just works®