Detecting Bufferbloat

Bufferbloat is topic which has been gaining broader attention, but is still not widely understood. This post will walk you through the basics of bufferbloat and how to determine if you are the victim of bufferbloat.

A Brief Synopsis of the Bufferbloat Problem

The topic of bufferbloat has been explained wide and far, but I'll add to the conversation too, focusing on brevity. This summary is based on the highly informative and technical talk Bufferbloat: Dark Buffers in the Internet, a Google Tech Talk by Jim Gettys. There is an assumption in the design of TCP that if there is network congestion, there will be timely packet loss. This packet loss triggers well designed TCP flow control mechanisms which can manage the congestion. Unfortunately, engineers designing consumer grade routers and modems (as well as all sorts of other equipment) misunderstood or ignored this assumption and in an effort to prevent packet loss added large FIFO (first-in-first-out) buffers. If users congest a network chokepoint, typically an outgoing WAN link, the device's large buffers are filled with packets by TCP and held instead of being dropped. This "bufferbloat" prevents TCP from controlling flow and instead results in significant latency.

Detecting Bufferbloat

All that's required to experience bufferbloat is to saturate a segment of your network which has one of these large FIFO buffers. Again, the outgoing WAN link is usually the easiest to do, but can also happen on low-speed WiFi links. I experienced this myself when installing Google's Music Manager, which proceed to upload my entire iTunes library in the background, at startup, using all available bandwidth. (Thanks Google!) I detected the latency using mtr. Windows and OS X does not offer such a fancy tool, so you can simply just ping your WAN gateway and see the lag.

Music Manager enabled, bufferbloat, slow ping to WAN gateway

Music Manager paused, no bufferbloat, fast ping to WAN gateway

Managing Bufferbloat

Unfortunately, there are no easy answers out there right now for many users. Often we cannot control the amount of bandwidth a piece of software will try to use or the equipment given to us by an ISP. If you are looking for a partial solution to the problem, checkout Cerowrt, a fork of the OpenWrt firmware for routers. It makes use of the best available technologies used to combat bufferbloat. Additionally, be on the look out for any software that might saturate a network segment, such as Bittorrent, Netflix streaming, or large file transfers over weak WiFi links.

Install SSL Certificate from Network Solutions on nginx

Despite nginx serving pages for 12.22% of the web's million busiest sites, Network Solutions does not provide instructions for installing SSL certificates for nginx. This artcle provides the exact steps for chaining the intermediary certificates for use with nginx.

Chaining the Certificates

Unlike Apache, nginx does not allow specification of intermediate certificates in a directive, so we must combine the server certificate, the intermediates, and the root in a single file. The zip file provided from Network Solutions contains a number of certificates, but no instructions on the order in which to chain them together. Network Solutions' instructions for installing on Apache provide a hint, but let's make it clear.

1	cat your.site.com.crt UTNAddTrustServer_CA.crt NetworkSolutions_CA.crt > chained_your.site.com.crt

This follows the general convention of "building up" to a trusted "root" authority by appending each intermediary. In this case UTNADDTrustServer_CA.crt is the intermediary while NetworkSolutions_CA.crt is the parent authority. With your certificates now chained together properly, use the usual nginx directives to configure SSL.

1 2 3 4

listen                 443; ssl                    on; ssl_certificate        /etc/ssl/chained_your.site.com.crt; ssl_certificate_key    /etc/ssl/your.site.com.key;

As always, make sure your key file is secure by giving it minimal permissions.

1	chmod 600 your.site.com.key

I hope this little note helps to ease nginx users looking to use a Network Solutions SSL certificate.