Saturday, November 30, 2013

Why Practice?

This evening while talking with my wife about my the TicTacToe game I'd been building as a practice, I found myself saying things about practicing I understood implicitly, but hadn't stated explicitly. So here it goes!

The Big Picture and the Start

Reflecting on the TicTacToe game revealed several parts of the utility of practice. The first was a common challenge; figuring out how to start. It wasn't immediately obvious to me how I would implement the computer player in the game. I practice to get exposed to common patterns and strategies so when I find myself in unfamiliar circumstances, there's a chance I'll be able to draw upon previous experience to find a way forward. While it's rare there is will be a perfect fit with previous experience, becoming comfortable with being asked to solve non-obvious problems is also of great value.

Although TicTacToe seems like a simple game, it can be difficult to explicitly state the strategies one would want to implement for a computer player. I took the opportunity to teach and play the game with my five year old daughter several times before deciding on a set of strategies for the computer. It made me realize that in much more complex systems, it would behoove me to experience new problem domains as much as possible before planning and implementation. Small iterations and user stories help to make explicit the assumptions customers often take for granted, but when possible, dog fooding and ghosting potential users is ideal.

The Details and the Grind

The other purpose of practice is to be as rigorous as possible with best practices and processes. While this sounds obvious, often times practice and the problem at hand may not seem important enough to be rigorous. After implementing the first classes of the TicTacToe game and being able to hold the whole solution in my head, I found it very easy to get more sloppy with my TDD cycles. Because I thought I knew the solution, I wasn't as dependent on listening to my tests for feedback as how to proceed. Clearly, this is a dangerous practice. One of the best solutions to this is pair program. Pairing reduces the chances that you'll cut corners and highlight implicit assumptions you've made that may not be shared by the team or customer.

While it's nice to say that pairing solves the problem of sloppy programing, who wants to pair with a sloppy developer? The purpose of practicing with simple problems and familiar domains is to develop the discipline to be rigorous in the face of complex problems and unfamiliar domains. If you cannot bring forth your best effort when things are easy, it's highly unlikely you'll be able to deliver the best possible production code.

Have Fun!

Although it seems trite, practice can be fun. By involving my daughter as a tester of the game, we both got something out of it. I could see more clearly what I needed to do to make the game useable and fun. It gave me a purpose for pushing myself to have fun with the practice.

Summing Up

  • Practice getting comfortable solving non-obvious problems.
  • Practice common strategies, patterns, structures, and algorithms for problems.
  • Practice looking at problem domains through the eyes of a developer.
  • Practice a rigorous workflow so when dealing with complex domains you're not distracted by the details of developing.
  • Practice for fun!

Sunday, November 3, 2013

Binary Tree in Ruby

I've returned back to Dave Thomas's katas and picked up number 11: Sorting. Harking back to the reuse we saw in the data munging kata we must sort through a pair of unordered lists efficiently without using built in libraries. I chose to implement a binary tree and enjoyed reminding myself of this algorithm very much. As usual, source is at the GitHub.

    def insert(node)
      self.count_all = self.count_all + 1
      insert_left(node) if node.word < word
      self.count = count + 1 if node.word == word
      insert_right(node) if node.word > word
    end

Sunday, October 27, 2013

Travel Itinerary Algorithm in Ruby

I've found a great new site for interesting and challenging programming exercises called TalentBuddy. Today I completed a fairly straight forward algorithm to reconstruct a travel itinerary from a pair of parallel arrays containing all the history departures and destinations. One of the neat things about TalentBuddy is that they expose your work to fairly large data sets (around 10,000 elements) and have a two second timeout, so implementations must be efficient.

My implementation begins by mapping the parallel arrays to a hash of destinations and departures. Then we calculate the first departure by getting the intersection of the two original arrays. Then a loop begins iterating through all destinations, retracing the history. This is made easy by the fact that no destination is visited twice. Finally, we have to handle the final case where the loop terminates when no additional destinations are found, printing the final destination. As always, full source is on GitHub

class PlaneTickets
  def get_journey(departure_ids, destination_ids)
    destination_for = map_departures_to_destinations(departure_ids, destination_ids)

    next_departure = (departure_ids - destination_ids).first

    while next_departure
      puts next_departure

      last_departure = next_departure
      next_departure = destination_for[next_departure]
    end

    puts destination_for[last_departure]
  end
end

Saturday, October 5, 2013

Product Pricing Kata in Ruby

Continuing along with Dave Thomas's kata series, I'd like to share my implementation of the Product Pricing kata where we're asked to implement a checkout system that accepts rules for different pricing strategies. This immediately reminded me of my FizzBuzz implementation. In the end it was actually a fairly straight forward implementation of the Strategy pattern. The more challenging part of the exercise was extracting the actual pricing rules from the supplied test suite; a user story is often easier to read than a test of very specific behavior.

In the end, I was pleased with the rule set I implemented. For the full solution, please visit GitHub.

def rules
    [
        QuantityDiscountRule.new(sku: "A", quantity: 3, price: 30),
        QuantityDiscountRule.new(sku: "B", quantity: 2, price: 15),
        RegularPriceRule.new({
                                 "A" => 50,
                                 "B" => 30,
                                 "C" => 20,
                                 "D" => 15
                             })
    ]
end

Sunday, September 15, 2013

Data Munging Kata in Ruby

Back in 2007, Dave Thomas posted a series of katas that continue to have an enduring appeal. The katas ranged from algorithm implementations to thinking exercises. This Data Munging kata had a particularly real world flare by focusing on data munging and algorithm reuse.

In short, the kata asks you to parse two data files and determine the minimum variance between two fields in each data set. The idea is to implement a solution for the first data set without reuse in mind. Next, implement a solution for the second data set and extract out the common, minimum variance algorithm between the two solutions. Nice!

I was pleased with my solution of passing in a data structure to my SpreadCalc class as well as the min_method and max_method the class could use to query the data structure to determine variance. This design would allow the class to be used on any data set that could respond to a method call and allow for the extension into all sorts of calculations related to minimums and maximums in a dataset.

class SpreadCalc
  def initialize(spread_data: spread_data, max_method: max_method, min_method: min_method)
    @spread_data = spread_data
    @max_method = max_method
    @min_method = min_method
  end

  def minimum
    min_index = 0
    min_spread = calc_spread(spread_data[min_index])

    spread_data.each_with_index do |data, i|
      spread = calc_spread(data)
      if spread < min_spread
        min_index = i
        min_spread = spread
      end
    end

    spread_data[min_index]
  end
end

I was also able to extract out a common algorithm for parsing the data files even though they were in slightly different formats. Although not intentional, I was in the mindset of extracting common functionality and the duplication was quickly obvious. I suppose that's the point of the kata! You can find the full source for my implementation on Github.

Monday, September 2, 2013

FizzBuzz in Ruby

FizzBuzz is a common and simple exercise in implementing the Strategy pattern. A brief synopsis of the Kata is as follows:

Print the numbers one through 100.
If the number is divisible by 3, print FIZZ.
If the number is divisible by 5, print BUZZ.
If the number is divisible by both, print FIZZBUZZ.
Using Ruby, the algorithm for this is quite straight forward using conditionals:
(1..100).each do |i|
  if i%3==0 and i%5==0
    puts "FIZZBUZZ"
  elsif i%3==0
    puts "FIZZ"
  elsif i%5==0
    puts "BUZZ"
  else
    puts i
  end
end

The kata only becomes interesting when you add an additional requirement; don't use conditionals. And of course, test drive your implementation. You can see my implementation on GitHub. I implemented two classes for processing a digit: Fizz and Buzz. These processors responded to process and processable? and were injected into a FizzBuzzRunner class along with the range. Additionally a default processor PassThrough was injected to the runner, which would be applied if no other processor was applicable and shared the same interface as the other processors.

The runner would query applicable_processors, asking each if the digit was processable?. If no processor was applicable, it would use the default, PassThrough. After collecting the applicable_processors, the results_from each processor were collected into an array. This allowed my final algorithm algorithm to read like well written prose.

class FizzBuzzRunner

  def initialize(range: range, processors: processors, default_processor: default_processor)
    @range = range
    @processors = processors
    @default_processor = default_processor
  end

  def as_string
    as_array.join(" ")
  end

  def as_array
    range.collect { |i| results_from(applicable_processors(i), i) }
  end
end

Wednesday, August 7, 2013

Buy One Get One Promotion with Spree

Implementing a "Buy One, Get One Free" promotion in Spree requires implementation of a custom promotion action and appropriate use of existing promotion rules. This article implements the promotion by automatically adding and removing immutable "get one" line items whose price is zero and whose quantity always mirrors its paid "buy one" counterpart. Although written and tested with Spree's 1-3-stable branch, the core logic of this tutorial will work with any version of Spree.

Promotion Eligibility

Begin by creating a new promotion using a meaningful name. Set the "event name" field to be "Order contents changed" so the promotion's actions are updated as the order is updated. Save this new promotion, so we can then configure Rules and Actions. In the Rules section, select the "Product(s)" rule and click Add. Now choose the products you'd like to be eligible for your promotion. If you'd like to include broader sets such as entire taxonomies (and have implemented the custom promotion rules to do so), feel free to use them. When we implement the promotion action, you'll be able to make things work.

You should now have a product rule that selects some subset of products eligible for your promotion.

Adding a Custom Promotion Action

We'll now add a custom promotion action that will do the work of creating the free line items for each eligible paid line item. Again, this implementation is specifically for the 1-3-stable branch, but the public interface for promotion actions has (amazingly) remained stable, and is supported from 0-7-0-stable all the way through 2-0-stable.

First, we begin by doing the basic wiring for creating a new promotion action. As the guide instructs, create a new promotion class in a file.

1
2
3
4
5
6
# app/models/spree/promotion/buy_one_get_one.rb
class BuyOneGetOne < Spree::PromotionAction
  def perform(options={})
    # TODO
  end
end

Then register this new class with Spree using an initializer.

1
2
3
# config/initializers/spree.rb
 
Rails.application.config.spree.promotions.actions << BuyOneGetOne

Then update your locales to provide translations for the promotion action's name and description.

1
2
3
4
5
6
7
8
# config/locales/en.yml
 
en:
  spree:
    promotion_action_types:
      buy_one_get_one:
        name: Buy One Get One
        description: Adds free line items of matching quantity of eligible paid line items.

And although the guide doesn't instruct you to, it seems required that you add an empty partial to be rendered in the Spree admin when you select the rule.

1
2
3
4
# app/views/spree/admin/promotions/actions/_buy_one_get_one.html.erb
 
# Empty File. Spree automatically renders the name and description you provide,
# but you could expand here if you'd like.

Pre-flight check

Before moving any farther, it's best to make sure the new promotion action has been wired up correctly. Restart your development server so the initializer registers your new promotion action and refresh your browser. You should now see a "Buy One Get One" promotion action.

Buy One Get One Logic

Now that we've got the promotion action wired, we're ready to implement the logic needed to create the new line items. Begin by collecting the order from the options.

1
2
3
4
5
6
# app/models/spree/promotion/buy_one_get_one.rb
class BuyOneGetOne < Spree::PromotionAction
  def perform(options={})
    return unless order = options[:order]
  end
end

Next we need to determine which line items in the order are eligible for a corresponding free line item. Because line items are for variants of products, we must collect the variant ids from the product rule we setup. If you've used something other the default Spree "Product(s)" rule, just make sure you end up with equivalent output.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
# app/models/spree/promotion/buy_one_get_one.rb
class BuyOneGetOne < Spree::PromotionAction
  def perform(options={})
    return unless order = options[:order]
 
    eligible_variant_ids = get_eligible_variant_ids_from_promo_rule
    return unless eligible_variant_ids.present?
  end
 
  def get_eligible_variant_ids_from_promo_rule
    product_rule = promotion.promotion_rules.detect do |rule|
      # If not using the Product rule, update this line
      rule.is_a? Spree::Promotion::Rules::Product
    end
    return unless product_rule.present?
 
    eligible_products = product_rule.products
    return unless eligible_products.present?
 
    eligible_products.collect { |p| p.variant_ids }.flatten.uniq
  end
end

Now that we've got the eligible variant ids from the promotion's rule, we'll identify the line items which have those variants.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
# app/models/spree/promotion/buy_one_get_one.rb
class BuyOneGetOne < Spree::PromotionAction
  def perform(options={})
    return unless order = options[:order]
 
    eligible_variant_ids = get_eligible_variant_ids_from_promo_rule
    return unless eligible_variant_ids.present?
 
    order.line_items.where(variant_id: eligible_variant_ids).each do |li|
      #TODO
    end
  end
  ....
end

We're now ready to process eligible line items. There are several cases we'll need to implement to have a working promotion:

  • When we find an eligible, paid line item:

    • If an existing corresponding free line item exists, update quantity to match the paid line item.

    • Else, create corresponding free line item with appropriate quantity.

  • When we find a Buy One Get One promotion line item:

    • If the corresponding paid line item still exists, do nothing.

    • Else, destroy the free line item.

These cases handle the creation, updating, and removal of promotional line items. Let's translate these cases into a skeleton of code which can then be implemented incrementally.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
# app/models/spree/promotion/buy_one_get_one.rb
class BuyOneGetOne < Spree::PromotionAction
  def perform(options={})
    return unless order = options[:order]
 
    eligible_variant_ids = get_eligible_variant_ids_from_promo_rule
    return unless eligible_variant_ids.present?
 
    order.line_items.where(variant_id: eligible_variant_ids).each do |li|
      if li.price != 0
        # It's an eligible variant and it's price is not zero, so we
        # found a "buy one" line item.
 
 
        matching_get_one_line_item = find_matching_get_one_line_item(li)
 
        # Create or update matching promo line item.
        if matching_get_one_line_item
          matching_get_one_line_item.update_attribute(:quantity, li.quantity)
        else
          create_matching_get_one_line_item(li)
        end
 
      else
        # It's an eligible variant and it's price is zero, so we
        # found a "get one" line item.
 
        # Verify "buy one" line item still exists, else destroy
        # the "get one" line item
        li.destroy unless find_matching_buy_one_line_item(li)
    end
  end
 
  def find_matching_buy_one_line_item(li)
  end
  def create_matching_get_one_line_item(li)
  end
  def find_matching_get_one_line_item(li)
  end
  ....
end

This well named and commented code reads nicely and clearly covers the create, update, and destroy cases we need to be concerned with. Now let's implement the helper methods.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
  ....
  def find_matching_buy_one_line_item(get_one_line_item)
    get_one_line_item.order.line_items.detect do |li|
      li.variant_id == get_one_line_item.variant_id and li.price != 0
    end
  end
 
  def create_matching_get_one_line_item(buy_one_line_item)
    # You may need to update this with other custom attributes you've added.
    new_line_item = buy_one_line_item.order.line_items.build
    new_line_item.variant = buy_one_line_item.variant
    new_line_item.currency = buy_one_line_item.currency
    new_line_item.price = 0
    new_line_item.quantity = buy_one_line_item.quantity
    new_line_item.save
  end
 
  def find_matching_get_one_line_item(buy_one_line_item)
    buy_one_line_item.order.line_items.detect do |li|
      li.variant_id == buy_one_line_item.variant_id and li.price != 0
    end
  end
  ....
end

We've now completed most of the implementation for the promotion action. Every time the order is updated, all line items are scanned for eligibility and the appropriate create/update/destroy actions are taken. This however, isn't the end of our implementation. Unlike other items in our cart, we need to prevent users from changing the quantity of "get one" line items or removing them from the cart. We also need some way of indicating that these zero price line items are complements of the Buy One Get One promotion.

Locked Line Items with Additional Text

While the concept of locked or immutable line items might rightly deserve its own, separate Spree plugin, we'll roll a quick one here to complete the implementation of our promotion. We'll need to add a few attributes to the spree_line_items database table, and tweak our implementation of create_matching_get_one_line_item.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
# db/migration/add_immutable_and_additional_text_to_spree_line_items.rb
class AddImmutableAndAdditionalTextToSpreeLineItems < ActiveRecord::Migration
  def change
    add_column :spree_line_items, :immutable, :boolean
    add_column :spree_line_items, :additional_text, :string
  end
end
 
# app/models/spree/promotion/buy_one_get_one.rb
  def create_matching_get_one_line_item(buy_one_line_item)
    # You may need to update this with other custom attributes you've added.
    new_line_item = buy_one_line_item.order.line_items.build
    new_line_item.variant = buy_one_line_item.variant
    new_line_item.currency = buy_one_line_item.currency
    new_line_item.price = 0
    new_line_item.quantity = buy_one_line_item.quantity
 
    new_line_item.immutable = true
    new_line_item.additional_text = "Buy One Get One"
 
    new_line_item.save
  end

Now that we know which line items aren't meant to be edited by users, we can update our UI to not render the options to remove immutable line items or update their quantity. We can also display the additional text in the line line item when it's available.

Missing from this implementation is a way to secure the immutable line items from manipulation of the immutable line items POSTed parameters. While this might be required in other cases using immutable line items, because our promotion action sets the "get one" line items quantity with every order update, we don't need to worry about this issue in this case.

Test Driving

At this point, you can begin manual testing of your implementation, but of course automated testing is best. Ideally, we would have TDDed this against a failing integration test, but the testing infrastructural setup required to do this is beyond the scope of the article. What's worth sharing though, is the syntax of the assertions I've developed to inspect line items, so that you can implement something similar for your specific needs. Here's a snippet from an integration test to give you a sense of the DSL we've built up.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
# spec/requests/buy_one_get_one_spec.rb
 
context "add eligible item to cart" do
  before do
    # setup cart
    visit_product_and_add_to_cart eligible_product
    visit "/cart"
  end  
       
  it "should add an identical, immutable, free item" do
    assert_line_item_count(2)
    assert_line_item_present(eligible_product.name, unit_price: undershirt_amount, unit_price: eligible_product.price, immutable: false)
    assert_line_item_present(eligible_product.name + " - (Buy One Get One)", unit_price: 0.00, immutable: true)
    assert_order_total(eligible_product.price)
  end
end

Of course you'd want to test all the cases we've implemented, but what's worth focusing on is the ability to assert specific attributes across many different line items. This is an extremely reusable tool to have in your testing suite. Good luck implementing!

Wednesday, June 26, 2013

Rails transposing day and month after upgrading Ruby 1.8.7

If you're wondering why your month and day are being transposed when saved in your database, you're likely:

  • Using a text field input for dates (likely with some JavaScript date picker)
  • Using American style date formatting (mm/dd/YYYY)
  • Upgrading from Ruby 1.8.7

If you meet these criteria you'll find that American style dates get parsed incorrectly in Ruby 1.9+ because of Ruby 1.9.x's new date parsing strategy. Unbelievably, this change effectively does away with American style date parsing in Ruby 1.9.x and Rails has happily followed suit!

american_date to the rescue!

After trying and failing to restore American style date parsing using timeliness, delocalize, and i18n_alchemy, I found american_date. If you look at the implementation, it is straight forward and restores backwards compatibility by simply adding a line to your Gemfile. Enjoy the return to sanity!

With Ruby 1.8.7 going EOL this month, and Rails 2.3.x and older losing support for even severe security issues, it's time to bust out your upgrade-foo for those old Rails apps. Of course, this is a tried and true topic, with many resources, most notably Railscast #225. Good luck with your upgrades!

Monday, June 17, 2013

Spree's New Release Policy

Spree has recently updated its documentation regarding contributions and has included (maybe for the first time?) an official Release Policy. This is an important step forward for the Spree community so that developers can communicate to clients the potential costs and benefits when upgrading Spree, understand how well Spree supports older releases, and gauge the overall speed of the "upgrade treadmill".

Deprecation Warnings

Deprecation warnings are to be added in patch releases (i.e. 2.0.2) and the code being deprecated will only be removed in minor versions. For example, if a deprecation warning is added in 2.0.2, 2.0.3 will still contain the same deprecation warning but in 2.1.0 the deprecation warning and the code will be gone.

Deprecation warnings are very helpful for developers, but without a robust test suite exercising your application, it's easy for deprecation warnings to go unnoticed. A strong test suite coupled with deprecation warnings helps you manage your client's expectations about how upgrades can affect your Spree customizations and extensions.

Master Branch Receives All Patches

Master branch receives all patches, including new features and breaking API changes (with deprecation warnings, if necessary).

No surprises here; the master branch is for developers and should not be used for production. If you think you've encountered a bug in Spree, make sure to create a failing test against master to make sure it hasn't be resolved by any existing patches. Read more about filing an issue for additional details.

Current Stable Release Branch

One branch "back" from master (currently 2-0-stable) receives patches that fix all bugs, and security issues, and modifications for recently added features (for example, split shipments). Breaking API changes should be avoided, but if unavoidable then a deprecation warning MUST be provided before that change takes place.

Continuing Spree's history of very aggressive refactoring, breaking API changes are permitted in the current stable branch. If you're looking for a truly stable release of Spree, you'll need to look back two stable branches behind master.

Two Releases Behind Master

Two branches "back" from master (currently 1-3-stable) receives patches for major and minor issues, and security problems. Absolutely no new features, tweaking or API changes.

In my opinion, this is the only branch that should be considered for use in production. With the API locked down and a greater chance of most bugs worked out while it was the current release branch, the "two-back" stable branch is a safe bet that's going to have the most up-to-date feature set.

Three and Four Releases Behind Master

Three branches back from master (currently 1-2-stable) receives patches for major issues and security problems. The severity of an issue will be determined by the person investigating the issue. Absolutely no features, tweaking or API changes. Four branches and more "back" from master (currently 1-1-stable and lesser) receive patches only for security issues, as people are still using these branches and may not be able to or wish to upgrade to the latest version of Spree.

It's nice to see a fairly strong commitment to accepting security patches, although if we look at this in absolute terms, the 1.1.x release was put into security-patches-only mode after just 13 months. Considering that the 1-1-stable branch is 2,127 commits behind the 1-3-stable branch (!!), it's clear that Spree is now formalizing it's very aggressive release culture.

Managing the Upgrade Treadmill

As stated previously, a strong test suite is the best tool available to be able to determine what upstream updates affect your Spree customizations. Coupled with deprecation warnings, it becomes a fairly straight-forward process for identifying breaking changes, creating estimates for fixes, and communicating these costs to clients. Following the stated guides for customizing Spree is also recommended. When visiting the Spree guides section on customization, you'll first be taken to the sub-section for authentication. Make sure to expand the Customization menu on your left to see additional guidance for customizing internationalization, views, JavaScript, stylesheet and image assets, business logic, and checkout.

Another important suggestion helpfully submitted by my colleague Mike Farmer was to never simply add gem 'spree' to your Gemfile. You should be very conscious about what version of Spree you want to use, and make sure you specify it in your Gemfile.

Monday, February 11, 2013

Detecting Bufferbloat

Bufferbloat is topic which has been gaining broader attention, but is still not widely understood. This post will walk you through the basics of bufferbloat and how to determine if you are the victim of bufferbloat.

A Brief Synopsis of the Bufferbloat Problem

The topic of bufferbloat has been explained wide and far, but I'll add to the conversation too, focusing on brevity. This summary is based on the highly informative and technical talk Bufferbloat: Dark Buffers in the Internet, a Google Tech Talk by Jim Gettys. There is an assumption in the design of TCP that if there is network congestion, there will be timely packet loss. This packet loss triggers well designed TCP flow control mechanisms which can manage the congestion. Unfortunately, engineers designing consumer grade routers and modems (as well as all sorts of other equipment) misunderstood or ignored this assumption and in an effort to prevent packet loss added large FIFO (first-in-first-out) buffers. If users congest a network chokepoint, typically an outgoing WAN link, the device's large buffers are filled with packets by TCP and held instead of being dropped. This "bufferbloat" prevents TCP from controlling flow and instead results in significant latency.

Detecting Bufferbloat

All that's required to experience bufferbloat is to saturate a segment of your network which has one of these large FIFO buffers. Again, the outgoing WAN link is usually the easiest to do, but can also happen on low-speed WiFi links. I experienced this myself when installing Google's Music Manager, which proceed to upload my entire iTunes library in the background, at startup, using all available bandwidth. (Thanks Google!) I detected the latency using mtr. Windows and OS X does not offer such a fancy tool, so you can simply just ping your WAN gateway and see the lag.


Music Manager enabled, bufferbloat, slow ping to WAN gateway


Music Manager paused, no bufferbloat, fast ping to WAN gateway

Managing Bufferbloat

Unfortunately, there are no easy answers out there right now for many users. Often we cannot control the amount of bandwidth a piece of software will try to use or the equipment given to us by an ISP. If you are looking for a partial solution to the problem, checkout Cerowrt, a fork of the OpenWrt firmware for routers. It makes use of the best available technologies used to combat bufferbloat. Additionally, be on the look out for any software that might saturate a network segment, such as Bittorrent, Netflix streaming, or large file transfers over weak WiFi links.

Tuesday, February 5, 2013

Install SSL Certificate from Network Solutions on nginx

Despite nginx serving pages for 12.22% of the web's million busiest sites, Network Solutions does not provide instructions for installing SSL certificates for nginx. This artcle provides the exact steps for chaining the intermediary certificates for use with nginx.

Chaining the Certificates

Unlike Apache, nginx does not allow specification of intermediate certificates in a directive, so we must combine the server certificate, the intermediates, and the root in a single file. The zip file provided from Network Solutions contains a number of certificates, but no instructions on the order in which to chain them together. Network Solutions' instructions for installing on Apache provide a hint, but let's make it clear.

1
cat your.site.com.crt UTNAddTrustServer_CA.crt NetworkSolutions_CA.crt > chained_your.site.com.crt

This follows the general convention of "building up" to a trusted "root" authority by appending each intermediary. In this case UTNADDTrustServer_CA.crt is the intermediary while NetworkSolutions_CA.crt is the parent authority. With your certificates now chained together properly, use the usual nginx directives to configure SSL.

1
2
3
4
listen                 443;
ssl                    on;
ssl_certificate        /etc/ssl/chained_your.site.com.crt;
ssl_certificate_key    /etc/ssl/your.site.com.key;

As always, make sure your key file is secure by giving it minimal permissions.

1
chmod 600 your.site.com.key

I hope this little note helps to ease nginx users looking to use a Network Solutions SSL certificate.

Tuesday, January 29, 2013

How to Apply a Rails Security Patch

With the announcement of CVE-2013-0333, it's time again to secure your Rails installation. (Didn't we just do this?) If you are unable to upgrade to the latest, secure release of Rails, this post will help you apply a Rail security patch, using CVE-2013-0333 as an example.

Fork Rails, Patch

The CVE-2013-0333 patches so kindly released by Michael Koziarski are intended for use with folks who have forked the Rails repository. If you are unable to keep up with the latest releases, a forked repo can help you manage divergences and make it easy to apply security patches. Unfortunately, you cannot use wget to download the attached patches directly from Google Groups, so you'll have to do this in the browser and put the patch into the root of your forked Rails repo. To apply the patch:

1
2
3
4
cd $RAILS_FORK_PATH
git checkout $RAILS_VERSION
# Download attachment from announcement in browser, sorry no wget!
git am < $CVE.patch

You should see the newly committed patch(es) at the HEAD of your branch. Push out to GitHub and then bundle update rails on your servers.

Patching without Forks

If you are in the unfortunate case where there have been modifications or patches applied informally outside version control or you are otherwise compelled to modify the Rails source on your server directly, you are still able to use the provided patches.

Before begining, take a look at the diffstat at the top of the patch:

1
2
3
4
.../lib/active_support/json/backends/okjson.rb     |  644 ++++++++++++++++++++
.../lib/active_support/json/backends/yaml.rb       |   71 +---
activesupport/lib/active_support/json/decoding.rb  |    2 +-
activesupport/test/json/decoding_test.rb           |    4 +-

As you can see the base path of the diff is "activesupport". (The triple dots are simply there to truncate the paths so the diffstats line up nicely.) However, when the activesupport gem is installed on your system, the version number is appended in the path. This means we need to use the -p2 argument for patch to "strip the smallest prefix containing num leading slashes from each file name found in the patch file." We'll see how to do this in just a second, but first, let's find the source files we need to patch.

Locating Rails Gems

To find the installed location of your Rails gems, make sure you are using the desired RVM installation@gemset (check with rvm current), and then run "gem env" and look for the "GEM PATHS" section. If you're using the user-based installation of RVM it might look something like this:

1
/home/$USER/.rvm/gems/ree-1.8.7-2012.02

Now that we know where the installed gems are, we need to get our patch and apply.

1
2
3
cd /home/$USER/.rvm/gems/ree-1.8.7-2012.0/gems/activesupport-2.3.15
# Download attachment from announcement in browser, sorry no wget!
patch -p2 < $CVE.patch

Often times these patches will include changes to tests which are not included in the ActiveSupport gem installations. You may get an error like this while patching CVE-2013-0333:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
patch -p2 < cve-2013-0333.patch                                                
patching file lib/active_support/json/backends/okjson.rb
patching file lib/active_support/json/backends/yaml.rb
patching file lib/active_support/json/decoding.rb
can't find file to patch at input line 768
Perhaps you used the wrong -p or --strip option?
The text leading up to this was:
--------------------------
|diff --git a/activesupport/test/json/decoding_test.rb b/activesupport/test/json/decoding_test.rb
|index e45851e..a7f7b46 100644
|--- a/activesupport/test/json/decoding_test.rb
|+++ b/activesupport/test/json/decoding_test.rb
--------------------------
File to patch:
Skip this patch? [y] y
Skipping patch.
1 out of 1 hunk ignored

This error is just saying it cannot find the file test/json/decoding_test.rb. It's OK to skip this patch, because the file doesn't exist to patch.

Verify the Patch is Installed

When doing any kind of security patching it is essential that you have confidence your actions were applied successfully. The strategies on doing this will verify based on the type of changes made. For CVE-2013-0333, it's a fairly simple check.

1
2
3
4
5
6
7
8
9
10
11
# Before applying patch
script/console
Loading development environment (Rails 2.3.15)
>> ActiveSupport::JSON::DECODERS
=> ["Yajl", "Yaml"]
 
# After applying patch
script/console
Loading development environment (Rails 2.3.15)
>> ActiveSupport::JSON::DECODERS
=> ["Yajl", "OkJson"]

Monday, January 28, 2013

Evading Anti-Virus Detection with Metasploit

This week I attended a free, technical webinar hosted by David Maloney, a Senior Software Engineer on Rapid7's Metasploit team, where he is responsible for development of core features for the commercial Metasploit editions. The webinar was about evading anti-virus detection and covered topics including:

  • Signatures, heuristics, and sandboxes
  • Single and staged payloads
  • Executable templates
  • Common misconceptions about encoding payloads
  • Dynamically creating executable templates

After Kaspersky Lab broke news of the "Red October" espionage malware package last week, I thought this would be an interesting topic to learn more about. In the post, Kaspersky is quoted saying, "the attackers managed to stay in the game for over 5 years and evade detection of most antivirus products while continuing to exfiltrate what must be hundreds of terabytes by now."

Separating Exploits and Payloads

Vocabulary in the world of penetration testing may not be familiar to everyone, so let's go over a few terms you may see.

  • Vulnerability: A bug or design flaw in software that can be exploited to allow unintended behavior
  • Exploit: Software which takes advantage of a vulnerability allowing arbitrary execution of an attacker's code
  • Payload: Code delivered to a victim by an exploit
  • Signature: Set of rules or pattern match against code
  • Sandbox: Protected segments in OS, where code can be run safely

Metasploit by design, separates the payload from the exploit. Payloads can come in two types. A single-stage payload includes all code intended for use in the attack. A staged payload has a small initial exploit which then connects back to a server using shell commands to download subsequent payloads. This is an important distinction because many anti-virus products have signatures for common first-stage exploits, but not for the much wider universe of secondary payloads. By building first-stage exploits that can evade detection, additional payloads can be installed and remain resident without detection.

A Unique Exploit for Every Target

To have unique initial exploits that will not have anti-virus signatures, Metasploit Pro includes tools to bundle exploits inside otherwise randomly generated executables. These tools create C code which assign variables in random orders and with random values. Functions are created at random to manipulate and perform calculations on these variables. The functions are then called randomly, building a random call tree, making it very difficult to develop a signature because the execution flow and memory maps are all random.

Of course, eventually, we want the random calculations to stop and the exploit to execute so a payload can be downloaded and executed. Amazingly, one of the key ways to hide the payload from the anti-virus is simply to wait to decode the encoded (obfuscated) exploit until after the anti-virus has completed its scan of the executable. Anti-virus vendors are keenly aware that their products hurt end user performance and so the amount of time which they can sandbox and scan an executable is limited. If the initial payload's random functions take a sufficient time, then the anti-virus releases the file from the sandbox. This delay is configurable and is very effective, allowing the exploit to be decoded and executed without detection.

The Next Generation of Exploits

It's been 8 months since these randomization generators were released with Metasploit Pro and anti-virus companies are starting to catch up. Still, only 8 of the 44 scanners used at VirusTotal detected one of these exploits bundled with randomized code. The next generation of generators are designed to avoid using shell code entirely, further reducing anti-virus products' ability to detect malicious behavior. Instead of shell code, system calls are starting to be used directly, pulling payloads directly into memory. Since anti-virus depends heavily on scanning writes to the file system, this also reduces the exploits surface area. PowerShell version 2.0 seems to be the vehicle of choice for these next generation of exploits and thus far has gone completely unnoticed by anti-virus vendors (according to David anyway).

Additional Resources

Thursday, January 10, 2013

Use Metasploit to Verify Rails is Secured from CVE-2013-0156

On January 8th, 2013 Aaron Patterson announced a major security vulnerability on the Rails security mailing list, affecting all releases of the Ruby on Rails framework. This vulnerability allows an unskilled attacker to execute commands remotely on any unpatched Rails web server. Unsurprisingly, it's getting a lot of attention; Ars Technica estimates more than 200,000 sites may be vulnerable. With all the hype, it's important to separate the facts from the fiction and use the attacker's own tools to verify your site is secure.

Within 36 hours of the announcement of CVE-2013-0156, the developers at Rapid7 released a metasploit exploit module. Metasploit lowers the barriers to entry for attackers, making the whole process a point and click affair with a slick web GUI. Fortunately, the Rails security team has provided many easy to implement mitigation options. But, how do *know* you've really closed the vulnerability, particularly to the most automated and unskilled attacks? No better way than to try and exploit yourself.

It's best to scan your unpatched site first so you can be certain the scan is working as expected and you don't end up with a false positive that you've eliminated the vulnerability. Here is the quick and dirty introduction to running Metasploit, and executing a scan:

UPDATE: I've changed the Metasploit instructions here a bit to include setting the VHOST option. Teammate Steph Skardal was using these instructions and together we found that without a VHOST set, the RHOSTS are resolved to an IP address. It's worth checking your Rails logs to verify a request is being received and processed. If you don't see anything there, check your nginx or Apache (or whatever) access logs for any possible 301 redirects.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
git clone git://github.com/rapid7/metasploit-framework.git
cd metasploit-framework
./msfconsole
use auxiliary/scanner/http/rails_xml_yaml_scanner
set RHOSTS mycompany.com
set VHOST app.mycompany.com
set RPORT 80
set URIPATH /rails_app
set VERBOSE true
show options
run
[+] mycompany.com:80 is likely vulnerable due to a 500 reply for invalid YAML
[*] Scanned 1 of 1 hosts (100% complete)
[*] Auxiliary module execution completed

If you don't get back a "likely vulnerable" message, it's probably because you're still running Ruby 1.8.7. As of this writing the metasploit module exploit states:

The technique used by this module requires the target to be running a fairly version of Ruby 1.9 (since 2011 or so). Applications using Ruby 1.8 may still be exploitable using the init_with() method, but this has not been demonstrated.

It's only a matter of time before Ruby 1.8 becomes supported, but this does give some folks a bit more time. Now let's review the mitgation strategies provided by the announcement to show you just how easy it can be to secure yourself.

Disabling XML Entirely

The nature of the vulnerability is in parsing XML in request parameters. If you don't parse XML, you should disable XML parsing entirely by placing one of the following snippets inside an application initializer.

1
2
3
4
# Rails 3.2, 3.1 and 3.0
ActionDispatch::ParamsParser::DEFAULT_PARSERS.delete(Mime::XML)
# Rails 2.3
ActionController::Base.param_parsers.delete(Mime::XML)

Removing YAML and Symbol support from the XML parser

I couldn't say it better than Aaron than myself, so I'll give it to you straight from the announcement:

If your application must continue to parse XML you must disable the YAML and Symbol type conversion from the Rails XML parser. You should place one of the following code snippets in an application initializer to ensure your application isn't vulnerable. You should also consider greatly reducing the value of REXML::Document.entity_expansion_limit to limit the risk of entity explosion attacks.
The entity_expansion_limit recommendation is not strictly part of CVE-2013-0156, but should be implemented as well to limit your exposure to entity explosion attacks.

To disable the YAML and Symbol type conversions for the Rails XML parser add these lines to an initializer:

1
2
3
4
5
6
7
#Rails 3.2, 3.1, 3.0
ActiveSupport::XmlMini::PARSING.delete("symbol")
ActiveSupport::XmlMini::PARSING.delete("yaml")
 
#Rails 2.3
ActiveSupport::CoreExtensions::Hash::Conversions::XML_PARSING.delete('symbol')
ActiveSupport::CoreExtensions::Hash::Conversions::XML_PARSING.delete('yaml')

Removing YAML Parameter Parsing

While it's *much* less common to parse YAML in request params than XML, Rails does support this, but not by default (except in version 1.1.0!). There is no fix for YAML params injection, so it must be disabled . The methods for doing this differ in Rails among rails versions.

1
2
3
4
5
6
7
8
9
10
11
12
13
# Rails 2.x: find and remove all instances
ActionController::Base.param_parsers[Mime::YAML] = :yaml
 
# Rails 3.x: add to initializer
ActionDispatch::ParamsParser::DEFAULT_PARSERS.delete(Mime::YAML)
 
# Rails 3.2, 3.1, 3.0: add to initializer
ActiveSupport::XmlMini::PARSING.delete("symbol")
ActiveSupport::XmlMini::PARSING.delete("yaml")
 
# Rails 2.3: add to initializer
ActiveSupport::CoreExtensions::Hash::Conversions::XML_PARSING.delete('symbol')
ActiveSupport::CoreExtensions::Hash::Conversions::XML_PARSING.delete('yaml')

Go Check your Site!

As you can see, with just a few lines of code, any site can manage their exposure to this risk. I strongly urge you to read the security announcement and avoid the hype. Then go patch your site!