Doug Bryant

Tech thoughts and notes

Fun With Ruby Hashes - Initializing With a Block

The alternate title for this post was Poor man’s email load balancer.

A less common method for creating and using Ruby hashes is to initialize the hash with a block. This allows you to return and/or assign a value for a missing key. You can think of this as a parallel to using method_missing in Ruby classes and modules.

The documentation for Hash.new with a block suggests

If a block is specified, it will be called with the hash object and the key, and should return the default value. It is the block’s responsibility to store the value in the hash if required.

Most often for the case where I wanted a default value returned, I would use fetch and set the default value for individual keys being looked for.

1
2
3
4
5
6
7
my_hash = {}
my_hash.fetch(:foo, "Not Set")
=> "Not Set"

my_hash[:foo] = 'bar'
my_hash.fetch(:foo, "Not Set")
=> "bar"

The same result can be achieved by initializing the hash with a block - any unknown keys requested from the hash will be initialized with a default value.

1
2
3
4
5
6
my_hash = Hash.new{|hash,key| hash[key] = "Not Set"}
my_hash[:foo]
=> "Not Set"

my_hash[:foo] = "bar"
=> "bar"

An alternative to initializing Hash with a block is to set the default_proc= on an existing hash.

1
2
3
4
5
6
7
8
my_hash = {}
my_hash[:foo]
=> nil

my_proc = proc{|hash,key| hash[key] = "Not Set"}
my_hash.default_proc = my_proc
my_hash[:foo]
=> "Not Set"

So now we have a way of always returning a default value for a hash. Not very interesting in and of itself. What else could we do with our block backed hash?

A simple loadbalancer…

This is one of my favorite hacks. At MileMeter, we were using our Google Apps account to send email (reminders, confirmation emails, etc). At the time (not sure what the limit is now) we were limited to 500 emails per day from a single account. At some point, we started bumping up against the 500 email per day limit.

It was easy enough to create another email account to send emails through, but ActionMailer only provided a single configuration and we didn’t want to upgrade Google Apps for all MileMeter users because of a single account limitation.

John Riney and I looked at our options went in search of a solution which would allows us to send more emails without spending any additional money.

John found the Hash documentation and realized we had a solution in what he deemed “Untrustworty Ruby Hashes”

It was easy enough to create additional emails accounts. Armed with a Hash initialized with a block, we can now tell ActionMailer to pick a random user_name from a list of accounts each time it sent an email, thus effectively load-balancing our email.

A typical ActionMailer configuration is

1
2
3
4
5
6
7
8
9
10
config.action_mailer.delivery_method = :smtp
config.action_mailer.smtp_settings = {
  address:              'smtp.gmail.com',
  port:                 587,
  domain:               'example.com',
  user_name:            '<username>',
  password:             '<password>',
  authentication:       'plain',
  enable_starttls_auto: true
}

Our new load-balanced ActionMailer config is initialize with a block and configured to return a different user_name each time the user_name key is accessed.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
# list of email accounts to send from
ACCOUNTS = %w{system1 system2 system3 system4}.map{|name| "#{name}@example.com"}
# => ["system1@example.com", "system2@example.com", "system3@example.com", "system4@example.com"]

# NOTE: user_name key not set
email_config = {
  address:              'smtp.gmail.com',
  port:                 587,
  domain:               'example.com',
  password:             '<password>',
  authentication:       'plain',
  enable_starttls_auto: true
}

config.action_mailer.delivery_method = :smtp
config.action_mailer.smtp_settings = Hash.new do |hash,key|
  ACCOUNTS[rand(ACCOUNTS.size)] if :user_name == key
end.merge!(email_config)

Assuming the password is set the same for all accounts you want to use, when an email is sent, the user_name for the account is looked up and set to a different value every time, effectively a simple load-balancing technique.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
irb(main):037:0> smtp_settings[:user_name]
=> "system1@example.com"
irb(main):038:0> smtp_settings[:user_name]
=> "system4@example.com"
irb(main):039:0> smtp_settings[:user_name]
=> "system2@example.com"
irb(main):040:0> smtp_settings[:user_name]
=> "system2@example.com"
irb(main):041:0> smtp_settings[:user_name]
=> "system3@example.com"
irb(main):042:0> smtp_settings[:user_name]
=> "system1@example.com"
irb(main):043:0> smtp_settings[:user_name]
=> "system3@example.com"
irb(main):044:0> smtp_settings[:user_name]
=> "system1@example.com"
irb(main):045:0> smtp_settings[:user_name]
=> "system3@example.com"

Moving From Subversion to Git

I just finished moving all our repositories from subversion to git.  There is lots of documentation on the web, but none of it ties everything together.  Specifically, once you get your subversion repository including all branches and tags imported into a local copy of git, how do you push your git repository including all branches and tags to a remote git server?   This was important to us because we have ongoing work in some branches which are not ready to be pushed to trunk yet.

I started out by following the excellent instructions at http://scie.nti.st/2007/11/14/hosting-git-repositories-the-easy-and-secure-way to setup our own git repository.  Follow these instructions and create a simple git repository with only a couple of files to make sure it works and become familiar with it.  At this point, you have a working central git repository but you are not yet ready to import your subversion repository.  If using github, you can skip this step.

Next, go grab git2svn from github.  This will download an entire subversion project, including branches and tags, into a local git repository.  Depending on the size of your codebase and number of branches/tags, etc, this may take a while.  Our smaller projects took about 10 minutes.  Our larger project took almost an hour.  It differs from git-svn in that it converts anything in subversion tags to actual git tags.

Now for the magic of pushing your local git repository imported from subversion to the central git repository your team will use.   At this point, you should have a working central/remote git repository you are going to use.

Inside the root of your local git project, run these commands, replacing everything which is in caps. git remote add origin git@YOUR_SERVER_HOSTNAME:YOUR_PROJECT_NAME.git git push origin master:refs/heads/master

After this runs, you have only pushed the trunk to the remote repository (now called master).  You will also want to push all your tags and branches.  With git, you have to explicitly push your tags.

git push --tags

And now for the branches.  This was the only piece I could not find on the interweb.  This is achieved by the command

git push --all

The --all command per the documentation specifies that all refs under $GIT_DIR/refs/heads/ be pushed rather than naming each ref to push.

That should be all there is to it. Our other developers were able to clone the newly minted git repository and start developing where they left off on their branch with the command:

git clone git@YOUR_SERVER_HOSTNAME:YOUR_PROJECT_NAME.git git checkout -b local_branch_name --track origin/remote_branch_name

Getting Git +svn Branches to Follow Remote Branch

Andy Delcambre and Robby Russell both have excellent articles about getting up and running with git as an subversion client.

The one thing that had been getting me was the fact that the branch I created locally did not always stick to following the remote branch. It would start out that way, but once I checked out the master branch again, it would not always and definitely not consistently point toward the remote branch once I re-checked out my branch.

I finally figured out that the local and remote branch must be named differently. The character case matters! So if I had a remote branch of ITR-FOO and a local branch of itr-foo, it would loose track of the remote branch. The easy solution is to name the branches differently by using a “-” remotely and a “_” locally for names or just name the branches differently. Now, I use ITR-FOO remotely and itr_foo locally, or shorten the local name to itr_promos if I am using ITR-PROMOTIONS remotely.

For merging branching, using git is so much simpler, even if it is very retarded sometimes.

MileMeter Has Launched

After a year of work, my company MileMeter has launched. We sell auto insurance by the mile with no vehicle tracking devices involved.

A big hats off to Chris Gay for working so hard for so many years so that we can could get to this point. Insurance rules had to be changed and it was difficult for many people to see the benefit selling insurance this way.

We are also one of very few, if not the only insurance company powered entirely by Ruby It certainly gave us an advantage as we were able to get a robust, well tested application out the door so quickly.

Ruby-mysql-osx

I keep having to look this up….. sudo env ARCHFLAGS=“-arch i386” gem install mysql — —with-mysql-config=/usr/local/mysql/bin/mysql_config

Pradipta's Rolodex

I woke up this morning to a 40+ thread of emails which originated by a recruiter sending 416 people job posting, all via TO:

It quickly turned into a somewhat fun conversation with lots of people and jokes. Rather quickly a google group named Pradipta’s Rolodex was created. Also followed was a Wikipedia entry (quickly deleted by Wikipedia admins), a FaceBook group, at least one domain registration as well as posts to Digg, Reddit and others.

Oh, the friday fun…..

Rspec Test Render(:nothing = True)

I was trying to test one of my controllers was successfully hitting the render(:nothing = true) block. I could not find anything specifically in the API which handled this senario, so I ended up using have_text with a space as an argument.

@response.should have_text(" ")

Vote for Milemeter at AWS Startup Challenge

Go vote for Milemeter at AWS Startup Challenge and keep me gainfully employed!

We are proud to be 1 of 7 Finalist for the Amazon.com AWS Startup Challenge and will give our presentation next week at Amazon.com HQ. Milemeter is an innovative insurance start-up that will offer “auto insurance buy the mile”.

Please Vote for Milemeter!

Update: Congratulations to Ooyala for winning the AWS Startup Challenge.

Testing Private and Protected Methods With Ruby

When I have to test my protected and private methods in ruby, I make the methods public for the scope of the test.

MyClass.send(:public, *MyClass.protected_instance_methods)
MyClass.send(:public, *MyClass.private_instance_methods)

Just place this code in your testing class substituting your class name. Include the namespace if applicable.