Setting up a MongoDB replicaset with AWS OpsWorks

I was recently tasked with setting up a MongoDB Replicaset.

Documentation for setting up a MongoDB Replicaset with OpsWorks was a bit on the sparse side and took a bit of trial and error. I ended up with this solution.

Fun with Ruby Hashes - Initializing with a block

The alternate title for this post was Poor man’s email load balancer.

A less common method for creating and using Ruby hashes is to initialize the hash with a block. This allows you to return and/or assign a value for a missing key. You can think of this as a parallel to using method_missing in Ruby classes and modules.

The documentation for Hash.new with a block suggests

If a block is specified, it will be called with the hash object and the key, and should return the default value. It is the block’s responsibility to store the value in the hash if required.

Most often for the case where I wanted a default value returned, I would use fetch and set the default value for individual keys being looked for.

my_hash = {}
my_hash.fetch(:foo, "Not Set")
=> "Not Set"

my_hash[:foo] = 'bar'
my_hash.fetch(:foo, "Not Set")
=> "bar"

The same result can be achieved by initializing the hash with a block - any unknown keys requested from the hash will be initialized with a default value.

my_hash = Hash.new{|hash,key| hash[key] = "Not Set"}
my_hash[:foo]
=> "Not Set"

my_hash[:foo] = "bar"
=> "bar"

An alternative to initializing Hash with a block is to set the default_proc= on an existing hash.

my_hash = {}
my_hash[:foo]
=> nil

my_proc = proc{|hash,key| hash[key] = "Not Set"}
my_hash.default_proc = my_proc
my_hash[:foo]
=> "Not Set"

So now we have a way of always returning a default value for a hash. Not very interesting in and of itself. What else could we do with our block backed hash?

Moving from Subversion to Git

Getting git +svn branches to follow remote branch

Andy Delcambre and Robby Russell both have excellent articles about getting up and running with git as an subversion client.

The one thing that had been getting me was the fact that the branch I created locally did not always stick to following the remote branch. It would start out that way, but once I checked out the master branch again, it would not always and definitely not consistently point toward the remote branch once I re-checked out my branch.

I finally figured out that the local and remote branch must be named differently. The character case matters! So if I had a remote branch of ITR-FOO and a local branch of itr-foo, it would loose track of the remote branch. The easy solution is to name the branches differently by using a “-” remotely and a “_” locally for names or just name the branches differently. Now, I use ITR-FOO remotely and itr_foo locally, or shorten the local name to itr_promos if I am using ITR-PROMOTIONS remotely.

For merging branching, using git is so much simpler, even if it is very retarded sometimes.

MileMeter has Launched

After a year of work, my company MileMeter has launched. We sell auto insurance by the mile with no vehicle tracking devices involved.

A big hats off to Chris Gay for working so hard for so many years so that we can could get to this point. Insurance rules had to be changed and it was difficult for many people to see the benefit selling insurance this way.

We are also one of very few, if not the only insurance company powered entirely by Ruby It certainly gave us an advantage as we were able to get a robust, well tested application out the door so quickly.

ruby-mysql-osx

I keep having to look this up….. sudo env ARCHFLAGS=“-arch i386” gem install mysql — —with-mysql-config=/usr/local/mysql/bin/mysql_config

Pradipta's Rolodex

I woke up this morning to a 40+ thread of emails which originated by a recruiter sending 416 people job posting, all via TO:

It quickly turned into a somewhat fun conversation with lots of people and jokes. Rather quickly a google group named Pradipta’s Rolodex was created. Also followed was a Wikipedia entry (quickly deleted by Wikipedia admins), a FaceBook group, at least one domain registration as well as posts to Digg, Reddit and others.

Oh, the friday fun…..

Rspec Test render(:nothing = true)

I was trying to test one of my controllers was successfully hitting the render(:nothing = true) block. I could not find anything specifically in the API which handled this senario, so I ended up using have_text with a space as an argument.

@response.should have_text(" ")

Vote for Milemeter at AWS Startup Challenge

Go vote for Milemeter at AWS Startup Challenge and keep me gainfully employed!

We are proud to be 1 of 7 Finalist for the Amazon.com AWS Startup Challenge and will give our presentation next week at Amazon.com HQ. Milemeter is an innovative insurance start-up that will offer “auto insurance buy the mile”.

Please Vote for Milemeter!

Update: Congratulations to Ooyala for winning the AWS Startup Challenge.

Testing private and protected methods with ruby

When I have to test my protected and private methods in ruby, I make the methods public for the scope of the test.

MyClass.send(:public, *MyClass.protected_instance_methods)
MyClass.send(:public, *MyClass.private_instance_methods)


Just place this code in your testing class substituting your class name. Include the namespace if applicable.

Transfer email from one google account to another

I recently signed up the netinlet.com email for Google apps for you domain

I had all my mail funneling to a gmail account to begin with, so I need to get all that email from the first google account to the new google apps account.

You can do with with POP, but you have to put in a different pop server name. Use pop.googlemail.com instead of pop.gmail.com. pop.gmail.com will not allow you to access via pop. Go figure.

Everything else stays the same, just make sure to use ssl on port 995.

The process takes a while. Mine has been running for about a half an hour and has only pulled over 300 emails so far. I have mine setup to pull over all email, so everything (INBOX, SENT, ARCHIVED) gets stuck into the inbox on the google apps account.

Google Apps - html verify takes a long time

I recently signed up the netinlet.com domain and a friends work domain for the Google Apps

You have to verify ownership of the domain to get the ball rolling. They allow you to either upload a unique html page to the domain or add a cname entry which points to google.com. I chose the former because it was simple and easy to do.

Google tells you up to 48 hours before the account is active. Well, 48 hours came and went. Then a whole week came and went and I heard nothing. Searches for why this was taking so long did not turn up much.

I finally stumbled upon some http log entries which show google was trying to verify. The entries looked like this:
216.239.36.136 – - [27/Mar/2007:22:58:25 +0000] “GET /googlehostedservice.html HTTP/1.0” 412 258 “-” “Jakarta Commons-HttpClient/3.0.1”
Look, a clue! Notice the 412 error code.

After some more hunting, it turns out that the verification process is triggering Apache mod_security. Luckily, the fix is simple.

If you don’t already have one, create a .htaccess file in you webserver root directory. In this file put
SecFilterEngine Off My host is textdrive and I found a textdrive specific fix here but this fix should apply to almost everybody who is getting 412 error codes in their log files.

Reinstall Solaris 10 on a T2000

My biggest gripe with the T2000 so far is actually with Solaris package management. pkgadd sucks because it does not handle dependencies and will not automagically download stuff from Sun. I really which they had something like the FreeBSD ports collection or apt-get or yum or something which automatically downloaded and install dependencies.

The T2000 ships (the one I received atleast) with Solaris 10 1/06 release. I want to play with ZFS which was included in the 6/06 release. Plus, I hope installing some of the other packages will be easier if I install the developer packages with some amount of libraries.

So not being able to find out how to upgrade the system from 1/06 to 6/06, I downloaded the 6/06 release and am doing a reinstall.

Since the T2000 does not ship with a video card, you have to install from the console. Not having used Solaris very much , it took a little while to figure out how to get the box to boot from the cdrom.

The trick is to get to the ok prompt. You can do this by logging in as root on the console and issuing the following command:

shutdown -y -g0 -i0 now

This leaves the server running but in init mode 0.

Now, you can boot from the cdrom and start the installer with the following command.

ok boot cdrom – nowin

This will start the installer in the console mode, rather than in X.

the T2000 has arrived

The T2000 has arrived!

I can’t wait to see what this thing will do with ruby and postgresql. This one has 6 cores and 8 gig of ram.

I will more than likely have to send it back at the end of the 60 day trial, but i’m very excited to get to use it as a test server in the meantime.

It’s time to put on my admin hat!

synergy - the kvm switch replacement

At work, I regularly use two computers, my primary development box (linux) and my other development box for the windows specific piece of the program I am working on.

I have one keyboard and mouse two monitors. I tried using a KVM switch, but that sucked because every time I would switch from linux to windows and back, the scroll wheel on my mouse quit working on the linux box.

I also tried using VNC for a while, but found the screen drawing was too slow.

On one of my rss feeds this morning, I ran across Synergy. This is one sweet program!

You designate one of your computers the primary computer and start a synergy server on that computer. On the other computer, you start a synergy client. Now, all you have to do to move the keyboard and mouse from one computer to the other is push the mouse to one side of the screen from either computer. So when I am on the linux box and want to use the windows box, I just push the mouse to the right of the screen and I am then using the windows computer. To get control back to the linux box, I just push the mouse to the left side of the screen, and voila! I’m working on the linux box again.

This setup only works if you have two monitors and one keyboard/mouse. And… it works very, very well.

TIP:
I start synergy daemon on the linux(master) box with crontab. On the windows box, there is a gui option to start looking for the master when the computer is turned on.

@reboot /usr/bin/synergys —daemon —config /path/to/synergy.conf

ruby 1.8.4 on FreeBSD core dumps

I spent the better part of the last couple of days trying to track down why after upgrading to Ruby 1.8.4 on FreeBSD, I started getting core dumps in one particular place in my application.

After much time on IRC and some gdb voodoo I finally figured out the source of the problem. There appeared to be a stack management problem. Even with 64 meg of stack space, ruby would core dump. This behavior did not occur on OSX.

I emailed the ruby-core mailing list and one of the FreeBSD guys got right back to me.

Apparently the problem is with the freebsd pthread implementation and ruby.

Solution – install the port ruby18-nopthreads

Magically the problem disappears.

quick script for adding new svn files to repository

svn status | grep ? | awk ‘{print $2}’ | xargs svn add

This is on a FreeBSD box. I have noticed xargs on linux is slightly different – on linux try changing the xargs line to codeprexargs -i -t svn add {}/pre/code

updated rails from .13.1 to .14.3

The update went very smooth with this guide

All my work thus far has been on the database side, mapping incoming xml documents to my model. But there have been some significant performance increases in active record. My unit tests were taking on average 78 seconds on a 3.4 ghz pentium box running FreeBSD 5.3. That time has now been cut to 62 seconds on the same box. And that is with transactional fixtures turned off and instantiated fixtures turned on. Not too shabby!

Great work guys! I can’t praise you enough.

obfuscated mailto - ruby one liner

I have always been annoyed at having to write email addresses online such as codefoo at bar dot com/code. Damn the spambots!

I recently ran across this technique for obfuscating email addresses on web pages so that it appears normal to the user, but not something which could be easily parsed by a spambot.

You convert the link text and href to html characters.

becomes (added line breaks for readability)

</a>

 Just a little bit more cryptic.

The core method for converting a string to an html character string is just a one line ruby method, of course.


def string_to_htmlc(s)
   s.strip.unpack(“C*”).map{|ch| “#” + ch.to_s + “;” }.to_s
end


A couple of things are going on here.

codes.strip.unpack(“C*)/code We first remove any leading or trailing whitespace with the strip method. The unpack method on string is used for decoding a string (possibly with binary data in it) into an array.  The argument code”C*"/code is an instruction that the next 0 to many characters should be turned into an unsigned integer, i.e. the number which represents the the ascii character.

We now need to convert our array of integers into the html code for each character.  The html code for a character is simply the ascii integer value for that character preceded by code#/code and ending with a code;/code.  The ruby map mehtod takes care of this for us.  Array#map invokes the block for each element in the array.

Finally we convert the array back into a string with the code.to_s/code for our html pleasure.  The code.to_s/code method on array is the same as calling codearray.join/code

All you have to do now is just put some code in your rails helper module to output a full href.


module MyHelper
def to_html_email(address)
email = string_to_htmlc(address)
“a href=\”#{string_to_htmlc(‘mailto:’)}#{email}\“#{email}/a”
end

def string_to_htmlc(s)
s.strip.unpack(“C*”).map{|ch| “#” + ch.to_s + “;” }.to_s
end

end


Now it should be safe to display an email address on a webpage.

another great thing about ruby - the unit testing

The more I use ruby, the more I love it. I ran across this today – the ability to run individual unit tests.

You can run an individual test method within a test case by providing the -n switch after the ruby file containing the tests. the -n argument accepts either an absolute test method name or a regular expression which will run all the tests which match the expression.

codepre
ruby test/foo_test.rb
/pre/code
This runs all the tests within foo_test.rb.

codepre
ruby test/foo_test.rb -n test_bar
/pre/code
This runs the individual test method named test_bar within foo_test.rb

codepre
ruby test/foo_test.rb -n /bar/
/pre/code
This runs all the test methods within foo_test.rb which contain the string “bar”.

identity 2.0 - great presentation

One of the best presentations from OSCON given as a keynote by Dick Hardt has been published online. It’s a must see. I was blown away when I watched this at the conference. It’s an incredibly entertaining and interesting presentation.

Identity 2.0 – OSCON 2005 Keynote

office revenge - aluminum foil desk

When I first started at my current job, I was out sick for a couple of days and came back to my office totally re-arranged as a practical joke. Pictures were upside down, glasses taped to the desk, shoes hanging from the ceiling, and about 100 other small re-arrangements.

It took a while to get back at one of the instigators, but the opportunity finally arrived :) Deanna went on vacation last week!

A while back I had seen an email where someone’s desk was “foiled” and really liked the result. Total project time about 2 1/2 hours. So here it is: (all picts at flickr)

Interesting app - the semantic web

A friend of mine pointed me to an interesting app this afternoon. Piggy-Bank

The description from the site is
bq. Piggy Bank is an extension to the Firefox web browser that turns it into a “Semantic Web browser�

It seems to take the various website data (via screen scrape) create an RDF document out of it. That information is stored in a central location which can be queried by the said application. For instance, one of the pieces of data it collects is location. So it could collect the location of all the apartments for rent from the apartment rental website. It would also collect the location from the bus terminal website. Ultimately you could do a query to find apartments within a given radius of all the bus terminal in you location and display it on a google map.

It’s a very interesting concept. I have looked at semantic web stuff before and understood the basic concepts of the underlying technology, but did not get what it could be used for. This app makes it fall into place a little more than before.

One of the things I still fail to understand about the semantic web stuff is how to go about using the data that is published. Nobody would publish the RDF/OWL model in the same way, just as to developers would come up with a slightly different database structure. It all stores the same data, but in a different way.

Using the apartment for instance. Two different property management companies publish an RDF document for the web.

Propery Management site 1 comes up with a model like this
codepre
Apartment:
Address:
line1
line2
city
…
Specs:
square_footage
number_of_bedroooms
…
/pre/code

And Property Management Site 2 comes up with a model like this:
codepre
Apartment:
addressLine1
addressLine2
city
…
squareFootage
numberOfBedroooms
…
/pre/code

They both have exactly the same data, just modeled differently. One is more normalized than the other and the attributes/elements of the document are are slightly different.

Perhaps I am missing something but it seems that if you wanted a central storage location for semantic web stuff, you would have to do alot of mapping between one document and another. This would seem to limit the number of sites you could query because of the time and labor involved in mapping.

I really want to believe in widescale use of this technology, but I fail to see it right now.

new toy

It was time for a new toy… Since the old digital camera was old and busted, it was time to bring in the new and shiny.

I wound up getting a Nikon Coolpix 4800

I really like this camera. They did a good job with controls on the camera. You don’t have to push too many buttons to do what you want and it’s pretty intutive.

And to christen the camera, I put created a flickr account

There is supposed to be a flickr feed on the sidebar, but it does not seem to be working now. Ah… something else to get working.

new server

My host has moved locations and everyone is now running on new hardware.

The move was painless. For me atleast. Everything was working just fine when I tested it out a little while ago.

Hopefully the combination of new hardware and hosting facility will fix the downtime problems I (they) have been having.

From textdrive’s flickr account…

Switched from Fedora to Ubuntu

I switched from Fedora to Ubuntu last week on my computer at work and I could not be happier.

Setup was simple. I really like the apt-get, gentoo portage and BSD ports model. RPM is just too much of a pain.

One interesting thing about Ubuntu is that there is not a standard root account. Just like you do with OS X, you have to sudo everything. That’s fine with me. And if you really, really need root, all you have to do is sudo bash and you are root.

This biggest annoyance is that I can not find a postgresql 8.0 package for ubuntu. I really don’t want to have to maintain all the startup scritps myself.

Otherwise, I’m very happy with it so far. Hopefully it will stay that way.

Ruby Proc

I was looking at ruby proc while researching the best way to do mapping in ruby (xml-db and db-xml mappings) and ran across this blog posting on procs. The flexibility of ruby never ceases to amaze me.

http://www.ntecs.de/blog/Tech/Programming/Ruby/RubyMethodNamingConsistency.rdoc

OSCON Slides

Update: Received an email from oreilly listing all the slides from the conference…

I have been meaning to go back and collect the ruby slides from OSCON and finally got around to it.

Dependency Injection: Vitally Important or Totally Irrelevant

Metaprogramming Ruby

Yield to the Block: The power of Blocks in Ruby

I could not find Dave Thomas’ slides on the net in 1 minute or less. But I did find a ruby presentation I did not attend…

10 Things Every Java Programmer Should Know About Ruby

And some ajax stuff too…

Learning Ajax

Postgres on OSX

I’m finally ramping up for longer term development on OSX. I have been impressed with the platform for the most part. Small things are annoying, like zip corrupting files you are zipping (a friend tells me this is in part due to backward compatibility with OS 9)

I’m doing all my database work with Postgres When I first went about setting up postgres, you basically had to compile it yourself or use something like fink. I was not really happy with those solutions. After coming back from OSCON, I discovered a postgres installer from druware.com. This rocks. Point, Click, Install. In recent years, I have grown tired of building software myself after doing it for so many years on linux.

Now to access postgres from ruby and rails, you need to install the postgres driver for ruby. Change the gcc complier to 3.3 and install the postgres gem. If you have not used used any ruby libraries with compiled C extensions, you must first fix the version of ruby that ships with Tiger. with RubyGems installed, do

precode
deathstar:~ doug$ sudo gem install fixrbconfig; sudo fixrbconfig (only if broken osx ruby)
deathstar:~ doug$ sudo gcc_select 3.3
deathstar:~ doug$ sudo gem install postgres — —with-pgsql-dir=/Library/PostgreSQL8
/code/pre

(You have to tell rubygems where druware installed postgres)

Now you are ready to start doing postgres development with ruby.

If you are using postgres on OSX, the installer from druware is definitely worth checking out.

Dependency Injection: Vitally Important or Totally Irrelevant

by Jim Weirich

Slides at http://onestepback.org/articles/depinj

Is Dependency Injection vitally important in a dynamically typed language

• Short version – no
• Medium version – maybe
• Long version – don’t know

Who are you? Perhaps not who you think you are.

Building a computer controlled coffee maker. When coffee in put – burner should be on. When pot not in or no coffee in put – burner should be off.

Talks about the problem of Concrete classes being tied together and show some solutions in java and ruby (ruby modeled off java code). Push problem out by using interfaces, constructor args, getter/setters, etc but problem is just moved from one place to another.

One solution – factory pattern. But… cumbersome and invasive (examples)
Another solution – Service Locators – invasive, order dependent (examples)

Goes through a typical dependency injection system in ruby: DIY module

Gains: Flexibility and Testability Losses: complexity and indirection

This makes sense in java, but what about ruby?

Based dependency injection example in ruby on how java classes work.

Java classes are hard

• Not objects
• (mostly) source code constructs
• unchangable at runtime
• Class name directly identifies class

Ruby Classes are soft

• Just objects
• Runtime live
• changable at runtime

Is Dependency Injection relevant in dynamic languages – perhaps on very large projects, but the jury is still out.

OSCON 2005 - MetaProgramming

presenation by Glenn Vanderburg

What is meta programming? It’s Programming your programming language

Rubyist have been discovering metaprogramming. Ruby style and idioms are still changing and adapting

Ruby good for metaprogramming b/c

• Dynamic and reflexive – everything is open – blocks allow writing new control structures – most declarations are executable statements – only slightly less malleable than lisp (no macros) – unobtrusive

Examples…

attr_reader, attr_writer, and attr_accessor.

if written in ruby attr_reader would be written like (actually written in C ) precode class Module def attr_reader(*syms) syms.each do |sym| class_eval %{ def #{sym} @#{sym} end } end end /code/pre

Speaker goes through several implementations over time of different ways different people did metaprogramming with ruby.

How to think about metaprogramming

• Definiting new constructs for your programming Language
  • so what do the constructs to? whatever you domain needs it to do.

Another way to think about metaprogramming is a new set of conceptual tools for eliminating duplication (and other smells) from your code.

And another way to think about it is how rails does it – almost as if you can talk you code – PersonTable has_a :name

Most DSLs also deal with other things ou don’t usually find in general-purpose languages

• Context dependence
• commands and sentences
• Units
• Large vocabularies
• Heirachy

Contexts – context for a new set of statements – a new scope (not in 1.8, but in 1.9) precode Struct.new(“Interval”, :start, :end) do def length @start – @end end end /code/pre

Backend code looks like if you wanted to add it to 1.8 precode class Struct initialize(*args, block) struct_class = #define struct using args struct_class_class_eval(block) if block_given? end end /code/pre

Another example of context from Systir system testing tool

precode add_user { name “Charles” password “secret” priviliges normal } /code/pre

Commands and Sentences

Multipart complex statements

ex. field(autoinc, :reg_id, pk) Overall, it’s just a methodcall – the first parameter – the type – is a method call precode def autoinc return FieldType::AutoInc.instance end /code/pre

Units

Domain specific – general purpose language deals with scalars – programs must maintain their knowledge ex 3.days.from_now Watch out for operator overloading

precode class Numeric def days self * 60 end end /code/pre

Large Vocabularies

override method_missing

Usage: Roman.XXII Roman.CCIX precode class Roman def self.missing_method(method_id) str = method_id.id2name roman_to_int(str) end def roman_to_int(string) … end end /code/pre

Resources:

http://www.vanderburg.org/Speaking/Stuff/oscon05.pdf http://hypermetrics.com/rubyhacker/coralbook/

OCON2005 - The Latest and craftiest attacks and penetration Techniques and tools

by Nitesh Dhanjani

Closed source tools not good for monitoring your own networks. Lots of times they give false positives and there is no way to verify whether the positive is correct or not.

• Methodology
• finding vulnerabilities using Google.com
• Using the Nessus framework
• Web application vulnerabilities
• Other useful AP tools

Attack and penetration methodology

• Discovery (whois, traceroute, search engines, etc)
• Scanning (ping sweeping, port scanning – find the live computers and scan)
• Enumeration (service Identification, banner grabbing)
• Exploiting known vulnerabilities (research on internet)
• Installing rootkits/cleaning logs (ador on linux)

Googling for Vulnerabilities

• Find private information that inadvertently have been made public
• Stealth: find info on google – does not tell site you are looking at them (especially w/ google cache)
• looking for patterns /*/admin
• looking for error messages
• find vnc desktops “VNC Desktop” inurl:5800 – no username – only passwords – user password generators to crack
• Webcams – /view/view.shtml axis /ViewerFrame?Mode=Motion /home/homeJ.html sample/LvAppl/ etc

Go to oreilly.net and search for his name for article on how to do this.

The Nessus Framework

• utomated vulnerability scanner
• Opensource
• Client Server Arch
• Extend (write plugins) using NASL (Nessus Attack Scripting Language)

Writing a simple NASL Plugin
Web application serves /src/passwd.inc

This file contains username and passwords

the plugin will scan for this vulnerability and report it as a security whole (severe)
precode
if(description)
BLOCK
script_category (ACT_GATHER_INFO)
script_family (english:“CGI abuses”)
script_copyright(english:“foo bar baz”)

include (“http_func.inc”); port = get_http_port(default:80) if (… vunerability…)

report it
end
/code/pre

SQL Injection

causes: Dynamic SQL and lack of input validation

Authors preference for SQL injection prevention are stored procedures.

There is also Blind SQL Injection. Does not rely on verbose SQL error messages. Attempt to fetch database data. Check out Absinthe ( http://0×90.org/ )

Cross Site Scripting (XSS)

• Occurrs when a webapplication does not html output encode user supplied data
• Example http://example.com/cgi-bin/welcome?cgi?text=lt;scriptgt;alert(document.cookie);lt;/scriptgt;
• Replace above example with any JS Code (steal cookies, hijack users sessions)

Burp Proxy

• Java based HTTP web proxy
• Alter http GET and POST requests on the fly
• get it from http://

Other tools -

• Metasploit (point, click,root) Framework for developing and testing exploit code – http://metasploit.com
• Wikto: automated google and webserver vulnerability scanning (and much more)
• http://sensepost.com/research/wikto/
• Ettercap Network MITM attacks, content filtering, sniffing, etc http://ettercap.sourceforge.net/
• Whax – live linux distro – Most AP tools/exploits included – http://iwhax.{com/net/org}?
• good book is Network Security Tools – O’Reilly

Update Doh! Guess what – typo does not escape the content of a blog post. So when I posted the straight text, I started getting javascript popups with my session id an name from the above javascript code – updated to escape with lt gt symbols.

Customizing Mac OSX using opensource

by Jordan Hubbard + Kevin Van Vechten

recompiling software which come pre-built w/ osx

Darwin is the os core of osx. Includes kernel, IOKit families, and BSD commands and libraries.

Darwin source – developer.apple.com/darwin || darwinsource.opendarwing.org
.tar.gz snapshots organized by release.

OpenDarwin – community run site w/ standard opensource tools (bugzilla, cvs, irc, etc)

Webkit – webkit.opendarwind.org. Based on khtml/kjs
can progress live

DarwinPorts – similar to FreeBSD ports system. 2,500+ ports. Easy customization wiht “Variants”

Fink – Based on debian packages. 5K+ ports – offers pre-built packages.

Building Darwin

• know objective
• only replace system when necessary
• beware of software updates (have to re-apply mods)
• make backups

Potential Problems

• Default compiler problems (different gcc version)
• environment variables
• build aliases
• no private headers
• no static libraries

user __gcc_select__ to change and report compiler versions

environment variables (where system looks for some things)

• SRCROOT (src files)
• OBJROOT (object files)
• SYSROOT (debug bin)
• DSTROOT (final bin)

Variables (compile)

• MACOSX_DEPLOYMENT_TARGET (10.4)
• RC_RELEASE (Tiger)
• UNAME_RELEASE (8.0)
• RC_ARCH (ppci386)
• RC_ProjectName (name of project being built)

And many more compile time variables

Missing header files – private headers (ex. /usr/local/streams/streams.h /usr/local/lib/libkld.a)

No internal tools /usrl/local/kextool? – for building kernel modules

DarwinBuild – handles all the above incompatibilities

precode

1. darwinbuild -init AC46 (ac46 is engineering build number)
2. darwinbuild project_name
3. darwinbuild bash
   /code/pre
   This will download sources build (missed point in which you could patch source) and install new binary

OSCON 2005 - Yield to the Block: The power of Blocks in Ruby

with Matz (ruby’s father)

Presentation Slides

Interesting presentation. Valuable information, but slides moved really quickly.

One of ruby’s biggest strengths is blocks.

Increasing in popularity. 65 attendees at rubyconf in 2004 – over 200 pre-registrations for rubyconf in 2005. Reflects growing popularity of ruby.

Why? Hacker preference and killer application

Rails brought ruby to the limelight.

Blocks are unique and powerful in ruby. You can think of blocks as high order function

Martin Fowler reference is a good intoduction to closures and blocks.

Most of the rest of the presentation were code examples which Matz went over very quickly.

Quote from Matz

You (the audience) should have better japanese than I have english.

OSCON 2005 - Thursday Keynote

Arrived about 15 minutes late…

Transforming Software Architecture into Internetwork Architecture

• Not tied to any platform
• Extensible
• Generic – general purpose
• Federated
• Identifier, Formats, and Protocol Standards

Internetwork Architecture of Global trade mirrors Internet/Sofware architecture. Standard containers and standard port protocol.

DHH

Secrets behind Ruby on Rails (same talk given at FOSCON)

Ruby on rails is an integrated stack of integrated frameworks. Ships with O/R mapper, MVC controller, etc. In other words a bunch of stuff that makes web developers happy!

Has had ~100k downloads in the last year.

Has created an ecosystem of people either partly or fully earning a living from developing w/ Rails.

250+ professionals from 36 countries. More than 5K+ weblog posts says Technorati. First book already has 6K+ orders. 7 more books coming. Definite interest and buzz around Rails.

Why is rails interesting?

• convention over configuration (no xml situps – how many time do you have to tell the computer to do the same thing?)
  • As long as you follow the rails conventions, you don’t really have to do any configuration. But if you need to step out of the convention, rails supports it.

precode
class Post ActiveRecord::Base
belongs_to :weblog
has_many :comments
has_one :author, :class = Person
end
/code/pre
For example, there is no Author object in the database, so you can override the default and map it to Person (has_one line above)

• Change is instant. Goal for rails was instant changes. Make a code or db change, refresh webbrowser, and you see your changes. No redeployment, no re-compilation, just save and referesh browser.
  • This is built in functionality of Ruby, not Rails. Ruby has Introspection, Open Classes, and you can execute code in class definitions.
• Rails ships as a complete, integrated, and familiar stack. Gives you everything you need out of the box. Might seem like that reduces your freedom and flexibility but actually gives you more. Kinda like Apple computer. You buy the way they do things from hardware to software. Everything works and works well. That eliminates many of the problems with OS such as windows or linux and allows you to run and solve problems. (may have paraphrased the Apple computer comparison from last night FOSCON)
• Flexibility is overrated. Too many technologies are chasing technologies as if flexibility were free. Rails trumps flexibility and you get alot in return. Constraints are liberating. Don’t have to worry about all that infastructure and can solve problems.

HP w/ Linux (more vendor wind…)

and more wind, and more wind, and more wind touting how important opensource software is and how cool linux is. Thank you Mr. Obvious. Now talking about how much HP does for opensource. Thank you, that’s very valuable but I don’t think it earns you a spot as a keynote speaker.

Computational Origami

Origami is the japanese art of paperfolding. Decorative abstract stapes. The modern extenstion is sculptural art achieved by folding paper, usually folded from one piece of paper.

Showing picture of origami folded from one piece of paper – incredible. One sheet, no cuts, only folding. Wow!

So what changed in the world of origami that change the old way of using several sheets of paper to the artsy form of today? Mathmatics.

Can model on computer and translate to paper. Take a stickfigure, measure all the lengths and construct a set of equasions. Then can solve for the crease pattern. Has a program called TreeMaker available for download that aids with this.

Applications in the real world

Space telescopes, automotive applications to name a few. Speaker designed a lense for a sattelite/space telescope. Needed to compact it to put it into space. How did they get it on the shuttle? Origami creases to reduce size.

Airbags – used to design how an airbag flattens inside the steering wheel of a car.

Mitchell Baker Chief lizard wrangler

Portland U. Now distributes mozilla/firefox downloads – bouncer – distributes load for mozilla/firefox downloads. Portland U. is the “hub” for all the downloads.

Started a commercial (for profit) mozilla arm to pay ongoing development, be able to accept money from online ads, etc.

Lastly, Dick ? from Identity20.com (perhaps .org) was speaking. Very good speaker about and good presentation about the next generation of identity management and why stuff like Passport did not work.

FOSCON2005 @ FreeGeek

For anyone who was or was not able to attend OSCON for Ruby and Rails related things, FreeGeek (think through the PDXLUG) sponsored some of the ruby/rails speakers from OSCON to come down and give a talk. This was very cool.

I arrived a few minutes late and DHH was speaking about rails. Very cool, especially because it was not one of the talks he had given so far at OSCON (atleast that I had seen).

Also there was Vanerburg speaking about metaprogramming in Ruby. The material he presented was pretty interesting. His talk plus Dave Thomas’ talk at the beginning of the week really gets me excited about Ruby. WOW! It is such an incredible language. it has so much more to offer than what you can get out of the Pickaxe book (Book is excellent, just does not push all the boundaries of Ruby. That may have require another 500 pages.)

West spoke about recreating the NextStep API for flash called ActiveStep and integrating it with Rails. Pretty interesting, although I did not get some of what he was talking about when it was flash specific. I have never done anything w/ flash. Very interesting.

And lastly there was White Lucky Stiff. They were slightly beyond description. Two guys and a girl showing homemade artsy computer animations, playing music and telling jokes. All material, w/ the exception of a couple of jokes, was about… Ruby. And most of it was funny. Definitely not politically correct, but that’s what made it so fun.

Met Robby Russell and talked to him for a bit. Great guy.

FreeGeek rocks. Turns out FreeGeek is a non-profit that recycles computer and teaches people how to build computers and run linux (job training) in the Portland area. Their location is very cool. Kinda like a computer garage. I was very comfortable there. It was a good fit.

Also in attendance was Matz – He did not give a presentation but accepted a gift from FreeGeek. A Japanese to English Dictionary.

Update Phil corrected me. Sorry for the misinformation. Rich Kilmer talked about the ActiveStep Flash API. Why the lucky stuff rather than “White Luck Stiff”. And Matz was given a dictionary of American idioms.

Don’t trust my memory!

OSCON 2005 - Extracting Rails from Basecamp

Last session of the day. More rails!

Extracting Rails from Basecamp

Basecamp came from 37 Signals.

• Less people, more power
• Less money, more value
• Less resources, better use
• Less time, better time

Trust – the best technical decisions come from technical people when it comes down to developing applications.

Start w/ least amount of resource and then start to scale.

Tried to build half a product rather than a half ass product.

Less Software

Approach to software engineering under constraints was to write less software.

So why another framework?

Experienced PHP background w/ some java.

Basecamp originally written in PHP. Hit wall with PHP – it fought back against what he wanted it to do.

Inspired by java but it was too much enterprise. Too much focus on the 99.999% and not enough on the 98%. Geard toward building the amazon.com of software, not the single developer hacking away.

So… Liberate the good ideas from the different languages. Ruby was the new place to liberate those ideas and make them not hurt.

Calls himself a shallow programmer

• Aesthetics
• Joy
• Less

So halfway through Basecamp he realized this might be useful to other people. And now we need an extraction.

Doesn’t work to design a framework before you start coding. That’s the cart before the horse. You are then limited to what you designed.

Need application driven development to make good frameworks.

So why opensource it.

• Selfish. Let other people do some of the work. Write it youself or let some one else help you. Only works when there are selfish reasons on the other end (they gain something too)
• (and two other reasons)

And it works. 1000 patches in 9 months. Extract, pass, reap, rinse, repeat.

Now need to get traction for your opensource project. If nobody is looking, it does not matter.

Greatest fear as an opensource developer is obscurity. Need a network to reap the effects.

Opensource programmers need to get rid of their academic humbleness and bring rave about the good stuff. If you are not passionate, nobody else will. Need to set a baseline of excitement. Passion is infectious. You decide how high to set the bar. David set the bar very high!

Self delusions work!

Go Looking for Trouble. Tout advantages over the known. “If it bleeds, it leads”

Dealing with Traction

Now you need to scale your culture

• Early influx can bend you out of shape.
• Release not so early, then often. (get it mostly like you want it then release and release often. That way there is less debate about the way things should work)
• Set a viral example of kindness. Care and show direction for newbies. Don’t tell anyone to RTFM. That will drive your users off and set and example for your culture.

Items will be added to rails by necessity. Example i18n. Not many people using it. Some people have tried it, but once enough people need and are using i18n, it will appear almost automagically and be useful.

Best Hacker of the Year

Congrats to David Heinemeier Hansson for the O’Reilly/Google best hacker of the year

OSCON 2005 - State of the opensource databases

Ingres _ by CA_

Ingres r3 is the version which was put into opensource.

Features

• Value based table partitioning (a-c go here d-x go here, etc)
  • can partition tables based on value of records
  • No application change required
• parallel query execution
  • may utlize more than one cpu on MP machines for a single query.
  • single processor machines also benefit
• Advanced query optimization techniques (query decomposition, flattening, and rewrite)
• Federation through ingres star
  • support two phase commits for distributed transactions
• Replication of master at one or more slaves
  • can also have multiple masters where the other master acts as a slave when doing updates to one or the other
• next release to support grids
• admittly driver support is lacking
• winblows only db gui

MySQL with David and Monty

• runs on 87 gazillion platforms
• Connector/MXJ – embedded jdbc within java (embedded java database???)
• Storage engine slides…
• Special storage engines
  • Archive – logging data you don’t want to delete or update
  • Blackhole – allows replication but throws away all data
  • MyISAM for logging: prefix locking allows fully concurrent inserts and reads

MySQL 5.0

<ul>
	<li>Stored procedures</li>
	<li>Triggers</li>
	<li>Views</li>
	<li>XA &#8211; distributed commits across different databases</li>
	<li>Data dictionary</li>
	<li>Server side cursors</li>
	<li>Precision Math &#8211; exact calculations with well defined rounding and atleast 56 digits precision.  Very fast w/ static memory allocation</li>
	<li>Strict mode &#8211; ability to get rollback/errors instead of closest value/warning messages (eh? can tell you when it is a bad date, 02/31/2005)</li>
	<li>Federated storage engine</li>
	<li>greedy optimizer (fast multi table joins)</li>
	<li>instance manager (replaces mysqld_safe script)</li>
	<li>extenstion to <span class="caps">LOAD</span> <span class="caps">DATA</span> for doing transformations/calculations at time of load</li>
	<li>5.0 still has 327 bugs but 16 really bad bugs at time of this writing.</li>
</ul><ul>
<li>Upcoming features
<ul>
	<li>partitioning (needed for 20 petabytes that one user is planning)</li>
	<li>replication additions &#8211; row based (physical) replication (normal is logical) + multi source replication</li>
	<li>global backup api</li>
	<li>mysql cluster w/ disk data (non indexed columns)</li>
	<li>Hash  Merge joins</li>
</ul></li>

</ul>

Firebird w/ Ann ?

• High concurrency, high performance
• low administration costs
• flexible architecture
• active project

Development seems to be fairly slow going. Borland reniged the opensource license and Firebird is a fork. First releases were mostly bug fixes. 1.0 and 1.5 can share same data file.

Firebird Strengths

<ul>
	<li>Flexible architecture</li>
	<li>processed based (fork)</li>
	<li>multi-threaded shared server</li>
	<li>embedded</li>
	<li>low admin cost
	<ul>
		<li>self balancing indexes</li>
		<li>cooperative garbage collection</li>
		<li>single file database</li>
		<li>transportable backup</li>
	</ul></li>
</ul><p><i>Firebird future (2.0)</i></p>

• currently in alpha 3
• global temporary tables
• execute block
• physical backup

Vulcan

• in parallel development
• fork to work on 64bit arch
• internal sql

Postgresql 8.1

8.1 features are frozen now. beta release in mid-august or early september.

Key New Features:

• Indexes combined using bitmaps – index any attributes you want to query on later. then when you do a join across different attributes, the engine bitmaps the indexes and can perform very fast lookups (knows where to find pages and other data structures)
• Two-phase commit for distributed transactions
• Automatic vaccuum process
• Global shared buffer lock removed, improves SMP support
• Functions returning multiple out parameters (oracle feature)
• Shared row locks

Pervasive, Green Plum and EnterpriseDB and Unisys building products/businesses around Postgres. Approx 1.5 million downloads of 8.0.×. and lots of new users (notable NWS and Ameritrade)

OSCON 2005 - Webwork vs. Spring spackdown

Matt Raible and Matthew Porter

Not much to report here. Mostly a bitch session about what’s wrong w/ java frameworks. I could not help but feel sorry after using rails for the last couple of months.

Matt and Matthew are both good presenters. Good light presentation for the day.

OSCON 2005 - Firefox extension development

Tools – XPIStubs (pronounced Zippy Stubs) Use this and forget half of what you need to know. Still needs some work.

Speaker looks like Hani (bileblog)

Looks like this tool stubs out everything you need to develop firefox extensions. Run the program to stub out program, do a ./configure make and it installs a plugin into firefox that pops up a window “Test”.

All you need to know to get started developing firefox extendsions are

• XPIStubs to get quickly stubbed out
• XUL and javascript for UI
• XulPlanet.com for reference

Extensions allow you to add new features to existing applications and add new functionality to browsers.

Why user firefox:

• Cross platform applications
• add content to the users browser
• provide the user w/ a better experience through client side customization
• easier than many alternatives
• b/c it’s cool

XUL
The xml interface language for mozilla/firefox

It provides

• layout and base widget set
• uses css for styling
• event handling via javascript command sets
• overlays

And… it may be loaded remotely or locally.

Command Event

• trap events like button clicks and key presses and allow you to respond to those events.

Broadcasters

• like command but can push out

Controllers

• Command sets can call on controllers to respond to events (like focus)
• Controllers are javascript classes that update state for commands

RDF and Templates

• allows you to define a template for displaying RDF data
• Difficult to use but very cool

XBL Bindings

• XML binding language
• allows ou to create new widgets or modify existing widgets

XPCOM

• cross platform components
• in C++, XBL or javascript. Also support scripting languages (PyXPCOM)
• Components are global to all chrome packages
• possibly the most important part of GRE

XPI: Cross platform Installer

• packaged in a jar file
• handles cross platform installs

Dev Tools

• typically use text based tools (vim, emacs, notepad??)
• Dom Inspector
• cview (XPCOM inspector)
• Venkman (javascript debugger)

Not Bad – rest seems like it will be demos, so shutting down to save batter power.

Update – ACK! It almost looks like jelly – programming in XML – WAY WAY Verbose

OSCON 2005 Keynote

Over the keynote was just OK. Not terribly exciting.

Tim O’Reilly is an awsome speaker as always. He seemed to keep it short this time for some reason. one thing he said that struck a chord, was once you create an standard, values moves up the stack. He believes opensource is doing this now, moving up the stack of standards (hardware).

Next up was Kim Polase (think I got her last name right). Good speaker but did not need to be delivering something as part of the keynote. Her talk seemed to be a plug for Spikesource.

Next up was Andrew Morton, linux kernel hacker extradonaire for OSDN. Difficult to understand b/c of how he spoke into the mic. Definitely a developer. Spoke a little in a drone about the integration of opensource and the commercial world.

Lastly was an interview with Jonathan Schwartz. Says he is happy to see an opensource implementation of Java (apache harmony project) but license for Java will not be open b/c they don’t want it to fork. Of opensolaris, says there is now no need ot compare big, bad closed source solaris to open linux. Can now compare the features of one to the other since they are both open. And to show that he was not 100% full of shit (maybe that is a little strong) said netbeans has the features it has today because of eclipse.

I think I would have rather have seen Tim give the entire keynote. I did not get much from any of the other speakers.

Update: Introduced myself to David HH (Rails) after keynote. Nice guy. Awsome framework.

OSCON 2005 - Learning XSLT

This update is late since my server was down during the track. Also my notes were pretty shoddy since there was so much info I was trying to digest plus I was getting sick during the track. My ribs ached and I my teeth were chattering b/c I was so cold. I believe I had a fever. Must have been some kind of virus.

The last time I tried using XSLT, I really, really did not like it. Lots of frustration and curse words. But given that I have to work with XSLT in my current project (to transform word xml templates) I decided to attend. I was going to attend Eric Hatchers Opensource gems, but that talk seemed to be mostly about Lucene and using a few opensource java projects. Interesting but given ruby has changed my interest in java and I need to use XSLT, well…

I’ll spare you my shoddy notes. Have a look at the speakers slides from the class. My notes mostly copied much of what was in the slides anyway. Plus the slides mirror much of what is going to be in the speakers upcoming Oreilly book.

Evan Lenz did an excellent job at presenting XSLT in a clear, very understandable way. I walked away feeling brain overload but feel like I learned something that I can apply.

Background on speaker, Evan Lenz

• Recommends Michael Kay’s XSLT Programmers Reference
• Participated on the XSL Working Group for a couple of years
• Wrote XSLT 1.0 Pocket Reference
• Preparing for PhD program in Digital Arts and Experimental Media

bidwell down (AGAIN!)

Well it was yesterday during OSCON and prevented me from blogging.

This is getting nuts. Seems to be occurring 2-3 times/week lately.

I really like textdrive but the downtime is getting really annoying. Particularly since I am trying to blog OSCON and the server goes down right in the middle of a track.

They are using FreeBSD, which I really like, but when you look at the status of bidwell when it is down (status.textdrive.com) it seems to be __fsck__ing for a long long time. Why doesn’t BSD have journaling file systems like linux. I understand why they can’t use linux’ but seems like they would have started development on one. On the linux boxes I use, it is so nice not to have to go through the fsck.

OSCON 2005 - Learning Ajax

Demos and slides

You’ll walk away with

• Enought code to be dangerous

Damn. I thought I was already dangerous. Crap.

Demo Life in a text area
Interesting – autosave in a textarea – much like autosave in a text editor or something like MS Word.

What is ajax

• still being defined – good portion still up to you
• No page refreshes – Don’t make me blink
• don’t make users and developers learn new idioms.
• Technology should not slow you down – ajax does not fundamentally change what you have been doing.
• Better interfaces without redesigning from ground up.

Should turn browser into a client for your protocol/api

Good, Bad, Ugly

• Good – more responsive, more intuitive interfaces
• Bad – Inaccessible by default (google web accellerator)
• ugly – debugging takes a village

When is ajax the answer

• give advantages of desktop apps in a web browser.
• when you need to do things that traditional request/refresh can not accomplish
• when the competition does it (you gotta do it too)
• when it makes the users experience better, not worse

Ajax basically manipulates the DOM. Everything is a node.

It’s moving a little slow at the moment. Talking about the javascript api for manipulating the dom and about how not everything is a node (text for example)

Easiest thing to make code portable is use getDocumentById() call

The speaker is sick! a big chunk of custom javascript for moving around paragraphs on a page. Glad there are toolkits for this. It would sound like a turrets party if I had to do it.

Did not know this – can set style propery of any node and can access css styles using CamelCase names.

Mozilla and MSDN have good dhtml references and speaker thinks MSDN has the best.

Limitations can’t request resources which are not on the same domain (XMLHttpRequests). you could proxy if needed

~4k is the usual typical upper and lower limit of what you can store in client cache (cookies) on a per domain level

Browser incompatibilities – write to the spec and use http://quirksmode.org as a reference for cross platform incompatibilities.

IE has conditional comments – block of code is commented out, but runs in appropriate browser. For example would only run in IE5 or IE5.5, or IE6, etc. Conditional Comments are not used very often, but very useful for determining which one of the many xmlhttprequest objects to load.

.innerHtml is a read/write attribute which modifies the dom. Should uses sparingly but is very useful.

.innerHTML example
precode
var fooNode = document.getElementById(“foo”);
var parentContents = node.innerHTML; // where innerHTML = ‘div id=’foo’/’
node.innerHTML = parentContents;
// fooNode is no longer a valid reference here
/code/pre
Point is if your program depends on a node being there, like an onclick handler, it may be going if you do not use innerHTML judiciously.

What to return

• html
  • easy to insert into document (fast)
  • can return a string – easy to debug
  • difficult to make portable
  • implies replacement, not updates

• xml
  • usually supported, MUST use “text/xml” document type
  • doesn’t tie your server to your rendering
  • you’ll need to buildUI on the client
  • xml is typically not fast b/c you have to traverse the document (usually w dom)
• Plain text
• javascript
  • fastest for large data sets
    • eval() is the fastest parser available
  • native language of you scripting environment
  • skips the xml-to-data structure step
  • app more likely to speak xml or html
  • JSON helps (standard proposed by ?? – a lightweight subset of what can be descript in javascript literal)

You could send back javascript for large data sets where the javascript you send back would have something like a large dataset in an array. Smaller in size and faster to render than xml/html.

Engineering for ajax

• Server-side design principals
  • ajax-style UIs should call the REST APIs you apps already expose – example Flickr
  • multiple points of entry
  • Single request, single action
  • more requests, smaller requests
• When retro-fitting existing apps, wrap at the module level, don’t write new code

Back from break – talking about autosaving

• xmlhttp for moving data back and forth. Use HTTP POST verb, idempotence matters.
• serInterval() for occasional execution
• event handlers wired to Nodes, IDs
• brittle – hard to maintain or reuse

XMLHTTP Basics

• synchronous or async communications (most calls will be asynchronous)
• simple callback scheme for status
• some verbs problematic (Safari KHTML)
  • HEAD
  • PUT
• File upload not available cross-browser
  • can be handled with iframes and form hacks

Now covering different ajax toolkits

Prototype by Sams Stevenson Ajax framework baked into Rails http://prototype.conio.net/

Dojo Ajax framework written by speaker, Alex Russell http://dojoframework.org

The toolkits (showing Dojo) have built-in graphics for UI display, example a progress indicator or spinner.

Whoa…. Dojo is pretty impressive. As I understand it, it’s sorta like tapestry but for ajax. you create widgets for say an autosaving text area. You can drop a widget on any part of your app and override default values by providing a dojo widget html fragment (div with some specific elements). Not tied to dom id, node, etc.

Debugging Tools

• Mozilla JS console
• Safari JS console
• Opera JS console
• IE Script Debugger
• Rhino or WSH – command line JS

Rhino command line example
java -jar js.jar

brings you into a javascript shell (like irb or python shell) poor interactive command line

example.js
precode
function foo() { print(“BAR”) };
/code/pre

java -jar js.jar example.js
precode
js foo()
BAR
js
/code/pre

Advanced Tools

• LiveHTTPHeaders – Mozilla/FF – Mozilla extension
• Venkman – Mozilla/FF JS debugger
• Squarefree JS Console bookmarklet
• MSE JS debugger for IE
• Konqueror
• VirtualPC/VMWare (for testing different browsers)
• Ethereal (w00t!)
• your tenaciousness

The talk was pretty good. Did not find myself getting bored through alot of it. I’m just glad someone else develops the JS libraries so I don’t have to.

Update Did not get to finish blogging while in the class b/c the server where I host netinlet.com went down (pretty much until the end of the conference that day). Was able to save all the content offline though and updated that. This is the repost of original + additional content.

OSCON 2005 - Day 2

On the agenda for today is “Learning Ajax” with Alex Russell and probably the XSLT track with Lentz. As much as I hate XSLT, I should probably go because I am having to use it for my current project. Perhaps I’ll learn something.

Was just looking over my blog from yesterday, and they pretty much suck as compared to this and this
Mine seem more like notes.

OSCON 2005 - Day 1 (Rails w/ DHH)

Kickoff – should be a code heavy presentation…

Install ruby dated 12/25/2004

gem install rails to install rails (requires ruby gems)

Demo based around creating a blog app.

David is giving a detailed explanation of directory structure and built in webserver. Seems to be standard stuff in docs. Major point is that everything is layed out for you so you can get started more quickly and do not have to make those decisions.

… Speaks about the ./script/generate program which stubs out controllers, models, etc.

if a rhtml file is named the same as a method on the appropriate controller, that rhtml file will be rendered by default as the return from the action method.

precoderedirect_to :action = “method_name”/code/pre will jump to another action method on controller.

./script/destroy can remove models/controller/etc just like it can generate them.

Rails has extended the to_s method to include parameters (for date) so you can say my_date.to_s(:long) or my_date.to_s(:short)

foreign key names are not plural – should be same as model name. fk or comments to posts is post_id not posts_id (sorry if that one is not in context of anything else)

can specify foreign keys when defining the relationship
precodehas_many :comments, :foreign_key = “crazy_dba_convention_id”/code/pre

with ./script/console you can interact directly with your model.

Pass in :locals { :variable = my_variable } you can render specific to what ever scope you are in.

Foreign key collections have extra convience methods such as create and build.
@post.comments.create(blah) where comments is a collections of comments on post.

layout the reverse of jsp include. view maps to controller name and is used as a template for any html that needs to be rendered for that controller.

Use ApplicationController which was generated as part of generating your project for extension points such as authentication (ApplicationController is a base class of all other generated Action Controllers)

Unit tests with fixtures, fixtures load the entire fixture file for that db class and loads the values into instance variables by their name.

precode
def test_fixtures_work
assert_equal “FooBarBaz”, @my_first_post[‘title’]
end
/code/pre
Where there is a fixture entry with the title my_first_post with the title of ‘FooBarBaz’

Unit tests usually only useful for testing your methods and the domain model.

Functional Testing Mocks already built for simulating gets and posts. Used for testing controllers and workflow of controllers. Can assert things such as responses (assert_response :success assert_template ‘foobar’)

When functional testing, you can test create or update methods similar to this: (code may make no sense, but methods will be like this)
precode
def test_create_post
post :create, :post = { :title = “foo”, :body = “bar” }
assert_response :redirect
assert_kind_of Post, Post.find_by_title(“This is my title”)
post :create, :post = { :title = "", :body = “bar” }
assert_response :success
assert_equal “don’t leave me out”, assigns(:post) .errors_on(:title)
end
/code/pre

AJAX
In template do
precode
%= javascript_include :defaults %
/code/pre
damnit, think the javascript_include is correct but may be wrong – DHH moved on.

use the remote call replacement (linkto_remote instead of link_to for example) and pass in parameters (controller, id of element in html, etc)

can call precoderender :nothing = true/code/pre in action if nothing needs to be rendered for ajax call

can distinguish ajxx from non ajax by using the method request.xml_http_request?

DHH must be tired – he’s ajaxing the wrong files and it’s getting slightly confusing.

Need to clear input fields when ajaxing a form. Example if you have inputs for name and address, you will have to clear the name and address input boxes.

OSCON 2005 - Day 1 (Ruby w/ Dave Thomas)

Finally…, it’s here. And the line up for today is Intro to ruby w/ Dave Thomas and Rails with DHH.

Dave Thomas is one of my most favorite technical authors – this should be good…

Introduction to Ruby

Is programming still fun? in and of itself it is fun – it’s the language and the tools. Rise of scripting language make it more fun b/c less time to run program.

Ruby born in japan 1994 – Mats

Ruby passes the 5 minute test.

Wait – this is funny – Dave is presenting in socks.

Ruby removes alot of the inherited cruft of other languages – no need to put () around method and class definitions – don’t need to put ; at the end of a line. These things are unnecessary -the compiler/interpreter can figure it out.

new is not a keyword. It is built into the object. Makes overriding the functionality of creating the object more flexible (perhaps by using class level variables?)

attributes begin w/ @

Attributes and methods are one and the same. Makes for universal access
precode
@my_var
attr_reader :my_var
/code/pre

is the same as

precode
def my_var=(x)
@a = x
end
def myvar
return @a
end
/code/pre

Blocks and iterators are pervasive in Ruby
The santa clause theory: precode3.times { puts “Ho!” }/code/pre

yield inside a method definition looks at code and stores in your “back pocket” which it executes later. (Thats a pretty crappy writeup…)

Ruby convention for iterators. Block should test for params. If block is not given , code should store in an array and return at the end of the method/iterator call.

Blocks as callbacks – use callback syntax as argument to initialize. If block not given, nil is stored. Can use later like
precode
callback.execute if callback
/code/pre

Resource management w/ blocks. Use block with resources to ensure resources are closed afterwards.

precode
File.open(‘/etc/passwd’) do
#… do something
end
/code/pre

File will be closed automagically at the end of the block.

Did not know this. Per-statement exception handling
precode
def x(name)
f = File.open(name)
yield f ensure f.close
end
/code/pre
where yield f ensure f.close is the per statement exception handling

Transparent block passing
precode
def x(name, callback)
File.open(name, callback)
end
/code/pre

Duck typing. Ruby has strongly-typed objects but untyped variables and methods. Type is determined by object protocol (by what object can do) Just have to make sure the types you are passing in support the methods that the object needs. If it walks like a duck and talks like a duck, it must be a duck.

Ruby community differentiates the type and class of an object

Metaprogramming mini DSL’s within you classes
How do we get there:

• Classes are open and can be extended – ex: can add stuff to string. ex – could add an encryption method to string so you could say cat.encrypt. yes it is dangerous to redefine core class methods like overriding the + method on Fixednum, but just b/c some idiot might do it doesn’t mean it should not be there. I you do something stupid, take ourself out to the parking lot and beat self w/ rubber hose.
• Definitions are active Add to classes with things such as adding tomorrow and yesterday methods to the Time class. One thing that was really driven home for me is that you can execute arbitrary code in you classes (functional). For ex. Caching. in one of your classes, you could read a file and store it in a string as part of the definition. Then methods on the call would have ready acccess to the string or array. Put another way, you can write code that executes during class definitions.
• All methods calls have a receiver “ruby”.length “ruby” is the receiver.

This is very powerful for metaprogramming
precode
class Doug
end
class Bryant
end

variable = day.even? Doug: Bryant
variable.do_something
/code/pre

Learned you can use the “inherited(subclass)” method on base classes and will automagically be called when you extend the base class.

Other stuff
Object space can tell you about objects at runtime. ex. give me all the strings in my program. Useful for such things as debugging – why is that person object still hanging around

Dave Thomas is an excellent speaker. I was very impressed. If you ever get a chance to hear him speak, do it. He has a very clear and concise way of explaining things. Not to mention a good sense of humor. He keeps things technical and assumes the audience are not all dumbasses – he knows we have all done some sort of software development before and does not stick on the syntax of the language.

ruby, ruby, ruby. learning ruby

I have been studying ruby very closely this week while trying build up my ruby chops.

Some interesting articles I ran across…

Explanation of class self At ruby garden here and
here

And another interesting article about ruby garbage collection

OSCON 2005

I just got signed up for OSCON 2005 in Portland. I’m totally stoked. There is a great line up this year.

Some of the tracks I intend to attend… (Very Ruby Heavy)

• Intro to Ruby w/ Dave Thomas
• Ruby on Rails: Enjoying the Ride of Programming
• Learning Ajax
• Integrate: Building a Site from Open Source Gems
• Application Development With Firefox and Mozilla
• WebWork vs. Spring MVC Smackdown
• States of the Databases
• Open Office Xml Doc Format
• SiteMesh: A Simple Approach to Web Site Layout
• Tapestry In Action
• Ruby Blocks
• Pragmatic Project Automation with Ruby
• Metaprogramming Ruby
• Dependency Injection: irrelevant?

Holy Cow! I knew it was ruby heavy, but did did not realize how heavy it was until I saw it listed here.