Notes on Life, ‘Puters and Hawaii

JRuby 1.1 Has Been Released

JRuby, the awesome Java meets Ruby project, has been released and tagged as 1.1. This is really good news. Thanks should be patted on the backs of each JRuby developer for this milestone. If you haven’t checked JRuby out, now is the time.

Here is the announcement http://docs.codehaus.org/display/JRUBY/2008/04/05/JRuby+1.1+Released

E-Trade: An Entire Company of Bumbling Baboons

Don’t even try opening a online business bank account at E-Trade. I finally got my business bank accounts open….. After a month of hassling with E-Trade I got Bank of America to open my accounts in less than an hour.

Here is the story: E-Trade messed up the account application. We had to send two faxes and our personal information and make about 20 calls over 4 weeks to try to resolve it. 4 weeks! 20 phone calls! OMG I wanted to shoot someone. It seems they have no ticket system to track customers problems and poor communication amongst their workers.

Every person we talked to gave us a new different story. It got to the point where we told a manager if they didn’t pull their head out we were going to go with another bank.

So we signed up with bank of america and 5 accounts took 30 minutes. Those commercials where the person clicks one button and says “I just bought some stock on e-trade” make me laugh now.

It’s About to Git Crazy: Rails Moves to Git

Rails-core announced yesterday that they are moving to Git source control hosted at github. Github is this really nice social-networking-meets-source-control site that helps support developers using Git help each other. It’s hard to explain but it is just awesome. It’s like Linked-In or Facebook for open-source developers using Git.

There was a lot of feet dragging and complaining by folks that like Mercurial better or use Windows (Ouch!) over at the rubyonrails blog entry.

I switched to Git 100% months ago for all my projects and haven’t looked back. It changes everything you know about source control. I think this is a really smart move by Rails-core team. I think the amount of contributions to Rails is just going to go ballistic. Acceleration to the next level. …Hive-mind…

Weekend Camping at Anini Beach

I went camping at Anini beach this last weekend. Got a sun-burn. Played frisbee until I was beat. Read some books. It was great. I had not camped at Anini in 13 years. It has to be one of the top 10 camp sites in the world. Being there in the shade in a hammock for more than 5 minutes shifts your mind into meditation mode. Take a look for yourself. :)

Anini Beach

Creating Blank Git Branches

Most of the time in git you will be creating branches of your main project and working on them. What if you wanted to create a ‘documentation’ branch or something? It doesn’t really deserve it’s own repository because it’s so closely related. The git project repository does this. The git project repository has separate branches for master, docs and man pages etc too. Here’s how you do it.

Go into your git project and type

git symbolic-ref HEAD refs/heads/empty
git rm -rf *
echo 'New blank branch' >README
git add README
git commit -m 'New blank branch'

That’s it - now you have a new branch ‘empty’.

Amazon Releases a New EC2

Amazon released their new EC2 API today.

The big news: You can have static IP addresses now. You can pick the data center where your machine instance is launched. Cool!

The minor news: Lots of little things were updated. They added tmpfs to the list of LOCAL_FS_TYPES so ubuntu volume images work without patching ec2-ami-tools. You no longer have to worry about having rsync lutime problems when creating a volume image. You can now use an upgraded Linux Kernel which comes with Fuse.

I was busy today upgrading all my images and notes. Gentoo and Ubuntu images were pretty easy to reconfigure. I have created a new public Ubuntu Hardy image that’s in my ubuntu-i686 bucket. Just list all public images to see it.

Experimental Jet Set, Trash And No Star

I am sitting here writing code on my laptop and listening to Sonic Youth. I have been on a SU marathon since yesterday. There is something unique about Sonic Youth for me. I don’t listen to them all the time, but when I do it really moves my soul. It sounds like random noise some of the time but it’s programmed deep in my brain. The music can be nostalgic/sad, angry, happy, melancholy, rowdy, haunting, defiant - so many moods. If I had never listened to them before now, in my *cough* later 30s, I don’t think I would like it. Having listened to them so much long ago and remembering them now, it triggers nostalgia-like feelings in me of when I was younger. That can be fun.

Viva Sonic Youth.

PS - They are playing in London on August 31 at the Roundhouse. Maybe I could stop by on the way to Germany! /me wants to go!

Rollerskating Ruby

Ever since I read Martin Fowler’s article on rollerskating and shared it with my partner in crime, we have been using the term and practice liberally. Fowler describes a customer service web form that a guy he knew put together. This customer service form wasn’t quite ready for the internets yet but they needed it yesterday. He talks about delivering iterative hacks and at one point they actually had someone on rollerskates going from one side of the building to the other to take information from a fax machine and input it into a database.

I have long been of the school of thought that you didn’t release anything until it was ready, tested in battle, accepted by the user and proven to supply the oxygen for a 911-call-center. This may have been a by-product of a decade and a half with static languages, consulting and writing code on projects for dead-serious large-businesses that would not stand for mediocrity. I don’t know. Now that I have been on Ruby for quite a while and experienced just how flexible dynamic languages are, I am sold on rollerskatin’….. for some applications.

I still think that a little planning and prefactoring, domain-driven design and behavior-driven development goes a long way towards crafting perfection. However, for systems that are behind the scenes at a start up and needed to be finished yesterday, roller-skating is smart business. It’s about writing the least amount of software possible to get the job done as soon as possible. Later when the jets cool, you’ll know more about what works and what doesn’t and you be in even a better place to refactor or rewrite.

This is not a license for a sloppy user experience whatsoever. The customer using your shiny new webapp had better be oblivious to your rollerskating behind the the curtain. Their experience with your service and with your business must be squeeky clean and enforce the brand of excellence you are striving to portray and deliver.

Rollerskatin

RailsConf ‘08 hotel for $7/night! Woot!

I scored a hotel for two weeks in Portland by the airport for $7/night! Woot! What a score.

railsconf08hotel.png

Creating the Perfect Gentoo Amazon EC2 AMI (image)

Update: I need to upgrade this for amazon ec2 2008-02-01 api.

I been playing with Gentoo again. I hadn’t been an active Gentoo user since it pissed me off in a emerge -u world snafu in 2004. I created some Gentoo EC2 images and thought I would share with you all.

I have recently stopped using Xen to create new images and started using Amazon EC2 AMIs to create new AMIs directly — “dog food”-style. The script below is an example of this. There is no need to have 32 & 64-bit Xen Dom0 machines around the house to get started creating custom AMIs. All you need is an Amazon EC2 account. Just fire up someone else’s Linux image and go to work creating a new AMI. I have been using Amazon’s Fedora 4 “developer” 32-bit “small” image to create a nice lean Gentoo image. Here is my script.

# Boot a developer image at EC2 && Login as root on the instance

# Move the /tmp dir to the big drive
mv /tmp /mnt && ln -sf /mnt/tmp /

# Bootstrap
mkdir /mnt/gentoo
wget -O - \
  http://gentoo.osuosl.org/releases/x86/current/stages/stage3-i686-2007.0.tar.bz2 | \
  tar xjC /mnt/gentoo
wget -O - http://gentoo.osuosl.org/snapshots/portage-latest.tar.bz2 | \
  tar xjC /mnt/gentoo/usr
wget -O - http://s3.amazonaws.com/ec2-downloads/linux-2.6.16-ec2.tgz | \
  tar xzC /mnt/gentoo/usr/src
zcat /proc/config >/mnt/gentoo/usr/src/linux-`uname -r`/.config

# FUSE module (has to be compiled with the same gcc as ec2's kernel)
cd /tmp
wget -O - \
  http://superb-west.dl.sourceforge.net/sourceforge/fuse/fuse-2.7.3.tar.gz | \
  tar xz
cd fuse-2.7.3
./configure --enable-kernel-module \
  --with-kernel=/mnt/gentoo/usr/src/linux-`uname -r`
cd kernel
make && make install
mkdir -p /mnt/gentoo/lib/modules/`uname -r`
cp -r /lib/modules/`uname -r` /mnt/gentoo/lib/modules/`uname -r`

# Setup
cat /proc/mounts >/mnt/gentoo/etc/mtab
mount -o rbind /proc /mnt/gentoo/proc
mount -o rbind /dev /mnt/gentoo/dev
mount -o rbind /sys /mnt/gentoo/sys
cp /etc/resolv.conf /mnt/gentoo/etc

# Chroot
chroot /mnt/gentoo /bin/bash
env-update
source /etc/profile
export PS1="(image) $PS1"

# Modules / Kernel
depmod -a
modprobe loop
echo 'loop' >>/etc/modules.autoload.d/kernel-2.6
echo 'fuse' >>/etc/modules.autoload.d/kernel-2.6
cd /usr/src && ln -sf linux-`uname -r` linux

# Cleanup
cd /
rm -rf tmp && ln -sf var/tmp tmp
rm -rf opt && ln -sf usr/local opt
rm -rf boot

# Root
usermod -p \
  `dd if=/dev/urandom count=50 2> /dev/null | md5sum | cut -d " " -f1-1` \
  root

# Rebuild
cat >/etc/make.conf <<\EOF
CFLAGS="-O2 -march=i686 -pipe -mno-tls-direct-seg-refs"
CXXFLAGS="${CFLAGS}"
CHOST="i686-pc-linux-gnu"
MAKEOPTS="-j2"
EOF
emerge --sync
emerge -e world
emerge --update --newuse --deep world ; # are these both needed ^ <-
etc-update
emerge eix gentoolkit
emerge --depclean
revdep-rebuild

# Locale
cat >/etc/locale.gen <<\EOF
en_US ISO-8859-1
en_US.UTF-8 UTF-8
EOF
locale-gen

# Timezone
cp /usr/share/zoneinfo/GMT /etc/localtime
cat >>/etc/conf.d/clock <<\EOF
TIMEZONE="GMT"
EOF

# Mounts
cat >/etc/fstab <<\EOF
/dev/sda1 /        ext3  user_xattr          0 1
/dev/sda2 /mnt     ext3  user_xattr          0 2
/dev/sda3 swap     swap  sw                  0 0
shm       /dev/shm tmpfs nodev,nosuid,noexec 0 0
EOF

# TTY
perl -p -i -e 's/^c([^1])/\#c$1/g' /etc/inittab

# Network
emerge dhcpcd ddclient net-misc/ntp
rc-update add net.eth0 default
rc-update add sshd default
rc-update add ntpd default
cat >/etc/ssh/sshd_config <<\EOF
Protocol 2
StrictModes yes
MaxStartups 10:30:60
Ciphers aes256-cbc,aes256-ctr
PasswordAuthentication no
ChallengeResponseAuthentication no
Subsystem sftp /usr/lib/misc/sftp-server
UseDNS no
EOF

# Boot
cat >/etc/conf.d/local.start <<\EOF
# /etc/conf.d/local.start
# Root SSH Public Key
[ ! -e /root ] && cp -r /etc/skel /root
wget --timeout 15 -q -O - \
  http://169.254.169.254/2007-12-15/meta-data/public-keys/0/openssh-key > \
  /root/.ssh/authorized_keys
chmod -R go-rwsx /root
# Userdata Shell Script
wget –timeout 15 -q -O - http://169.254.169.254/2007-12-15/user-data | sh
EOF

# EC2 tools
emerge ruby curl unzip symlinks
cd /tmp
wget http://s3.amazonaws.com/ec2-downloads/ec2-ami-tools.zip
cd /usr/local
unzip /tmp/ec2-ami-tools.zip
ln -sf ec2* ec2-ami-tools
chmod -R go-rwsx ec2*
rm -rf /tmp/ec2*
# Recompile rsync (lutimes doesn’t work with old ec2 kernel)
cd /tmp
wget -O - http://www.samba.org/ftp/rsync/src/rsync-2.6.9.tar.gz | \
  tar xz
cd rsync-2.6.9
perl -pi.bak -e ’s/\blutimes\b//’ ./configure
./configure –prefix=/usr/local/ec2-ami-tools
make
make install
cd ..
rm -rf rsync*

# Bundle
export AMAZON_USER_ID=’FIXME’
export AMAZON_ACCESS_KEY_ID=’FIXME’
export AMAZON_SECRET_ACCESS_KEY=’FIXME’
cat >/mnt/pk.pem <<\EOF
-----BEGIN PRIVATE KEY-----
FIXME
-----END PRIVATE KEY-----
EOF
cat >/mnt/cert.pem <<\EOF
-----BEGIN CERTIFICATE-----
FIXME
-----END CERTIFICATE-----
EOF
export EC2_PRIVATE_KEY=/mnt/pk.pem
export EC2_CERT=/mnt/cert.pem

cat >/usr/local/sbin/image <<\EOF
#!/bin/bash
export EC2_AMITOOL_HOME=/usr/local/ec2-ami-tools
PATH=$EC2_AMITOOL_HOME/bin:$PATH
BUNDLE=`date '+%y%m%d%H%M%S'`
ec2-bundle-vol -r i386 -u $AMAZON_USER_ID \
  -k $EC2_PRIVATE_KEY -c $EC2_CERT \
  -b -d /mnt -s 10000 --fstab /etc/fstab \
  -e /root -p $BUNDLE
ec2-upload-bundle -b $HOSTNAME -m /mnt/$BUNDLE.manifest.xml \
  -a $AMAZON_ACCESS_KEY_ID -s $AMAZON_SECRET_ACCESS_KEY
rm -rf /mnt/$BUNDLE* /mnt/img-mnt
EOF
chmod 700 /usr/local/sbin/image

export HOSTNAME=gentoo-i686
rm -rf /var/tmp/* /usr/portage/distfiles /usr/portage/packages
symlinks -crsdv /
image

# Register & make the ami public (on another machine)
ec2-register $HOSTNAME/$BUNDLE.manifest.xml
ec2-modify-image-attribute ami-xxxxxx --launch-permission -a all

#
# Below is an example of a boot script that you might pass in as "userdata"
# You would configure the hostname and dyndns and/or maybe puppet or cfengine
#

#!/bin/bash
# Hostname
echo 'HOSTNAME="fqdn.example.com"' >/etc/conf.d/hostname
/etc/init.d/hostname restart
echo ‘127.0.0.1 ‘`hostname -f`’ ‘`hostname -s`’ localhost’ >/etc/hosts
echo ’search ‘`hostname -d` >/etc/resolv.conf
echo ‘nameserver 172.16.0.23′ >>/etc/resolv.conf
echo ‘dhcp_eth0=”release nodns nontp nonis”‘ >/etc/conf.d/net
/etc/init.d/net.eth0 restart
# DynDNS
cat >/etc/ddclient/ddclient.conf <<\EOF
daemon=300
syslog=yes
mail=root
mail-failure=root
ssl=yes
use=web, web=169.254.169.254/2007-12-15/meta-data/public-ipv4
protocol=dyndns2, server=members.dyndns.org, custom=yes, \
login=FIXME, password=FIXME \
EOF
hostname >>/etc/ddclient/ddclient.conf
/etc/init.d/ddclient start
rc-update add ddclient default
fi

#
# After the new instance is booted, you may want to login and configure some
# basic tools or whatever
#

# Extras Tools
cat >>/etc/portage/package.keywords <<\EOF
dev-util/git
sys-fs/encfs
sys-fs/fuse
sys-fs/sshfs-fuse
EOF
emerge dev-util/git
emerge sys-fs/fuse sys-fs/encfs sys-fs/sshfs-fuse