Amazon AWS EC2 and Cluster SSH

A devops engineer at work

Cluster SSH, or its OSX incarnation CsshX are great tool to quickly connect to a collection of machines and issue interactive commands in parallel. You just configure them with a collection of machines called 'clusters', set your .ssh/config correctly for all those boxes and voila, you are now in the matrix.

The problem arise when you run your stuff on Amazon's AWS on EC2 dynamic instances. It's impossible to know at a given time which instances are effectively running, and it makes your cssh (or CsshX) configuration useless.

The good news is, Amazon provides an API to access and manage EC2 instances. The even better new is there's a Perl package to access that programmatically. So..

Lets put EC2 API Access and Cluster SSH together: ec2-cssh

ec2-cssh is a small command line utility that queries the EC2 API for instances using configured sets of criteria (by tag, by whatever AWS accepts for filtering instances) and generates a cssh (Cluster SSH) or a csshX (CsshX) shell command that connects you to all those maching boxes.

What's even better is that ec2-cssh builds its configuration from several config file, at system level, user level, and current directory level.

I'm not going to enter into details, but the idea is that you can do this:

- Define the sets of instances (by tag, etc...) at project level, shared with all the devs in the project checkout in a file .ec2cssh.conf

- Define your AWS credentials, your favourite cluster ssh command or your own preferred instances sets in the safety of your $HOME/.ec2cssh.conf.

- Stuff can also be configured system wide in /etc/ec2cssh.conf

Of course all of this is optional, and you can perfectly have one config file with everything in it.

O, and by the way, ec2-cssh is written in Perl.

Hope you'll enjoy it!

Jerome.