The case for manual regression testing.

In our team, as part of our release process, we have a manual regression testing part. We have a rather long regression suite document that consists of sections and bullet action points.  As a quality insurance step before each release, we split this between members of the team and go through all the action points manually in our staging environment, hunting for regressions on the way and making sure the reality agrees with the document.

Very often members of other teams who see us doing that are quite sceptical about the benefits of doing such a thing. The standard argument against manual regression testing is the following:"We are developers, this should be automated. Look at phantom.js, and selenium and <insert any cool testing framework here>".

These critics have a point. Automated testing is good. And if fact we do have some quite extensive phantom.js automated tests. They make sure we don't break the most critical and essential features of our product's frontend (we use Ember.js). I will not dwell on selenium as over the time, I have never seen it being effectively useful for anyone. Tell me I am wrong in the comments.

However, human driven manual regression testing is very useful. Here are some reasons why:

Spreads product knowledge

As a positive side effect of manual regression testing, you will spread the product knowledge across your team. This is highly beneficial for new joiners, as it gives them a regular occasion to look in details at parts of the product they have never worked on. Also, having fresh pairs of eyes looking at your regression document will point out very quickly its flaws and inaccuracies.

Makes the team understand software development better

It's not always easy to explain the concept of regression to someone who is not accustomed with software development. Usually when a developer finds a bug, the first natural reflex is to fix it, and if you're lucky to open a ticket describing it correctly. But under the pressure of releasing on time, it is the perfect moment to say "If it is broken on live, it's not a regression, open a ticket and we will fix it later. Don't let that block our release".

A good occasion to bug chase

When was the last time you had the occasion to go systematically through all your product and find bugs pro-actively? Manual regression time is perfect for that. Don't wait until you get issues from customers, capture bugs before they suffer from them and stay on top of the game.

All regressions are not equal

A lot of things can go wrong when it comes to software. Automated testing can protect you against the biggest mistakes, by testing things you know about in advance. This is the key point here. You should also be able capture the regressions you cannot know about in advance, or are very very difficult to automatically test.

Some examples: The scroll bar of a dynamic drop down is covered by some other element when the page is halfway scrolled down. Some new design element colour look positively ugly against a specific background.

The list can go on and on. And only humans can detect unknown issues. Automatic tests can only detect what they were programmed for. This is good, but certainly not enough if your customers are human.

Make the team do some work as a team

As developers, we tend to try shining as a single star, doing complex things in isolation from others. Going through a simple bullet point list of rather boring things as a team; coordinating and reporting towards the common goal of releasing is in my mind a very useful exercise. Experience proves that such a thing is not so trivial to coordinate.

Conclusion

Yes we are developers, and in our wildest fantasy, we live in a work where machines do all the work for us. Even the work of being users of our products. But if you are developing products where your users are living creatures, you definitely want to be in their shoes regularly. What we techies see as 'not a bug' can have a huge impact on the perception and experience of users. On the other hand, what we see as a bug is sometimes just OK for users to live with and can be fixed later.

Check you are providing not only a "machine perfect" system, but also something that makes sense to mere human beings. Use manual regression as part of your release process.

Amazon AWS EC2 and Cluster SSH

A devops engineer at work

Cluster SSH, or its OSX incarnation CsshX are great tool to quickly connect to a collection of machines and issue interactive commands in parallel. You just configure them with a collection of machines called 'clusters', set your .ssh/config correctly for all those boxes and voila, you are now in the matrix.

The problem arise when you run your stuff on Amazon's AWS on EC2 dynamic instances. It's impossible to know at a given time which instances are effectively running, and it makes your cssh (or CsshX) configuration useless.

The good news is, Amazon provides an API to access and manage EC2 instances. The even better new is there's a Perl package to access that programmatically. So..

Lets put EC2 API Access and Cluster SSH together: ec2-cssh

ec2-cssh is a small command line utility that queries the EC2 API for instances using configured sets of criteria (by tag, by whatever AWS accepts for filtering instances) and generates a cssh (Cluster SSH) or a csshX (CsshX) shell command that connects you to all those maching boxes.

What's even better is that ec2-cssh builds its configuration from several config file, at system level, user level, and current directory level.

I'm not going to enter into details, but the idea is that you can do this:

- Define the sets of instances (by tag, etc...) at project level, shared with all the devs in the project checkout in a file .ec2cssh.conf

- Define your AWS credentials, your favourite cluster ssh command or your own preferred instances sets in the safety of your $HOME/.ec2cssh.conf.

- Stuff can also be configured system wide in /etc/ec2cssh.conf

Of course all of this is optional, and you can perfectly have one config file with everything in it.

O, and by the way, ec2-cssh is written in Perl.

Hope you'll enjoy it!

Jerome.

Image::Magick with Perlbrew.

Image::Magick is great, but it's a pain to install if you are using perlbrew, or if your system's Image Magick library is out of date. After some googling I found that this was the best technique, but cpanm support is missing, making using Image::Magick very difficult to include in your application's dependencies.

So I wrote a perlbrew + cpanm compatible Image::Magick perl package.

Note that it still depends on your perl to be build to generate the shared library libperl.so. So give it a go and if your perl is not compatible, it should tell you what to do:

cpanm -v Alien::ImageMagick

Then use Image::Magick as usual. Your application will have to depend on Alien::ImageMagick, not Image::Magick.

Happy coding!

Blobs in Postgresql with Perl

Are you using, or do you want to use Postgresql Blobs with Perl?

Providing you are in the Moose ecosystem, here is Pg::Blobs. Pg::Blobs is a role that adds blobs handling methods to any Moose based package.

Here is how to use it:

package My::App;
use Moose;
with qw/Pg::Blobs/;
# Just provide:
sub pgblobs_dbh{ .. return the DBH .. }
1;

Now your package My::App is capable of managing Postgresql blobs:

my $o = .. and instance of My::App ..
my $blob_id = $o->pgblobs_store_blob($binary_content);
print $o->pgblobs_fetch_blob($blob_id);

A couple of guidelines:

Blobs in Postgresql are just numeric Object IDs. You WILL have to store them in a OID column for later retrieval. If you don't, there is no guarantee they will persist, as any unreferenced blob can get vacuumed away.

You MUST wrap any blob operation in a transaction.

Happy blobing!