S3 backup script in a single binary

Everyone has a backup script that takes a tarball/zipfile/etc and uploads it somewhere for safe-keeping. In a lot of places where I’ve worked, the “somewhere” winds up being an Amazon S3 bucket (or lately a DigitalOcean Space). These scripts are lovingly crafted and sometimes quite clever, using the aws cli or s3cmd, or something custom. What I wanted was to encode my process for encrypting a backup and sending that backup to an S3 bucket....

Convert PEM to JKS

SSL/TLS in Java is a pain in the behind. Not only is the setup verbose, but the format for certificates and keys is unique. Nginx/Apache/Go/etc seem to be happy using certifiates and keys encoded as PEM files, but Java has its own special KeyStores, with the JKS format being the default. There are a number of questions on forums, and custom recipes that involve openssl, to convert PEM certificates and keys into formats that can be imported into a Java keystore....

New Developer Booklist

Welcome friend. Here’s something for you to read so that we can have some awesome arguments. The Pragmatic Programmer Design Patterns: Elements of Reusable Object-Oriented Software Refactoring: Improving the Design of Existing Code Domain-Driven Design: Tackling Complexity in the Heart of Software Working Effectively with Legacy Code Growing Object-Oriented Software, Guided by Tests Release It!: Design and Deploy Production-Ready Software Continuous Delivery: Reliable Software Releases through Build, Test, and Deployment Automation Building Microservices Pragmatic Thinking and Learning: Refactor Your Wetware Drift into Failure: From Hunting Broken Components to Understanding Complex Systems Building Evolutionary Architectures ...

Git revision of a single file

git --no-pager log -1 --pretty=%h <filepath> ...

Consumer-Driven Contract Tests

The most useful way I’ve seen such contract tests work is that the team that consumes the messages creates and publishes an artifact in their build pipeline for use by the creators of the messages. For this example let’s have it create a tarball with a shell script entry point. The inputs to the shell script can be a URL to the api-server and any other parameters required, like user IDs, oauth tokens, etc....

Building Clouds

I’ve spent this year building networks using Amazon Web Services and teaching people how to do it. So I’d like to share the code that I’ve used as teaching examples and as seeds for the creation of some pretty cool environments. AWS PY was my first published attempt at interacting with AWS in python & Puppet to instantiate, provision and control EC2 instances, as well as the seed for an incredibly cool project at the start of this year....

Default HTML-escape using Freemarker

Most java developers have at least heard of Freemarker. FreeMarker is a “template engine”; a generic tool to generate text output (anything from HTML to autogenerated source code) based on templates. It’s a Java package, a class library for Java programmers. It’s not an application for end-users in itself, but something that programmers can embed into their products. It is the “generic” nature of Freemarker that trips up java web developers....

Provision EC2 instance using boto

Sam Newman recently published a very interesting blog entry on using fabric to apply puppet scripts on remote machines. He left the provision_using_boto() method as an exercise to the reader. That just sounded tempting enough to be a challenge since I hadn't gotten around to looking at boto. You can find the result of my attempt on GitHub. To be precise aws.py implements the provisioning using boto and fabfile.py drives fabric and puppet....

RPMs and Effing Package Management

I've been using FPM to build native packages for applications for the last few months and so far I cannot believe just how cool it is. It can create RPMs and DEBs from ruby gems, python modules, node packages and even directories. The last one is very useful for packaging up stand-alone java apps. Check it out, you may like it too....

AWS CloudFront invalidation

It is now possible to invalidate objects (files) in AWS CloudFront distributions. Handy when someone, like me, occasionally publishes files with the wrong content type. Here is how I implement this invalidation in python....