After 10 years or 10,000 hours, most SRE & DevOps practitioners are YAML expert witnesses. We've written tens of thousands of lines of YAML - Kubernetes manifests, GitLab CI pipelines, Ansible playbooks, Docker Compose files, infrastructure definitions. We understand the format intimately. We know its quirks, its strengths, its …

Merry Christmas & Happy Holidays, Everyone!

My wife prompts students with this fun holiday writing idea:

"If you could gift the world anything, what would it be?"

I love to read responses in the comments!

As for me, I have wonderful news, my dream gift to the world will come true …

tl;dr if you just want something like aws s3 cp cli, try gsutil rsync.

At work one key item of team's sprint is properly utilizing and securing Google Cloud Platform (GCP).

For one of my projects, I'm learning GCP's Object Store called Google Cloud Storage (GCS).

I have prior …

A pre-signed request grants a semi-trusted user temporary access to a private resource.

Let's unpack that statement ...

Pre-signed means, we bless a specific action on a specific private resource for a short duration of time.

Semi-trusted means, we have authenticated the user, but we don't trust them to have full …

Recently at work I transitioned our fleet from Ubuntu 14.04 LTS to Ubuntu 18.04 LTS. During the process I noticed an issue with our newer generation AWS EC2 "nitro" based instance types (specifically c5.2xlarge).

AWS was presenting my root block device as /dev/nvme1n1 and my data …

tl;dr use a custom entrypoint in your CircleCI 2.0 config to limit Java memory to 1G.

The new CircleCI 2.0 docker configuration supports a "primary image" (listed first) which runs all the "steps" as well as zero or many "service images" (listed subsequently). The "service images", although …

This blog post assumes you already have:

• a Jenkins master and none or many build servers
• FPM installed on the build servers
• Node.js installed on the build servers

I add jenkins-build.sh in the root of the Node.js code repo:

# example usage: JOB_NAME=example-api BUILD_NUMBER=101 bash jenkins-build …

This took me about 2 hours to figure out, hopefully it saves you time.

/etc/sysconfig/network:

HOSTNAME=desired-hostname.example.com

/etc/hostname:

desired-hostname.example.com

/etc/cloud/cloud.cfg:

preserve_hostname: true

At work today I needed an easy way to collect private IP addresses of every instance in one of our production VPCs.

I ended up adding a tool to https://botoform.com to perform this task.

bf --profile <aws_profile> dump <vpc_name_tag> instances --output-format json

For example:

bf --profile customer3 dump …

Library work around.

You only need a two lines of code to access package names and versions.

References to take you from filtering novice to expert.

Don't reinvent the wheel, use Botocores Config facilities for work with AWS.

This weekend I finished my first sprint on s3p which is a Python library and CLI application that manages release pipelines on AWS S3. I put a lot of effort into the readme.rst file, so look there for usage and examples.

The main purpose of s3p is to use …

So I'm working with an API (AWS ElastiCache) that offers mandatory pagination of results. I need to get all results, so I took some time to work out this logic.

def combine_results(function, key, marker=0, **kwargs):
    """deal with manditory pagination of AWS result descriptions"""
    results = []
    while marker != None:
        result …

I'm currently refactoring a script that tags AWS resources and I came up with this one liner to generate pretty output. It basically turns {'tag1':'value1','tag2':'value2'} into tag1=value1, tag2=value2. Here is the code:

', '.join(['='.join(key_value) for key_value in {'a':'1','b':'2'}.items() ])

Oh, and …

When operations first became a thing, system administrators stood up servers using a base image from their favourite distribution. Things were done manually. Some administrators created their own distros, some wrote customised shell scripts to be run once-and-only-once to provision software and settings. This method worked, but it was slow …

Python Lambda functions are very powerful but I often forget how they work or the fun things they do. This post will document how to use a lambda to provide different default arguments to a function.

We will use the human function found in ago.py as an example - because …