Random   •   Archives   •   RSS   •   About   •   Contact   •   > Tag: Python

who owns yaml?

YAML Logo

After 10 years or 10,000 hours, most SRE & DevOps practitioners are YAML expert witnesses. We've written tens of thousands of lines of YAML - Kubernetes manifests, GitLab CI pipelines, Ansible playbooks, Docker Compose files, infrastructure definitions. We understand the format intimately. We know its quirks, its strengths, its …

Continue Reading






Introducing SLOP

Introducing SLOP: A Simple Language Open Protocol for Machine Learning Interaction

This post introduces the SLOP project, a community-driven effort to standardize Machine Learning API interactions. I'm Russell (fxhp), and I'm excited to share how this project came together and how you can get involved!

A Community Effort Kicked Off …

Continue Reading







Pre-signed GET and POST for Digital Ocean Spaces

A pre-signed request grants a semi-trusted user temporary access to a private resource.

Let's unpack that statement ...

Pre-signed means, we bless a specific action on a specific private resource for a short duration of time.

Semi-trusted means, we have authenticated the user, but we don't trust them to have full …

Continue Reading

AWS nvme to block mapping

Recently at work I transitioned our fleet from Ubuntu 14.04 LTS to Ubuntu 18.04 LTS. During the process I noticed an issue with our newer generation AWS EC2 "nitro" based instance types (specifically c5.2xlarge).

AWS was presenting my root block device as /dev/nvme1n1 and my data …

Continue Reading

Yuletide Trains and Homegrown Video Games

pixel art style seasonal wrapped gift

Each holiday season I find myself drawn to a side passion of mine.

While some build model trains and others create Christmas light shows with synchronized music you'll find me on my sofa where I build, explore, and tinker on my own video game engine.

Maybe the sound of wrapping …

Continue Reading


Quickstart to DKIM Sign Email with Python

For a long time I have put off DKIM signing email sent from my web services because I couldn't wrap my head around all the indirection Postfix requires to make it work.

Honestly, I put it off for over 5 years...

Today a thought sprung into my head:

"Could I …
Continue Reading

Fulfilling Childhood Dreams: Solar

Ever since I was an 8 year old boy I have wanted solar. I remember reading about the environment and alternative energy sources in a monthly "socal studies" flyer my school subscribed our classroom to. I questioned, even at my young age, why the world wasn't actively switching over to …

Continue Reading

How-to Work From Home

"I work from home" — a phrase I have uttered hundreds of times and is often met with instant amazement, envy, or jokes about pants. My goal for this book is to teach you the hacks I have learned in my career to help you land your dream job. If you …

Continue Reading


Build RPM or DEB packages for Node.js using Jenkins and FPM

This blog post assumes you already have:

  • a Jenkins master and none or many build servers
  • FPM installed on the build servers
  • Node.js installed on the build servers

I add jenkins-build.sh in the root of the Node.js code repo:

# example usage: JOB_NAME=example-api BUILD_NUMBER=101 bash jenkins-build …
Continue Reading

Register Super Powers with Pyramid add_request_method

The Pyramid web application framework uses a request object to hold state regarding an inbound HTTP connection. A view must accept a request object as the first argument which makes it always available to our views and templates.

This behavior rocks, but Pyramid makes it even better by allowing us …

Continue Reading

Sharing a Pyramid cookie with Flask or Tornado

Do you have a Pyramid application which authenticates users and uses a signed cookie as a session? Do you want to build a microservice using another framework and allow it to use the same cookie and session? Me too!

First we will review a bit of Pyramid code which describes …

Continue Reading






Dealing with pagination in Python

So I'm working with an API (AWS ElastiCache) that offers mandatory pagination of results. I need to get all results, so I took some time to work out this logic.

def combine_results(function, key, marker=0, **kwargs):
    """deal with manditory pagination of AWS result descriptions"""
    results = []
    while marker != None:
        result …
Continue Reading

Turn python dict into a key=value string and back again

I'm currently refactoring a script that tags AWS resources and I came up with this one liner to generate pretty output. It basically turns {'tag1':'value1','tag2':'value2'} into tag1=value1, tag2=value2. Here is the code:

', '.join(['='.join(key_value) for key_value in {'a':'1','b':'2'}.items() ])

Oh, and …

Continue Reading

Heka, World2!

This article expands on my “Hello World” for Heka blog post. Check that one out first if you are new to Heka.

In this guide we introduce using Heka over the network by utilizing two Hekad processes on localhost. For discussion purposes we name one of the Hekad processes "sender …

Continue Reading

Mailpile Salt States for Ubuntu or Debian

I wrote these Salt States to install Mailpile on an Ubuntu host. Fun fact, it took me 20 minutes to write these states and they worked the first time I ran them. Disclaimer - I used a throw away server and wasn't concerned that buckets of packages were installed to the …

Continue Reading

You can hack on FreeNAS 9

This post analyses the FreeNAS 9 code base and discusses the various places users may feel confident to hack on.

FreeNAS uses the following software stack:

Django
A Python Web Application Framework which complies with WSGI
Nginx
A very fast web server which may act as a reverse proxy server …
Continue Reading


Filter Salt Stack Return Data Output

Sometimes you only want to see what has changed, and that is OK.

Create a file like this:

filter.py

#!/usr/bin/python

from json import loads
from json import dumps

import fileinput

stdin_lines = [line for line in fileinput.input()]

ret = loads(''.join(stdin_lines))

for minion_id, data in ret.items …
Continue Reading

Replace the Nagios Scheduler and NRPE with Salt Stack

Note: I will update this post as I progress.

So the idea is to use Salt Stack's remote execution to communicate with all nodes and run the Nagios checks and collect the return output instead of using the NRPE client/service protocol. This reduces the number of agents running on …

Continue Reading


tar-back

tar-back is a backup utility to tar and gzip target filesystems.
It supports a custom retention, filter exclusions, and backup directory.

I use tar-back in combination with cron to perform regular backups of all localhost filesystems into /archive/fs. I then have a central long term storage server that collects …

Continue Reading


Hackathon 2013 Virtualization

As a warning before we dive into things, this post is less of a formal publication and more of a stream of conscience.

My employer newcars.com has allowed the technical staff to host hackathon! Over the past couple weeks I have had quite a few ideas tumbling around in …

Continue Reading





Guido name dropped tornado python tulip and pep-3156

Pycon 2013 was excellent, in fact it was my first one I have attended.

I found it odd that django and Pyramid had plenty of talks but nobody mentioned tornado.

The only person that brought up tornado was Guido himself, who has been researching and developing async python since December …

Continue Reading




Explaining cache with python

What is cache? I define cache as "a saved answer to a question". Caching can speed up an application if a computationally complex question is asked frequently. Instead of the computing the answer over and over, we can use the previously cached answer. This post will present one method of …

Continue Reading



What do you name your python virtualenv?


What do you name your python virtualenv?

I name my virtualenv 'virtpy'. Is there a standard name being used out there?

Maybe we can come to a consensus as a standard name? Please feel free to post your virtualenv names here as a sort of poll.

Continue Reading


Block cipher lab


Consider the following block cipher. Suppose that each block cipher T simply reverses the order of the eight input bits (so that, for example 11110000 becomes 00001111).
Further suppose that the 64-bit scrambler does not modify any bits. With n = 3 iterations and the original 64-bit input equal to 10100000 …
Continue Reading

Monoalphabetic Cipher and Inverse Written in Python

introduction and background

A monoalphabetic cipher uses fixed substitution over the entire message.

You can build a monoalphabetic cipher using a Python dictionary, like so:

monoalpha_cipher = {
    'a': 'm',
    'b': 'n',
    'c': 'b',
    'd': 'v',
    'e': 'c',
    'f': 'x',
    'g': 'z',
    'h': 'a',
    'i': 's',
    'j': 'd',
    'k': 'f',
    'l': 'g …
Continue Reading


Symmetric Encryption vs Public Key Encryption

How many keys are involved for symmetric key encryption? How about public key encryption?

Suppose you have N people who want to communicate with each other using symmetric keys. All communication between any two people, i and j, is visible to group N. Only person i and person j can …

Continue Reading

LinkPeek.com, webpage to image, was a by-product

tldr; When faced with pivoting or killing a project, take a good look at all possible by-products. Don't miss the hidden gem in a project's slag!

Last year I built yoursitemakesmebarf.com, a novelty web application which allowed anonymous link submission. The software would automatically take screenshots of submitted links …

Continue Reading









© Russell Ballestrini.