Random   •   Archives   •   RSS   •   About   •   Contact > Tag: Code

Nginx throw HTTP 503 maintenance JSON for all requests

I found this technique after stumbling on Aaron Parecki's blog. You can read his post here:

Lets pretend you have an API and you need to turn on maintenance for a major change. All your …


Capability driven Presentation

A web page does not need to look the same on every browser or device. We cannot control the capabilities of a user's browser or device. As web designers, we have the duty to give the viewer the best experience possible. A user will come with what they have and …


Build RPM or DEB packages for Node.js using Jenkins and FPM

This blog post assumes you already have:

  • a Jenkins master and none or many build servers
  • FPM installed on the build servers
  • Node.js installed on the build servers

I add jenkins-build.sh in the root of the Node.js code repo:

# example usage: JOB_NAME=example-api BUILD_NUMBER=101 bash jenkins-build …

Register Super Powers with Pyramid add_request_method

The Pyramid web application framework uses a request object to hold state regarding an inbound HTTP connection. A view must accept a request object as the first argument which makes it always available to our views and templates.

This behavior rocks, but Pyramid makes it even better by allowing us …


Sharing a Pyramid cookie with Flask or Tornado

Do you have a Pyramid application which authenticates users and uses a signed cookie as a session? Do you want to build a microservice using another framework and allow it to use the same cookie and session? Me too!

First we will review a bit of Pyramid code which describes …


My first Systemd Service Script and Override

At work we mostly run Centos and I have some NodeJS services to deploy. I feel most familiar with Ubuntu / Upstart so this post serves as my notes on systemd.

In this contrived example, we define a service for our taco-api application. The taco-api source code lives in /opt/taco-api …


Set static hostname for RHEL Centos 7 on AWS

This took me about 2 hours to figure out, hopefully it saves you time.

/etc/sysconfig/network:

HOSTNAME=desired-hostname.example.com

/etc/hostname:

desired-hostname.example.com

/etc/cloud/cloud.cfg:

preserve_hostname: true

If I swallow lots of air will I be lighter?

After dinner tonight, Carter, my four year old asked me, "If I swallow lots of air will I be lighter?". I thought about the question for a moment and then told him it depends on what surrounds you.

If you are surrounded by:
  • air and you swallow lots of air …

Output all instance identifiers of an AWS VPC to JSON

At work today I needed an easy way to collect private IP addresses of every instance in one of our production VPCs.

I ended up adding a tool to http://botoform.com to perform this task.

bf --profile <aws_profile> dump <vpc_name_tag> instances --output-format json

For example:

bf --profile customer3 dump …

Migrating from WordPress to Pelican

Five hints to save time during your migration from WordPress.


Boto3 get main route table

Library work around.


List all installed package names in Python

You only need a two lines of code to access package names and versions.


Filtering AWS resources with Boto3

References to take you from filtering novice to expert.


Working with botocore's ~/.aws/config

Don't reinvent the wheel, use Botocores Config facilities for work with AWS.


A Python script which searches for available interpreters


Setting region programmatically in Boto3


Custom Rundeck HipChat notification templates

Today I built a GUI and workflow around Ansible using Rundeck. Tonight I started diving into sending HipChat notifications and after a bit of research, I managed to create a custom notification template for each Rundeck project.

Modify your project's configuration file, on Ubuntu it was in /var/rundeck/projects …


Build release pipelines on S3 with s3p

This weekend I finished my first sprint on s3p which is a Python library and CLI application that manages release pipelines on AWS S3. I put a lot of effort into the readme.rst file, so look there for usage and examples.

The main purpose of s3p is to use …


Dealing with pagination in Python

So I'm working with an API (AWS ElastiCache) that offers mandatory pagination of results. I need to get all results, so I took some time to work out this logic.

def combine_results(function, key, marker=0, **kwargs):
    """deal with manditory pagination of AWS result descriptions"""
    results = []
    while marker != None:
        result …

Turn python dict into a key=value string and back again

I'm currently refactoring a script that tags AWS resources and I came up with this one liner to generate pretty output. It basically turns {'tag1':'value1','tag2':'value2'} into tag1=value1, tag2=value2. Here is the code:

', '.join(['='.join(key_value) for key_value in {'a':'1','b':'2'}.items() ])

Oh and …


You can hack on FreeNAS 9

This post analyses the FreeNAS 9 code base and discusses the various places users may feel confident to hack on.

FreeNAS uses the following software stack:

Django
A Python Web Application Framework which complies with WSGI
Nginx
A very fast web server which may act as a reverse proxy server …

Nginx with SSL and mixed content errors with upstream WSGI servers

Mixed content errors occur because Nginx (the front-end server) communicates to the upstream WSGI server using http. WSGI does not know (or care) about the SSL session between Nginx and the user. The WSGI server will naively generate URIs and serve assets as http.

To fix mixed content errors, we …


IRC Bot (Foxbot) runs canned remote executions using Salt Stack

I extended my IRC Bot Foxbot today to allow it to run canned remote executions on behalf of users in an IRC channel. This is only a prototype or proof-of-concept. Be very careful not to allow users to inject their own commands. Foxbot must be running on the Salt Master …


Filter Salt Stack Return Data Output

Sometimes you only want to see what has changed, and that is OK.

Create a file like this:

filter.py

#!/usr/bin/python

from json import loads
from json import dumps

import fileinput

stdin_lines = [line for line in fileinput.input()]

ret = loads(''.join(stdin_lines))

for minion_id, data in ret.items …

Replace the Nagios Scheduler and NRPE with Salt Stack

Note: I will update this post as I progress.

So the idea is to use Salt Stack's remote execution to communicate with all nodes and run the Nagios checks and collect the return output instead of using the NRPE client/service protocol. This reduces the number of agents running on …


Automatic Backups

My tools for creating automatic backups for various systems


tar-back

tar-back is a backup utility to tar and gzip target filesystems.
It supports a custom retention, filter exclusions, and backup directory.

I use tar-back in combination with cron to perform regular backups of all localhost filesystems into /archive/fs. I then have a central long term storage server that collects …


mysql-back

mysql-backis a backup utility script to dump (backup) and gzip every MySQL database on a host.

I use mysql-back in combination with cron to perform regular database dumps of MySQL servers to the /archive/db partition on localhost. I then have a central long term storage server that collects …


Test Game Engine with Python and SFML

Over this holiday season, Christmas and New Years, I took the time to mess with some Game Development. I wrote a demo game engine using Python and SFML. I plan to use this post to track my progress.

Video Evolution Playlist

2014-01-01


virt-back's Domfetcher class returns doms from libvirt API


How to overload default function arguments in python using lambda

Python Lambda functions are very powerful but I often forget how they work or the fun things they do. This post will document how to use a lambda to provide different default arguments to a function.

We will use the human function found in ago.py as an example - because …


ago.py human readable timedelta 0.0.4 release


We have released ago.py 0.0.4

Special thanks to David Beitey for supplying ideas and python code for this update!

All changes are backward compatible.

Change log:

  • added support for future dates
  • added optional past_tense …

Tips for getting pull requests approved

Pull rejection sucks!

You have just coded, implemented, and submitted a pull …


Explaining cache with python

What is cache? I define cache as "a saved answer to a question". Caching can speed up an application if a computationally complex question is asked frequently. Instead of the computing the answer over and over, we can use the previously cached answer. This post will present one method of …


miniuri parser and ago human timedelta

I just packaged and published a couple of python modules to pypi:

To install them, run:

Install:

pip install --upgrade ago miniuri

If you want to view their source code, look here:

I hope you enjoy them.


Prevent a certain program from running too long in bash

Update - I opensourced this script here: bash kira

I came up this this script to kill certain programs after they run for too long. This works like similar to a timeout. Warning this script is pretty harsh and kills the program.

#!/bin/bash
PROGRAM=replace-with-program-name
PIDSFILE=/tmp/kill-these.pids

for …

Zenoss or Nagios monitoring of HTTPS using client certificate authentication

I recently needed to monitor an HTTPS API for response time and availability. At first I planned to just use the Nagios check_http command.

After gathering more requirements I learned that the API was protected by client certificate authentication. After some research I quickly found that no solution existed to …


Block cipher lab


Consider the following block cipher. Suppose that each block cipher T simply reverses the order of the eight input bits (so that, for example 11110000 becomes 00001111).
Further suppose that the 64-bit scrambler does not modify any bits. With n = 3 iterations and the original 64-bit input equal to 10100000 …

Monoalphabetic Cipher and Inverse Written in Python

introduction and background

A monoalphabetic cipher uses fixed substitution over the entire message.

You can build a monoalphabetic cipher using a Python dictionary, like so:

monoalpha_cipher = {
    'a': 'm',
    'b': 'n',
    'c': 'b',
    'd': 'v',
    'e': 'c',
    'f': 'x',
    'g': 'z …

Why does a Hash provide better message integrity then an Internet checksum?

Why does a Hash provide better message integrity then an Internet checksum?

Hash function and checksum function both return a value which cannot be reversed.

An Internet checksum (TCP checksum or IP checksum) is designed to detect common errors quickly and efficiently. An Internet checksum does not attempt to prevent …

How to Incorporate Custom Configuration in a Pyramid Application

Note: This post shows an old way to modify the request object. I discuss a new way in my Pyramid add_request_method post.

Imagine that you have just built a wiki, blog, or cms web application that will be deployed multiple times by different people. You would like to provide the …


CSS frameworks not rendering properly on all browsers

Read on for the one line fix.


LinkPeek.com web address thumbnail api alpha release

Convert any webpage to an image.


Python Image Grabber pig.py

Download all the images from a URI with this simple tool!


virt-back: restoring from backups

In a perfect world we should create backups but never need them. Although this statement holds truth, creating guest backups provides many more benefits.

The most common reasons system administrators restore from a virt-back guest backup:

  • recovering from data corruption
  • recovering deleted files
  • recovering from a virus infection
  • recovering from …

The Great Gist Heist

Please listen to my story before jumping to conclusions.


a hack to gain 80 percent efficiency when creating github projects


Add a Breadcrumb Subscriber to a Pyramid project using 4 simple steps


Google Bot Attempts to Crawl Shortest Urls First


Porting the ChaosTheory Wordpress theme to Pylowiki


virt-back: a python libvirt backup utility for kvm xen virtualbox

backup your virtual maching guests.


Response to L-Theanine: a 4000 Year Old Mind-Hack

Solve problems and write code in your sleep.


A homegrown python bread crumb module

I have placed bread.py, a python breadcrumb module, into the public domain.

The bread object accepts a url string and grants access to the url crumbs (parts) or url links (list of hrefs to each crumb).

Tutorial: Add a Breadcrumb Subscriber to a Pyramid project using 4 simple steps …



© Russell Ballestrini.