Random   •   Archives   •   RSS   •   About   •   Contact   •  

who owns yaml?

YAML Logo

After 10 years or 10,000 hours, most SRE & DevOps practitioners are YAML expert witnesses. We've written tens of thousands of lines of YAML - Kubernetes manifests, GitLab CI pipelines, Ansible playbooks, Docker Compose files, infrastructure definitions. We understand the format intimately. We know its quirks, its strengths, its …

Continue Reading








Introducing SLOP

Introducing SLOP: A Simple Language Open Protocol for AI Interaction

This post introduces the SLOP project, a community-driven effort to standardize AI API interactions. I'm Russell (fxhp), and I'm excited to share how this project came together and how you can get involved!

A Community Effort Kicked Off by Nathan …

Continue Reading






Integrating OpenAI with Dry's Sample Chat Game

This tutorial demonstrates enhancing Dry's Sample Chat Game by integrating OpenAI's language models, enabling the game to provide intelligent, AI-driven responses to user queries. Dry, the successor to Urho3D, offers a comprehensive framework for developing 2D and 3D games. Leveraging LLMs within the Dry engine opens up new possibilities.

Background …

Continue Reading

Optimizing Memory Management for Plausible Docker on Ubuntu

Running Docker containers on a server with limited memory can lead to out-of-memory (OOM) issues, which can disrupt services and lead to downtime. This guide will show you how to increase swap space on an Ubuntu server to provide a buffer against OOM errors, using a real-world example from a …

Continue Reading


Installing YaCy on Ubuntu

This guide will walk you through installing YaCy on Ubuntu.

By default YaCy is configured to bind to 0.0.0.0 but it's admin interface is only accessible by default to a white list which includes localhost and 127.0.0.1, since my install is headless on a …

Continue Reading


Ubuntu 22.04 Letsencrypt Docker Hints

letsencrypt certbot is now installable via snap (the deb apt repository is no longer maintained).

alternatively you can use certbot via docker if you plan to use the certonly mode.

I did run into some issues & I will document my workarounds here:

domains=(
    example.com
    shop.example.com
)

for domain …
Continue Reading


Vertically Scaling GitLab Server For As Cheap As Possible

Today's essay acts as a power-up love story for the underdog.

A living document & quickstart for:

  • bootstrappers
  • small business under 99 employees
  • solo ops or devs
  • entrepreneurs
  • hackers
  • tinkerers

You also deserve a quick start to a GitLab Server power house!

Regardless of my intended audience, this strategy should scale …

Continue Reading




I tried to install GitLab on TrueNAS and failed

Ok, so a month ago I changed employment and the new company uses GitLab exclusively for centralized code version control system.

This is my first time using GitLab and my first projects was integrating a dou/cloudmapper with a GitLab runner on a schedule.

After a couple weeks I've learned …

Continue Reading

WeeChat on-boot in a tmux session

Chronicles of a washed up systems administrator.

In the basement, M. Bison (mbision.foxhop.net.), a T430 with a cracked screen runs quietly with his lid closed, acting as a SmartOS hypervisor to 3 Solaris derived zones and 4 KVM ubuntu guests.

One of these guests is the oldest of …

Continue Reading

Dreaming of unlimited home computer storage capacity

Unlimited computer storage capacity is a common science fiction trope and fun thought experiment.

What would be possible if we could potentially store near infinite data at your house in a normal sized desktop computer?

Even better using today's tech, what ideas could we dream up?

This post will explore …

Continue Reading




Pre-signed GET and POST for Digital Ocean Spaces

A pre-signed request grants a semi-trusted user temporary access to a private resource.

Let's unpack that statement ...

Pre-signed means, we bless a specific action on a specific private resource for a short duration of time.

Semi-trusted means, we have authenticated the user, but we don't trust them to have full …

Continue Reading


AWS nvme to block mapping

Recently at work I transitioned our fleet from Ubuntu 14.04 LTS to Ubuntu 18.04 LTS. During the process I noticed an issue with our newer generation AWS EC2 "nitro" based instance types (specifically c5.2xlarge).

AWS was presenting my root block device as /dev/nvme1n1 and my data …

Continue Reading

Yuletide Trains and Homegrown Video Games

pixel art style seasonal wrapped gift

Each holiday season I find myself drawn to a side passion of mine.

While some build model trains and others create Christmas light shows with synchronized music you'll find me on my sofa where I build, explore, and tinker on my own video game engine.

Maybe the sound of wrapping …

Continue Reading

All Local Heros need a Gig Side Kick

Five months ago during my preparation and ramp up to gardening season, I started thinking about what it would be like to have a partner, a side kick, a secretary to keep me honest and on task.

Somebody who could review old journal notes from previous years and give me …

Continue Reading

Running DynamoDB Local service container on CircleCI 2.0

tl;dr use a custom entrypoint in your CircleCI 2.0 config to limit Java memory to 1G.

The new CircleCI 2.0 docker configuration supports a "primary image" (listed first) which runs all the "steps" as well as zero or many "service images" (listed subsequently). The "service images", although …

Continue Reading


Quickstart to DKIM Sign Email with Python

For a long time I have put off DKIM signing email sent from my web services because I couldn't wrap my head around all the indirection Postfix requires to make it work.

Honestly, I put it off for over 5 years...

Today a thought sprung into my head:

"Could I …
Continue Reading

Fulfilling Childhood Dreams: Solar

Ever since I was an 8 year old boy I have wanted solar. I remember reading about the environment and alternative energy sources in a monthly "socal studies" flyer my school subscribed our classroom to. I questioned, even at my young age, why the world wasn't actively switching over to …

Continue Reading

How-to Work From Home

"I work from home" — a phrase I have uttered hundreds of times and is often met with instant amazement, envy, or jokes about pants. My goal for this book is to teach you the hacks I have learned in my career to help you land your dream job. If you …

Continue Reading


So You're Planning a Beta Test

I'm running a beta for Remarkbox and here is my biggest take away:

Always collect information from potential customers as soon as possible. Catch customers while they pursue a solution to their problems.

What do I mean?

Well for starters, don't just ask for an email address, have them fill …

Continue Reading


Selenium grid on Kubernetes

This post continues from where we left off on the Minikube guide. If you do not already have a Kubernetes cluster, you should read that first.

Selenium Grid allows you to build a cluster of Selenium nodes. Today we will create a Selenium cluster with 1 hub and 4 nodes …

Continue Reading



Capability driven Presentation

A web page does not need to look the same on every browser or device. We cannot control the capabilities of a user's browser or device. As web designers, we have the duty to give the viewer the best experience possible. A user will come with what they have and …

Continue Reading

Build RPM or DEB packages for Node.js using Jenkins and FPM

This blog post assumes you already have:

  • a Jenkins master and none or many build servers
  • FPM installed on the build servers
  • Node.js installed on the build servers

I add jenkins-build.sh in the root of the Node.js code repo:

# example usage: JOB_NAME=example-api BUILD_NUMBER=101 bash jenkins-build …
Continue Reading

Register Super Powers with Pyramid add_request_method

The Pyramid web application framework uses a request object to hold state regarding an inbound HTTP connection. A view must accept a request object as the first argument which makes it always available to our views and templates.

This behavior rocks, but Pyramid makes it even better by allowing us …

Continue Reading

Sharing a Pyramid cookie with Flask or Tornado

Do you have a Pyramid application which authenticates users and uses a signed cookie as a session? Do you want to build a microservice using another framework and allow it to use the same cookie and session? Me too!

First we will review a bit of Pyramid code which describes …

Continue Reading

My first Systemd Service Script and Override

At work we mostly run Centos and I have some NodeJS services to deploy. I feel most familiar with Ubuntu / Upstart so this post serves as my notes on systemd.

In this contrived example, we define a service for our taco-api application. The taco-api source code lives in /opt/taco-api …

Continue Reading


If I swallow lots of air will I be lighter?

After dinner tonight, Carter, my four year old asked me, "If I swallow lots of air will I be lighter?". I thought about the question for a moment and then told him it depends on what surrounds you.

If you are surrounded by:
  • air and you swallow lots of air …
Continue Reading

















Risk, Process, and Balance

The operations of a company will have intrinsic risk. Risk occurs each time we decide to take an action or an inaction. This means that anything we choose to do, or not do, has associated risk.

An organization which has an unhealthy aversion to risk has a much higher chance …

Continue Reading

autofs /net automount stopped working

So autofs randomly stopped working on one of my Ubuntu hosts (this issue has been found on Arch as well so its most likely a change upstream). I found this error in the logs:

attempting to mount entry /net/freenas.example.net
get_exports: lookup(hosts): exports lookup failed for freenas …
Continue Reading

Custom Rundeck HipChat notification templates

Today I built a GUI and workflow around Ansible using Rundeck. Tonight I started diving into sending HipChat notifications and after a bit of research, I managed to create a custom notification template for each Rundeck project.

Modify your project's configuration file, on Ubuntu it was in /var/rundeck/projects …

Continue Reading


Dealing with pagination in Python

So I'm working with an API (AWS ElastiCache) that offers mandatory pagination of results. I need to get all results, so I took some time to work out this logic.

def combine_results(function, key, marker=0, **kwargs):
    """deal with manditory pagination of AWS result descriptions"""
    results = []
    while marker != None:
        result …
Continue Reading

Turn python dict into a key=value string and back again

I'm currently refactoring a script that tags AWS resources and I came up with this one liner to generate pretty output. It basically turns {'tag1':'value1','tag2':'value2'} into tag1=value1, tag2=value2. Here is the code:

', '.join(['='.join(key_value) for key_value in {'a':'1','b':'2'}.items() ])

Oh, and …

Continue Reading

Migrating MongoDB from Ubuntu to SmartOS

First, I installed the mongodb 14.2.0 (uuid a5775e36-2a02-11e4-942a-67ae7a242985) dataset:

imgadm avail | grep mongo
imgadm import a5775e36-2a02-11e4-942a-67ae7a242985

Next, I launched a new zone with this image.

Then I grabbed the uuid of the zone (211b992b-a448-40b4-94c9-00fa82615cec) and I connected into the zone

zlogin 211b992b-a448-40b4-94c9-00fa82615cec

The zone automatically creates a username …

Continue Reading

Set Root Password SmartOS Percona MySQL Zone

I used project-fifo to launch the percona (14.2.0) MySQL dataset. I couldn't get into the MySQL instance so I reached out on IRC. Johngrasty, a friendly guy in the #smartos IRC channel, provided a command to display the randomly generated MySQL password emitted to the zone-init log:

cat …
Continue Reading



Heka, World2!

This article expands on my “Hello World” for Heka blog post. Check that one out first if you are new to Heka.

In this guide we introduce using Heka over the network by utilizing two Hekad processes on localhost. For discussion purposes we name one of the Hekad processes "sender …

Continue Reading

Mailpile Salt States for Ubuntu or Debian

I wrote these Salt States to install Mailpile on an Ubuntu host. Fun fact, it took me 20 minutes to write these states and they worked the first time I ran them. Disclaimer - I used a throw away server and wasn't concerned that buckets of packages were installed to the …

Continue Reading

You can hack on FreeNAS 9

This post analyses the FreeNAS 9 code base and discusses the various places users may feel confident to hack on.

FreeNAS uses the following software stack:

Django
A Python Web Application Framework which complies with WSGI
Nginx
A very fast web server which may act as a reverse proxy server …
Continue Reading




Filter Salt Stack Return Data Output

Sometimes you only want to see what has changed, and that is OK.

Create a file like this:

filter.py

#!/usr/bin/python

from json import loads
from json import dumps

import fileinput

stdin_lines = [line for line in fileinput.input()]

ret = loads(''.join(stdin_lines))

for minion_id, data in ret.items …
Continue Reading

Replace the Nagios Scheduler and NRPE with Salt Stack

Note: I will update this post as I progress.

So the idea is to use Salt Stack's remote execution to communicate with all nodes and run the Nagios checks and collect the return output instead of using the NRPE client/service protocol. This reduces the number of agents running on …

Continue Reading

Configuration Management and the Golden Image

When operations first became a thing, system administrators stood up servers using a base image from their favourite distribution. Things were done manually. Some administrators created their own distros, some wrote customised shell scripts to be run once-and-only-once to provision software and settings. This method worked, but it was slow …

Continue Reading


tar-back

tar-back is a backup utility to tar and gzip target filesystems.
It supports a custom retention, filter exclusions, and backup directory.

I use tar-back in combination with cron to perform regular backups of all localhost filesystems into /archive/fs. I then have a central long term storage server that collects …

Continue Reading

mysql-back

mysql-backis a backup utility script to dump (backup) and gzip every MySQL database on a host.

I use mysql-back in combination with cron to perform regular database dumps of MySQL servers to the /archive/db partition on localhost. I then have a central long term storage server that collects …

Continue Reading

The Three Deployment Management Strategies

There are three deployment management strategies that could be used to maintain a system. Each has pros and cons which I outline in this document.

run once

A proceedure that is run once and only once to setup a system's configuration values and settings. A semaphore or flag generally blocks …

Continue Reading


How I added two Seagate 240G SSDs as SmartOS L2ARC

How I added two Seagate 240G SSDs as SmartOS L2ARC

  1. removed icepacks from two western digital velociraptors
  2. installed ssds into icepacks
  3. installed icepacks into HP hotswap trays
  4. installed trays into HP prolaient g6 server

How to list all drive installed in Solaris, Open Solaris, or SmartOS

iostat -eE
format

AVAILABLE …
Continue Reading




Hackathon 2013 Virtualization

As a warning before we dive into things, this post is less of a formal publication and more of a stream of conscience.

My employer newcars.com has allowed the technical staff to host hackathon! Over the past couple weeks I have had quite a few ideas tumbling around in …

Continue Reading




Control a MongoDB collection in configuration management

This post explains how to use configuration management (Salt Stack) to completely control a MongoDB collection. In our example we want to control a store's collection of plans.

First we create a JSON representation of the collection.

mongodb/plan.json:

{
  "_id" : { "$oid" : "4ef8b9e2be329f491d98f74b" },
  "cost" : 20, "description" : "development",
  "name" : "good", "count …
Continue Reading


Understanding Salt Stack user and group management

This state will create a user:

russell:
  user:
    - present

This state will create a user and a group. This also makes the user part of the group, and handles creating the group first:

russell:
  group:
    - present
  user:
    - present
    - groups:
      - russell
    - require:
      - group: russell

This state handles user and group generation …

Continue Reading






Automatic event hangout with cron

Create an online only, hangout event

Create a new event with a date far into the future, like the year 2015. Go to the event's options > advanced and enable 'this event is online only' which will create a unique Hangout URI.

Create a cronjob

Create a cronjob on each device …

Continue Reading

Guido name dropped tornado python tulip and pep-3156

Pycon 2013 was excellent, in fact it was my first one I have attended.

I found it odd that django and Pyramid had plenty of talks but nobody mentioned tornado.

The only person that brought up tornado was Guido himself, who has been researching and developing async python since December …

Continue Reading




Tips for getting pull requests approved

Pull rejection sucks!

You have just coded, implemented, and submitted a pull request. A short while later the request is declined by an upstream maintainer and you feel crushed. We have all been there. Today I'm going to show you a better way. This article will teach you how to …

Continue Reading


Explaining cache with python

What is cache? I define cache as "a saved answer to a question". Caching can speed up an application if a computationally complex question is asked frequently. Instead of the computing the answer over and over, we can use the previously cached answer. This post will present one method of …

Continue Reading





Always attempt to scale vertically first

I spent the weekend fretting because one of my servers was basically being DOS'd by paying customers. During the outage I started thinking about the best way to scale and how I could make the code-base more efficient.

Linux top reported high load, in the 20's. Eventually I figured out …

Continue Reading





Trouble mounting filesystem on KVM guest after reboot

Just found this out the hard way...

It looks like the attachment of /KVMROOT/guest-dev-app.img on guest-dev did not persist when the KVM host rebooted for patching.

As it appears the virsh attach-disk command works a lot like the mount command.

In order to have a disk attachment persist …

Continue Reading

nosslsearch cname is a bad idea and solution


Google SafeSearch and SSL Search for Schools suggests implementing the following changes to the network:
To utilize the no SSL option for your network, configure the DNS entry for www.google.com to be a CNAME for nosslsearch.google.com.

Here are the reasons why this is a bad idea …
Continue Reading


What do you name your python virtualenv?


What do you name your python virtualenv?

I name my virtualenv 'virtpy'. Is there a standard name being used out there?

Maybe we can come to a consensus as a standard name? Please feel free to post your virtualenv names here as a sort of poll.

Continue Reading


How to capture HTTPS SSL TLS packets with wireshark

This article will explain how to use wireshark to capture TCP/IP packets. Specifically I will show how to capture encrypted (HTTPS) packets and attempt to document the "dance" a client and server do to build an SSL tunnel.

What is Wireshark?

Wireshark is a network protocol analyzer for Windows …

Continue Reading




Block cipher lab


Consider the following block cipher. Suppose that each block cipher T simply reverses the order of the eight input bits (so that, for example 11110000 becomes 00001111).
Further suppose that the 64-bit scrambler does not modify any bits. With n = 3 iterations and the original 64-bit input equal to 10100000 …
Continue Reading

Monoalphabetic Cipher and Inverse Written in Python

introduction and background

A monoalphabetic cipher uses fixed substitution over the entire message.

You can build a monoalphabetic cipher using a Python dictionary, like so:

monoalpha_cipher = {
    'a': 'm',
    'b': 'n',
    'c': 'b',
    'd': 'v',
    'e': 'c',
    'f': 'x',
    'g': 'z',
    'h': 'a',
    'i': 's',
    'j': 'd',
    'k': 'f',
    'l': 'g …
Continue Reading


Symmetric Encryption vs Public Key Encryption

How many keys are involved for symmetric key encryption? How about public key encryption?

Suppose you have N people who want to communicate with each other using symmetric keys. All communication between any two people, i and j, is visible to group N. Only person i and person j can …

Continue Reading

Attributes of an 8-block cipher

Consider an 8-block cipher and answer the following:

How many possible input blocks does this cipher have?

How many possible mappings are there?

If we view each mapping as a key, then how many possible keys does this cipher have?

To find the input blocks of this cipher we raise …

Continue Reading



Today I lost a customer

Today I lost a customer.

I added some new code to LinkPeek to accept coupons and I didn't think of an edge case. This ended up creating an uncaught exception in my server side code which ultimatly served the newly subscribing customer an HTTP 500 error page.

The damage was …

Continue Reading

LinkPeek.com, webpage to image, was a by-product

tldr; When faced with pivoting or killing a project, take a good look at all possible by-products. Don't miss the hidden gem in a project's slag!

Last year I built yoursitemakesmebarf.com, a novelty web application which allowed anonymous link submission. The software would automatically take screenshots of submitted links …

Continue Reading

flash mob office meeting definition

Flash Meeting

In a office or cubicle environment a group of uninvited people gather and hover around your desk to talk to you. A meeting forms in immaculate conception as you sit bewildered at your desk.

Other names: Flash Meeting, Flash Mob Meeting, Flash Office Meeting

Continue Reading




I'm petrified of launching my web application


I'm petrified of launching my web application because I'm fearful that I won't ...
  • acquire users
  • support my users well
  • scale in a timely manner
  • react quickly to feedback
  • monetize the application

But most of all I'm scared that nobody will like me. I'm scared of failure.

Now that I got …

Continue Reading


Occupy Wall Street Stack vs Queue

Occupy Wall Street contributors claim to use a "stack" to determine speaking arrangements.

I plan to explain how the term "stack" used in this scenario does not align itself with the mathematical or computer science definition.

The term stack means First In Last Out or "FILO". For example: a person …

Continue Reading















virt-back: restoring from backups

In a perfect world we should create backups but never need them. Although this statement holds truth, creating guest backups provides many more benefits.

The most common reasons system administrators restore from a virt-back guest backup:

  • recovering from data corruption
  • recovering deleted files
  • recovering from a virus infection
  • recovering from …
Continue Reading









Voice Over IP with TeamSpeak

This article will cover running a Voice Over IP service like TeamSpeak on a VPS.

Voice Over IP allows users to communicate using audio over the Internet.

When planning for this article I originally was going to cover ventrilo, but their download link was obfuscated behind a heinous php session …

Continue Reading






How did Stack Overflow get initial traction?

Stack Overflow was a progressive and natural evolution of the standard clunky forum.

Using ajax it created a more fun and clean user experience.

Using badges and karma to gain responsibility allow forums post to become a game. People naturally like to see progression and growth, being able to watch …

Continue Reading



© Russell Ballestrini.