Random   •   Archives   •   RSS   •   About   •   Contact

Heka, World!

This post serves as a "Hello World" for the data collection and processing software called Heka. Heka is written in Go and was open sourced by Mozilla, the same fabulous group that brings us Firefox!

I intend to use Heka to replace Logstash agents by sending logs directly to ElasticSearch and continuing to use Kibana3 for visualizations. Also I aim to start collecting metrics and sending to a central Whisper back-end to fuel Graphite charts. All that we need to make Heka take on these responsibilities is one binary and some custom configuration.

Heka: Hello, World!

This mostly contrived example will show how to use Heka to watch /tmp/input.log and write to /tmp/output.log.

Step 1: install Heka

Compile from source or install the Heka package for your operating system.

Step 2: create a Heka TOML configuration file


type = "LogstreamerInput"
log_directory = "/tmp"
file_match = 'input\.log'

type = "FileOutput"
message_matcher = "TRUE"
path = "/tmp/output.log"
perm = "664"
encoder = "hello_heka_output_encoder"

type = "PayloadEncoder"
append_newlines = false

Step 3: start Hekad and test!

  1. in terminal 1:

    sudo hekad -config=/tmp/hello-heka.conf
  2. in terminal 2:

    tail -f /tmp/output.log
  3. in terminal 3:

    echo 'Heka, World!' >> /tmp/input.log

Like magic the data appended to input.log will appear in output.log

Heka also has great docs and a number of input and output plugins, but don't take my word for it, try it yourself!

Looking for a better comment system?

You should try Remarkbox — a hosted comment service that embeds in your pages to keep the conversation in the same place as your content. It works everywhere, even static sites!

Remarks: Heka, World!

© Russell Ballestrini.