Update: The issue might be memory bound. Check out this graph that show much higher swap.
After much research this appears to be a load calculation and display problem with the newer Linux kernels. The community has found Commit-ID: c308b56b5398779cd3da0f62ab26b0453494c3d4 to be the problem. The commit causes incorrect high reported load averages can be reported under conditions of light load and high enter/exit idle frequency conditions (greater then 25 hertz).
A nice fellow at http://www.smythies.com/~doug/network/load_average/new.html researched the topic between tick and tickless linux kernels and the effect they had on load averages. You should check it out.
You should check out my latest project, Remarkbox. It's a comment system that works everywhere, even static sites!