Sun, 03 Nov 2013

Coffee-Explicated Sidebar Item

Because my caffeine rush is hitting, and because this is my blog, and I can bore the shit out of whomever I want, I thought I’d do a short (but probably still eye-glazing) explanation of how I populate a small area of my blog’s left sidebar.

Rolling Top 10 Sidebar snapshot

TL;DR: I grab XML from a few times a day and render the HTML statically into the sidebar.

At one point, I was going to make the sidebar live with whatever was playing in iTunes at the particular moment when the blog is being accessed, but that’s a really silly thing to do, for reasons both technical and social.

This sidebar contains a rolling top 10 of my most-listened-to musical artists in any given week. I feel compelled to funnel as much of my listening as possible through applications that scrobble to Because caters to sad folks like me who love to keep track of things like “which artists did I listen to most during the second week of February 2008?”, they provide lots of ways for applications to access this data.

Usefully, for example, I can get an XML feed of artists I’ve listened to, and I can request various time-slices of this data. In this case, I grab the raw XML and parse the bits I care about from the XML tree. I then write an HTML fragment into a cache. You don’t want to see the code that does this, because I’m a real tyro at using ElementTree, so it’s really ugly.

Though Blosxom can be configured to generate its pages statically (e.g. most Movable Type blogs are run this way), I’ve pretty much always run it dynamically (i.e. every blog page request runs a script which generates a page on the fly to serve to the user-agent – most Wordpress blogs are run this way.) There are pros and cons to both approaches, of course, but in general:

  1. Static pages are faster to serve (and place less load on the web server), but introduce complications when you want to include conditional / changing content
  2. Dynamic pages are more flexible and allow more on-the-fly customizations per user-agent, but can bring a webserver to its knees when traffic spikes

Because this blog is very low traffic, it’s almost never been a problem running it dynamically. In the course of its history, it was linked from Slashdot once, and Daring Fireball once, and stayed up in both cases. To be honest, over-enthusiastic spidering by poorly-coded web crawlers has been a bigger issue.

One strategy I’ve adopted is to pre-render as much “dynamic” content as possible. Many of the sidebar elements that contain changing content (the Pinboard links, the geolocation stuff) are actually rendered into static HTML periodically and then transcluded into the page, rather than being fetched/rendered on every pageload.

With that in mind, I grab the XML from once per hour, and cache the transformed results. When the page loads, it grabs that already rendered cache and plops it into the sidebar.

:: 10:07
:: /administrivia/weblog | [+]
::Comments (1)

posted by Seth Williams at Mon Aug 4 01:05:45 2014

Hey Dave, thanks a lot for sharing your thoughts on the XML tree. We have some tutorials as well that may benefit your readers at

The Magic Word:
Which planet is closest to the sun? (hint -- it's Mercury...)

Thus spake the master programmer:
“When you have learned to snatch the error code from
the trap frame, it will be time for you to leave.”
— Geoffrey James, “The Tao of Programming”