Unfortunately the export format of readitlater isn’t compatible with the import format required by Delicious; so here’s a quick python script to convert a ReadItLater exported html file to a file that Delicious’s import will accept.

Get the export

First, visit readitlaterlist.com/export and download that file somewhere; it should create ril_export.html

Convert to Delicious format

Copy the script below to ril2del.py (or whatever you feel like) then run:

./ril2del.py ril_export.html converted_for_delicious.html

Import to Delicious

Then visit the Delicious import page at export.delicious.com/settings/bookmarks/import and point it at your generated file (converted_for_delicious.html) and hey presto, you’ve got all your readitlater bookmarks in Delicious complete with tags. Read items from RiL will have an extra tag “read” and unread items have an “unread” tag.

The script

(requires python2 and BeautifulSoup4 to be installed; you can install BeautifulSoup4 with pip)

#!/usr/bin/env python

from bs4 import BeautifulSoup
import sys
import os

def usage():
    print "r2d <inputfile> <outputfile>"
    return 1

def first_child(list):
    for item in list:
        return item

def process(ul, extraTag, output):
    for li in ul.find_all("li"):
        a = li.a
        replacements = dict()
        replacements[":tags"] = a["tags"] + "," + extraTag
        replacements[":time_added"] = a["time_added"]
        replacements[":href"] = a["href"]
        replacements[":content"] = first_child(a)
        outputString = """<DT><A HREF=":href" ADD_DATE=":time_added" PRIVATE="1" TAGS=":tags">:content</A>\n"""
        for key in replacements:
            outputString = outputString.replace(key, replacements[key])
        output.write(outputString.encode("UTF-8"))

def main():
    args = sys.argv[1:]
    if len(args) != 2:
        return usage()
    infile = args[0]
    outfile = args[1]
    if not os.path.isfile(infile):
        print "Cannot read input file:", infile
    soup = BeautifulSoup(file(infile).read())

    output = file(outfile, "w")
    output.write("""<!DOCTYPE NETSCAPE-Bookmark-file-1>
<META HTTP-EQUIV="Content-Type" CONTENT="text/html; charset=UTF-8">
<TITLE>Bookmarks</TITLE>
<H1>Bookmarks</H1>
<DL><p>
""")

    uls = soup.find_all("ul")
    unread = uls[0]
    read = uls[1]
    process(ul=unread, extraTag="unread", output=output)
    process(ul=read, extraTag="read", output=output)

    output.write("</DL><p>")

if __name__ == "__main__":
    sys.exit(main())

It pumps out a .crx (signed with your .pem, ready for hosting on your site) and a .zip (ready for uploading to the Chrome Store).

Here it is – it should work under linux and cygwin, let me know if you have any issues in the comments – instructions at the top of the file.

Click here to download a copy of it.

There’s an example of how to use it at the end of this post.

crx_build.sh Chrome extension build script

#!/bin/bash
# Author Seb Maynard, 2012
# http://5eb.me
#
# Script to generate and package CRX chrome extensions and zip files ready for upload
# requires updates.xml, manifest.json in current folder - these shouldn't reference a version number,
# but instead have xxxxxxx where the version number will be replaced in.
#
# This should work on cygwin and *nix
#
# required variables are
# $version (appended to the built filenames - like 0.0.1)
# $files (should be an array of files to include - like (updates.xml somefile.txt ../someotherfile.txt) )
# $modulename
#
# when this script has finished running, 2 variables will be availabe for use:
#   $filename_crx
#   $filename_zip
# which can then be used (for example) to automatically update your server with the new files

# dependency checks
if [ "$version" == "" ] ; then
	echo "Need a version variable (version=...)"
	exit
fi

if [ "$files" == "" ] ; then
	echo "Need a list of files (files=...)"
	exit
fi

if [ "$modulename" == "" ] ; then
	echo "Need a modulename (modulename=...)"
	exit
fi

# Copy in the files we need
rm -r dist_tmp
mkdir dist_tmp
for i in "${files[@]}"
do
	cp "$i" dist_tmp/
done

# Sort out the version numbers
echo Building dist_tmp folder
cd dist_tmp
sed -i "s/xxxxxxx/$version/g" updates.xml
sed -i "s/xxxxxxx/$version/g" manifest.json
cd ..

# build the Chrome crx for distribution
echo Building Chrome crx
chrome_run=`which google-chrome 2>/dev/null`
cygwin=
if [ ! -f "$chrome_run" ] ; then
	cygwin=yes
	chrome_run=`cygpath -u "$APPDATA/../Local/Google/Chrome/Application/chrome.exe"`
fi
if [ -f "$chrome_run" ] ; then
	path_dist_tmp=`pwd`/dist_tmp
	path_key=`pwd`/$modulename.pem
	if [ "$cygwin" == "yes" ] ; then
		path_dist_tmp=`cygpath -w $path_dist_tmp`
		path_key=`cygpath -w $path_key`
	fi
	echo Building dist_tmp.crx in $path_dist_tmp
	$chrome_run --no-message-box --pack-extension=$path_dist_tmp --pack-extension-key=$path_key
	filename_crx=$modulename-$version.crx
	if [ -f "$filename_crx" ] ; then
		echo $filename_crx already exists\! Skipping rename
	else
		mv dist_tmp.crx $filename_crx
		echo Built $filename_crx
	fi
else
	echo "Couldn't find Chrome!"
fi

# Build the zip for uploading
cd dist_tmp
filename_zip=$modulename-$version.zip
if [ -f "../$filename_zip" ] ; then
	echo $filename_zip already exists\! Skipping zip
else
	echo Removing update site line from packaged zip
	sed -i "s/\"update_url\"\:.*$//g" manifest.json
	zip -r ../$filename_zip *
fi
cd ../

Example usage

This is my build.sh for the raytracer:

#!/bin/bash

# version
version=0.0.3

# file list
files=(
	updates.xml
	manifest.json
	sebtracer-16.png
	sebtracer-48.png
	sebtracer-128.png
	../sebtracer2.html
	../sebtracer2.nmf
	../sebtracer2_x86_32.nexe
	../sebtracer2_x86_64.nexe
	)

modulename=SebTracer

source /s/git/Code/chromeextensions/crx_build.sh

However, sometimes it’s good to be able to listen to your music when you haven’t got an internet connection – the Android Music client supports “make available offline” for tracks, but the webpage doesn’t yet – so I wrote a small Chrome extension that generates a list of wget commands (and mv commands) to automate the process of downloading specific tracks from Google Music, ready for play offline.

The extension adds a button to the Google Music webpage’s interface that when clicked will (hopefully!) copy a series of shell commands (linux or cygwin) to your clipboard. You can also click a link to view the commands it generates – these commands will download each track, then rename them in the form <track-number> – <track name>.mp3

For example:

wget 'http://t.doc-0-0-sj.sj.googleusercontent.com/stream/001?id=aaaaaaaaaaaaaaaa&itag=25&source=skyjam&o=aaaaaaaaaaaaaaaaaaaa&ip=0.0.0.0&ipbits=0&expire=0000000000&sparams=id,itag,source,o,ip,ipbits,expire&signature=aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa.aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa&key=sj2' -O track1.mp3; mv track1.mp3 '01 - My track name.mp3' ;

If you run that in a shell, you’ll end up with your MP3.

The tracks don’t have any ID3 info embedded but for those times when you just need 1 or 2 albums (which is what this should be used for!) that doesn’t matter too much. This is *not* intended to batch-dump your entire Google Music library.

Obviously, this shouldn’t be used for evil – please use responsibly

Download

You can download the latest version here.

Changelog

2012-03-08 v0.0.1
First build

2012-03-08 v0.0.2-4
Slightly more user-friendly interface, support for copying directly to the OS clipboard

2012-03-08 v0.0.5
Supports downloading tracks from playlists now too

Then I found out about NaCl - Native Client – Google’s system for building extensions for Chrome using native code – and noticed the requirements for building things in it are that it compiles in GCC 4. Sounded like a good test-case…

I’ve built a NaCl version of the simple raytracer – it’s not fast (it’s only single threaded) and because it’s compiled for NaCl, that’s got a small performance hit too, but it works.

It’s pretty slow at the moment, single threaded, no acceleration structures (kd-trees et al), no ray averaging, etc but now it has triangles (!) so next step is .obj loading or at least simple scene graph editing in the page it’s loaded into… Once I’ve figured out pthreads in Pepper and made them work in NaCl, that should speed it up a bit on multicore processors anyway.

It’s about 4mb for the NaCl .nexe (unfortunately, most of that is 0′s added as padding by the modified gcc) so takes a little while to download but if you’d like to try it, you need a recent(ish) version of Chrome and you can find it in the Chrome store (with a rather daft name too):

https://chrome.google.com/webstore/detail/keikppoeceeaceabgdiommjljipfjmng

Depending on the speed of your PC, it may be unuseably slow, but on my work I7 it’s responsive enough… (drag the bits around x and y; hold ctrl to drag around x and z).

For reference, here’s a native (Windows) version compiled using VC++ (I said it was portable C++!) – on my PC, it runs considerably faster – I’m not 100% convinced where the major slowdowns are, but that’s something to look into.

Future

Eventually (when I get time – something I don’t seem to have much of at the moment) I’ll expand on it, and add some features to it.

• add some kind of scene description support
• acceleration structure for the scene description (kd-tree, octree perhaps)
• multi-threading

Changelog

2012/03/08 – v 0.0.2
first build in NaCl, first build of Chrome extension, and upload to Chrome store. Woo!

Warning! If you get your iptables config wrong, you can end up blocking yourself out of your server, so be careful! Always best to have serial console access to the box just in case…

IPTables rules can be configured one-by-one; you can list the current iptables with:

sudo iptables -L

This should print something like:

Chain INPUT (policy ACCEPT)
target     prot opt source               destination

Chain FORWARD (policy ACCEPT)
target     prot opt source               destination

Chain OUTPUT (policy ACCEPT)
target     prot opt source               destination

Chain fail2ban-ssh (0 references)
target     prot opt source               destination

This basically means accept everything, from everyone. Incidentally, if you need to flush (get rid of) all your iptables rules:

iptables -F

Running an iptables command like this:

-A INPUT -i lo -j ACCEPT

will allow (-j ACCEPT) all incoming connections (-A INPUT) on the localhost interface (-i lo). On the other hand, this:

iptables -A INPUT -j DROP

will drop (i.e. reject) everything incoming. What we’d like to end up with is a firewal which allows everything on localhost, allows certain incoming ports on tcp and udp, a host which is always allowed no matter what, and block everything else (whilst also allowing existing connections) – for an iptables rule set which does that for these ports and this host:

Ports to allow
22 – SSH (tcp)
80 – HTTP (tcp)
443 – HTTPS (tcp)
110 – POP3 (tcp)
995 – POP3S (tcp)
1194 – OpenVPN (tcp and udp)

Host to allow:
123.123.123.123

The iptables commands to generate that list are:

sudo iptables -A INPUT -i lo -j ACCEPT
sudo iptables -A INPUT -m state --state RELATED,ESTABLISHED -j ACCEPT
sudo iptables -A INPUT -p tcp -m tcp --dport 22 -j ACCEPT
sudo iptables -A INPUT -p tcp -m tcp --dport 443 -j ACCEPT
sudo iptables -A INPUT -p tcp -m tcp --dport 80 -j ACCEPT
sudo iptables -A INPUT -p tcp -m tcp --dport 110 -j ACCEPT
sudo iptables -A INPUT -p tcp -m tcp --dport 995 -j ACCEPT
sudo iptables -A INPUT -p tcp -m tcp --dport 1194 -j ACCEPT
sudo iptables -A INPUT -p udp -m udp --dport 1194 -j ACCEPT
sudo iptables -A INPUT -s 123.123.123.123 -j ACCEPT
sudo iptables -A INPUT -j DROP

If you run all of these, then run iptables -L again, you should get something like:

Chain INPUT (policy ACCEPT)
target     prot opt source               destination
ACCEPT     all  --  anywhere             anywhere
ACCEPT     all  --  anywhere             anywhere            state RELATED,ESTABLISHED
ACCEPT     tcp  --  anywhere             anywhere            tcp dpt:ssh
ACCEPT     tcp  --  anywhere             anywhere            tcp dpt:https
ACCEPT     tcp  --  anywhere             anywhere            tcp dpt:www
ACCEPT     tcp  --  anywhere             anywhere            tcp dpt:pop3
ACCEPT     tcp  --  anywhere             anywhere            tcp dpt:pop3s
ACCEPT     tcp  --  anywhere             anywhere            tcp dpt:openvpn
ACCEPT     udp  --  anywhere             anywhere            udp dpt:openvpn
ACCEPT     all  --  123.123.123.123          anywhere
DROP       all  --  anywhere             anywhere

Chain FORWARD (policy ACCEPT)
target     prot opt source               destination

Chain OUTPUT (policy ACCEPT)
target     prot opt source               destination

If you’re still connected, then everything’s working ok so far!

Next up, we’d like to get these rules automagically loaded on startup, so we’ll create a save-state for iptables:

sudo iptables-save > /etc/iptables.rules

Now flush your rules, (iptables -F) so we can then restore our rules with:

sudo iptables-restore < /etc/iptables.rules

Running iptables -L again after that should give you the same rules active you had before. HOORAY!

To get this to happen automatically on startup, create a file in /etc/network/if-pre-up.d – everything in this folder gets run automatically on startup just before the network interface is brought online.

#!/bin/sh
# this file is /etc/network/if-pre-up.d/iptables-restore
iptables-restore < /etc/iptables.rules
exit 0

then make sure it's executable! Otherwise, it won't do anything:

sudo chmod a+x /etc/network/if-pre-up.d/iptables-restore

If you reboot now, your box should come back eventually, you should still be able to login, and running iptables -L should give you your list of configured iptables rules.

fail2ban

Fail2ban will help block some ips if they appear to be hammering your server trying to login. It's quite easy to get going on ubuntu:

sudo apt-get install fail2ban

That's pretty much it... just make sure that the /etc/fail2ban/jail.conf file includes the [ssh] section, and that it's enabled. Then, restart the fail2ban daemon:

sudo /etc/init.d/fail2ban restart

then you should now have some extra iptables rules:

Chain INPUT (policy ACCEPT)
target     prot opt source               destination
fail2ban-ssh  tcp  --  anywhere             anywhere            multiport dports ssh
...

And that's that. Your server is now just a little bit more secure good luck!

Box (used to be box.net) is an online storage service that’s been around for a while – it’s quite popular and gives you more storage space than Dropbox usually does. And, if you’ve got an Android or iPhone, getting the mobile Box app (for Box.com) unlocks 50gb of storage immediately, for free. 50GB of storage. Unfortunately, it doesn’t have a linux syncing client yet – wouldn’t it be nice if you could create your own?

Contents

• Intro
• Signup for Box!
• Mount the Box folder locally using webdav
• Setup Unison to keep a local folder and your Box folder in sync
• Create a cron job to run unison automagically

Box supports webdav; webdav is an extension to HTTP that allows handling of remote filesystems – thankfully, there are a number of packages in linux available to let you mount these webdav folders locally and have them behave as local folders. However, performance on these usually is a bit slow (each read/write involves a few round-trips to the web server) so what we’ll get here is:

• a locally mounted webdav folder that will have the “live” contents of what Box has, but quite slow
• a local copy of the contents of the webdav folder that gets kept in sync with the online Box copy every, or automatically when you change a file – but because it’s a local copy, it’s no different to normal files

(Anywhere I’ve used vim, feel free to use your editor of choice – just replace it with something like gedit if you’d rather a gui)

Sign up for Box!

So, first up, you’ll need a Box account.

Just visit box.com to create one, or signup from your mobile (then you get 50gb space from the getgo!). Once you’ve got your account all set up and confirmed, continue!

Mount the Box folder locally using webdav

Next we need to mount the Box web folder as a webdav folder – the davfs2 package will do the job:

sudo apt-get install davfs2

Now create somewhere for the webdav folder to mount:

sudo mkdir /media/box.net

Then we’ll add an entry in fstab (the file that Ubuntu reads to work out what’s mountable)

sudo vim /etc/fstab

and put this in it at the end:

https://www.box.net/dav /media/box.net davfs defaults,rw,user,noauto 0 0

Now we’ll change the davfs config so that users in the users group can mount the webdav share whenever they need, and also to disable the use of file locks – they don’t seem to work properly with the Box.com webdav service

sudo vim /etc/davfs2/davfs2.conf

and put this in it (there might already be something similar to this somewhere in the file – just replace it with this):

dav_group users
use_locks 0

Next, we need to add our user to the users group – (my user’s called seb, replace seb with you obviously!)

sudo addgroup seb users

Now we need to store our Box.com username and password so that it doesn’t prompt every time we try and use it:

sudo vim /etc/davfs2/secrets

and put this at the end:

https://www.box.net/dav <your_user> <your_password>

At this point, everything should be ready to test the webdav setup; run this to mount it:

mount /media/box.net

If that worked, then you should be able to open a filemanager (like nautilus or thunar) and visit /media/box.net and view the contents of your Box account! However, you might notice that it’s really not very responsive – and sometimes freezes up the file manager… which is what the next section will fix.

Setup Unison to keep a local folder and your Box folder in sync

Unison is a 2-way synchronization tool that can do all sorts of powerful things; here though we’re just going to use it to keep 2 folders in sync with each other – I’m going to put my local Box folder in /home/seb/box.net and keep it in sync with the existing /media/box.net. So first make sure you’ve got the box.net folder:

cd ~
mkdir box.net

Now you need to install unison (if you’ve not got it already) then run unison once to create the .unison config folder:

sudo apt-get install unison
unison # if you've already used unison before to create a profile, this might run it! So be careful...

You should now have a .unison folder in your home directory. We’re going to create a new unison profile to keep our folders in sync:

vim .unison/box.prf

and put this in it (obviously again replacing seb with your username!):

root = /home/seb/box.net
root = /media/box.net

ignore = Name *~
ignore = Name .*~

auto = true

retry = 2

logfile = /tmp/unisonlog

batch = true

That profile ignores any files with filenames ending in ~ (like vim and emacs’ backup files), sets automatic and batchmode to true (so it’ll try and do everything automatically, without asking for confirmation all the time) and write out a log of what happened to /tmp/unisonlog.

So let’s try it:

unison box

This should print out a load of stuff about what it’s doing, what it’s copying, conflicts etc, then write something (hopefully!) like:

UNISON 2.32.52 finished propagating changes at 08:30:46 on 24 Feb 2012
Saving synchronizer state
Synchronization complete at 08:30:46

If it did, then the contents if your box.net account should be in the box.net folder in your home directory. Brilliant.

So, now you can edit files, move them around, change, rename, whatever files in the /home//box.net foder, run unison box, then your box account will update itself. Likewise, you can put files in your box account on your phone, upload files through the website, email files in, run unison box again and your box.net folder in your home dir should update itself. AMAZING!

It’s a bit of a pain in the bum having to manually run unison everytime we want updates though…

Create a cron job to run unison automagically

Cron lets your system run commands automatically in the background on timed intervals, without you having to do anything. So we’ll create a cron job to automatically run our unison box command every hour. You can fiddle with that number to suit, but bear in mind that if you need some files sync’d immediately, just run unison box again.

Edit your crontab (your user’s list of scheduled cron jobs):

crontab -e

and put this at the bottom of it:
0 * * * * unison box

Cron’s format is a bit odd – it’s:

m    h    dom    mon    dow    command

So the crontab line we put in will run unison box on the 0th minute of * hours (every hour), * days of the month, * months, * days of the week – which is exactly what we want.

And with that, you should be done! Feel free to drop me a line if you have any problems…

	dnotify -CDMr --background /home/seb/sync -e "unison box"

A standard package in Ubuntu is the Vino VNC server – it’s the one that’s used when you select “allow users to remotely control my pc” which is quite handy. However, it only lets you log in once your user has logged into Gnome or XFCE (i.e. after you’ve got past the login promt in GDM or LightDM) – so if your PC reboots whilst you’re away, it’s a bit of a pain as you can’t VNC into it.

In XUbuntu 11.10, the team removed GDM (Gnome Display Manager) and replaced it with LightDM (Light Display Manager) so these instructions are specific to that, although they may work on other setups.

Instructions

LightDM apparently supports a VNC server out of the box; however, it’s designed to work with tightvncserver (or similar) which create separate X sessions (the 2nd lot of VNC servers I mentioned above) so not quite what we’re after¹.

Ubuntu derivatives use Upstart (upstart.ubuntu.com) to manage startup processes - LightDM (and GDM!) creates a login-session-start event when the display manager is up (i.e. when X is all loaded, but before you’ve logged in) so we’ll create an upstart job that listens for that, and starts a VNC server on the existing X session, allowing connections before login.

Step 1 – we want the VNC server to have a password, just in case – the VNC port shouldn’t really be exposed to the outside world – it’s easy to wrap it in an SSH connection to encrypt everything for you (see here for a good explanation of how to set that up).

sudo x11vnc -storepasswd /etc/x11vnc.pass

This will prompt you for a password to (lightly) secure your VNC server.

Step 2a (for LightDM – XUbuntu) – create the upstart job. Put this in /etc/init/x11vnc.conf :

start on login-session-start
script
/usr/bin/x11vnc -xkb -auth /var/run/lightdm/root/:0 -noxrecord -noxfixes -noxdamage -rfbauth /etc/x11vnc.pass -forever -bg -rfbport 5900 -o /var/log/x11vnc.log
end script

Step 2b (for lxdm – LUbuntu) – LXDM doesn’t seem to emit the right event, but instead is has /etc/lxdm/LoginReady which lets you specify pre-login events. However, we need to first get the xauth setup properly. Edit /etc/lxdm/lxdm.conf and uncomment the line:

xauth_path=/tmp

This specifies that we’re going to use a different xauth path; we also need to update the /etc/lxdm/LoginReady file with our x11 command, using the updated xauth path.

Put this in /etc/lxdm/LoginReady:

/usr/bin/x11vnc -xkb -auth /tmp/.Xauth1000 -noxrecord -noxfixes -noxdamage -rfbauth /etc/x11vnc.pass -forever -bg -rfbport 5900 -o /var/log/x11vnc.log

Step 3 – Done!

Reboot! That’s all there is to it – hopefully, you’ll now have a shared VNC server which connects to your main desktop X session, running on port 5900 using the password you gave, all started automatically as soon as LightDM asks you to login. Hooray!

[VNCServer]
enabled=true
port=5901 # I've used port 5901 so it doesn't interfere with the setup from above

Ubuntu’s normal repositories have Wine 1.3 which apparently works ok, but the latest builds (1.4RC4+) have a few stability fixes and performance tweaks so I decided to try and run that. This may not be the quickest way to install these things, but it was the way I did it and it all works

Instructions

Install Winetricks – a helper tool which downloads and installs things automatically for Wine.

From a terminal:

sudo apt-get install winetricks

Next up, we’ll install the official version of Steam – the --no-isolate parameter makes sure that Steam is installed in the shared wine environment.

winetricks --no-isolate steam

That will download and install Steam and a few other libraries – follow through the installer then let it run and log in. Easy peasy! After you’re sure Steam itself is working (don’t try and run any games yet), exit Steam.

As I mentioned, the latest versions of Wine have a lot of compatibility fixes for games – I noticed an increase in framerate and less crashes after I installed the latest Wine (although this could be the placebo effect…!)

Add the official development Ubuntu Wine repository (again, from a terminal):

sudo add-apt-repository ppa:ubuntu-wine/ppa

and press enter – that adds the Ubuntu Wine repository to your apt sources list, and gets the GPG key for it too.
(this comes from here: https://launchpad.net/~ubuntu-wine/+archive/ppa)

Once that’s done, install the latest versions of Wine using our new repository:

sudo apt-get update
sudo apt-get install wine1.3 wine1.3-gecko

and that’s it! Now you should have the latest version of Wine that will be automatically kept up to date, and a Steam icon on your desktop (if you chose to put one there) – run it and install all your favourite games

This works brilliantly on my laptop to play Oblivion – it’s a tiny bit slower than when run natively under Windows, but I just turned down a couple of the graphics settings and it’s all great. If you have any issues with sound (sometimes it cuts out) then setting Audio -> Direct Sound to “emulation” apparently fixes it.

Requirements

Your own domain (example.com)
Your have a server on the outside world, running PHP (for this example anyway)
A way of requesting a remote web page automatically

Introduction

Our previous internet provider gave me a static IP (2 actually!) on a standard ADSL broadband connection. I also own a domain (for this, I’ll say example.com) – so I setup a zone specifically for my house (subdomain.example.com) – this was nice and easy using any DNS provider as it never changed. I also created NS records pointing subdomain.example.com at my home IP, so that I could create requests for blah.subdomain.example.com

We’ve recently moved house. We’re now with Virgin, and they don’t provide static IPs – so I thought I could use something like dyndns.com or no-ip.com (which runs a small daemon which updates your IP to point to something like yourname.no-ip.com). However, these don’t support recursive wildcard DNS requests without paying for a subscription (i.e. something.yourname.no-ip.com) so I’d only be able to have a single domain name for my home, which makes using Virtual Hosts in apache not really possible…

Solution!

Find a free DNS host which support an API, and supports wildcard DNS records for a fixed zone. I’ve done the hard work for you, and found Zerigo

Setup a free account on Zerigo and creat a new zone for subdomain.example.com – my primary DNS is hosted elsewhere for example.com, so I just created an NS record for subdomain.example.com on my main DNS to point at a.ns.zerigo.com. You could probably host both on Zerigo if you wanted.

Then, create 2 hosts on this new zone – “” (empty hostname) and “*” (the wildcard host) – give these any old IP; they’ll be updated in a minute!

So, now if you do something like (from your machine):

nslookup banana.subdomain.example.com a.ns.zerigo.com

you should get the IP you’ve given both these hosts. If you do, everything’s working fine.

Now, the best bit about Zerigo is that it has a rather nifty REST api for updating DNS zones and hosts, and they provide a PHP client library along with some fairly simple examples. So I created a small PHP script (using their provided API – thank you!) which just updates your zone to point to the client IP requesting the script, provided it’s not the same as it used to be (with very simple authentication):

The PHP script to do it for you

<?php
$WILDCARD_ZONE="subdomain.example.com"

// rather nasty "authentication" - does the job though... (and it's fine provided the logs on your server are private!)
// just check the MD5 of a GET param is the same as one we're expecting
if (isset($_GET["key"]) && "abc123abc123abc123abc123abc123ab"===md5($_GET["key"])) {
        // get the last IP we updated it to - if it's not the same, write the new one into the file
        // for use next time.
        $oldIP=file_get_contents("oldIP");
        $remoteIP=$_SERVER['REMOTE_ADDR'];
        if ($oldIP != $remoteIP) {
                file_put_contents("oldIP", $remoteIP);
        }
        else {
                // IP hasn't changed, so nothing to do - save superfluous calls to the Zerigo API
                die("No change: $oldIP, $remoteIP");
        }
        // include the zerigo API
        require_once("zerigo-dns_api_php/zerigo_ns.php");

        ZerigoAPI::$api_user = 'you@zerigouser.com';
        ZerigoAPI::$api_key  = 'abc123abc123abc123abc123abc123ab';

        // get all zones on your account (up to 50 of them anyway)
        $zones = NSZone::find_all(array('per_page'=>50, 'page'=>1));

        // Now get the zone we're trying to update
        $homeZone = null;
        foreach ($zones as $zone) {
                if ($zone->domain == $WILDCARD_ZONE) {
                        $homeZone = $zone;
                }
        }
        // found the zone we want?
        if ($homeZone) {
                // update all the hosts that match what we're fudging (i.e. the empty host and the wildcard)
                foreach ($homeZone->hosts as $host) {
                        if ($host->hostname == "" || $host->hostname == "*") {
                                // bit of debug to go in my cron logs
                                echo "Updating " . $host->hostname . "." . $homeZone->domain;
                                echo " from " . $host->data . " to " . $remoteIP . "<br/>\n";
                                $host->data = $remoteIP;
                                // call the Zerigo API to update the remote IP for this host
                                if ($host->save()) {
                                        echo "Updated.<br/>\n<br/>\n";
                                }
                                else echo "Failed.<br/>\n<br/>\n";
                        }
                }
        }

        exit();
}
header("Status: 404 Not Found");

(You’ll also need the Zerigo PHP api stuff too, which can be found here: Zerigo PHP Libs)

Getting it to automatically update

I just set a box on my home network to fetch this every now and again – at the moment, I’ve just got a cron job that does it hourly and it keeps my remote IP up to date  (with a low (1 hour) TTL) on Zerigo’s DNS servers. I presume there’s a way in Windows to create a scheduled job that does it too, so this method is pretty cross-platform (and because PHP can run on pretty much anything, it works on all client/server setups).

Hooray!

:set nocompatible

seems to fix the problem. You can make this permanent by sticking

set nocompatible

in .vimrc in your home directory. Anyone know why?!