Lanyards

Blog update

I'm trying to resurrect this extremely crusty blog. As a result of the ongoing twitterpocalypse I have moved to Mastodon. I've removed my half-assed tweet mirror from here, since it wasn't very interesting, and I don't intend to mirror my Mastodon posts here either, but I will occasionally copy posts manually for posterity.

Arts and crafts with conference lanyards

Do you have a sad graveyard of conference lanyards? Cannibalise them for parts! That's free hardware you can use for all kinds of stuff, like a DIY sewing bird / third hand, a keyring with portions that unclip, or a clip to stop your phone case from falling out of a small pocket.

Two crocodile clips connected by a piece of elastic DIY sewing bird made from a small clamp, a snap clip and a binder clip
Compartments in a storage container containing assorted lanyard straps and clips A keyring with some items attached with snap clips

Noether

Noether

Today Hodgestar and I had to say goodbye to a very old friend. Noether was an excellent cat and a force of nature, known to all our friends for her unique personality. She's irreplaceable, and I will miss her terribly, but I'm very grateful for every moment that I got to spend with her.

Installing Not Tetris on a recent Ubuntu version

Not Tetris is an awesome game which is just like Tetris except not. Everyone should play this game. Unfortunately, it was built with a very old version of the Löve game engine, which makes it tricky to install on a modern system.

Building Löve 0.7.2 on Ubuntu Zesty is failing for me for some reason, although hopefully this is a fixable issue. Edit: you fix it by installing headers for SDL 1.2 and Lua 5.1 -- then you can check out and build the source, which is much simpler. The executable from the 0.7.2 Ubuntu Precise package works just fine, however -- here are streamlined instructions for extracting it and instaling it locally in your own bin directory.

(You can also just install the package with dpkg, copy the executable and then uninstall it (so that it isn't overwritten by the updated version of the same package), but it's much less tedious to extract the package without installing it.)

# Work in a subdirectory to make it easier to clean up
mkdir tmp
cd tmp

# This is the package for amd64; replace with other architecture if applicable
wget https://launchpad.net/ubuntu/+archive/primary/+files/love_0.7.2-1build1_amd64.deb

# Extract
ar x love_0.7.2-1build1_amd64.deb
tar -xzf data.tar.gz

# Copy executable
cp usr/bin/love-0.7 ~/bin/

# Clean up
cd ..
rm -rf tmp

Make sure your ~bin is in your path.

Now you can get Not Tetris!

# Get the ZIP and put it in a sensible place
mkdir -p ~/Games/NotTetris
cd ~/Games/NotTetris
wget 'http://stabyourself.net/dl.php?file=nottetris2/nottetris2-linux.zip'

# Unpack
unzip nottetris2-linux.zip
rm nottetris2-linux.zip

First let's make an executable wrapper you can run on the commandline:

echo 'love-0.7 ~/Games/NotTetris/Not\ Tetris\ 2.love' > bin/nottetris
chmod +x bin/nottetris

Now you should be able to launch the game with the command nottetris. You can also integrate it into the XDG menu. Put this into ~/.local/share/applications/nottetris.desktop:

[Desktop Entry]
Encoding=UTF-8
Value=1.0
Type=Application
Name=Not Tetris
GenericName=Not Tetris
Exec=nottetris
Categories=Game

Now it should also appear in your menu under Games.

Happy Not Tetrising!

Using Calibre to catalogue a physical speculative fiction book collection

The problem

I have a large collection of paper books, which I am slowly entering into Calibre as empty book records, so that I can have a more easily accessible record of what I do and do not own.

I initially thought that I could make the data capture trivial by scanning the ISBN barcodes of the books with a phone app. Then I would enter a list of ISBNs into Calibre, push a button to fetch metadata for them automatically, and everything would Just Work, like magic.

I was wrong for various reasons:

  • A lot of my books predate the existence of ISBNs.

  • I want my virtual book records to match my physical books as closely as possible -- but an ISBN is not an unambiguous identifier of a particular edition of a book. Multiple editions with completely different covers can share the same ISBN.

  • Calibre wasn't really designed for this kind of pedantic cataloguing of physical objects, so it doesn't really care about these distinctions. It also helpfully smushes metadata records together if their ISBNs match, and there is no way to make it stop.

  • The default public metadata sources that Calibre uses don't even have metadata for the vast majority of old editions of SFF books.

The solution

A really good source of metadata for old SFF books is ISFDB, the Internet Speculative Fiction Database. And there is a plugin for Calibre which scrapes metadata from it! Unfortunately it hasn't been updated in several years, and ISFDB's HTML periodically changes. So I am maintaining a fork, which I tweak whenever I enter a new batch of books and discover that something has broken. I have also started a reimplementation, in which I hope to include all the little bits of data entry glue that I am about to describe.

Edit: I am now focusing on the reimplementation -- the fork is pretty much abandoned.

My current workflow

  1. I take a pile of books and look them up on ISFDB in my browser (Firefox). I usually process one author at a time, since it's the most efficient way to find multiple book titles at once. I search each title page for the specific edition which most closely matches the physical copy I have. This record is uniquely identified with an ISFDB ID which appears in the URL and on the publication page.

  2. At this point I would previously laboriously copy the ISFDB IDs from all the open ISFDB pages by hand into a text file, and then run a script to create entries in Calibre with these identifiers. The manual copying became very annoying very quickly, so I hacked together a Python script which automatically extracts these identifiers from the currently open tabs in a running Firefox session. Now I can pipe the output of this script to the script which creates records.

  3. At this point I have some empty records with only the ISFDB ID set (and also some custom columns which are not related to the metadata). Now to avoid the record-smushing issue I disable all metadata sources except ISFDB (important!) and fetch metadata for all the records. If an ISFDB ID is present, my fork of the plugin will ignore all other data (like author and title) and use only the ID in its search, so assuming that I have found the correct records in step one the download is guaranteed to fetch the correct data.

  4. Now I do some manual cleanup, like fetching or correcting cover images which were missing from ISFDB.

The Horrible Firefox Hack

Edit: The latest versions of Firefox store the session in an lz4-compressed json file rather than an uncompressed json file, which necessitates the update below (thanks, StackOverflow!). You will need to install the lz4 library.

Edit: Recent versions of the lz4 library require you to import lz4.block explicitly.

isfdb_ids_from_firefox.py:

#!/usr/bin/env python3
firefox_session_path="/home/confluence/.mozilla/firefox/mhxsxkg0.default/sessionstore-backups/recovery.jsonlz4"
import lz4.block
import json
import re
f = open(firefox_session_path, "rb")
magic = f.read(8)
session = json.loads(lz4.block.decompress(f.read()).decode("utf-8"))
f.close()
tabs = []
for w in session["windows"]:
    tabs.extend(w["tabs"])
urls = [t["entries"][-1]["url"] for t in tabs]
for u in urls:
    #print(u)
    m = re.search("www\.isfdb\.org/cgi-bin/pl\.cgi\?(\d+)", u)
    if m:
        print(m.group(1))

The Record Creation Script

calibre-add-from-isfdb.sh (marvel at my consistent naming conventions):

#!/bin/bash
while read id
#for id in `cat $@`
do
  calibredb add -e -I isfdb:$id
  added_id=`calibredb search -l 1 identifiers:isfdb:$id`
  if [ -n "$added_id" ]
  then
    calibredb set_custom "shelf" $added_id "SFF"
    calibredb set_custom "read" $added_id "1"
  fi
#done
done < "${1:-/dev/stdin}"

In addition to creating a new record with the ISFDB ID filled into the identifier field with the appropriate prefix, I also set two custom columns to mark the book as read and file it in the correct category, which I have called a shelf. You can edit this to set whatever custom values you want, or remove it entirely.

Assuming that both scripts are executable and in your path, you can put them together like this:

isfdb_ids_from_firefox.py | calibre-add-from-isfdb.sh

Future work

In the reimplementation I hope to incorporate an entry field for ISFDB IDs and the magical Firefox session scraping directly into the plugin, so that the whole process is more streamlined, not operating system-dependent, and more usable by other people. In the meantime, if you use some flavour of Unix, you may be able to use my very messy current setup with minor modifications.

If you have comments, questions or rotten tomatoes, contact me on Twitter or file an issue against the reimplementation on GitHub (even though it currently doesn't exist).

Hello world

I'm finally ditching the WordPress blog on my old server in favour of a statically generated one on my new server. I'm using Nikola, which seems pretty cool so far. I have imported my embarrassing old posts and some aggregated tweet backups. I'm hoping that if I make this blog usable again I will actually use it, so that I can stop cramming everything into 160-character telegrams stored on other people's servers. No comments for now; hardly anyone ever commented anyway. ;)

Yet another way to play videos in an external player from Firefox

I spent some time today trying to figure out how to get Firefox to play embedded videos using anything other than Flash, which is an awful, stuttering, resource-devouring mess. The internet is littered with old browser extensions and user scripts which allegedly make this possible (e.g. by forcing sites like YouTube to use the vlc media plugin instead), but I was unable to get any of them to work correctly.

Here's a quick hack for playing videos from YouTube and any other site that youtube-dl can handle in an external mplayer window. It's based on several existing HOWTOs, and has the benefit of utilising a browser extension which isn't dead yet, Open With, which was designed for opening websites in another browser from inside Firefox.

I wrote a little script which uses youtube-dl to download the video and write it to stdout, and pipes this output to mplayer, which plays it. Open With is the glue which sends URLs to the script. You can configure the extension to add this option to various context menus in the browser -- for example, I can see it if I right-click on an URL or on an open page. You may find this less convenient than overriding the behaviour of the embedded video on the page, but I prefer to play my videos full-screen anyway.

This is the script:

#!/bin/sh
youtube-dl -o - $1 | mplayer -

Make it executable. Now open Open With's preferences, add this executable, and give it a nicer name if you like. Enjoy your stutter-free videos. :)

(Obviously you need to have youtube-dl and mplayer installed in order for this to work. You can adapt this to other media players -- just check what you need to do to get them to read from stdin.)

Why phishing works

Let me tell you about my bank.

I would expect an institution which I trust to look after my money to be among the most competent bureaucratic organisations I deal with. Sadly, interactions which I and my partner H have had with our bank in recent years suggest the opposite.

Some of these incidents were comparatively minor: I once had to have a new debit card issued three times, because I made the fatal error of asking if I could pick it up from a different branch (the first two times I got the card, my home branch cancelled it as soon as I used it). More recently, when I got a replacement credit card, every person I dealt with had a completely different idea of what documents I needed to provide in order to collect it. When H bought a used car, it turned out that it was literally impossible for him to pay for it with a transfer and have the payment clear immediately -- after he was assured by several employees that it would not be a problem.

Some incidents were less minor. H was once notified that he was being charged for a replacement credit card when his current card was years away from expiring. Suspicious, he called the bank -- the employee he spoke to agreed that it was weird, had no idea what had happened, and said they would cancel the new card. H specifically asked if there was anything wrong with his old card, and was assured that everything was fine and he could keep using it. Of course everything was not fine -- it suddenly stopped working in the middle of the holiday season, and he had to scramble to replace it at short notice. All he got was a vague explanation that the card had to be replaced "for security reasons". From hints dropped by various banking employees he got the impression that a whole batch of cards had been compromised and had quietly been replaced -- at customers' expense, of course.

What happened last weekend really takes the cake. The evening before we were due to leave on a five-day trip, well after bank working hours, H received an SMS telling him that his accounts had been frozen because he had failed to provide the bank with some document required for FICA. This was both alarming and inexplicable, because we had both submitted documents for FICA well before the deadline years ago. The accounts seemed fine when he checked them online. When he contacted the bank, he was assured that he had been sent the SMS in error, everything was fine, and he didn't need to provide any FICA documents.

So we left on our trip. I'm sure you can see where this is going.

On Thursday evening H's accounts were, in fact, frozen. He tried unsuccessfully during the trip to get the bank to unfreeze them, but since it was closed for the weekend there was pretty much nothing he could do until we got back home.

Hilariously, although employees from the bank made a house call this morning to re-FICA him, he had to go to the branch anyway because they needed a photocopy of his ID (scanning and printing is magically not the same as photocopying).

Again, what actually happened is a mystery -- the bank claims that they have no FICA documents on record for H (as an aside, why are customers not given receipts when they submit these documents to the bank? If there is no record of the transaction, the bank can shift blame to the customer with impunity if it loses vital documentation).

We're very fortunate that none of these incidents had devastating consequences for us, since they only impacted one of us at a time. If we relied on a single bank account, we could easily have ended up without food, without electricity, or stranded in the middle of the Karoo with no fuel. This is pretty clearly not an OK situation.

The common thread running through all these incidents is a lack of communication: both between the bank and its customers, and within the bank. We rapidly discovered through our dealings that while the bank maintains the facade of a single, unified entity, it in fact comprises several more-or-less autonomous divisions, and communication between these divisions often resembles diplomacy in the Balkans.

I would not be surprised to discover that various aspects of customers' information are distributed over several disconnected databases. There appears to be no way for certain types of information to be linked to accounts, which necessitates the use of bizarre hacks. When H's credit card was disabled, this was in no way reflected in the online interface -- he just had an enormous amount of money reserved on the card. When his accounts were frozen, this was again not apparent in the interface (which was recently updated) -- his available credit balance was just zeroed, and he got a non-specific error whenever he tried to transfer funds. I believe that the back-end infrastructure for managing this information effectively and making it available to customers and employees simply does not exist.

As a result of this, I have learned not to believe a word that a bank employee says if there is any chance that the issue crosses some internal jurisdictional boundary. In the best case scenario they are aware of their own lack of information and are able to direct you to the appropriate department. In the worst case scenario, they dispense outright misinformation -- something which can have devastating consequences (for you).

We live in an age with an abundance of instant communications methods. In spite of this, it appears to be beyond the bank's abilities to inform people timeously about what is going on with their accounts. It has backed off from electronic communications presumably because of fears of phishing, and has fallen back to two obsolete and inefficient channels: voice phonecalls, which are prone to misunderstandings and human error and leave the customer with no written record, and SMSes, which have a character limit. Both these channels are vulnerable to blips in the cellphone network infrastructure, customers having no airtime, or customers just not being available to answer their phones at certain times (since voice phonecalls are synchronous). The bank also uses these channels as if it believes that they are intrinsically secure and trustworthy, which is ludicrous, as anyone who keeps getting calls from "the Microsoft Support Centre" can attest.

In order for this dysfunctional system to work, the bank appears to expect its customers to accept unexpected additional charges unquestioningly, and to follow instructions issued by unknown persons calling from unverified internal numbers, even if they are nonsensical or suspicious and cannot be corroborated by other bank employees contacted through more reputable public-facing channels. It is standard procedure for such callers to demand personal information from customers in order to verify their identities, but they offer no evidence of their own legitimacy.

In short, the bank expects us to be the kind of credulous chumps who fall prey to phishing scams. It's easy to see why phishing works when genuine communications from the bank are so unprofessional and follow such laughable security practices that they are virtually indistinguishable from phishing.

I haven't named my bank, although it's pretty easy to find out what it is if you know where to look, because I'm sceptical that there are significant differences between the way South African banks operate; particularly the Big Four. I've certainly heard the same kinds of horror stories from customers of other banks.

This is the final straw which has led us to investigate other banking options. Whether the pastures we're moving to are greener or just differently green remains to be seen.

If you feel strongly that there is a bank which is notably better or worse than the others, or you just want to share your own tale of banking woe, let me know in the comments. I'll be over here, stuffing my money into a sock to hide under my mattress.

Polish fridge cake recipe

I'm reposting this with better instructions and more specific ingredients. This dessert was made in Poland during the communist era as a chocolate substitute, and will probably be familiar to a lot of Poles. The recipe is idiot-proof (or at least idiot-resistant up to 200m). If you have burned fudge (*cough* like me), don't worry -- this is a lot easier.

Ingredients:

  • 1 cup sugar
  • 1 cup water
  • 250g unsalted butter
  • 4 tablespoons cocoa powder
  • 500g full cream milk powder
  • Vanilla and/or almond essence
  • 200g plain biscuits (e.g. Marie biscuits; one packet)
  • 300g of your favourite nuts (3 packets)

Equipment:

  • Large pot
  • Spoon
  • 2l tupperware container with lid (e.g. 2l ice-cream tub)
  • Aluminium foil for lining
  • Cutting board
  • Large knife

If you have no foil you can take your chances with a bare plastic container, but it will be harder to unmould the cake later. If you're going to do that, at least make sure it's flexible. You can probably use one of those springform cake tins as well.

Instructions:

Put the sugar, water, butter and cocoa powder in the pot, making sure that there are no lumps in the cocoa powder (you can sift it through a sieve, or just pre-mix it into the sugar). Heat the mixture until the butter has melted and the sugar has dissolved, stirring until you have a uniform brown goo. Don't overheat it; when it starts boiling you should be done.

Remove the mixture from the heat and leave it to cool down slightly. While it is cooling, break up the biscuits into quarters, and if you're using walnuts or pecans pick through them to make sure there are no bits of shell. Line the container with foil -- you can mould a foil lining over the outside and then transfer it to the inside.

Once the mixture is no longer boiling hot, add a few drops of vanilla/almond essence and the milk powder and mix thoroughly until you have a uniform mass. Add the nuts and stir them in. Then add the biscuits and stir them in. The mixture should now be pretty solid. Spoon it into the lined container, smooth the top with a spoon, put the lid on and put it in the fridge overnight. Pat yourself on the back, and eat all the chocolate left over in the pot.

The next day, unmould the cake from the container, peel off the foil, flip it over on the cutting board, cut it into slices about half a centimetre thick and cut the slices into halves or thirds. Stack up the pieces back in the container, and eat any that don't fit.

Variations:

If you're one of those unfortunates allergic to nuts, you can use more biscuits instead. Some people put in raisins and other dried fruit -- there's no accounting for taste. If you don't eat gluten, leave out the biscuits. If you leave out the biscuits try to put in a bit less water.

It should be possible to make this with margarine and some kind of vegan milk powder substitute -- if you try it and it works, let me know what you used in the comments.

Goodbye, Sober Counsel

Sober Counsel

We got her and her sister Furious Purpose as kittens. She was always the calmer, less adventurous one. She was a skilled hunter, and liked to have her belly rubbed. I regret not having a more recent photo of her. We couldn't be there when she had to be put to sleep, and we will always be grateful to our friend Marita for looking after her.

Finding articles on Twitter

I have strong opinions about retweeting, which seem to be the opposite of many other people's opinions about retweeting. I like new-style retweets. If I retweet someone, I want it to be clear that they are the original source of the tweet. If someone else tweets something terribly insightful, I don't want to see fifty old-style retweets of it. Finally, I would like "RT @joe RT @bob OMG, totally RT @alice I agree! RT @cheesemonger OMG you guys I loled!!1 RT @somedude this is lank insightful RT @dave The" to die in a fire.

Which it mostly has. When new-style retweeting was first introduced there was a lot of wailing and gnashing of teeth as some people were outraged that they were seeing tweets from people they weren't directly following in their timeline. Today, nobody seems to care anymore, and I seldom see a manual old-style retweet. Unfortunately, I see a similar problem with the automated sharing of articles.

As far as I'm concerned, the best way to share an article on site Foo on Twitter is this: 1) go to Foo's Twitter stream, 2) find the tweet which links to the article, 3) retweet it. Unfortunately, what most "share this article on Twitter" buttons that sites display (and encourage you to use) actually do is make you create an old-style retweet. It usually automatically includes the site's Twitter username, and of course it's a link to the site, so credit is not really a problem -- but duplication still is, and in my OCD opinion this form of sharing is far less elegant than retweeting the original, official tweet.

I made a bookmarklet which searches Twitter for the title of the page you're currently reading. It tries to autodetect and chop off the site name if it exists, and it loads the results in a new window (or tab, depending on your browser preferences). It currently doesn't limit the search to the site's Twitter account, because determining what that is is less trivial than extracting the title from the page. I may write a more complicated bookmarklet when I have more time. In the meantime, if you're as obsessive about Twitter etiquette as I am, enjoy:

Find on Twitter