Matthew's Journal

Systemd is in many ways an improvement over System V init (sysvinit), but I worry about the direction it seems to be going in (and how upstream interact with folk), and I think distributions need to work a bit harder to preserve user choice in this area. ISTM there is space for a more tightly-focused new init system.( Read more... )

I have a google calendar associated with my work email address, and I mostly use Thunderbird as my MUA. If you have a similar set up, you've probably been using the provider for google calendar - it's the second most popular Thunderbird add-on.

That stopped working for most people late last year, or at least the ability to respond to calendar invitations did. Thunderbird are moving to a new "MailExtensions" framework for add-ons, and I think some part of the legacy framework for older add-ons (like the google calendar provider) has managed to mess up the relevant settings.

Google Calendar does, however, support CalDAV, so you can instead connect Thunderbird to your google calendar thus. The settings to do so aren't entirely obvious, though, so I thought I'd briefly outline how to get this working.

You'll need to know the email address you sign on to google with, and your calendar ID. If you're using your default calendar, that's also your email address (otherwise see [0] below). Then proceed as follows:

• File menu -> New -> Calendar
  
• select On the Network; click Next
  
• select CalDAV
  
• Username is the email address you sign in to google with
  
• Location is https://apidata.googleusercontent.com/caldav/v2/CALID/events replacing CALID with your calendar ID
  
• Click Next
  
• Put something sensible into Calendar Name
  
• De-select Show Reminders (otherwise you get reminded about all the past events when it first loads)
  
• Make sure its associated with the correct email address (this is the one you'll reply to meeting invitations with)
  
• Click Next
  
• Click Finish
  

You should be set; a window will probably pop up and ask you to sign into your google account and allow Thunderbird access to your calendar (say yes!). I found that I couldn't set "Prefer client-side email scheduling" during the creation process and had to enable it via calendar preferences once I'd set the calendar up. Set this if you want to be able to respond to calendar invitations by email.

The bit of this that took a bit of work was the Location setting, which I found in Google's CalDAV API Developer's Guide.

[0] If not, find it in google calendar thus: click the gear and then "settings". On the left-hand pane find the calendar you want under "Settings for my calendars" and click on it. In the menu that appears, click on "Integrate calendar" (or scroll the right pane until you get there). The first item under the heading is the calendar id.

I needed to fill in a PDF form recently (the proper editable sort, not a flat PDF with empty spaces); me being me I wanted to use Free tools, and ideally do so in a way that I could use version control on the bits I was adding to the form.

Thankfully, with pdftk, this is pretty doable. It was mostly only a bit fiddly because there wasn't a coherent answer to some of the edge cases (e.g. round tick boxes). Basically, the process is:

1. pdftk generate_fdf to get the file you want to edit
2. pdftk dump_data_fields if necessary to learn about tickboxes
3. edit the fdf file
4. pdftk fill_form to fill the form in

As an example form, here's a publically-available one. If you run pdftk n1-eng.pdf generate_fdf output fields.fdf, you get an .fdf (Form Data Format) file. It looks a bit weird, but basically you can edit the relevant bits of contents as text.

An excerpt:

<<
/V ()
/T (In the)
>> 
<<
/V /
/T (Check Box34)
>> 

So if you put some text in the parentheses above "In the", then that text appears in the relevant box on the form. In this case, it's pretty obvious where on the form that's going to be, but not all the fields are so obvious - where is Text28, for example?

You can, obviously, put dummy data in each place, fill the form out and observe the results. That's fine, but obviously doesn't help with the Check Boxes. To find out more, you want pdftk dump_data_fields, which includes for this form:

---
FieldType: Text
FieldName: Text28
FieldNameAlt: Brief details of claim
FieldFlags: 12587008
FieldJustification: Left
---
FieldType: Button
FieldName: Check Box34
FieldFlags: 0
FieldJustification: Left
FieldStateOption: Off
FieldStateOption: tickbox No - Does, or will, your claim include any issues under the Human Rights Act 1998?
FieldStateOption: tickbox Yes - Does, or will, your claim include any issues under the Human Rights Act 1998?
---

From this, we can see that Text28 is where we put the "Brief details of claim". It also tells us how to tick the relevant tickbox about the HRA 1998, though it's a bit non-obvious. What you do is include the entire FieldStateOption of the box you want, thus:

<<
/V (tickbox No - Does, or will, your claim include any issues under the Human Rights Act 1998?)
/T (Check Box34)
>> 

So you can edit everything into the FDF file, and keep it in git (and get useful diffs), and then turn it into a filled-out PDF with pdftk form.pdf fill_form data.fdf output form.filled.pdf - the form is neatly filled out, including the tick-boxes!

Git has become the de-facto standard revision control system, so one of the things I've been doing recently at work is moving our old Subversion repositories to git[0]. git svn is a plausible tool, but it's more designed for continued inter-operation between git and svn than for a one-way export. For example, because tags in svn are basically specially-named branches, they end up as branches in the resulting git repository, which isn't really what you wanted.

Thankfully, the internet has plenty of handy tutorials on how to improve on git svn's default operation, so the resulting git repository is a better starting point for future development. Probably the best of these is The git book's article. Two of the steps are to turn the "tags as branches" into proper tags, and to rename the remotes/* branches to local branches:

#tag-branches into tags
$ cp -Rf .git/refs/remotes/origin/tags/* .git/refs/tags/
$ rm -Rf .git/refs/remotes/origin/tags
#remotes/* branches into local branches
$ cp -Rf .git/refs/remotes/* .git/refs/heads/
$ rm -Rf .git/refs/remotes

It may[1], however, turn out that there is nothing in .git/refs/remotes/origin/tags/, even if there were tags in your svn repository and git branch -a shows the relevant branches! This will be vexing. Furthermore, there will only be a few branches in .git/refs/remotes/. What is going on?

The answer is that git has tidied up .git/refs because it has run git pack-refs for you. That means that there is now a file .git/packed_refs which has lines of the form:

7c8a5686b698a61a33c4a6c73776da20b5ede635 refs/heads/origin/protocol2

This is the equivalent to having the file .git/refs/heads/origin/protocol2 containing 7c8a5686b698a61a33c4a6c73776da20b5ede635 (i.e. that the tip of the origin/protocol2 branch is 7c8a568...).

So what? Well, it now means you can achieve the tidying up that the above shell runes do by editing .git/packed_refs (carefully!). For example, I used emacs' query-replace function to replace refs/remotes/origin/tags/ with refs/tags. And joy was unconfined.

So I wrote this in case anyone else trips over this feature (so feel free to link to this post). If I have some tuits I might try and get the git book updated, too...

[0] Also taking advantage of this opportunity to re-structure them somewhat with git filter-branch and friends, but that's not really relevant right now.
[1] More likely if your repository was large or complex

I encountered this problem yesterday, and it took a bit of yak-shaving to fix. Further, I didn't find much of immediate help by googling, so I thought I'd note down what I did.

A little background

Back in the day, you configured your X session by writing a small script that would run a few programs (say a clock, maybe an xterm or two, something to fettle your background), and finally execute a window manager. You'd call this ~/.xsession, and all was good. As time went by, linux distributions started shipping increasingly fancy desktop environments like GNOME and KDE, but you could always write yourself an .xsession, and keep your old-style setup.

I've never quite been convinced by fancy window managers; I can't afford really massive monitors, so am jealous of my screen real-estate, and all these taskbars and widgets and so on take up too much of my screen. Also, I'm very used to my X setup, so it's efficient for me to carry on using it. If you care, I run fvwm with a 4x3 set of virtual screens, either xplanet (the earth as viewed from the moon) or xphoon (the moon in its current phase) as backdrop, and a little buici-clock.

The problem

Anyhow, atreic has been very kindly letting me use their computer to work from. It's running Ubuntu 12.04 LTS, which has unity as the default window manager, and lightdm as the default display manager. I wanted to get my usual setup going, but with minimal disruption to atreic's computer (which implies minimal system-level changes). I installed fvwm, and that meant that lightdm would then offer me an fvwm option when I logged in, but that still didn't run my .xsession.

The solution

A bit of digging found me this Debian bug report, which was forwarded upstream to Ubuntu. Debian resolved the problem by adding another .desktop file to their lightdm package that would run the user's .xsession. Ubuntu's developers, in their wisdom, decided that anything resembling a "run my .xsession" option would be far too confusing for their users. Instead, as a Ubuntu user, you have no way of getting your .xsession run without admin access to the machine.

Thankfully, I do have this, so I was able to pinch lightdm-xsession.desktop from Debian's package, and install it into /usr/share/xsessions/. And all was well with the world again!

A little grumbling

It seems to me that Ubuntu have chosen to break a traditional linux behaviour with not a great deal of justification; I ought to be able to get my .xsession run without needing rootly powers. Further, the man-pages for unity, lightdm, and dm-tool all leave a lot to be desired; the configuration files for lightdm, for example, are pretty poorly documented.

A couple of footnotes

I gather that Ubuntu's package of gdm does include a .desktop file that runs the user's .xsession. That's quite a heavyweight workaround!

If you want to know what all these .desktop files are about, there's a specification online.

You might well want to be able to get back to the Ubuntu login screen without logging out (if, for example, you wanted to let atreic log in to their own computer ;-). You can do this from the command-line thus:
dm-tool switch-to-greeter

I have concluded that I really ought to learn some Perl. In my experience, while reading the fine manual is a good start, having some sort of project in a new language is the way to really cement my learning.

I have also been thinking for a while that it might be nice to have some record of beers I have drunk. Partly so I can avoid the real stinkers, and partly so that when faced with a beer festival list, I can try and pick new beers. And also because I'm a bit of a sad geek.

Things to note about a beer would be: name, brewery, abv, rating (I am persuaded by Ben that /5 is the way to go here, rather than the /10 I've used in the past), comment, and maybe date and occasion of tasting and country of origin? I guess you might want the opportunity to add repeat tastings in the event of changing one's mind substantially.

Obvious features would be: interactive or bulk import, various search and sort routines for output (cli and maybe web-based too?), and the ability to feed in a beer list (from e.g. a festival) and get back new beers, and perhaps beers I've liked a lot previously.

Less obvious features would be: tweeting new beers as they are added to the system, links to Open Beer DB, beermad, Cyclops, etc., export in odd formats...?

I don't think there's much of great difficulty in implementation - some sort of SQLish storage should get most of the searching and sorting pretty much for free, and the rest is just some text processing (with a bit of thought about reliable updates, future spec changes and the like). What have I missed?

I'm sure this is a wheel that has been invented before, but maybe someone else will want to use it :)

And finally, now would be a good time to pass on any pearls of wisdom that you wish someone had imparted to you when you started to write Perl ;-)

Is it a bug that while (GNU) tar -cjf foo.tar.bz2 creates foo.tar.bz2, tar -cjf user@host.name.fqdn:/path/to/foo.tar.bz2 won't create /path/to/foo.tar.bz2 on the target host, instead failing with ENOENT, or merely a bug in the man-page that it doesn't mention this fact?

ETA: Please Switch NoBoydie, those of you to whom that means something...

atreic and I are off to Portugal later in the year. In a probably-vain attempt to avoid being one of those annoying monoglot English tourists, and because I can only remember about two words from my previous trips to Portugal (with MWW, years ago), I thought I should try and pick up a bit of the language before then. There's a bewildering range of teach-yourself courses available (although the number is slightly reduced if you want European Portuguese), but I eventually settled upon Teach Yourself Instant Portuguese, since 35-45 minutes a day for 6 weeks seemed about the time I had available. Perhaps this redefinition of "instant" should have been a warning...

Anyhow, I finished week one on Sunday, and manage a respectable 89% in the test for the week. Having had yesterday off (you get one day off a week), I'll be starting week two later today. I don't yet feel I can say anything terribly useful, but I guess that's to be expected. It's obviously aimed at a certain type of person, though: one of the things I learned how to say is "A minha esposa tem um Mercedes", or "Trabalho para três bancos grandes. O trabalho é chato, mas o dinheiro é bom." It's pretty grammar-light, which is probably sensible, but can be a bit frustrating: I ended up looking ahead to see what the difference between "sou" and "estou" is, for example, and I'm not clear why sometimes it's "Tem uma casa em Londres", and other times "Sim, tem casa em Londres", or why sometimes you say "Estive em [place]", and other times "Estive no [place]". The author's view is that the little words aren't necessary to being comprehended, and that one is better off saying something approximately right but plausibly pronounced and confidently. She may well be right! I guess the test will be to see how useful it is in Portugal, or whether everyone will hear my badly-mangled Portuguese, wince, and reply in English...

Whilst looking around for a suitable course, I noticed Rosetta Stone[0], and Linguaphone, both of whom claim to be able to teach one to a plausibly-proficient level in a not-very-large period of time (although both shy way from claiming exactly how long, I note). I've always wondered if they are actually any good - if not, how do they stay in business, but if so, why don't we use them in schools, rather than taking 5 secondary years to get to GCSE level?

Finally, the course I have includes some flash-cards. I didn't want to cut my nice new book up, so I LaTeXd up some of my own (this involved debugging the slightly-bitrotted-flashcards package). pm215 suggested that I consider Anki, as a more intelligent way to do flashcard-type learning. So, having used only the old-fashioned flashcards last week, I'm going to give this a go this week. One difficulty, though, has been entering accented characters - LaTeX lets you specify them in ASCII, so \~a for ã, but that doesn't really work for Anki (you can feed it LaTeX, but it looks very odd having one LaTeX'd character in the middle of a word). After a bit of googling, I used xev to find out what X thought my Alt-Gr key was (ISO_Level3_Shift), and told X to use it as the Compose key: xmodmap -e "keysym ISO_Level3_Shift = Multi_key". This means I can generate most of the symbols in the WP page, although the rune for generating ǎ, for example, still doesn't work, and there should be a nicer way of setting this up.

This should be documented more usefully, I feel (I shouldn't be looking at WP, for example), and it would be nice to have something like the OSX app that shows you what the keyboard "looks" like (e.g. I'd hold down Alt-Gr, and it would highlight the keys that I could press to generate an accented character, as well as what accent would be about to appear)...

[0] who only offer the wrong sort of Portuguese

Over dinner last night, we talked a little about RSI. One thing that came up was that using the shift key and another key at the same time with the same hand (e.g. left-shift + q) puts your hand in a funny shape, and often makes RSI worse. People taught to touch-type don't do this, but others do.

It occurred to me that you could get some bit of X to only make opposite-shift work (at least for letters, sorting the punctuation characters out might be a bit harder), and this would fairly rapidly educate typists to DTRT. How hard would it be?

"time" is a reserved keyword in bash, not a builtin as one might naively expect[1,2]. Even more amusingly, many UN*X systems also have /usr/bin/time which has rather less nice output.

This leads to some irritating edge-cases if you forget this (or didn't realise it in the first place). Consider:

bash-3.2$ time sleep 1

real    0m1.009s
user    0m0.000s
sys     0m0.002s
bash-3.2$ FOO=bar time sleep 1
        1.00 real         0.00 user         0.00 sys
bash-3.2$ time FOO=bar sleep 1

real    0m1.002s
user    0m0.000s
sys     0m0.002s

...and things only get worse if you actually start chaining things together...

bash-3.2$ time sleep 1 | time sleep 4
        4.00 real         0.00 user         0.00 sys

real    0m4.004s
user    0m0.001s
sys     0m0.005s

Not perhaps the best design decision ever...

[1] #chiark thinks this was to make it possible to time entire pipelines, although there would be several obvious idioms for doing that if time were a builtin
[2] See the "Pipelines" section in bash(1) for more details