emperor: (Default)
Add MemoryShare This Entry
posted by [personal profile] emperor at 11:35am on 21/12/2007 under , ,
At work, I have a 2x 2.66 GHz Dual-Core Xeon Mac Pro with 12 GB of RAM, that I use for some fairly intensive disease simulation work. I deal with reasonably large data-sets; whilst my own code will happily build 32- or 64-bit on OS X, I use other tools (python, R, gnuplot, etc.) that come as 32-bit by default on OSX (or through fink). Building them 64-bit on OS X has proved painful - even with Leopard, the 64-bit stuff is a bit half-arsed (presumably because Apple don't want to ship two versions, one for 64-bit platforms, one for 32-bit), and there's only so much fighting with build-systems I can do before I want to smash things.

So, I figured I might as well try installing a proper 64-bit OS; it would enable me to have a completely free-software system to use as well. Debian wiki has some instructions, which seemed like a good starting point. The 'net is, however, full of a lot of confused and confusing discussion of the issues. I won't duplicate those instructions here (although I will be editing that page in due course). Suffice it to say that up to the "reboot" point, everything went smoothly. I have a second hard-disk which I wasn't using, so I installed onto that, leaving about half of the disk free for future deployment.

The machine rebooted, rEFIt offered me a linux option, I selected that, and after a pause, was offered "Missing operating system" on a line by itself. That error message isn't found in the rEFIt source, which was confusing - I spent some time thinking it was an error from a bit of the Mac's firmware, so fiddling with different partition schemes, partitioning using Disk Utility, etc. These were all blind alleys. The key observation (which was [livejournal.com profile] pm215's) was that rEFIt contains syslinux_mbr.h, and, indeed, the following code snippet:

    if (!HaveBootCode) {
        // no boot code found in the MBR, add the syslinux MBR code
        SetMem(SectorBuffer, MBR_BOOTCODE_SIZE, 0);
        CopyMem(SectorBuffer, syslinux_mbr, SYSLINUX_MBR_SIZE);
    }

So I went looking at the beginnings of partitions (and indeed, in the MBR of the disks themselves) to see what was there. Sure enough, there was LILO on the second disk drive, but Syslinux's MBR on the first disk drive's MBR (i.e. the disk I boot OSX from)!

rEFIt does seem to rather assume you're only using one disk drive (see the gptsync utility, which only looks at the first disk), so it seems that when asked to boot linux, it observed the lack of MBR on the first disk, stuck a syslinux MBR there, and tried to boot it. syslinux then produced the "Missing operating system" error that had so frustrated me.

The work-around is to install your bootloader of choice into the MBR of the first disk (OS X doesn't use the MBR for booting, so this is safe), and then all is well. rEFIt now starts LILO on the first disk, which will happily boot Linux off /dev/sdb2.

That's all good. There are a few wrinkles remaining, though:
  • Filesystems - linux will read hfsplus filesystems, but can only write to them if the journal is disabled. Leopard doesn't let you un-journal a filesystem from the GUI, although you can do so using the command-line diskutil disableJournal [force] command. You don't want to do this on a volume that isn't carefully backed-up! Similarly, although ext2fsx exists, that only manages read-only access to ext3 unless journalling is disabled. It strikes me that these are things that ought to be fixible, without overly endless pain.

  • X - X.org doesn't fully support the NVIDIA GeForce 7300 GT card in the mac pro - you can't make any use of a second monitor currently. I've no idea if this is likely to get fixed any time soon

  • Hardware autodetect - Linux isn't quite a smooth as OSX for this - I had to work a bit at getting the trackball correctly configured, and I need to spend some time getting my other mouses' buttons working (they've been re-mapped as well as the track-balls). I had to hand-hold the X.org install rather more than I'd like, too. Nothing too taxing, though.


When creating users on linux, I made sure to match the uid and gids from OSX, meaning that users can access (albeit read-only) their Leopard homespace on Debian. I'm generally pretty impressed with the linux install, the fully 64-bit toolchain is a real boon, and it's all DFSG-free! :)
There are 9 comments on this entry. (Reply.)
 
posted by [identity profile] wellinghall.livejournal.com at 12:55pm on 21/12/2007
What size datasets do you work with?
emperor: (Default)
posted by [personal profile] emperor at 01:04pm on 21/12/2007
Well, the Cattle movement data is 20GB or so, but I also have to handle the results of multiple simulation runs, do statistics on them, and so on. Most recently, comparing the results from 2 different simulation models, with 100,000,000 runs of each...

Not giant, but big enough that the 2G memory limit you get in 32-bit land bites from time to time.
 
posted by [identity profile] wellinghall.livejournal.com at 01:48pm on 21/12/2007
Thanks for that. I'm getting interested in the sort of modelling other people do cf my own.
emperor: (Default)
posted by [personal profile] emperor at 09:15pm on 21/12/2007
I'm going to be submitting a paper in the New Year, so I could let you have a copy of the draft we submit if you like.
 
posted by [identity profile] wellinghall.livejournal.com at 09:19pm on 21/12/2007
Yes pls
 
posted by [identity profile] deliberateblank.livejournal.com at 01:28pm on 21/12/2007
X - X.org doesn't fully support the NVIDIA GeForce 7300 GT card in the mac pro - you can't make any use of a second monitor currently. I've no idea if this is likely to get fixed any time soon
I find nVidia's own X.org drivers work pretty well. (I think it's even a 7300 I'm running dual head at work.)
emperor: (Default)
posted by [personal profile] emperor at 01:32pm on 21/12/2007
I used to use an nVidia proprietary driver, and it leaked memory like a sieve. Might be worth a try again, I guess...
 
posted by [identity profile] deliberateblank.livejournal.com at 01:40pm on 21/12/2007
Hadn't noticed it leaking - Firefox 2 on the other hand...

I'm pretty impressed with the just-worksness of their driver installer too - much easier than VMware's often-broken kernel module builder.
gerald_duck: (duck and computer)
posted by [personal profile] gerald_duck at 11:25am on 22/12/2007
LInux HFS+'s inability to write to journaled filesystems is the result of a single line of code put there explicitly because of someone's paranoia. Many people are of the opinion that it's in fact perfectly safe to comment out that check — provided you leave the dirty-flag check. People keen to write files from Linux on an HFS+ iPod have little choice, sadly.

October

SunMonTueWedThuFriSat
      1
 
2
 
3
 
4
 
5
 
6
 
7
 
8
 
9
 
10
 
11
 
12
 
13
 
14
 
15
 
16
 
17
 
18
 
19
 
20
 
21
 
22
 
23
 
24
 
25
26
 
27
 
28
 
29
 
30
 
31