Matthew's Journal

I have some code that spends a few hours talking to my cattle movement database, and building two dictionaries of movements. I thought it would be good to be able to store the resulting data structures using cPickle, so that in future I could analyse the results on my faster computer, and other suchlike things.

Except I get errors when attempting this:

Looking up movements: . . . . . . . . . . . . . . . . . . . . . done.
Writing data structure to pickle file...Traceback (most recent call last):
  File "./code/python/breeds.py", line 122, in ?
    cPickle.dump(cows,intf,cPickle.HIGHEST_PROTOCOL)
MemoryError

A bit lame (it seems a not unreasonable assumption that if you can hold some data in memory, you can pickle-dump it to file), but there you go. I figure I'll try building a 64-bit python, to give me more address space (it was getting no-where near the available RAM on this system). But, python's configure doesn't have a handy 64-bit switch, nor does it seem to do anything sensible with CFLAGS Indeed, even doing CC="gcc -mcpu=b9 -m64" ./configure didn't quite do what I wanted, and I had to frob LD_LIBRARY_PATH to get any modules built. Indeed, the resulting command-line to run the code with the 64-bit python (and a compiled cx_Oracle module also built 64-bit stylee) ended up being:
LD_LIBRARY_PATH=/opt/local/lib/sparcv9:/export/home/oracle/product/10.1.0/db_2/lib PYTHONPATH=/export/home/matthew/software/cx_Oracle-4.1/build/lib.solaris-2.9-sun4u-2.5:$PYTHONPATH ~/software/Python-2.5.1/python postdoc/code/python/breeds.py

I'm sure it should be easier than that (indeed, 64-bit compilation on solaris often seems to be a bit of pain). I'll find out later whether that's actually worked or not!