WibbleBlog

So I’ve had the new machine running for a couple of weeks now and it is rather nice.

My home made benchmarks are for conversion of photos from RAW to JPEG with Bibble Pro and conversion of video from DV to MPEG2 from within Cinelerra.

In the photo benchmark the old 2.4Ghz Celeron D/1Gb system converted 27 RAW images from my 20D to JPEG in 11 min 38s. The 2.4Ghz Core 2 Quad/2Gb system did the same set in 1 min 48 - a 6.5x speedup. Not bad. Of course this is biased by Bibble being multi-processor aware, but that is half the point of having multiple cores.

Since the video conversion was not multi-threaded, this is more a test of the raw clock-for-clock performance of the two chips. The result was a 2.33x speedup with the quad (from 8 min 34s to 3 min 40s).

All round pretty good I thought.

Or something like that.

I was all set to buy an iMac. Then intel were evil and dropped their prices. Given that I could now get a Q6600 (yes, a quad core 2.4Ghz Core 2 chip) for less than $300 and with whatever silly offer Crucial or OCZ are having today get 4Gb of decent (4-4-4-16 DDR2-800 for example) RAM for about $150 why would I pay St Steve $1500 for a 2.16Ghz Core 2 with only a gig of RAM.

I admit that a fully specced machine with monitor etc would pan out at around a grand, that is still rather a lot cheaper. With academic pricing, I could throw in Windows XP 64bit edition ( or Vista Home Premium), Photoshop Elements 5.0, Premiere Elements 3.0, Photoshop Lightroom and Office Professional and still come out under the $1500.

For comparison, getting the equivalents on the Mac (Photoshop Elements 3.0, Final Cut Express, Aperture and Office) would take the Apple total to $2000.

Of course, in the short term I’m going to stick it out with linux, but the temptation to head Windows-wards is getting stronger. Especially given that a number of people who’s opinions I respect reckon that Vista is pretty good.

Next up, what I’ve actually bought.

Everyone talks about the Digital Photography Workflow. This usually means the process of getting photos from the camera through to print with stops on the way for cataloging and archiving of the digital data.

\n

My current work flow is thus:

\n

(1) Copy files from CF card with USB card reader to \’DropBox\’ folder.

\n

(2) Use GQView to go through them and discard the obviously terrible ones (focus, composition etc)

\n

(3) Batch rename the files using GQView

\n

(4) Move them to the appropriate directory in my hierarchy. I store them by camera then in a folder named as YYYY-MM-DD with the RAW images in a RAW sub-folder.

\n

(5) Batch convert the RAW images using a template in LightZone as a first pass. Tweak those that deserve it.

\n

(6) Get prints using either RitzPix for simple 6×4\’s and EZPrints for enlargements and panoramas.

\n

For step 4 I have a perl script that I call from GQView as an \’editor\’ the script makes use of the wonderful Image::Exiftool Library (see bottom of post)

\nI could probably use some better colour management - I\’ve never really got the hang of that - and getting the files off the compact flash could be better streamlined.

\nThe real surprise to me has been that this process is simpler than the palaver I had on the Mac. Who\’d have thought it.\n\n

\n#!/usr/bin/perl -w\n\nuse Image::ExifTool \':Public\';\nuse File::Copy;\nuse File::Basename;\nuse File::Path;\n\nmy $Canon20Ddirstring = \'/home/ajj/Photos/20D/\';\nmy $CanonA75dirstring = \'/home/ajj/Photos/A75/\';\nmy $CanonA70dirstring = \'/home/ajj/Photos/A70/\';\n\nmy $filepath = shift or die "Please specify filename";\n\nmy $exifTool = new Image::ExifTool;\n$exifTool->Options(DateFormat => q{%Y-%m-%d});\nmy $info = $exifTool->ImageInfo($filepath);\n\nmy $datestring = $$info{CreateDate};\nmy $modelstring = $$info{Model};\nmy $dirstring;\n\nif ($modelstring =~ /20D/){\n        if ($$info{Quality} =~ /RAW/){\n                print "RAW Image from 20D\\n";\n                $dirstring = $Canon20Ddirstring.$datestring."/RAW/";\n                &movefile($filepath,$dirstring);\n        } else {\n                print "JPEG Image from 20D\\n";\n                $dirstring = $Canon20Ddirstring.$datestring."/";\n                &movefile($filepath,$dirstring);\n        }\n} elsif ($modelstring =~ /A75/){\n        print "JPEG Image from A75\\n";\n        $dirstring = $CanonA75dirstring.$datestring."/";\n        &movefile($filepath,$dirstring);\n} elsif ($modelstring =~ /A70/){\n        print "JPEG Image from A70\\n";\n        $dirstring = $CanonA70dirstring.$datestring."/";\n        &movefile($filepath,$dirstring);\n} else {\n        print "No idea. I give up\\n";\n}\n\nsub movefile\n{\n        local($dirst);\n        local($fname);\n        local($fpath);\n\n        ($fpath,$dirst) = @_;\n        $fname = basename($fpath);\n\n        if (-d $dirst) {\n                move($fpath,$dirst.$fname) or die "Argh. File move failed: $!";\n                print "Moved ".$fpath." --> ".$dirst.$fname."\\n";\n        } else {\n                mkpath($dirst, 0,0755) or die "Argh. Can\'t make dir: $!";\n                move($fpath,$dirst.$fname) or die "Argh. File move failed: $!";\n                print "Moved ".$fpath." --> ".$dirst.$fname."\\n";\n        }\n}\n\n\n

We recently moved H into her own room and this precipitated my moving all my junk out of it. This meant finding a new home for the PC that had languished up there not getting much use because, well, it is nicer to spend time with L downstairs.\n

\nHowever, with my burgeoning photo collection and the arrival of a Mini-DV camcorder, hooking a plethora of external devices to my laptop was getting increasingly annoying.\n

\nSomething needed to be done. \n

\nMy first thought, given how much I like OS X, was to ditch the PC and get a Mac Mini. They look cheap until you actually spec them decently, at which point they basically cost the same as an IMac. And I don\’t have a spare $1200 floating around to spend on *yet another* computer, when we have several perfectly good ones.\n

\nSo I\’m stuck with the hardware I have (see “Linux Hell” post). The only question then is whether to go with Linux or Windows (as previously discussed, getting OS X to run is possible, but not good). I just couldn\’t bring myself to use XP or run the Vista gauntlet. Given that L will still be using the Powerbook, I don\’t have to worry about linux quirks annoying her.\n

\nSo, linux it is. The gimp works pretty well, Evolution is a pretty good mail/calendar client, Firefox is a great web browser, there are very good video editing tools (Cinelerra is a very serious app). I\’m posting this with BloGTK. \n

\nI did succumb to a couple of upgrades. I got another Hyundai Q17 monitor on ebay for dual headed goodness. Great for photo/video editing! Also I had to get a firewire card to attach external disks and the camcorder to. Best Buy were my friend there. I expect I\’ll need to put more RAM in at some point, and will probably want to move to one of the new quad-core Intel chips when they are cheap enough and I\’m fed up of waiting ages for video to render. \n

\nSo far my feeling is that everything is useable and tolerable, but not as nice as OS X - either in look/feel or function. We\’ll see how well it goes.

‘Arghh.\n\nI recently bought a Mini-DV camcorder (I will post about that at some point). In theory these work well with linux - the firewire (IEEE1394) kernel drivers are good and solid these days and there are some good tools for handling getting the video off (Kino, dvgrab) and working with it (Kino, Cinelerra).\n

\nIt *should* just be a question of plug and play. But no. I get a permissions error on the device (/dev/raw1394). Lots of we instructions tell you that actually you should be using /dev/dv1394/some/other/guff, but reading the linux1394 website you find that the dv1394 driver is deprecated in favour of the raw1394 driver. OK. Probably good technical reasons for that.\n

\nSo a bit of digging and you find that you need to edit the udev config at /etc/udev/rules.d/40-permissions.rules (obviously). Then you find this:\n

\n# IEEE1394 (firewire) devices\n# Please note that raw1394 gives unrestricted, raw access to every single\n# device on the bus and those devices may do anything as root on your system.\n# Yes, I know it also happens to be the only way to rewind your video camera,\n# but it\'s not going to be group "video", okay?\n

\n

\n

\nGreat. So basically, the people who develop the drivers have said “You have to use raw1394″ and the Ubuntu people (actually probably the Debian people) have said “Woah there. Can\’t have you using your firewire ports for anything.”\n

\nGah. \n

\nSo I ignored the warning and set the group of raw1394 to “video”, of which all users are members by default (to permit such wild behaviour as using a firewire port).\n

\nHowever, this is yet another classic example of OSS being totally user unfriendly. I\’m an experience Unix geek so it is like water off a ducks back to me, but I\’d have thought that most people would expect to plug it in and have it work - the way that all the help files (when they exist) say it does.

OK. Hand up which moron thought this was a good idea? Hmm?

If you don’t know, gnome-vfs is a system whereby rather than using the tried and tested underlying unix mechanism for mounting filesystems, Gnome - through Nautilus, its desktop manager (think Finder, or Explorer) has deployed it’s own virtual filesystem for things like network shares.

This has a serious problem.

Unless apps are made gnome-vfs aware (and it seems very few are - including the Gimp, which is a major Gnome app) they can’t see those shares. At all. No chance.

Brilliant. Possibly one of the worst decisions in Linux history. Just as everyone is getting network aware, make people go “Hmm. Where the hell are my files?” Worse still, *some* applications can see them and others can’t - wonderful inconsistency.

The best bit of course is that this means that if you run the Gnome desktop and happen to use a Java app or Qt/KDE based app or a Python app or you get the idea, then they will *never* be able to access those shares as gnome-vfs will only work with Gnome/Gtk apps.

What happened to Free software? Whilst I applaud the technical concept behind gnome-vfs, in a practical sense it is very silly.

Having accepted that I won’t be able to afford a Mac desktop system any time soon, I’ve been playing around with trying to get OS X running on the homebrew PC that was lying around gathering dust in the corner of the spare room. The osx86 project provides lots of oblique references to getting hold of hacked install DVDs and a browse of some bittorrent servers produced such isos. I can confirm that it is possible to install OS X on random PC hardware (in my case a Celeron D 2.4Ghz in an MSI PM8M motherboard, Seagate 160Gb SATA HDD and PNY Nvidia FX 5200 graphics card). It is however, not simple. I couldn’t get graphics to work properly (i.e. with Quartz Extreme and Core Image) and it was all horrible. That and it just felt, well… Wrong.

God (well, OK, his representative here on earth, St Jobs) has decreed that the One True OS will run only on the One True Hardware Platform and thus so it must be.

So. I figured that I should try to use the machine for something useful and so decided that the answer was clearly a dual boot Linux and Windows machine (Win 2K).

Then I hit the first problem of linux - the massive choice of distros. I had previously used Ubuntu and that seemed OK, but the new release of Fedora was out and I thought I’d try that.

I’m using Fedora Core 5 at work on our dual 2 core Xeon server and it works nicely as a server OS. They’ve added a funky looking new window manager in FC6 which I thought might be good to try.

Having waited what seemed like an eternity to download two CD images, I discovered upon installation of the basic distro that they don’t support my wireless card (a Beklin Atheros jobbie). FPS it worked out of the box with Ubuntu. Who are they guys trying to kid - the FC6 install and setup process feels no better than the last version of RedHat I installed about 5 years ago. I can’t be arsed with this “download some third party things and bodge them in” approach any more.

Anyway - looking at DistroWatch I like the look of Foresight so I’m downloading it as I type. Fspot looks good, Beagle sounds neat (I like Spotlight on OS X, so having that will be great). It also includes the madwifi drivers for my network card. Fingers crossed.

We shall see if it is any good…

Well, not quite.

I got a Macintosh Error (-51) whilst trying to save a PDF in Acrobat Pro that I had edited.

Not terribly helpful. Still a bit of a google turned up a list of OS 9 error codes. Once again, not obviously helpful, but I figured that Apple wouldn’t have bothered to change their error codes.

-51 is, apparently, “Reference number invalid”. Being a twisted geeky sort of chap, this made me realise immediately that it was complaining that I’d moved the file while it was open. I moved it back and all was well.

Normally, of course, OS X takes such behind-the-scenes jiggery-pokery in its stride. However, I had opened the file in Preview, dragged the icon from the top of the preview window and dropped it onto Acrobat Pro when I decided to edit it. I suspect this indirection caused the grief.

Still. Nice to know that the OS bothers to translate error messages usefully. Grr. I would say “I’m sure Apple will fix this in Leopard”, but given that their coders are probably engaged on making a phone/PDA/mp3 player that no-one will buy…

I sat down this evening to quickly check my email only to be bombarded by an onslaught of several thousand delivery failure notices.

Now I’m not the sharpest tool in the box at the moment, owing to the recent arrival of a baby daughter. However, even in my sleep deprived state I realised that I hadn’t sent thousands of messages.

The truth was that some *$%!#@} spammer had sent out a lot of mail with the from address spoofed as being @wibblefoo.com. I, being lazy, had a catch-all set on my VPS (virtual private server) so that @wibblefoo.com was all coming to me.

This had the virtue of meaning that I could use a new email address for every website that I sign up to, so I can easily search for mail from a particular source, or work out who is selling my details to spammers. I knew when I started that this day would come, but the tsunami of email was a bit of a shock.

Since ditching my old compsoc account which got about 10,000 spams a month [1] I had gotten used to the quite life of 10’s of spams a week, which Mail.app was merrily dealing with for me quite effectively.

I’ve now set the catch-alls for my domains to route to /dev/null and set up forwarders for the addresses that I do actually use. A grep of my mail directory [2] revealed about 70 odd addresses that I’d received email on this year. I figure that anything older probably isn’t of much interest. Managing this is a bit more work that before - I have to add the forwarder before I use a new address on a website - but is clearly better than getting thousands of spam bounces a day!

I’d like to add SPF to my domain, but it seems that bluehost don’t let me do that kind of monkeying around with my zone file. Shame as SPF looks like quite a good way forward to (at least partially) solve this problem of spoofing.

[1] I got that email address in 1994, in the good old days of the internet. Thanks to my use of it on newsgroups etc before the pro spammers existed it seems that it is present on just about every spam list sold as far as I can see.

[2] Ah, the wonders of unix. Mail.app could sort by To: address, but couldn’t give me a list of the addresses. grep was my friend thus:

grep -Eiho "\b[A-Za-z0-9._%-]+@wibblefoo\.com" * |sort -u 

. For those of you unfamiliar with grep options we have: E extended regexp; i case insensitive (though that didn’t work for some reason, hence the two cases of a-z in the regexp); h suppress filename output; and o show only the matching text. Of course this only worked because the INBOX.mbox folder I was in did not contain any of the spam to random addresses.

I have to log into a VAX and the program I use on it expect that keys F1-F20 will be available.

Terminal.app does not help me here. Nominally the F1-F20 keys run along the top of my Apple wireless keyboard. Unfortunately two things coincide to get in the way of VAXy goodness,

Firstly, F14-F20 are allocated as keys for the likes of screen brightness, sound volume and CD/DVD eject. Thus, whatever modifiers are used the result gets nowhere near Terminal.app.

Secondly, Terminal.app is a little braindead. For some reason F1-F4 are assigned to P, Q, R and S. This I discovered thanks to this handy little snippet of python from NPC :

#!/usr/bin/python
import curses
import curses.ascii

def mainloop(scr):
while 1:
scr.keypad(1)
ch = scr.getch()
try:
scr.erase()
scr.addstr(0,0, “%s, %s, %s” %
(curses.keyname(ch), curses.ascii.unctrl(ch), ch))
except:
pass
scr.refresh()

if __name__ == “__main__”:
curses.wrapper(mainloop)

Also, option-F1 to F16 and shift-F1 to F12 are assigned to a bizarre combination of other F-key values. I’m sure there is a reason, I just can’t see it.

Anyway, I want the keys assigned as they are on our PCs - so that I can use F1-F12 as F9-F20. Being able to send F1-F4 would be nice too. So the solution is to edit the Keyboard section in the Window Settings of Terminal. This is a bit fiddly as you have to use their delete button rather than press delete - as pressing the key sends the value of delete to the window. The result is this:

F1 -> \O33[11~
F2 -> \O33[12~
F3 -> \O33[13~
F4 -> \O33[14~
F15-F20 -> Leave as default

option-F1 -> \O33[20~
option-F2 -> \O33[21~
option-F3 -> \O33[23~
option-F4 -> \O33[24~
option-F5 -> \O33[25~
option-F6 -> \O33[26~
option-F7 -> \O33[28~
option-F8 -> \O33[29~
option-F9 -> \O33[31~
option-F10 -> \O33[32~
option-F11 -> \O33[33~
option-F12 -> \O33[34~

And lo, the Function keys were passed correctly to the terminal (at least as far as curses and the VAX is concerned)