Panic -- 42 is NOT the answer!

Once upon a time, information technology tools were simple: you had a stone tablet, a hammer and a chisel. And you could reasonably expect your information to stay safe in storage and be easily retrievable/understandable for hundreds if not thousands of years! But things aren't so simple these days.

I've been struggling recently with how to manage and retrieve information from my work system. It's an AMD Athlon 64-bit notebook with 1 GB of memory and a built-in 80GB hard drive plus three additional external USB-connected 80GB drives. (Total cost for this was around Australian $4000. Contrast this with, say, the initial model of the IBM System/3 introduced in late 1969, which with max hard disk of 10 MB and max memory of 16 KB would in today's dollar terms would have cost maybe 100 times as much for a tiny fraction of the processing power.)

Stored on the various drives are backups and software distributions and tens of GB of documentation in various forms such as PDF and HTML files. I'm in the middle of testing various Windows-based desktop search products to see how they each handle all that information: how quickly they index it, and how powerful/accurate are their search capabilities.

Mixed results so far. I eliminated ISYS:desktop early on, because IMHO -- while it offers the most comprehensive search capabilities of all -- I found it rather complex than all the others to install, configure, and use for searching. But the real clincher for me was that ISYS:desktop is quite expensive (hundereds of dollars).

I trialled X1 Desktop Search but uninstalled it when the two-week trial period expired -- not because it doesn't work well, but because it isn't free (although not exorbitantly priced).

The other products that I'm evaluating are free, and they are: Microsoft's Windows Desktop Search (which recently became entangled with MSN Toolbar), blinkx and Copernic Desktop Search. Each has its advantages and disadvantages in terms of usability, search features, performance, indexing space consumed, reliability, customization and so on. I'll probably publish a comparison when I finish my tests (which won't be for some weeks or months, mainly because the indexing rate is laboriously slow).

[UPDATE - 25 August 2005] ... I see that Google Desktop Search Version 2 (Beta) has just been released, so I'll probably give that a try too.

My gut feel at this early stage is that I'll never really be able to manage the information that I already have and what I'll be adding to it as time goes on. Using one of the above products, or something else, it will always be a struggle to retrieve the information I need at the time I need it -- although, following Murphy's Law, I often come across the information after I need it! Then there's the question of backup and recovery for all those gigabytes of data, which is such a huge topic that I will avoid it here.

And all of this relates to my tiny personal computing world. Expand the above to the information requirements of a large enterprise, multiple enterprises, the worldwide community, and the implications are mind-blowing. All this is illuminated by the article Pack-rat Approach to Data Storage is Drowning IT which makes you wonder where it will all end.

- - - - - - - - - - - - - - - - - - - - - - -
Apologies to Douglas Adams, who indicated that the answer is 42 but I'm not convinced that it is! (Also see H2G2.)