YOU KNOW you're in an advanced civilisation when one human being
can change the world. But what are you in when one computer can change
And Moses went up from the plains of Moab unto the mountain of Nebo, to the top of Pisgah, that is over against Jericho. And the LORD shewed him all the land ... unto the utmost sea ... and the LORD said unto him, This is the land which I sware unto Abraham, unto Isaac, and unto Jacob, saying I will give it unto thy seed.IF AN alien had landed on earth thirty years ago and said, not "Take me to your leader", but "Take me to your knowledge" (and why are all fictional aliens politicians, not academics?), then he would swiftly have been shown towards a copy of the Encyclopaedia Britannica. There was no place you could point to and call Knowledge, but Britannica was the closest mankind had to offer.
If he landed today and made the same demand, though, we would produce for him a computer with an Internet connection, and we'd enter the address http://www.altavista.digital.com. And (I'll say straightaway) he still wouldn't get something we could point to and say "All human knowledge is here" -- but he'd be a lot closer.
And what is Alta Vista? Well, if you've already got here you probably know, but for completeness I'll say that Alta Vista is the most powerful computer that Digital have ever built. It contains ten processors and 6Gb of RAM (at current prices that's £150,000 worth, though that's without reckoning on a quantity discount). It is used to run Digital's database software, and in fact was conceived as a way of publicising this software (and also Digital's ability to build computers this powerful, should anyone else ever ever need one).
Its mission, though, is what's important. It could have been adapted from The Hitch-Hiker's Guide To The Galaxy:
Alta Vista's mission is to create a searchable index of every word on every page on the whole of the World Wide Web. A bunch of less awesome computers take upon themselves the task of fetching once each Web page there is, and updating the database accessed by the main server.
This is an endeavour of utter grandeur. Mankind still has a large body of what might be called "legacy knowledge", knowledge that exists only on paper (for instance, the 1966 alien's Britannica). But already most truly new knowledge appears on the Web as it is created. Alta Vista's effect is wider than that of a new gizmo on a computer somewhere. Its breadth of "reading" means it is becoming nothing less than a instantly searchable concordance to the entire of human knowledge. It's not "a" database. It's the database.
If you work with the Internet as much as I do, you'll need, like me, to step back and pause awhile as the enormity of that prospect sinks in. Never before in human history could such a thing have existed. Like the atomic bomb, it cannot be unmade. The collective consciousness of humanity (or, if you don't feel like being that mystic, the individual's sense of community and importance) must adjust to encompass it.
In the olden days, when a physicist wanted a partial differential equation solved, a student was called in to solve it. At the end of three years of appalling and unremitting slog, the physicist had the answer and the student had a PhD in physics. In modern times a computer can give the answer in a fraction of the time, freeing physics PhD students to do more entertaining things, such as read their friends' Web pages.
The Web, and Alta Vista, promise an overthrow of the current world order in the textual and subjective disciplines matching that brought in the numerical disciplines by the single computer. No longer does it take a PhD's worth of work to discover what Shakespeare thought about astronomy: a search on Alta Vista can provide references to each time he mentions it. (Of course, Alta Vista automates only the data-gathering phase of such a PhD, just as Fortran automates only the results-gathering phase of a PhD in theoretical physics. But historically these, and not the demanding and intellectually interesting phases, have often been the greater parts in time and effort of both species of PhD.)
Alta Vista also provides a "gold standard" of knowledge: a place where you can be sure something is, if it exists at all. This in itself is not new: Great Britain (like most countries) operates "copyright libraries" which is an unfortunate name for a system whereby a certain set of libraries (the British Library, the National Libraries of Scotland and Wales, and the libraries of Cambridge and Oxford universities and of Trinity College, Dublin) receive one copy each of every book which is published.
As I'm a graduate of Cambridge University, I'm in theory allowed to use the University Library. I don't reckon I ever will, though. I went in there once, when I was an undergraduate, seeking a certain piece of information. The UL is a vast building, which reputedly served as the inspiration for the design of Deep Thought in the television Hitch-Hiker's Guide To The Galaxy. It is grim and Orwellian in architecture, and, once you get in, the staff prove to be equally grim and Orwellian. I had a lot of time on my hands, so I eventually found the right section, pulled out in turn each of the five or so books on the topic, and found nothing. Searching the books took ten minutes; gaining admittance and finding them had taken all afternoon.
Fortunately, this most reclusive piece of information was nothing to do with my degree; I was reading Alan Garner's novel Red Shift and wanted to know how to use the Lewis Carroll cipher to decode a message given in the endpapers.
(At least I never felt in fear for my life in the UL, which I certainly did in the lift [I was told it was called a paternoster lift] in Birmingham University library. Is that still there? What is their accident insurance like?)
Searching Alta Vista for "+carroll +cipher", on the other hand, gives the right answer immediately. By the time Alta Vista had been invented, though, I'd eventually noticed some odd things about letter frequencies in the message and derived for myself the still, small fact which the UL hadn't provided: the Lewis Carroll cipher is the same as the Vigenère cipher.
Sadly, though, Alta Vista is not complete, and for reasons more depressing than the unavailability of the three big commercial information sources: Britannica, the OED and Larousse Gastronomique. It seems that the Alta Vista team underestimated the sheer size of the undertaking. Their help pages still optimistically say that the database is continually being rebuilt and is completely refreshed every "few days"; some documents it serves, though, are six months old (one way you can tell is by documents which contain CGI script includes giving the server's local date and time), and www.iota.co.uk has been submitted several times over a month with no sign that Alta Vista has ever visited here.
But this is not to detract from the magnitude of Alta Vista's influence. Whether Alta Vista is halfway there, or only a tenth, it provides compelling reason for completing the task. If Digital are feeling they've proved their point and are disinclined to spend further money on this "advertising campaign", then surely someone else will take up the challenge; perhaps even the British Government -- has there yet been suggested any better project for the Millenium Commission to spend the Lottery money on? (Enhancing the provision of Knowledge with the proceeds from the Lottery, a direct tax on Stupidity: now that's creative government!)
Knowledge has changed. I don't think it's over-egging the pudding to say that the availability of an instantly accessible full-text index of all human textual output is the biggest change to come over knowledge since the invention of the printing press. And maybe it's bigger, because you can mass-produce information all you like if it doesn't end up on the desk of the person who needs it, who might never have known it existed.
In days of long eld, an academic might have made a pilgrimage of knowledge to Rome or another great library of antiquity; now that printing has been invented, the months that would take have been cut to the day or half-day it takes to visit the UL. Two orders of magnitude improvement. But a search on Alta Vista takes seconds: three orders of magnitude further improvement.
We are faced with an age in which knowledge is no longer parochial. It must be like this being a Catholic: the idea that there is an ultimate recourse, a highest court of knowledge, a point where answers become definite. Alta Vista isn't quite an omnipope, of course, as it will sometimes say
No documents match the query.
and I suspect his Holiness the Pope isn't in the habit of turning round to enquiring cardinals and telling them he just doesn't know. But at least when Alta Vista gives in, we will know -- as no enquirer of any previous generation could ever have known for certain -- that what we ask is beyond the ultimate limits of the corpus of human knowledge. The era in world history in which it's credible or useful to say "I don't know" is passing. In the future the only opposite of "I know" will be "Nobody knows".
-- Peter Hartley, 9th April 1996
Afterword: 7th May 1996 Digital have themselves put together a more prosaic, but by the same token more informative, explanation of Alta Vista.
Afterword: 3rd June 1996 The people who built Alta Vista: do they lie awake at night thinking, "We've changed the shape of knowledge, we've changed world civilisation"? I suppose, seeing as I lay awake last night thinking about them thinking that, that they must.
Afterword: 5th August 1996
Date: Sun, 04 Aug 1996 21:05:45 -0700 From: Timothy McGinnis <firstname.lastname@example.org> To: email@example.com Subject: No documents match the query. Yours is the only page that Alta Vista returns to the query (No documents match the query.) I just thought you might find that interesting.