Jonathon Fletcher: The Forgotten Father of the Search Engine 95
PuceBaboon writes "If you were under the impression that Brin and Page invented the search engine while working out of a garage somewhere in Silicon Valley then think again. The first practical web-crawler with a searchable index, JumpStation, was running out of Stirling University, Scotland, twenty years ago this year, long before Google came into existence. In a tale all too typical of the U.K. tech industry through the years, JumpStation's creator, Jonathon Fletcher, was unable to find funding for his brainchild and commercial exploitation of the idea fell to others. Jonathon, who was a panel member at the ACM SIGIR conference in Dublin earlier this year is now quite serene about the missed opportunity, despite his frustration at the time. Meanwhile, Stirling University is quoted as 'now looking at a way to mark' Jonathon's achievement."
Tribute (Score:5, Funny)
How about with a Google Doodle?
It doesn't pay to be the first (Score:5, Interesting)
Re: (Score:1)
Geez - just Alta Vista? Lycos, Yahoo, and a host of others existed at the time as well.
You've got Jobs wrong, he didn't drive execution and find Woz. He and Woz were members of a club and they worked on some things together and Steve saw the value in it and drove it from there. You make it sound like Steve had a plan. He didn't. He fell into a situation where he saw the potential, and he was smart and lucky enough to be able to execute on it and pull in others as needed.
Re: (Score:2)
Well, clearly the definitive history needs to be written -- I don't remember about Lycos, off the top of my head -- but I'm pretty sure altavista, then a project at DEC, was the first automated web-crawler index of the web (It was the idea of a friend of mine, who died in 2006: Paul Flaherty) . Altavista was up and running when Yahoo was still a hierarchical list edited by human beings.
Some history of Altavista: htt [searchenginewatch.com]
Re: (Score:2)
And when Google was new it was not better than the others, or not obviously so. Maybe there was something better behind the scenes or had a better way to monetize the search, but to the average user you couldn't tell that. What seems to have happened is that the other search engines fell behind, they were slower to use, went out of business, whatever. I always felt that Altavista was the better engine. Google ended up being my search engine because it was the only one left when none of the others I reme
Re: (Score:2)
The big win for google, if I recall, was that it was a lot faster.
Re: (Score:2)
Google had two big improvements over AltaVista (which at that point was the market leader by far). Relevance and speed. By that stage SEO was alive and well (although probably not called that). The big trick for fooling search engines of the time was spaff filler at the bottom of a page that contained search terms to make the page seem more relevant. When Google started page rank was mostly immune to this technique so it returned much more relevant results, and that is why it spread so quickly and overtook
Re: (Score:2)
+1
Yes, google beat altavista by using "Page Rank" (i.e. analyzing the graph of the way pages were linked together, to estimate popularity). It was pretty remarkable, altavista was clearly more flexible (like other people here, I remember the "NEAR" keyword fondly), but you had to read through a page or two of links to find what you wanted, and google had a knack for putting it up top in the first few links (hence the "I'm Feeling Lucky" button, for when you were pretty sure you wanted just the top link)
Re: (Score:2)
Luck is also under-appreciated. For most of the big successes out there, there is someone else with the same idea and talent at about the same time who didn't happen to know anyone with the resources to carry the idea through or get it noticed. The subsequent successes are not necessarily due to the nature of the person so much as being able to build on the first success and the resources it brings.
Re: (Score:1)
Re: (Score:2)
"It pays to be the first when critical mass is achieved. Who remembers JCR Licklider?"
Except Google didn't do that, either. "Critical mass" was achieved with Yahoo! Search, Dogpile, and others.
Search engines were alive and well, and getting millions of hits a day, long before Google came along. The only thing different about Google was that they had a better way of telling which sites were more popular. So their results were better.
So it wasn't a matter of "being first when critical mass is achieved" at all. It was a matter of "doing it better than the big players" at the time.
Too b
Re: (Score:2)
They also weren't a portal. Remember what the Excite home page came to look like?
Re: (Score:2)
No, it pays to be the best.
Short memories (Score:5, Insightful)
Re:Short memories (Score:5, Informative)
Google wasn't the first by a long shot. By several years in fact.
But, they were one of the first ones to solve the problem of all of those web sites which had polluted every search by adding random words to meta tags or whatever they did.
When I first discovered Google, Yahoo had devolved into pretty much nothing but spam and irrelevant search results. It had become somewhat useless to use most search engines, because they never actually retrieved anything relevant to the search, just stuff which showed up due to those SEO idiots.
Google's page ranking managed to discard a lot of unrelated crap and actually get you something useful, and I never used Yahoo as a search engine again.
Of course, since then, Google's adherence to their own "do no evil" mantra has become a bit of a joke [pressherald.com], and they've become really annoying about trying to force you to use more of their services even when you don't want to.
So much so that if I was ever within a few feet of Sergiy Brin he might get a kick in the nuts just for the fun of it. You know, just to show him what it's like and to show we care.
Re:Short memories (Score:4, Interesting)
I wish I could use the Google I first found. It now ignores all kinds of information and meaningful symbols.
Google code search should be an interface to normal google.
Re:Short memories (Score:4, Informative)
I wish I could use the Google I first found.
You don't, actually. That version of Google was way too susceptible to gaming, er, SEO.
It now ignores all kinds of information and meaningful symbols.
Have you tried verbatim mode? That doesn't help with searches that include less-common symbols, but it does help with a lot of searches. AFAIK, Google always stripped special characters from searches and from its index, though, so I think you may be remembering an engine that didn't actually exist.
Re:Short memories (Score:4, Insightful)
Valid, but I wish quotes would work correctly.
I don't think I have tried that. I just wish it did not strip things inside of quoted strings. If I wanted it to strip them I would not have put it in a quoted string. I might be remembering something that never existed or combining attributes of search engines here.
Re: (Score:3)
and the minus sign. It almost never works for me when I need to remove useless links from the results.
Re: (Score:3)
I just wish it did not strip things inside of quoted strings
Back when it was at google.stanford.edu, the tokenizer simply threw out certain symbols and stop words. So, you could never search on those, which was immediately frustrating on the 'linux search'! If they've changed things, then they're actually indexing all that stuff now and deliberately throwing parts of your search query out just to return worse results. I guess that's possible, but it seems more likely that they're modifying your string fo
Re: (Score:2)
I don't actually know, but I don't think most symbols are indexed. There are a few [google.com], but I don't think the others are present in the index, so searching on them isn't possible. They could be indexed, certainly, but I'm sure plenty of testing has been done, and it works better for more people the way it is.
Re: (Score:2)
You may already understand this, but a good method for map-reduce optimization is token combination (combining synonyms and misspellings and such). I'm not a googler, but have presumed that is behind google being good at similar terms (merging tokens) and poor with quoted strings.
If I'm not wrong, you'd in effect be saying 'damn the specialized system for not allowing an impossible output'.
Re: (Score:2)
Yes, but for many of my searches that behavior is near useless. I really do want to search for ntop, not top or nettop or anything not related to ntop.
Re: (Score:2)
And what was your problem with Altavista?
Re: (Score:3, Informative)
^ What he said. Altavista was pretty amazing for its time, and had some really interesting tools that went in a different direction than Google ended up going. Google went down the path of "our algorithm is magical, you will like our list of results". Altavista had a lot of neat algorithms correlating things, but they gave the user more insight. They had a graphical search-map app, for instance, that let you visually see clusters of related topical areas relevant to your search and drill into them...
Re: (Score:2)
And what was your problem with Altavista?
My problem with it was that it wasn't really a search engine. It did no web crawling and sites were added by hand -- and it was damned hard to get your site on it. I was on all the rest of them, and I argued with them about the poor selection of Quake sites they had, some of the worst, content-free crap out there, while mine was actually excellent (other webmasters listed by Altavisa told me "your site puts mine to shame").
Infoseek was IMO the best one before Google
Re: (Score:2)
Are you confusing Yahoo with AltaVista?
There was a link on the AltaVista front page to add a site that was not in the index, and its robot spidered all the links from that starting URL. Their big boast at the time was they had a bigger index than any other search engine (something like 8 billion pages). That was one of the reasons that Google used to crow about how many pages they could spider in their first few years.
Re:Short memories (Score:4, Insightful)
From what I recall of those days, I don't believe Yahoo wasn't really a search engine at all. Website operators had to submit their sites to Yahoo, who would then manually review the site and then decide whether or not it should be included in their "best of the web" listings.
Lycos, AltaVista, Webcrawler... those were search engines.
Re: (Score:2)
If I remember correctly they had a directory (which was fee based and free) and a separate crawler. For the crawler you just needed to enter a seed URL while for the directory you were required to enter more information.
Re: (Score:1)
Re: (Score:2)
Re: (Score:2)
Though google was better at searches, I think where it was really better was its search input parameter processing.
Altavista back in the day was an engineering exercise. (Remember how long it was before they had Altavista.com). As such, it's full of nice geekisms like boolean operators and parenthetical constructs. Most people couldn't understand that.
Google was always more of a natural language input system. The only "operators" most people use are + and -, and use their algorithms to kind of guess the re
Re: (Score:2)
HTDig. I remember trying to get it work on my site, then my boss went a new direction.
There were various spiders. There was even URouLette, which was a random-link-from-a-web-spiders-db. You couldn't do that now, with all the porn and driveby malware sites.
There were various web frontends to WAIS [wikipedia.org], which never really caught on.
And if you want to expand search engine past web search engines, there was Archie [wikipedia.org] for ftp. Oh, you mean searching a Hypertext-ish system? Well there was Veronica and [wikipedia.org] Jughead [wikipedia.org], for litt
Yes, but... (Score:5, Insightful)
Brin and Page were the ones who made a profitable search engine.
Henry Ford didn't invent the automobile (that award went to Karl Benz a few years earlier), but he was the first to make a fortune building automobiles.
Re: (Score:2)
Re: (Score:2)
Henry Ford didn't invent the automobile (that award went to Karl Benz a few years earlier)
Don't say it out loud, or Cugnot will be steaming with rage in his grave.
Re: (Score:3, Interesting)
The most credit to the success of Benz, that resulted in the big Mercedes-Benz thing that is still very much relevant today basically goes to his wife.
There is somewhat of a startling co-incidence with Apple Products. There were a lot of people who build "cars" before Benz.
The "Patented Benz Motorcar" was basically a failure with no customers, until his wife loaded their two kids on board (without telling him) and went on a 212km (132 miles) round trip. That was basically the "Hey, a motorcar is not just a
Re: (Score:1)
Henry Ford didn't invent the automobile (that award went to Karl Benz a few years earlier)
Don't say it out loud, or Cugnot will be steaming with rage in his grave.
Yes, it should read "the modern automobile". Then the above statement is correct.
Re: (Score:2)
Brin and Page were the ones who made a profitable search engine.
Yahoo, Lycos, even Altavista (in its way) were all profitable search engines before Google.
Brin and Page just created a search engine that was an order of magnitude better than its competitors.
Google won because it was BETTER... (Score:1)
... not because it was first.
Re: (Score:2)
That's true (Score:5, Informative)
Google won because it was BETTER ... not because it was first.
I remember when I first tried google. I had been using AltaVista and I was amazed at how much more relevant the Google results were. Primitive search engines seemed to just bring up any page that had a lot of the words in, Google's page ranking, and looking up related terms (you ask for "secured lending" and also get pages that say "mortgage") made a real difference.
Re:That's true (Score:5, Informative)
I was also amazed at the relevance of the hits, but I still missed AltaVistas "near"-operator. It allowed you to find only results where one term was close to another.
Google does support wildcard searches. You can search for "foo * bar" (the quotation marks are part of the search string) and you'll get pages that have "foo" followed by some stuff followed by "bar". In that order, so it's not exactly the same as "near", but pretty close. You can also use OR, so:
"foo * bar" OR "bar * foo"
is pretty close to "foo near bar".
https://support.google.com/websearch/answer/136861?hl=en&ref_topic=3081620
Re: (Score:2)
and looking up related terms (you ask for "secured lending" and also get pages that say "mortgage") made a real difference.
Google does that now, but it didn't originally. It seemed like it did, though, because if there were a lot of pages with "secured lending" on them which all referred to one page about mortgages, giving it a high pagerank score, then that mortgage page was likely to be in the results.
Re: (Score:2)
I remember when I first tried google. I had been using AltaVista and I was amazed at how much more relevant the Google results were. Primitive search engines seemed to just bring up any page that had a lot of the words in, Google's page ranking, and looking up related terms (you ask for "secured lending" and also get pages that say "mortgage") made a real difference.
That's one of my frustrations with google. If I wanted to search for mortgage, I'd have searched for "mortgage." Just give me the words that I specify and I'd be happy.
Re: (Score:2)
This exactly, 100%. The youngsters don't believe me when I tell them about that revelatory experience of using Google for the first time and getting WAY more relevant results than all those clunky late-90s search engines. The only reason the complete switchover to Google didn't happen faster is we just couldn't believe at first how good it was by comparison.
And you know what? It's still really good. Not perfect, they'll probably get their lunch eaten by some clever startup one of these days. But right
Re:Google won because it was BETTER... (Score:4, Interesting)
AC brings up a legitimate point: All search engines are involved in a race between themselves and those trying to spam the results.
In the very early days of the WWW, there were a smattering of sites, with actual content, so the basic word-counting approach was fine. Then the spammers showed up, saw the potential of spamming search engines, saw that they were doing word-counting, and just filled their pages with search terms repeated about 300 times, and poof, those search engines were useless.
Then Google came in, and for the first time focused not on what the contents of the page were but instead on what the links to that page said. This was vulnerable too, to Google bombing, but it was far less vulnerable to SEO spammers, so it was a big improvement. As Google grew, it put a lot of resources into trying to prevent SEO spamming. It's not wholly successful, but the fact is that it's better at it than anyone else.
Thats about right (Score:3, Funny)
The Guy from the U.S. funds, markets and makes the money.
Re: (Score:1)
Yes. Because there are no businessmen in England, Scotland or Ireland. Those are the magical lands of innovation. It's the rotten Americans who are making business happen at their expense. Throw in some orcs and ents and we're well on our way to another Lord of the Rings.
Re: (Score:2)
Re: (Score:2)
What about Konrad Zuse?
http://en.wikipedia.org/wiki/Konrad_Zuse [wikipedia.org]
Re: (Score:2)
and thank you for posting that. They never talk about him on the BBC...
Re: (Score:2)
Zuse's accomplishments, especially in the privations of war, were remarkable.
He could theorise, build and deliver in a way that very few are capable.
I don't think he has a present day equivalent of whom I'm aware. Perhaps only Danny Hillis comes close but Thinking Machines didn't succeed.
Re: (Score:2)
The Englishman, the Irishman and the Scotsman all agree.
That how expensive it is getting!
archie (Score:4, Interesting)
While not for HTTP resources, I believe the first search engine (for FTP) was `archie` at McGill.
Re: archie (Score:3)
Veronica?
20 years ago there was barely any web to index (Score:2)
So unless his engine indexed a load of other services such as gopher and ftp servers too then it really was too far ahead of its time so you can see why it didn't garner much interest. Also I wouldn't be surprised if people didn't just think it was archie on steroids so why bother?
doesnt really count... (Score:2)
...but a software company I started, right out of collage, developed an online search engine (for BBSs) called MagiSearch in 1989 or so. It could crawl up to 32000 (i think i remember that number was related to how big the stack memory was in relation to our data structure in pre 32bit days) text files and pre-index every tokenized word and phrase into an btrieve "database" (pre-RDBMS and SQL) for lightning fast online searches and retrievals...the tech was pretty primitive but the damn thing worked really
Re: (Score:2)
Dumb question anyway because what you really want to know is who created the first crawling search engine. That's what we're really talking about here, web crawlers feeding search engines.
Re: (Score:2)
well i guess but it a web page is nothing but a text file (with markup of course) connected with some pipes...there is no doubt that the moment we connected to the WWW through a gateway (this tech was just being developed at the time...Mosaic hadn't been created yet) we would have started to figure out how to start crawling the webpages it was such an obvious need...
Re: (Score:3)
Yea, I created the 'Classroom Door' for BBS's back in the 80's. It was a display of 12 or 16 virtual desks with space for 10 or so lines. So you could post a short, 2 line message on your desk or put messages on friends' desks. The older messages would scroll off the virtual desk as new ones were posted.
Precursor to Twitter maybe? Facebook?
[John]
Re: (Score:3)
Not to detract from your accomplishment, but to clarify, 1989 was not pre-SQL; SQL dates back to the early 70s and became an ANSI standard in 1986. Commercial RDBMS's based on SQL were available as early as 1979. There might not have been any open source SQL databases back then, though.
Page Rank? (Score:2)
Was his first search engine using page-rank or something like that to bring relevant searches, or was it just a web crawler + grep?
Re: (Score:2)
"Page Rank" (get the capitals right) is Google's secret sauce, named after Larry Page, not web pages.
Others, too... (Score:2)
The first search engine I used was the World Wide Web Worm (probably in 1994, I think). Before that, I used to use Archie quite a lot, which was a search engine for FTP sites (which you accessed via telnet).
The World Wide Web Worm found me a quite a few research papers which I needed to read to prepare the dissertation we had to do in the final year of our degree course. It saved many many hours of shuffling through paper in the library.
Ain't that the history of "inventions" ? (Score:1)
by what definition? (Score:2)
There were many attempts to index the web and create search engines around that time. I find it a stretch to call JumpStation "the first practical" search engine.
In any case, the article is right that trying to get a high-tech company off the ground in Europe is an exercise in frustration. The UK is probably still better than the rest of Europe.
Who was first in page ranking? (Score:2)
So the real question is:
Who wrote the first page ranked search engine and if they weren't Google then why didn't they end up dominating?
And the first web-crawler was... (Score:3)
Didn't mkgray code the Wanderer like four months before JumpStation? That's eons in Big Bang time.
When you go back in time past net.Genesis, hours can seem like days.
Time for your afternoon nap, chillen.. (Score:1)
Search before the Web (Score:1)