How Search Engines and Directories Work
How Search Engines and Directories Work
By Tracy Barbour
Search tools are constantly evolving, which can make it confusing to understand how they work. Today’s search tools fall into three main categories: crawler-based search engines, human-powered directories and hybrids. What distinguishes them is how they gather their search listings.
Search Engines
Crawler-based search engines, like Google, create their listings automatically. They “crawl” or “spider” the Web, then people search through what they have found. If you change your Web pages, crawler-based search engines eventually find these changes, and that can affect how you are listed. Page titles, body copy and other elements all play a role.
Crawler-based search engines have three major elements. First is the spider, also called the crawler. The spider visits a Web page, reads it, and then follows links to other pages within the site. Everything the spider finds goes into the second part of the search engine, the index. The index, sometimes called the catalog, is like a giant book containing a copy of every Web page that the spider finds. If a Web page changes, then this book is updated with new information.
Search engine software is the third part of a search engine. This is the program that sifts through the millions of pages recorded in the index to find matches to a search and rank them in order of what it believes is most relevant.
Directories
Directories, such as Yahoo! and the Open Directory (DMOZ), depend on people to generate search listings instead of automated spiders. You submit a short description to the directory for your entire site, or human editors will write one for the sites they review. A directory search looks for matches only in the descriptions that are submitted.
Things that are useful for improving a listing with a search engine have nothing to do with enhancing a directory listing. Also, changing your Web pages has no effect on your listing. That is, unless you enhance the content.
Hybrid Engines
In the Web’s early days, it used to be that a search engine either presented crawler-based results or human-powered listings. Today, it’s extremely common for both types of results to be presented. Usually, a hybrid search engine will favor one type of listings over another. For example, MSN Search is more likely to present human-powered listings from LookSmart. However, it does also present crawler-based results - especially for more obscure queries.
Article Source: http://ezinearticles.com/?expert=Tracy_Barbour
http://EzineArticles.com/?How-Search-Engines-and-Directories-Work&id=1575547
