The leading search engines, Google, Yahoo! and Microsoft, use crawlers to find pages for their algorithmic search results. Pages that are linked from other search engine indexed pages do not need to be submitted because they are found automatically. Some search engines, notably Yahoo!, operate a paid submission service that guarantee crawling for either a set fee or cost per click.[22] Such programs usually guarantee inclusion in the database, but do not guarantee specific ranking within the search results.[23] Yahoo's paid inclusion program has drawn criticism from advertisers and competitors.[24] Two major directories, the Yahoo Directory and the Open Directory Project both require manual submission and human editorial review.[25] Google offers Google Webmaster Tools, for which an XML Sitemap feed can be created and submitted for free to ensure that all pages are found, especially pages that aren't discoverable by automatically following links.[26]
Search engine crawlers may look at a number of different factors when crawling a site. Not every page is indexed by the search engines. Distance of pages from the root directory of a site may also be a factor in whether or not pages get crawled.[27]
No comments:
Post a Comment
Leave Post