Search engines

A search engine is a tool used to find specific documents/media/information on a particular network, more specifically the World Wide Web. We will only focus on web-search engines. Most search engines use a constantly updated index of information/web pages/documents to quickly display results and URLs of the source of that information. The search engines fill up their indexes using a variety of methods:

Crawling – An automated program will read a web-page and follow any hyperlinks leaving the page. Every new page that it finds is indexed and all the links from that page are followed. It is an extremely slow process because it will inevitably read the entire internet. Each proprietary search engine normally has its own crawler although sometimes this data is shared.

Sharing Data with Directories - Traditionally search engines used a service provided by massive directories, which are user submitted links pointing at websites with descriptions about what that website is about. This practice is waning in its popularity.