AO3 Investigate how search engines work

Authors Avatar by Ajibola

Unit 2: collaborative working AO3-Investigate how search engines work 

Web crawlers-  This is a type of search engine, it  basically a bot that automates all sites on the web and keeps in them in a neat order according to your search. They by visiting sets on the web and then processing the details on the sites such as key words, hyper-links and such and then copy the websites for later use. I t basically starts with a bunch of names of Url it needs to visit as i said before it processes all the information on the websites particularly keywords, it then uses that information to determine things on what the website is about. for example it processes a page about football matches and player statistic I then type in "where to find sites on football matches" the search then finds relevant sites and used sites that uses keywords such as football matches and ranks them in a list according to relevance. Web crawlers though often have trouble duplicating the same website, this purposes a real problem when there is little change in url.

Example of webpages that use web crawler software include




These search engines use crawlers everyday to keep up with the changes on websites and the information on them. Though most save copies on the webs browsed and then index them for later use. Web crawling can be used for other application but is mostly use for search engines.

directories-  Directories are a search engines that have links and websites organized and categorized by humans rather than bots and software, these are usually more organized and provide better quality sites this because the sites are regularly inspected and tested by human, the listing of websites were put there by people and categorized and put in list relevance by people. This sort of system often provides less results than an crawler and can also be bit slower since its done by humans. Although these are looked at as search engines(they are technically) a better way of putting would be a collection of sites put together by human editors and arranged according the subject the website is specializing in. examples include:


Best of the web


The websites on these sites are usually very high quality and informative and are almost never duplicates. This is because directories such as Yahoo and best of the web inspect their websites and pre screened by humans.

example of using directory would be if you were searching for a good restaurant around your area, this is were directories come in as they often easily categorized making it easy for you to navigate thought the searches specifically set in the area you search for.

Hybrid- hybrid search engines are both crawler based results and directory based results, most search engines are moving on to this system as provides the best of both worlds, basically the features of a hybrid search is just combining the two, such as Google for example is hybrids search engine, google uses the crawler system, but also has some sites that were inspected by humans from the directory. Yahoo is probably the best example for this as it has both Yahoo search and Yahoo directory. If you go Yahoo and search you can get results from a combination of the two sites as yahoo would have section for the directories and then at the bottom would be the crawler based results. This usually makes the websites on the search engine more reliable and makes it easier to navigate through the web having a second option of search through the directories instead of the crawler based search results

Meta-  this uses different types of search engines to generate a overall listing of the best sites from the search engines e.g Google, Bing, Yahoo. Search engines that use the system of meta search is not a search engine on its own but instead borrows the results from other search engines. Some may use major search engines while others might add less popular search engines. Most search engines using this system work on the bases on that the web is too big and large for any single search engine to obtain. Also some meta search engines don't list their results in the traditional way such as listing the results in order of relevance, some just instead just list the result based on the quality of the search engines they are using. Though some just use the the traditional listing method. Examples include dog-pile and browsey these types of  search engines are best used when you aren't having luck with finding what you want or the subject your searching for is obscure

Join now!

Children search engines- These are search engines specifically made for children. These search engines filter out filthy things on the internet such as swear words and sexual content and instead is dominated by children sites displaying facts and information's kids can ask simple questions such as "why how far is the moon from the earth ?and can also go on games and stuff. and all sorts. Examples include Ask for kids, akol and kids click and yahooligones,search engines are extremely useful for kids and all the search engines have different ways of filtering out the results.


This is a preview of the whole essay