Search Engine Ranking Explained
A search engine's goal is to generate a set of relevant, ranked results for every search query. Generating the search results is a two stage process. Initially, web pages which are relevant to the search term are extracted from the search engine's index. The initial results set is then ranked based on each page's relevance to the search term. The results are presented in ranked 'relevance' order, with the most relevant page at the top of the results and the least relevant (of the relevant sites) at the bottom.
Search Term Subjectivity
The relevance of any given web page to a search term is subjective. The person conducting the search will have a preconceived idea of the type of result he is expecting, but a second searcher using the same search term may be hoping for an entirely different set of results. Consider the search terms SERPS. A high proportion of searchers using the search term will be looking for Search Engine Optimization information relating to the generation of Search Engine Results Pages (SERPs). However, the vast majority of searchers from the UK searching for SERPS will be expecting results relevant to State Earnings Related Pensions (SERPS). In this example, the top ten ranking positions in Google are divided equally between pages relevant to pensions and those relevant to Search Engine Optimization.
Determining Relevance
Search engines gather information about web pages using software programs called robots, also known as spiders or crawlers. The information that a robot collects by crawling a web page is broken down into snippets of information which are stored in a vast array of indexes maintained by the search engine. The information relating to a single web page will be widely dispersed across hundreds, possibly even thousands, of indexes. However, search engine research has determined which web page elements are most likely to contain information relating to the subject or topic of the page. These elements are given most weight when determining the relevance of a page to a particular search term. Google has stated that its algorithm (a mathematical formula used to determine relevance, and ultimately rankings) uses over 100 different factors in its ranking calculations. Many of these ranking factors will be directly derived from the web page, such as the text used in the H1 tag, or from the page's relationship with other pages, an example being the anchor text of inbound links (IBLs). A few ranking factors may be quite dissociated from the website itself, such as whether or not the site is included in the Google Directory. Each of the 100+ ranking factors has an associated weighting in the algorithm, so that for example the text in an H1 tag will have a greater effect on ranking than the same text in an alt text of an image.
Generating Relevant Results
Search engine users require relevant results, but equally importantly they require results quickly. Even with the computing power available to a search engine, it would not be possible to carry out the algorithmic ranking calculations on a large data set in order to generate a complete set of results within a timeframe acceptable to the searcher. If you carry out a search in Google for the term Search Engine Optimization, the SERPs states that there are 12+ million results. The total results figure may be reasonably accurate, since the information required to calculate it resides in the vast array of indexes, but Google has not calculated ranking positions for each of the 12+ million relevant pages. Google actually presents a maximum of 1,000 results for each search term. These 1,000 results are generated by performing algorithmic ranking calculations on a data set comprising 40,000 relevant pages drawn from Google's indexes. How the initial data set is drawn up depends on the number of relevant pages contained across the indexes. If the number of pages required for the initial data set can be drawn from the indexes storing page title and IBL anchor text data (considered the two prime indicators of relevance), then no other indexes are used. If the page title and IBL anchor text indexes fail to return sufficient pages for the initial data set, additional pages are drawn from other, secondary indexes. Having compiled the initial data set of 40,000 pages, these are ranked for relevance to the search term using all 100+ weighted ranking factors in the algorithm. The first 1,000 results are then presented, in ranked order, as the search results for the search term.
Search Engine Optimization
Search Engine Optimization requires the ability to 'reverse engineer' a web page so that it conforms to the search engine relevance model for a particular search term. Effective Search Engine Optimization therefore requires an understanding of the mechanisms and algorithms used by search engines to rank web pages. Since each search engine uses a different algorithm, and all the algorithms are continually being refined and adjusted, Search Engine Optimization is as much an experimental science as it is an art.
