You may be wondering - how do search engines work?
Search engines have three primary functions:
1. Crawling - Crawling is the process where search engines discover the most important pages on the web. The search engine uses a program called a 'crawler', 'bot', or 'spider' which follows an algorithmic process to determine which websites to crawl and how often. As a search engine's crawler moves through a website it also detects and records any links it finds on these pages and adds them to a list that will be crawled later.
2. Indexing - Once a search engine crawls and registers a website, it compiles an index of all the words it sees and their location on each page. It is essentially a database of billions of web pages to be used for later retrieval.
3. Ranking - Once a keyword is entered in the search box, search engines will check for pages within their index from most relevant to least relevant. There are a variety of factors including language, location, and device type.
When it comes to search engine market share, Google is the industry leader. So, it makes the most sense to optimize a website for Google's search algorithm.
Google's algorithm searches web pages that contain keywords used in a search, then assigns a rank to each page based on several factors, including how many times the keywords appear on the page.
Higher ranked pages appear further up in Google's search engine results page (SERP), meaning that the best links relating to a search query are theoretically the first ones on Google lists.
Knowing the purpose of crawling, indexing, and ranking helps website owners improve their websites to make it easy for search engines to read and understand, and to better target them to the right search results.