Written by Benjamin Denis
Search engines including Google, but also including search engines on sites such as Amazon, Netflix or YouTube and search on your computer or email software – work to the same principles:
- The content you want to search is indexed first
- A user enters a search query as words
- The search engine returns results from the index that best match the query
- The results are ranked with the intention of putting the most relevant results at the top of the results
If you are a WordPress site editor in almost any country of the World, your main interest in search engines – and for reading this article – is ranking on Google. Always remember, though, that other search engines exist, and that Google may not always be the most popular search engine.
By learning how search engine work, you can optimize your site for better ranking and more traffic!
Unique marketing opportunity
Visibility in a search engine can generate traffic, but it is also a unique marketing opportunity: being visible in the very moment that a person is interested in finding you, your product or your content. If you have an Indian restaurant in Seattle, for example, then being visible in Google when people in Seattle search “Indian restaurant” is an amazingly effective way of earning a new client. With most alternative marketing tools – such as handing out flyers on a street-corner – you are pushing messages out to people who have not asked for this information and there is much less chance of converting them to clients. There is even the possibility you may annoy one of them and lose a potential client!
Google does not search the web
From early search solutions used to find files on a computer, it became obvious that opening and reading each file on the computer, one after the other, was not an efficient way to find matching results. As the storage capacity of computers increased searching directly through all the files on a disk was too slow and it also increased the risk of damaging fragile disk-drives.
Did you know? Because search was not an option, early computer users were encouraged to organize files in directories and sub-directories. Applied to the web, this led to the creation of Internet directories such as Yahoo or DMOZ whose purpose was to organize the World’s websites in a directory structure. In the 1990s these sites were much more popular tools than search engines. DMOZ was even integrated into Google as Google Directory from 2000 to 2011.
Search engine software uses data stored in indexes to help it find results more efficiently. An index is part of a database that stores information in a format that makes it quicker to search. For example, rather than searching in multiple files for a word, an index will organize data by word: Listing all the files that contain a specific word in one index.
Google created and maintains its index by crawling the web. When it discovers a new page, it indexes the content of that page. It then regularly goes back to check to see if that page has changed – and if so, it indexes it again. When you search on Google you are searching Google’s index and not on the web itself. It is therefore possible that Google proposes a link to content that is no longer online, or that it omits content that is online but has not yet being indexed.
Pause for a moment and think of how big Google’s index is and how big a job it has keeping it up to date. Google says it crawls and indexes hundreds of billions of web pages (and as reported by Kinsta, WordPress powers 37% of all the websites on the Internet which is a cool thought). The index is well over 100,000,000 gigabytes in size and requires large industrial buildings to house the hardware that store the data. Crawlers, which are computer programs, are regularly re-visiting and re-indexing millions of web pages every day.
Keyword queries
Searches are performed using queries that are expressed in words. You can type these words into the Google search bar or use voice recognition instead of typing.
Different people have different methods for searching Google. For example, a person in Seattle looking for a restaurant may just type “restaurant” into Google (knowing that it will provide only local, relevant results), some may type “Seattle restaurant”, others “restaurants in Seattle” and so on. Someone, somewhere, may type – or say – something like “please can you suggest a nice indian restaurant for me in downtown Seattle please, thank you very much” and they all get quick, relevant results (which proves that it pays to be polite!).
Google is quick at returning the results. We can therefore deduce that it is not searching through all the pages it found on the web to find pages that match the keywords you typed. It is possible that it has indexes for popular searches (this means maintaining a list of URLs that correspond to exact multi-word queries). This is certainly the case for “Restaurants in Seattle”. When a user types this query, Google already has a list of relevant pages that answer that question.
However the query ““please can you suggest a nice indian restaurant for me in downtown Seattle please, thank you very much” has probably never been searched before and it is unlikely that Google has a pre-prepared a list of relevant pages for this query. This is not a rare occurrence, Google indicates that every day, 15% of searches are for queries that have never been searched before.
In this case Google breaks down the query to the most relevant words and expressions to match it to its different indexes. Over the years, Google has got better at interpreting these more complex queries and currently does it better for other search engines in most languages.
Algorithms to rank results
It is believed that for any query, Google first creates a list of only 1000 pages and then ranks these from 1 to 1000, eliminating some results along the way. There is an algorithm for finding the best 1000 pages and another for ranking these 1000 results from 1 to 1000.
SEO expert Jason Barnard has a theory that Google has many algorithms that calculate different ranks for each page based on different signals (topicality, quality, page speed, etc). It then multiplies them together to calculate a bid. The page with the highest bid is ranked highest. He also points out that because scores are multiplied, a very low score for any factor can significantly reduce a bid and the page’s rank.
This is only educated guess-work though. The details of how Google works exactly are a closely guarded secret and Google remind us that they change and improve the algorithm every day.
The main factors, though, are certainly the use of words in individual pages on your site and the links that those pages receive from other pages on the web (whether they may be links from your own site or links from pages on other sites). Minor factors, but potentially with a big impact for some keywords, may be linked to user experience, page speed, social networks, and online reputation.
After ranking the pages, Google may also decide to filter out some results. This could simply be to reduce the number of results from one site, but it could also be filters against spam or adult content.
Usually less than a second after sending your query, you get results back from Google.
Also bear in mind that search results are also personalized to your geographic location and your search history. These factors can make a big difference on search from one computer to another.
SERPs and snippets
For each query, Google returns results as a SERP (Search Engine Results Page). It is very rare that Google finds no results for a query.
As well as results from the web search, Google may add advertising from Google Ads and results from other Google search engines such as Google News, Google My Business, Google Images, …
The results from web search are typically listed as 10 results per page. Each result has a link to as web page and a short description. This result is called a Snippet. The Snippet may also be enriched with the websites’ logo, reviews, images, prices and other useful information.
The information for the snippet can be provided by the website itself using specific tags such as KEYWORD, META DESCRIPTION or Schema.org structured data. The quality of this information may influence Google in ranking your page, but it is also important in influencing the users who are looking at the results to click on your snippet and generate traffic. You may only rank third but have a very eye-catching snippet that means that most searchers will click on your link rather than the first two results.
In 2013, Dr Peter J. Meyers did a mock-up of 24 different SERP features. Although it is a bit old now, it is still a good introduction to the variety of results possible in Google.
What is SEO?
SEO is an acronym for Search Engine Optimization. This is the work you do to optimize your website for search engines – so that you can get listed (for free) in Google when there is a relevant search for your site.
From better understanding how search engines work, you can see that the main tasks in SEO are:
- Make sure that Google can index your site correctly, can find all your content and links between pages. Providing a sitemap is a good idea here,
- Understand what queries your target audience use on search engines and make sure that you have pages or posts that use the keywords of those searches and answer the question the search engine was asking. Most SEO plugins give you tools to analyze pages for keywords,
- Ensure that tags important for SEO are correctly completed. Most importantly TITLE and META Description tags. You will need a SEO plugin to do this,
- Where relevant, provide Schema.org structured data to describe your site or the content of your pages to obtain rich snippets. Again, using a SEO plugin that allows you to add relevant Schema to your posts and pages to gain rich snippets,
- Develop external links from other web sites,
- Analyze your ranking in search engines and the traffic you receive from them.
SEO is also used in another definition to mean Search Engine Optimizer, i.e. the person who is going to do the work above. If this person is you, then continue reading our articles to learn more about getting better visibility and more traffic for your WordPress site!
Written by Benjamin Denis
