Design a site like this with
Get started

Search Engines

A search engine, also known as a web search engine, is a system software that discovers relevant contents on the internet based on given queries. It is one of the most widely used softwares all over the world.


WHOis was the first domain search engine with limited capabilities invented in 1982. Alan Emtage, together with Bill Heelan and Peter Deutsch developed a content search engine named Archie that could make a directory list of files on FTP sites and produced a searchable database. The first full-fledged web search engine was W3Catalog developed at the University of Geneva in 1993.

A search engine has three steps to implement to produce a result. These three steps are detailed below:

  1. Crawling: AKA spidering, is a process in which an automated script or computer program is used to crawl through the web for specific
    content. This helps in producing an index of relevant contents. A crawler is also sometimes called a Spiderbot. It can detect HTML errors and broken links.
    Robots.txt file contains data on how to crawl or if not to crawl the website.
  2. Indexing: After a web crawler crawls through the web, an index of related sites is produced and displayed in paging. Keywords and metadata
    are specifically searched to index webpages. Formats that are supported by search engines to index are:
    ASCII text file
    MS Word, Excel & PowerPoint
    Lotus Notes
    ZIP etc.
  3. Ranking: Finally, relevant sitse are given ranks based on factors such as quality, uniqueness, mobile optimized , accessability, backlinks, loading speed etc. Google Search Console
    helps to find out if a particular website is properly accessible.
Search Engines (Tishan’s Wall)

Next, webpages are displayed as what is called SERP (Search Engine Result Pages). This is published in paging depending on the browser’s configuration.

Custom Search Engines

A custom search engine helps to view specific results as required by the users. Currently, these are the two popularly active custom search engines:

  1. Google Custom Search Engine: Developed at Google in 2006, it provides a beautiful way of customising searches, refine and view as expected by a user.
    Official Website:
  2. Bing Custom Search: Developed by Microsoft Corporation, it has a powerful API to refine search results and show the user. It is totally free to use at any scale with no ads.
    Official Website:

Popular Search Engines

As of today, there are many vendors who provide search engine services. Google’s search engine has the largest market share worldwide on all the platforms. However, Yandex leads the market in Russia whereas Baidu tops in China.

Credit: Webfx (Feb, 2020)

Writing a Search Engine

Since a search engine is a multi-level complex software to make, its internals can be divided into three layers:

  1. Frontend: JavaScript
  2. Middle Layer: General-purpose languages (ex. Java, Python, PHP)
  3. Backend: C++, C, Rust, GO


Search engines are a crucial part of today’s technology. The more relevant and neutral contents users get, the more they benefit.


Published by Farial Mahmod Tishan

Life-long learner. Developing Flutter apps on Parrot Linux OS .

Leave a Reply

Please log in using one of these methods to post your comment: Logo

You are commenting using your account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: