What are Search Engines and How do they Work?
A search engine is a software program that provides information according to the user query. It finds various websites or web pages that are available on the internet and gives related results according to the search. For example, a student wants to learn C++ language so he searches the “C++ tutorial GeeksforGeeks” in the search engine. So the student gets a list of links that contain the tutorial links of GeeksforGeeks. Or we can say that a search engine is an internet-based software program whose main task is to collect a large amount of data or information about what is on the internet, then categorize the data or information and then help user to find the required information from the categorized information. Google, Yahoo, Bing are the most popular Search Engines.
How do Search Engines Work?
Search engines are generally working on three parts that are crawling, indexing, and ranking
1. Crawling: Search engines have a number of computers programs that are responsible for finding information that is publicly available on the internet. These programs scan the web and create a list of all available websites. Then they visit each website and by reading HTML code they try to understand the structure of the page, the type of the content, the meaning of the content, and when it was created or updated. Why crawling is important? Because your first concern when optimizing your website for search engines is to make sure that they can access it correctly. If they cannot find your content you won’t get any ranking or search engine traffic.
2. Indexing: Information identified by the crawler needs to be organized, Sorted, and Stored so that it can be processed later by the ranking algorithm. Search engines don’t store all the information in your index, but they keep things like the Title and description of the page, The type of content, Associated keywords Number of incoming and outgoing links, and a lot of other parameters that are needed by the ranking algorithm. Why indexing is important? Because if your website is not in their index it will not appear for any searches this also means that if you have any pages indexed you have more chances of appearing in the search results for a related query.
3. Ranking: Ranking is the position by which your website is listed in any Search Engine. (There are three steps in which ranking works).
- Step 1: Analyze user query – This step is to understand what kind of information the user is looking for. To do that analyze the user’s query by breaking it down into a number of meaningful keywords. A keyword is a word that has a specific meaning and purpose, for example when you type how to make a chocolate cupcake search engines know that you are looking for specific information so the results will contain recipes and step-by-step instructions. They can also understand the meaning of how to change a light bulb is the same as how to replace a light bulb search engines are clever enough to interpret spelling mistakes also.
- Step 2: Finding matching pages – This step is to look into their index and find the best matching pages, for example, if you search dark wallpaper then it gives you the result of images, not text.
- Step 3: Present the results to the users – A typical search results page includes ten organic results in most cases it is enriched with other elements like paid Ads, direct answers for specific queries, etc.
Performance of Search Engine
The performance of search engine is determined by 2 requirements. They are:
- Effectiveness (quality of result).
- Efficiency (Response time & through put).
Components of Search Engine
There three components in search engine. They are web crawler, data base, and search interface:
- Web crawler: A search engine uses multiple web crawlers to crawl through world wide web and gather information. It is basically a software which is also known bat or spider.
- Data base: The information which is gathered by web crawler by crawling through internet is stored on the database.
- Search Interface: Search interface is just an interface to the data base which is employed by the user to search through the data base.
Basic building blocks of search engine:
There are basically two building blocks which perform various activities.
- Indexing: Indexing Indexing performs mainly 3 activities text acquisition, text transformation index creation.
i)Text acquisition: Text acquisition basically identifies and stores documents into data base for indexing.
- It convert variety of documents into a consistent data Format.
- It also stores text meta data and other related information of document.
ii) Text transformation: It transforms document into indexed terms.
- Parser: It recognizes the “words’ in the text with the help of tokenizer and process the sequence of text tokens to recognize structural pattern.
- Stopping: Removes stop words like “and”, “or”, “the”.
- Stemming: It groups together all the words derived from same stem.
- Link analysis: It is used identify the popularity page. It uses links & of & anchor text from web pages.
- Information extraction: Information extraction identifies classes of index terms which are important for some application.
- Classifier: Identifies class related data of document.
iii) Index creation:
- Document statistics: It collects the features like position & count of words.
- Weighing: Calculates weights of index terms.
- Inversion: As the format of inverted files is fast for query processing it converts document term information to term document information
2. Querying: It consists following three tasks
- User interaction: User interaction provides a query input which gives an interface and parser for query language. Then it transforms the query by improving query. Then it shows the output by Constructing the display of ranked documents for a query.
- Ranking: It first calculates the score of document by using ranking algorithms. It processes query in distributed environment.
- Score: qi*di , Where qj & di are term weights for term i query and document
- Evaluation: It this step it logs user queries & interaction for improving search engines efficiency & effectiveness.
Usage of Search Engine
Search engines have so many usages and some of them are:
- Searching for information: People use a search engine to search for any kind of information present on the internet. For example, Rohit wants to buy a mobile phone but he does not know which one is the best mobile phone. So he searches “best mobile phones in 2021” in the search engine and gets the list of best mobile phones along with their features, reviews, and prices.
- Searching images and videos: Search engines are also used to search images and videos. There are so many videos and images available on the internet in different categories like plants, animals, flowers, etc., you can search them according to your need.
- Searching location: Search engines are also used to find locations. For example, Seema is on a Goa trip but she doesn’t know the location of Palolem beach. So she searches “Palolem beach” on the search engine and then the search engine gives the best route to reach Palolem beach.
- Searching people: Search engines are also used to find people on the internet around the world.
- Shopping: Search engines are also used for shopping. Search engines optimize the pages to meet the needs of the user and give the lists of all the websites that contain the specified product according to the best price, reviews, free shipping, etc.
- Entertainment: Search engines are also used for entertainment purposes. It is used to search videos, movies, games, movie trailers, reviews of movies, social networking sites, etc. For example, Rohan wants to watch a movie named “Ram”, then he searches this movie on a search engine and the search engine returns a list of links (of the websites) that contain the Ram movie.
- Education: Search engines are also used for education. With the help of search engines, people can learn anything they wanted to learn like cooking, programming languages, home decorations, etc. It is like an open school where you can learn anything for free.
How do We Use a Search Engine?
Search engines are easy to use. There are billions of searches are performed using search engines each day. It’s estimated that more than 5.6 billion searches are made per day. For example, searching on Google, so to this simply open your web browser. Then type “www.google.com” in the search bar of your web browser and press “Enter”. Then the google search engine will open and now we are ready to search any information on the google search engine. Always remember the result returned by the search engine may not all be relevant to search because it will return search results that have the search words, they are not necessarily in the same order you typed them in.
For example, we need to open a website named GeeksforGeeks to learn Python language. So we write “geeksforgeeks Python” in the Google search bar and press enter and you will see a list of links that will redirect you to the tutorial of Python on the GeeksforGeeks portal.