How to detect search engine bots with PHP ?

Search engine bots (sometimes called spiders or crawlers) are computer programs(bots) that crawl the web pages. In other words, they visit webpages, find links to further pages, and visit them. Often they map content that they find to use later for search purposes (indexing). They also help developers diagnose issues with their websites.
As we know, the ever-growing usage of JavaScript on the web is certainly beneficial to users, but rendering JS is a challenge for search engines. If your website is not properly handled by bots, or your content changes frequently, you should render your pages dynamically, and serve rendered HTML to crawlers instead of JavaScript code. Thus, in order to do so, you have to know if a request was made by a real user, or by a crawler(search engine bot).
PHP doesn’t have any built-in function to detect search engine bots. However, the following function can be used for this purpose.

Example:

filter_none

edit
close

play_arrow

link
brightness_4
code

<?php
  
function is_bot($system) {
      
    // Bots list 
    $bot_list = array(
        'Googlebot', 'Baiduspider', 'ia_archiver',
        'R6_FeedFetcher', 'NetcraftSurveyAgent',
        'Sogou web spider', 'bingbot', 'Yahoo! Slurp',
        'facebookexternalhit', 'PrintfulBot', 'msnbot',
        'Twitterbot', 'UnwindFetchor', 'urlresolver'
    );
      
    // If it is search engine bot
    // returns true, else returns false
    foreach($bot_list as $bl) {
        if( stripos( $system, $bl ) !== false )
            return true;
    }
      
    return false;
}
  
echo is_bot('Googlebot');
  
?>

chevron_right


Output:



1

This function compares the PHP user agent with a list of common spiders from search engines, more than 180 bots, spiders and crawlers.

When ‘Googlebot’ is given as input, the function returns true(1) as the provided input is the name of a search engine bot.

My Personal Notes arrow_drop_up

Check out this Author's contributed articles.

If you like GeeksforGeeks and would like to contribute, you can also write an article using contribute.geeksforgeeks.org or mail your article to contribute@geeksforgeeks.org. See your article appearing on the GeeksforGeeks main page and help other Geeks.

Please Improve this article if you find anything incorrect by clicking on the "Improve Article" button below.