Block Search Engines from Indexing Your Website

Share this article :

Open the robots.txt file and add / change as per followings rules.

Block your entire website

User-agent: *
Disallow: /

Block directory (with everything inside)

User-agent: *
Disallow: /itsiti/images/

Block a page

User-agent: *
Disallow: /itsiti_index.html

Block a specific file

User-agent: *
Disallow: /*.jpeg$

If you want to enable the blocking for certain user agents, you can put the user agent name instead of * (* is applies to all user agents). You can find the list of user agents from this website,

Related posts

Add to Bookmark Site Javascript
HTML Text Tags Attributes
Fantastico: Cannot install more than one script in domain root directory
Enable Gzip Compression via .htaccess
© 2018 ITsiti. All Rights Reserved
Powered by KEEM