Block Search Engines from Indexing Your Website



Share this article :

Open the robots.txt file and add / change as per followings rules.

Block your entire website

User-agent: *
Disallow: /

Block directory (with everything inside)

User-agent: *
Disallow: /itsiti/images/

Block a page

User-agent: *
Disallow: /itsiti_index.html

Block a specific file

User-agent: *
Disallow: /*.jpeg$

If you want to enable the blocking for certain user agents, you can put the user agent name instead of * (* is applies to all user agents). You can find the list of user agents from this website, Robotstxt.org.

Related posts

HTML Text Tags Attributes
This site may harm your computer
HTTP Status & Error Codes
Find Broken Links (Images, URLs) in Website using Xenu's Link Sleuth
© 2017 ITsiti. All Rights Reserved
Powered by KEEM