.htaccess
Author: Udo E.
Visited 3159,
Followers 1,
Shared 0
How can .htaccess or robot.txt be integated in my sites with website X5 ev. 11? I want to do this, to block crawlers like semalt.com etc. from visiting my homepages i.e. www.cerrillares.com
Posted on the
for .htacces you can best check the faq section of your webhost, you can then upload this to your website.
For the robot.txt file you can check internet, just google it, you will find plenty site's that gives you examples how to create the file itself with the options you want. You can upload it to your website using the upload file button in the section where x5 normally updates your website to your provider. (step 5 -> upload website -> next, then top button wit green arrow in it)
Goodluck !
Udo,
if you want to block a crowler you need to add
create section for bad bots, and add code like this inside it, remember if there are more than 1, you need to [or] and for the last one do not use the "or", here is the sample
# Block Bad Bots
RewriteCond %{HTTP_USER_AGENT} ^BlackWidow [OR]
RewriteCond %{HTTP_USER_AGENT} ^Bot\ mailto:*** [OR]
RewriteCond %{HTTP_USER_AGENT} ^ChinaClaw [OR]
RewriteCond %{HTTP_USER_AGENT} ^Custo [OR]
RewriteCond %{HTTP_USER_AGENT} ^DISCo [OR]
RewriteCond %{HTTP_USER_AGENT} ^Download\ Demon [OR]
RewriteCond %{HTTP_USER_AGENT} ^eCatch
RewriteRule .* - [F]
for robot.txt
User-agent: *
Disallow: /admin
Disallow: /captcha
Disallow: /menu
Disallow: /imemail
Disallow: /pcss
Disallow: /res
Disallow: /style
Sitemap: https://www.nuluhost.com/sitemap.xml
anything with deisallow means do not look, if you want to be looked it, change it to " Allow"
if you are looking for specific agent, the add the name of it in front of User-agent. "*" means all agent from any search index website who uses robotxt and follows that rules.
if this post is the correct answer, please mark this answer as correct answer.
------------------------------------------------------------
Good Luck
http://www.usx5host.com (6 months free Web Host for x5 users, promo code: usx56m)
http://www.usx5.com (Templates and support for x5)
http://www.wdesigners.us
remeber to use your own sitemap address inside your robotxt.
if this post is the correct answer, please mark this answer as correct answer.
------------------------------------------------------------
Good Luck
http://www.usx5host.com (6 months free Web Host for x5 users, promo code: usx56m)
http://www.usx5.com (Templates and support for x5)
http://www.wdesigners.us