WebSite X5Help Center

 
Udo E.
Udo E.
User

.htaccess  en

Autor: Udo E.
Visited 3215, Followers 1, Udostępniony 0  

How can .htaccess or robot.txt be integated in my sites with website X5 ev. 11? I want to do this, to block crawlers like semalt.com etc. from visiting my homepages i.e. www.cerrillares.com

Posted on the
3 ODPOWIEDZI
Andre E
Andre E
Moderator
Najlepszy Użytkownik miesiąca EN

for .htacces you can best check the faq section of your webhost, you can then upload this to your website.

For the robot.txt file you can check internet, just google it, you will find plenty site's that gives you examples how to create the file itself with the options you want. You can upload it to your website using the upload file button in the section where x5 normally updates your website to your provider. (step 5 -> upload website -> next, then top button wit green arrow in it)

Goodluck !

Czytaj więcej
Posted on the from Andre E
Anthony A.
Anthony A.
User

Udo,

if you want to block a crowler you need to add

create section for bad bots, and add code like this inside it, remember if there are more than 1, you need to [or] and for the last one do not use the "or", here is the sample

# Block Bad Bots
RewriteCond %{HTTP_USER_AGENT} ^BlackWidow [OR]
RewriteCond %{HTTP_USER_AGENT} ^Bot\ mailto:*** [OR]
RewriteCond %{HTTP_USER_AGENT} ^ChinaClaw [OR]
RewriteCond %{HTTP_USER_AGENT} ^Custo [OR]
RewriteCond %{HTTP_USER_AGENT} ^DISCo [OR]
RewriteCond %{HTTP_USER_AGENT} ^Download\ Demon [OR]
RewriteCond %{HTTP_USER_AGENT} ^eCatch

RewriteRule .* - [F]

for robot.txt

User-agent: *
Disallow: /admin
Disallow: /captcha
Disallow: /menu
Disallow: /imemail
Disallow: /pcss
Disallow: /res
Disallow: /style

Sitemap: https://www.nuluhost.com/sitemap.xml

anything with deisallow means do not look, if you want to be looked it, change it to " Allow"

if you are looking for specific agent, the add the name of it in front of User-agent. "*" means all agent from any search index website who uses robotxt and follows that rules.

if this post is the correct answer, please mark this answer as correct answer.

------------------------------------------------------------

Good Luck
http://www.usx5host.com (6 months free Web Host for x5 users, promo code: usx56m)
http://www.usx5.com (Templates and support for x5)
http://www.wdesigners.us

Czytaj więcej
Posted on the from Anthony A.
Anthony A.
Anthony A.
User

remeber to use your own sitemap address inside your robotxt.

if this post is the correct answer, please mark this answer as correct answer.

------------------------------------------------------------

Good Luck
http://www.usx5host.com (6 months free Web Host for x5 users, promo code: usx56m)
http://www.usx5.com (Templates and support for x5)
http://www.wdesigners.us

Czytaj więcej
Posted on the from Anthony A.