WebSite X5Help Center

 
Adrian B.
Adrian B.
User

Is my robot file correct?  en

Author: Adrian B.
Visited 1142, Followers 1, Shared 0  

This is my robot file content - is it correct?:

User-agent: *

Disallow: *

Disallow: /*

Thank you

Posted on the
4 ANSWERS
Aleksej H.
Aleksej H.
Moderator

Hello.If you want to prohibit indexing the entire site, then write this:

User-agent: *

Disallow: /

Read more
Posted on the from Aleksej H.
Adrian B.
Adrian B.
User
Author

No I want whatever normal should be, nothing blocked on the site

Read more
Posted on the from Adrian B.
JJ. JUAG
JJ. JUAG
User

The recommendation for robots.txt in wx5 is current:

User-agent: *
Disallow: /admin
Disallow: /captcha
Disallow: /imemail

This has the effect that these 3 directories are excluded for all search engines (agents), since the contents of these directories are not suitable for searches

JJ.

Where: 1 Settings - Website Optimization, Seo - Include file robots.txt (in pro version)

Read more
Posted on the from JJ. JUAG
Adrian B.
Adrian B.
User
Author

Thank you I see the robot file on my server is exactly that

Read more
Posted on the from Adrian B.