WebSite X5Help Center

 
Gert D.
Gert D.
User

Robots.txt problem  en

Author: Gert D.
Visited 1891, Followers 2, Shared 0  

Hello Website XS 11 team,

I got a message from Google with the headline: Googlebot cannot get permission tot enter my CSS- en JS-data files on http://www.piggybanks.nl/

I tested robots.txt on Google Search console and all was cleared as OK.

There was also a message that Googlebot could not find the sitemap. That I also don’t understand because (I checked) there is a sitemap!.

Can you help me out? It has all to do with ranking on Google, as you know.

Greetings from The Netherlands,

Gert

Posted on the
1 ANSWERS - 1 USEFUL - 1 CORRECT
Paul M.
Paul M.
Moderator

Hello Gert,

A default WebSite X5 robots.txt contains the following lines:

User-agent: *
Disallow: /admin
Disallow: /captcha
Disallow: /menu
Disallow: /imemail

However, your existing robots.txt file on your server contains three additional lines as follows:

Disallow: /pcss
Disallow: /res
Disallow: /style

If you remove these lines then Googlebot should be able to gain access.

I am not so sure about the sitemap issue as everything looks OK to my eyes as it does to yours...  but check in Google Webmaster Tools that you have added both versions of your site as below:

http://www.piggybanks.nl/

http://piggybanks.nl/

...and that you have specified a preferred version, and that the sitemap is set up correctly on that particular version.

Kind regards,

Paul

Read more
Posted on the from Paul M.