WebSite X5Help Center

 
Graham O.
Graham O.
User

Anti ai Mining code  en

Author: Graham O.
Visited 1104, Followers 1, Shared 0  

Hi guys

I'm on a photosharing website that recently made this announcement:

We’ve blocked the ChatGPT crawler​

Google says that AI systems should be able to mine publishers’ work unless companies opt out. We don’t agree that that is the case, so we have opted out!


I then contacted the admin of another photo sharng site I'm a member at and got this reply:

It's just adding a robots.txt file to the root and telling chatGPT not to index the site.

Both of my Website X5 websites are photo sites so I'd like to block ChatGPT and others from mininig the images too, so can you give me the code and where it should be added to my websites.  I'm assuming it will be in the template somewhere.

Posted on the
17 ANSWERS - 1 USEFUL - 1 CORRECT
Claudio D.
Claudio D.
Moderator
Best User of the month IT

https://platform.openai.com/docs/gptbot

so add this two lines of codes at your robots.txt (having you EVO version you have to do it by hand with a text editor ... ):

User-agent: GPTBot

Disallow: /

Read more
Posted on the from Claudio D.
Graham O.
Graham O.
User
Author

Many thanks Claudio  laughing

Read more
Posted on the from Graham O.
Graham O.
Graham O.
User
Author

Sorry to be dumb but I can't see where the robot.txt file lives.  The only thing I could find is for an old version of X5:

  • Go to Step 1- Website Settings > SEO.
  • Open the Base section and select the Include the robots.txt file option.
  • Use the field underneath to manually edit the instruction to include in the robots.txt file.

So the only place I could see where code could be entered is 1 > Settings and Code and placed it in the upper section.

Is this the correct place fo rthe code to be?

Read more
Posted on the from Graham O.
Claudio D.
Claudio D.
Moderator
Best User of the month IT

yes the place is that , but  no..

as I wrote you before , you have the EVO version which does not have the functionality to edit the robots.txt file, so you have to do it by hand.


The file
robots.txt
is located in the root of your online site, you download it to your pc and edit it with the NOTEPAD


if It wasn't there on your host , you can create it from scratch on your PC , again with the NOTEPAD and then export it to the root of your online site, via the website X5 functions , export site to internet , browse , upload file.

if I am wrong and the feature is present in EVO go ahead ! wink

Read more
Posted on the from Claudio D.
Graham O.
Graham O.
User
Author

I misunderstood your post Claudio.  Thiught you menat I could do it in Evo.

I know next to nothing about the terminology or where this stuff is.

Where is the root and the robot.txt file if ther is one?

Read more
Posted on the from Graham O.
Axel  
Axel  
User

Hey Incomedia, good be nice to include this 2 lines into robot.txt files provides by WSX5  to block ChatGPT crawler by default.wink

Read more
Posted on the from Axel  
Claudio D.
Claudio D.
Moderator
Best User of the month IT
Graham O.
I misunderstood your post Claudio.  Thiught you menat I could do it in Evo.

you can do this but not directly from the program.

Graham O.

I know next to nothing about the terminology or where this stuff is.

Where is the root and the robot.txt file if ther is one?

Website X5 is a program that allows even those who know nothing about coding to create their own Web site, but, in my opinion, this should not make , you soften up, but should be a stimulus to grow by learning new things.
Knowing the meaning of certain terms is , again in my opinion, indispensable if you want to create WebSite.
The root is the root folder, at the highest level of your Web site structure on the host.

this is the root 

robots.txt is positioned here, if It wasn't there on your host , you can create it from scratch on your PC , again with the NOTEPAD and then export it to the root of your online site, via the website X5 functions , export site to internet , browse , upload file.

@Graham, I hope now it is clearer , explained in this way.

Axel  
Hey Incomedia, good be nice to include this 2 lines into robot.txt files provides by WSX5  to block ChatGPT crawler by default.

@Axel, even ? come on it seems to me a bit exaggerated to create a feature for this !

Read more
Posted on the from Claudio D.
Axel  
Axel  
User

@Claudio,

ChatGPT is too popular now, and it's not complicated for Incomedia to modify the robots.txt file by default... into WSX5 package... just a voluntee foot-in-mouth

Why to wait for this very little fix...

The best should be to have a radio box to accept or not this protection, like others robots.txt file options ... And by this way each user can creates his own robots.txt file

Read more
Posted on the from Axel  
Claudio D.
Claudio D.
Moderator
Best User of the month IT

@Axel... 

If Incomedia will do it, welcome...


but, I sincerely hope that these "little things" (solvable in less than 5 minutes by themselves and requiring a minimum of experience that you can't not have if you want to make WebSite), won't distract them from much more important and full-bodied changes/improvements (I've been waiting for a long time for at least fifty of them).

Read more
Posted on the from Claudio D.
Graham O.
Graham O.
User
Author
Claudio D.
Graham O. I misunderstood your post Claudio.  Thiught you menat I could do it in Evo. you can do this but not directly from the program. @Graham, I hope now it is clearer , explained in this way.

This is the key Claudio.  If you had said that it was in the host before.  I was looking in website X5!  As I said. I know next to nothing of website/hosting structures which is why I use Website X5.

Today I have learnt a bit more.  Thank you :)

Read more
Posted on the from Graham O.
Graham O.
Graham O.
User
Author

Just got to work out how to acceass the host now.  I've been into the hosting part of my account with no luck.

Read more
Posted on the from Graham O.
Claudio D.
Claudio D.
Moderator
Best User of the month IT
Graham O.
This is the key Claudio.  If you had said that it was in the host before.  I was looking in website X5! 

Actually I had already told you that the file
robots.txt
is in your host , even though it may not be there
Also how to access it I showed you before with detailed pictures.
And I repeat , if it is there on the host you edit it , if it is not there you create it with NOTEPAD.

Read more
Posted on the from Claudio D.
Graham O.
Graham O.
User
Author

Apologies Claudio.  In my total confusion I overlooked this and now it seems to have clicked into place in my head.

I have now uploaded the file and have learnt from this experience.

Many thanks for your patience

Read more
Posted on the from Graham O.
Claudio D.
Claudio D.
Moderator
Best User of the month IT

if the site is this
http://www.lutondunstablepc.org/
the file
robots.txt
is not present !!!
And you need to delete the code you put in stats and code as from this error

---


PS: also activate SSL certificate and redirect from http to https (you have to entire your Host)

Read more
Posted on the from Claudio D.
Graham O.
Graham O.
User
Author

Yes - I've done that when uploaded the txt file I createsd into the root.  It is showing in the attached screenshot.

Read more
Posted on the from Graham O.
Graham O.
Graham O.
User
Author

you need to delete the code you put in stats and code as from this error

I removed code when I uploaded the txt file soo screenshot.  That error should not ne there.

Read more
Posted on the from Graham O.
Graham O.
Graham O.
User
Author

I just checked with Google robots.txt Tester and got a 404 witha message that my site does not have a robots.txt file!

Read more
Posted on the from Graham O.