I have a robot.txt file i wana ask you to look at. I'm sure mine is too long. Please help me get the best one for my sit
Autor: Tommy Du PreezPlease help me to tweek this robot.txt
I'm sure it's an overkill but wana get the best search results possible out of the internet:)
User-Agent: *
User-agent: Googlebot
Disallow:
User-agent: Googlebot-image
Disallow:
Allow: Googlebot
Disallow:
Allow: Googlebot-image
Disallow:
User-agent: APIs-Google
Disallow:
User-agent: mediapartners-Google
Disallow:
User-agent: AdsBot-Google-Mobile
Disallow:
User-agent: AdsBot-Google-Mobile
Disallow:
User-agent: AdsBot-Google
Disallow:
User-agent: (Various mobile device types) (compatible; Mediapartners-Google/2.1; http://www.google.com/bot.html)
Disallow:
User-agent: AdsBot-Google-Mobile-Apps
Disallow:
Allow: /*?hl=
Allow: /?cd=
Allow: /?hl=
Allow: /adwords/
Allow: /apt/AdPreview
Allow: /apt/anon/AdPreview
Allow: /d/AdPreview/
Allow: /home*
Allow: /keywordplanner
Allow: /ko/KeywordPlanner/Home
Allow: /support/
Allow: /um/home$
Allow: /um/identity
Allow: /um/KeywordPlanner
Allow: /um/signin
Allow: /um/Signup/Home$
Allow: /um/StartNewLogin
Allow: /um/StartNow
Allow: /um/Welcome
Allow: /um/Welcome/
Disallow: /*?
Disallow: /api
Disallow: /um/*
Disallow: /um/StartAccountSetup
Hi, you can solve with NO INDEX,
read here https://support.google.com/webmasters/answer/93710?hl=en let me know