Blog

How To Make A Robots.txt File

How To Make A Robots.txt File

What is a Robots.txt File?

It’s a small text file that tells Google what it should or should not put in its search results.

Why You Need a Robots.txt File

The search engines like it when you have a robots.txt file. It tells the search engines what you want to be indexed (listed) in the search results.

This is How You Make a Robots.txt File

1. Open notepad
2. Copy and paste the bolded text below into the empty notepad file

User-agent: *Allow: /
Disallow: thankyou.html

3. Save it as robots.txt
4. Upload it to the root directory of your hosting account
5. You are done!

For everything you don’t want someone to or the search engines looking up (i.e. = not relevant to your website), type new Disallow line for each thing, like this…

User-agent: *Allow: /
Disallow: thankyou.html
Disallow: /myporncollection
Disallow: /this-old-web-page-i-dont-want-anyone-seeing.html
Disallow: /thispornvideo.mpeg

That is how you make a robots.txt file

If you are using WordPress, I would suggest downloading this XML Generator WordPress Plugin.

Pinterest
LinkedIn