Free Tool

Robots.txt Generator

Build a correct robots.txt file with simple checkboxes. Control crawlers, block AI bots, add your sitemap, and download instantly. Free, no signup.

Configure Your robots.txt

Your robots.txt

Download

Save this as robots.txt and upload it to the root folder of your website (e.g. orexagency.com/robots.txt).

Why It Matters

Control How Search Engines Crawl Your Site

A robots.txt file tells search engine crawlers which parts of your site they can and can't access. It's a small file that lives at the root of your domain, and getting it right helps search engines spend their time on the pages that actually matter.

You can also point crawlers to your sitemap, slow down aggressive bots with a crawl delay, and — increasingly relevant — block AI training crawlers if you'd rather your content wasn't scraped for AI models.

FAQ

Questions About This Tool

It's a plain text file at the root of your website that gives instructions to search engine crawlers about which pages or folders they should or shouldn't crawl.

In the root directory of your domain, so it's accessible at yourdomain.com/robots.txt. Search engines look for it there automatically.

Not necessarily. robots.txt stops crawling, but a page can still be indexed if linked elsewhere. To reliably remove a page, use a noindex meta tag instead.

That's your choice. Blocking them stops your content being used to train AI models, but has no effect on your normal search rankings. Many sites now block them; many don't.
More Free Tools

Try These Next

Website Speed Test
SEO Checker
Meta Tag Generator

Ready to Grow Your Business?

Let's build your online presence and turn your visitors into customers.

Get Started WhatsApp Us