Where do I put the robots.txt file?

At the root of your domain — e.g. https://example.com/robots.txt. It won't be recognized anywhere else.

Does robots.txt stop pages from appearing in search results?

Not reliably — it only asks crawlers not to fetch the page. A disallowed page can still be indexed by URL if other pages link to it. Use a noindex meta tag for that instead.

SEO · Free · No account required

Robots.txt Generator

Build a robots.txt file for your site — control which crawlers can access which paths.

Rule 1

User-agent

Disallow (one path per line)

Allow (one path per line, optional)

Sitemap URL (optional)

Output

User-agent: *

How to use it

01Set the user-agent for each rule (use * for all crawlers).
02List paths to disallow or allow, one per line.
03Add your sitemap URL if you have one.
04Copy or download the result as robots.txt.

Is my file uploaded?

This generator doesn't need a file — it's built entirely from what you type, locally in your browser.

FAQ

Where do I put the robots.txt file?: At the root of your domain — e.g. https://example.com/robots.txt. It won't be recognized anywhere else.
Does robots.txt stop pages from appearing in search results?: Not reliably — it only asks crawlers not to fetch the page. A disallowed page can still be indexed by URL if other pages link to it. Use a noindex meta tag for that instead.

More SEO Tools

Meta Tag Generator Sitemap Generator Meta Description Length Checker

Built by Ni Wayan Astari — Performance Marketer with a strong interest in data and technology. See more free tools or selected work.