Robots.txt Generator
Generate a robots.txt file with custom rules.
Runs entirely in your browser. Your data is never uploaded.
About Robots.txt Generator
Robots.txt Generator builds a valid robots.txt with allow/disallow rules, user-agents and a sitemap reference to control how crawlers access your site.
How to use
- 1Choose what to allow or block.
- 2Add your sitemap URL.
- 3Copy the robots.txt.
Frequently asked questions
Where does robots.txt go?
At the root of your domain: https://example.com/robots.txt.