Robots.txt Generator
A free robots.txt generator that builds a valid robots.txt from checkboxes: choose which crawlers (Google, Bing, Yahoo, DuckDuckGo, Baidu) to target and which Allow/Disallow paths to include. Optionally add a Sitemap URL. Copy the result and upload it as robots.txt to your site root. Runs entirely in your browser. No server required.
Why Use This Robots.txt Generator?
- Visual: No need to remember syntax; select crawlers and rules with checkboxes.
- Common rules: Predefined paths like /admin, /api, /private, and /sitemap.xml.
- Sitemap: Optional field to add a Sitemap line.
- Copy: One click to copy the full robots.txt content.
Key Features
- User-agent blocks: Include one or more of Googlebot, Bingbot, Yahoo Slurp, DuckDuckBot, Baiduspider.
- Allow/Disallow: Toggle Allow: /, Disallow: /admin, /api, /private, and Allow: /sitemap.xml.
- Sitemap URL: Optional Sitemap: http://www.tiptinker.com/sitemap.xml
- Live output: The text updates as you change options.
How it Works
The tool builds one User-agent block per selected crawler. Each block lists the selected Allow and Disallow lines. If you enter a Sitemap URL, a Sitemap line is appended. The result is plain text in standard robots.txt format, ready to save as robots.txt at your domain root.
Try the Robots.txt Generator
Select the crawlers and rules you want, optionally add a Sitemap URL. The output updates as you change options. Copy the result and paste it into your robots.txt file.