Build a correct X-Robots-Tag HTTP header to control how search engines index non-HTML files like PDFs and images, where a meta robots tag is impossible. Pick your directives, optionally target a specific crawler or file type, and copy the rule as an Apache, Nginx or raw-header snippet. 100% in your browser — nothing is uploaded or stored.
1. Choose directives
Select the indexing rules you want to apply. They combine into a single header value.
2. Target (optional)
Leave the crawler blank to apply to all robots, or name one (e.g. googlebot). Optionally scope the rule to a file type.
robots.txt, or the directive will never be seen. Apache rules need mod_headers enabled. Test on a staging copy and verify with your browser's network inspector.🔒 Everything runs in your browser. Nothing you enter is uploaded, logged or stored.
The X-Robots-Tag Generator builds the HTTP response header that lets you control how search engines index files where a meta robots tag simply will not fit. You can add a <meta name="robots"> tag to an HTML page, but you cannot add one to a PDF, an image, a spreadsheet or any other non-HTML file. The X-Robots-Tag header solves this by carrying the same indexing directives — noindex, nofollow, noarchive and more — at the server level, applied to any file type you choose. This tool assembles the directives correctly and outputs ready-to-paste rules for Apache, Nginx or as a raw header.
It is part of the Robots & Directives group in our free SEO Toolkit. Pair it with the Meta Robots Tag Generator for HTML pages, the Robots.txt Generator and Robots.txt Tester to manage crawling, the Nofollow Link Generator for individual links, and the .htaccess Redirect Generator for server-level redirects. Everything runs in your browser — nothing you enter is uploaded or stored.
Directives that meta tags cannot reach.
noindex, nofollow, noarchive, nosnippet and more.
Apply rules to PDFs, images or any extension.
Apache, Nginx and raw-header output.
Rules are built in your browser — nothing leaves your device.
From a few clicks to a ready-to-paste rule.
Choose noindex, nofollow and others.
All files or specific extensions.
Apache, Nginx or raw header.
Paste into your server config.
An Apache rule that noindexes every PDF on the site.
# Noindex all PDF files <IfModule mod_headers.c> <FilesMatch "\.(pdf)$"> Header set X-Robots-Tag "noindex, noarchive" </FilesMatch> </IfModule>
The X-Robots-Tag is an HTTP response header that tells search engines how to index and serve a particular resource. It carries exactly the same directives as the familiar <meta name="robots"> tag — noindex, nofollow, noarchive and the rest — but because it travels in the HTTP header rather than the page body, it can be applied to files that have no HTML to put a meta tag in. That makes it the only way to control indexing for PDFs, images, videos, spreadsheets and other non-HTML resources. This tool builds the header correctly and outputs it as a server configuration snippet you can paste in place. It is part of the Robots & Directives group in the SEO Toolkit.
These three controls are often confused but do different jobs. robots.txt controls crawling — whether a bot may request a URL at all — but it cannot reliably keep a page out of the index. The meta robots tag controls indexing for HTML pages, placed in the <head>. The X-Robots-Tag does the same indexing job as the meta tag but at the server level, so it works for any file type and can be applied to many URLs at once with a single rule. The crucial catch: an indexing directive only works if the crawler can fetch the file, so never block in robots.txt something you have told to noindex.
Controls crawling, not indexing — a blocked URL can still rank.
Indexing control for HTML pages, set in the head.
Indexing control for any file, set in the HTTP header.
A noindexed file must stay crawlable to be seen.
Each directive instructs the crawler differently. noindex keeps the resource out of search results entirely and is the most common use. nofollow tells the crawler not to follow links found in the resource. noarchive and nocache suppress the cached copy, while nosnippet hides the text preview. noimageindex stops images on the resource being indexed, and notranslate declines translation offers. The shortcut none equals noindex plus nofollow. Two take values: max-snippet caps the preview length (use -1 for no limit), and unavailable_after sets a date when the page should drop from results.
Paste the generated block into the .htaccess file in your site's root, or into the server or virtual-host configuration. It requires the mod_headers module, which most shared Apache hosts enable by default; the IfModule wrapper means the block is ignored rather than erroring if the module is missing. Always back up the file before editing.
Search engines only see the X-Robots-Tag header when they actually fetch the file. If you also disallow that URL in robots.txt, the crawler never requests it, never sees the noindex, and the URL can still appear in results as a bare link. To remove something from the index, leave it crawlable and let the noindex directive do its job.
Yes. Enter a crawler name such as googlebot or bingbot and the directive is prefixed with it, so only that bot obeys the rule while others ignore it. Leave the field blank to apply the directive to all robots, which is the usual choice unless you specifically need different behaviour per engine.
No. The header and configuration snippets are assembled entirely in your browser with JavaScript, and the tool never makes a network request with your input. Nothing you type is uploaded, logged or stored, so it is safe to use for unreleased sites and confidential client projects.
Keep brochures and documents out of search results.
Stop specific images being indexed with noimageindex.
Drop time-limited content with unavailable_after.
Hide internal or temporary resources from the index.
Use the X-Robots-Tag Generator with these tools from the SEO Toolkit: set page-level rules with the Meta Robots Tag Generator, manage crawling with the Robots.txt Generator and test it with the Robots.txt Tester, mark individual links with the Nofollow Link Generator, set server redirects with the .htaccess Redirect Generator, and validate your sitemap.
Anyone controlling how files get indexed.
Everything about the X-Robots-Tag header.
It builds a correct X-Robots-Tag HTTP header that controls how search engines index a resource. You choose directives like noindex and nofollow, optionally target a crawler or file type, and the tool outputs a ready-to-paste rule for Apache, Nginx or as a raw header, with a plain-language preview of the effect.
Yes, completely. There is no cost, no sign-up and no limit on how many headers you generate. The tool runs entirely in your browser, so there are no server costs and nothing to pay for.
Use it whenever the resource is not an HTML page. You cannot add a meta robots tag to a PDF, image, video or spreadsheet, so the X-Robots-Tag header is the only way to noindex those files. It is also useful when you want to apply one indexing rule to many URLs at once at the server level.
robots.txt controls crawling — whether a bot may request a URL — but cannot reliably keep a page out of the index. The X-Robots-Tag controls indexing. They must not conflict: if you block a URL in robots.txt, the crawler never fetches it and never sees the noindex header, so the URL can still appear in results.
The tool supports noindex, nofollow, noarchive, nosnippet, noimageindex, notranslate, none and nocache, plus the valued directives max-snippet and unavailable_after. You can combine several into one header value; the generator joins them in the correct comma-separated format that search engines expect.
For Apache, paste the block into your .htaccess file or server configuration; it needs the mod_headers module, which most hosts enable. For Nginx, add it to the relevant server or location block and reload the configuration. The raw-header option is for any setup where you can set response headers directly.
Yes. Switch the target to specific file types and enter extensions such as pdf, doc or jpg. The generator wraps the directive in a FilesMatch block for Apache or a location block for Nginx so the header is sent only for those file types, leaving the rest of your site untouched.
Yes. Enter a crawler name such as googlebot and the directive is prefixed with it, so only that bot obeys the rule. Leave the field blank to apply the directive to all robots, which is the typical choice unless you need different indexing behaviour for different search engines.
No. The directive takes effect the next time the crawler fetches the file and reprocesses it, which can take days or weeks. To speed removal of a page you control, you can also use the removals tool in Search Console, but the file must stay crawlable for the noindex to be seen at all.
No. The header and configuration snippets are assembled entirely in your browser with JavaScript, and the tool never sends your input over the network. Nothing you type is uploaded, logged or stored, so it is safe to use for unreleased sites and confidential client work.
Yes. The generator is fully responsive, so you can build and copy an X-Robots-Tag rule from a phone or tablet. The directive checkboxes, the target options, the format tabs and the output box all work the same on small screens as they do on desktop.
Convert, compress, and resize images in multiple formats — JPG, PNG, WebP, ICO, and more.
Calculate in-hand salary, CTC breakup, tax deductions, and more for Indian employees.
Convert length, weight, temperature, speed, volume, and 200+ measurement units instantly.
Count words, convert case, generate lorem ipsum, find & replace text, and more writing utilities.
Control how search engines index PDFs, images and any non-HTML file — pick your directives and copy a ready-to-paste Apache, Nginx or raw-header rule. Free, private and instant in your browser.
🤖 Open the Generator ← All Robots & Directives Tools