X-Robots-Tag

Like meta robots tags, this tag provides crawlers instructions for how to crawl or index web page content.