rbowen / centos / centos.org

Forked from centos/centos.org 5 years ago
Clone

Blame content/robots.txt.erb

e8ec7a
---
e8ec7a
is_hidden: true
e8ec7a
path:      "/robots.txt"
e8ec7a
---
e8ec7a
Sitemap: http://www.centos.org/sitemap.xml
e8ec7a
e8ec7a
User-Agent: *
e8ec7a
Crawl-delay: 1.0
e8ec7a
Disallow: /scripts/
e8ec7a
Disallow: /themes/
e8ec7a