areguera / centos / centos.org

Forked from centos/centos.org 4 years ago
Clone

Blame static/robots.txt

e8ec7a
Sitemap: http://www.centos.org/sitemap.xml
e8ec7a
e8ec7a
User-Agent: *
e8ec7a
Crawl-delay: 1.0
e8ec7a
Disallow: /scripts/
e8ec7a
Disallow: /themes/
c2ad4e
Disallow: /docs/2/
c2ad4e
Disallow: /docs/3/
c2ad4e
Disallow: /docs/4/
9c8982
Disallow: /forums/posting.php
9c8982
Disallow: /forums/search.php
9c8982
Disallow: /forums/app.php
9c8982
Disallow: /forums/ucp.php
9c8982
Disallow: /forums/memberlist.php
9c8982
Disallow: /forums/report.php