# page for controlling search engine spider behavior
# disallow all spiders from intranet pages

User-agent: *
Disallow: /