# All Robots User-agent: * Disallow: /images/ Disallow: /scripts/ Disallow: /*.pdf$ Disallow: /*.swf$ # Google Image User-agent: Googlebot-Image Disallow: / # Google AdSense User-agent: Mediapartners-Google* Disallow: / # Yahoo! Slurp User-agent: Slurp Crawl-delay: 1 # Internet Archiver Wayback Machine (web.archive.org) User-agent: ia_archiver Disallow: # digg mirror User-agent: duggmirror Disallow: /