# ------------------------------------------------------------------------------ # Robots Exclusion File # http://www.robotstxt.org/wc/exclusion.html # # A sub-domain is considered as an entirely different site and should be treated as such by placing the robots.txt file in the root of the subdomain. # # An asterisk * can be used to identify collections of entries (i.e. /projects/client*.htm). # Multiple *'s may be included in any line. # # A dollar sign $ can be used to control access to a specific directory. # For example, Disallow: /project/text$ will disable access to the file /projects/text but will still allow access to /projects/text.doc and /projects/text/home.htm # The entry must exactly match for this to be effective. # ------------------------------------------------------------------------------ # Google Search Engine Robot # http://www.google.com/support/webmasters/ # User-agent: Googlebot # Yahoo! Search Engine Robot # http://help.yahoo.com/us/ysearch/slurp/ # User-agent: Slurp # Microsoft Search Engine Robot # http://help.live.com/help.aspx?project=wl_webmasters # User-agent: msnbot # ChangeDetection # http://www.changedetection.com/webmasterhelp.html User-agent: ChangeDetection Disallow: / # 80legs # http://www.80legs.com/spider.html User-agent: 008 Disallow: / # All Other Search Engine Crawlers User-agent: * Disallow: /images/ Disallow: /scripts/ Disallow: /cgi-bin/ Disallow: /slideshows/ Disallow: /about/copyright/ Disallow: /about/terms/ Disallow: /journal/share/ User-agent: Googlebot Disallow: /*.gif$ Disallow: /*.jpg$ Disallow: /*.png$ Disallow: /*.psd$ Disallow: /*.pdf$ Disallow: /*.tif$ Disallow: /*.doc$ Disallow: /*.ai$ Disallow: /*.zip$ User-agent: Googlebot-Image Disallow: / Sitemap: http://apostrophestudios.com/sitemap.xml