# search engine robot file. # Use this file to prevent areas # of your website from being indexed # by web robots. # Required for all crawlers to crawl your site ; User-agent: * # followed by Disallow: /whatever name User-agent: * Disallow: /Code%20from%20Host/ Disallow: /ISP/ Disallow: /Misc%20Docs/ Disallow: /images/ Disallow: /cgi-bin/ Disallow: /private/ Disallow: /includes/ Disallow: /education/ Disallow: /John%20Wade/ Disallow: /John%20Wade%20August/ Disallow: /Misc%20Docs/ Disallow: /modules/ Disallow: /phpBB/phpBB%20Userguide_files/ Disallow: /Representing Disallow: /SWCRN-Wade Disallow: /test Disallow: /Announcements/images/ Disallow: /Announcements/annc Disallow: /Banner%20Maybes/ Disallow: /BondUniv/ Disallow: /test/ Disallow: /Invalid/