# /robots.txt file for http://webcrawler.com/ # mail webmaster@webcrawler.com for constructive criticism # For help see http://www.robotstxt.org/wc/exclusion-admin.html User-agent: * Disallow: /goo/ Disallow: /photoview/ Disallow: /port/ Disallow: /preview/ Disallow: /test/ Disallow: /test2/ Disallow: /compliance/ # BEGIN XML-SITEMAP-PLUGIN Sitemap: http://www.fordfam.com/sitemap.xml.gz # END XML-SITEMAP-PLUGIN