# See http://www.robotstxt.org/wc/norobots.html for documentation on how to use the robots.txt file # # To ban all spiders from the entire site uncomment the next two lines: Sitemap: http://www.isafetrade.com/sitemapindex.xml User-Agent: * Disallow: /companies/ Disallow: /leads/ Disallow: /users/ Disallow: /networks/ # Not crawlable Disallow: /redirect Disallow: /signup Disallow: /login Disallow: /logout Disallow: /transactions Disallow: /subscriptions Disallow: /credit_accounts #Disallow: /companies-suppliers-manufacturers-importers-exporters # Legacy links Disallow: /faq Disallow: /payment_terms Disallow: /aboutus