# # Standard robots.txt file for TPDirect, undated July, 2005 # # URL and Link Search Engine Bot Exclusions # The * tells all bots to follow the guidlines listed # Wildcards (*,?) are not allowed in Dissallow statements # Edit this file to exclude any folders needed or specific files # within allowed folders that are to be excluded. # Example File exclusion for nobots.html in the test folder # Disallow: /test/nobots.html # # Email Bots are disallowed from all folders and files within the site # and must be identified by the bot name. This list must be updated to # remain operational. # User-agent: Titan Disallow: / User-agent: EmailCollector Disallow: / User-agent: EmailSiphon Disallow: / User-agent: EmailWolf Disallow: / User-agent: ExtractorPro Disallow: / User-agent: * Disallow: /activeerror Disallow: /analog Disallow: /flash Disallow: /gallery Disallow: /images Disallow: /nav Disallow: /video