User-agent: *
User-agent: InternetSeer.com
User-agent: Syntryx ANT Scout Chassis Pheromone
User-agent: Toutatis 5-9.28 (hoppa.com)" 
User-agent: PEAR HTTP_Request class 
User-agent: Wget/1.9.1
User-agent: NutchCVS/0.7.1
User-agent: PlantyNet_WebRobot_V1.9
User-agent: IRLbot/2.0
User-agent: Exabot/2.0
User-agent: SBIder/0.8-dev
User-agent: Sensis Web Crawler
User-agent: RufusBot
User-agent: MJ12bot/v1.0.5 
User-agent: NG/2.0
User-agent: MJ12bot/v1.0.6
User-agent: findlinks/1.1-a4 
User-agent: lwp-trivial/1.38
User-agent: StackRambler/2.0 
User-agent: voyager/1.0
User-agent: FAST Enterprise Crawler
User-agent: iCCrawler
User-agent: LapozzBot/1.4
User-agent: boitho.com-dc/0.81
User-agent: SquidClamAV_Redirector 1.6
User-agent: Pompos/1.3
User-agent: WebIndexer/1-dev
User-agent: Exabot NG/MimeLive Client
User-agent: IP2LocationBot/1.0 
User-agent: Dumbot(version 0.1 beta - http://www.dumbfind.com/dumbot.html)
User-agent: ConveraMultiMediaCrawler/0.1
User-agent: DomainsDB.net MetaCrawler v.0.9.7b
User-agent: ia_archiver-web.archive.org
User-agent: Miva (AlgoFeedback@miva.com)" 
User-agent: larbin_2.6.3 (sneha@iitk.ac.in)" 
User-agent: wwwster/1.4
User-agent: LWP::Simple/5.803
User-agent: NG-Search/0.8.2_(http://www.ng-search.com/)
User-agent: ::_detectURL
User-agent: EuripBot/0.4 
User-agent: MJ12bot/v1.0.1
User-agent: IRLbot/1.0 (+http://irl.cs.tamu.edu/crawler)
User-agent: HeinrichderMiragoRobot (http://www.miragorobot.com/scripts/deinfo.asp)
User-agent: Regiochannel/1.0 (http://www.regiochannel.de/)
User-agent: Speedy Spider (Beta/1.0; www.entireweb.com)
User-agent: NutchCVS/0.06-dev (Nutch; http://www.nutch.org/docs/en/bot.html; nutch-agent@lists.sourceforge.net)
User-agent: Amfibibot/0.07 (Amfibi Robot; http://www.amfibi.com; agent@amfibi.com)
User-agent: PROext/1.0.01beta (compatible; Win16; S)
User-agent: Yandex/1.01.001 (compatible; Win16; I)
User-agent: Gulliver/1.3
User-agent: Slurp.so/1.0 (slurp@inktomi.com; http://www.inktomi.com/slurp.html)
User-agent: Cha-Cha/v1.0b1
User-agent: Mercator-1.0
User-agent: HenryTheMiragoRobot
User-agent: WhizBang! Lab
User-agent: Spider/MaxBot.com admin@maxbot.com
User-agent: Scooter/2.0 G.R.A.B. V1.1.0
User-agent: AVSearch-3.0(test/berkeleypage)
User-agent: Openfind Robot/1.1A2
User-agent: LEIA/2.90
User-agent: InfoSeek Sidewinder/0.9
User-agent: CCU_GAIS Robot/1.0B2
User-agent: NG/1.0
User-agent: EuripBot/0.2 (+http://www.eurip.com) GetRobots
User-agent: EasyDL/3.04  http://keywen.com/Encyclopedia/Bot
User-agent: findlinks/0.912 (+http://wortschatz.uni-leipzig.de/findlinks/) 
User-agent: Twiceler www.cuill.com
User-agent: cfetch/1.0
User-agent: aipbot/1.0
User-agent: Sensis.com.au Web Crawler 
User-agent: Xenu Link Sleuth 1.2e
User-agent: Xenu Link Sleuth 1.2d
User-agent: YottaShopping_Bot/4.12
User-agent: Java/1.4.2_05
User-agent: Links SQL
User-agent: NaverBot-1.0 (NHN Corp. / +82-2-3011-1954 / nhnbot@naver.com)
User-agent: SeznamBot/1.0
User-agent: Wildsoft Surfer
User-agent: findlinks/0.89 (+http://wortschatz.uni-leipzig.de/findlinks/)
User-agent: ZipppBot/0.11
User-agent: findlinks/0.901g (+http://wortschatz.uni-leipzig.de/findlinks/)
User-agent: Websquash.com (Add url robot)
User-agent: NP/0.1 (NP; http://www.nameprotect.com; npbot@nameprotect.com)
User-agent: thesubot-beta-www.thesubot.de
User-agent: DLMAN
User-agent: NextGenSearchBot 4 
User-agent: Muhsearch.de
User-agent: PrimeFind Spider - http://www.primefind.com
User-agent: W3C-WebCon/5.4.0 libwww/5.4.0
User-agent: ColdFusion
User-agent: Holmes/1.0
User-agent: OmniExplorer_Bot/1.09 (+http://www.omni-explorer.com) Rentals Crawler
User-agent: Computer_and_Automation_Research_Institute_Crawler (crawler@info.ilab.sztaki.hu)
User-agent: Seekbot/1.0 (http://www.seekbot.net/bot.html) HTTPFetcher/0.3
User-agent: ObjectsSearch/0.06
User-agent: findlinks/0.901g (+http://wortschatz.uni-leipzig.de/findlinks/)" 
User-agent: SeznamBot/1.0 (+http://fulltext.seznam.cz/)" 
User-agent: NuSearch Spider www.nusearch.com
User-agent: Java/1.5.0_02
User-agent: WebFerret
User-agent: 'Ahoy! The Homepage Finder'
User-agent: Arachnophilia 
User-agent: ArchitextSpider
User-agent: ASpider/0.09 
User-agent: AURESYS/1.0
User-agent: BackRub/*.* 
User-agent: Big Brother 
User-agent: BlackWidow 
User-agent: BSpider/1.0 libwww-perl/0.40 
User-agent: CACTVS Chemistry Spider
User-agent: Digimarc CGIReader/1.0
User-agent: Checkbot/x.xx LWP/5.x
User-agent: CMC/0.01
User-agent: combine/0.0
User-agent: Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1) 
User-agent: conceptbot/0.3 
User-agent: root/0.1 
User-agent: CS-HKUST-IndexServer/1.0 
User-agent: CyberSpyder/2.1 
User-agent: Deweb/1.01 
User-agent: DragonBot/1.0 libwww/5.0 
User-agent: EIT-Link-Verifier-Robot/0.2 
User-agent: Emacs-w3/v[0-9\.]+ 
User-agent: EmailSiphon
User-agent: EMC Spider 
User-agent: explorersearch 
User-agent: ExtractorPro
User-agent: FelixIDE/1.0 
User-agent: Hazel's Ferret Web hopper 
User-agent: ESIRover v1.0 
User-agent: fido/0.9 Harvest/1.4.pl2 
User-agent: Hämähäkki/0.2 
User-agent: KIT-Fireball/2.0 libwww/5.0a 
User-agent: Fish-Search-Robot 
User-agent: Mozilla/2.0 (compatible fouineur v2.0; fouineur.9bit.qc.ca) 
User-agent: Robot du CRIM 1.0a 
User-agent: Freecrawl 
User-agent: FunnelWeb-1.0 
User-agent: gcreep/1.0 
User-agent: GetURL.rexx v1.05 
User-agent: Golem/1.1 
User-agent: Gromit/1.0 
User-agent: Gulliver/1.1 
User-agent: AITCSRobot/1.1 
User-agent: wired-digital-newsbot/1.5 
User-agent: htdig/3.0b3 
User-agent: HTMLgobble v2.2 
User-agent: IncyWincy/1.0b1 
User-agent: Informant 
User-agent: InfoSeek Robot 1.0 
User-agent: Infoseek Sidewinder 
User-agent: InfoSpiders/0.1 
User-agent: inspectorwww/1.0 http://www.greenpac.com/inspectorwww.html 
User-agent: 'IAGENT/1.0' 
User-agent: IsraeliSearch/1.0 
User-agent: JCrawler/0.2 
User-agent: Jeeves v0.05alpha (PERL, LWP, lglb@doc.ic.ac.uk) 
User-agent: Jobot/0.1alpha libwww-perl/4.0 
User-agent: JoeBot 
User-agent: JubiiRobot
User-agent: jumpstation 
User-agent: Katipo/1.0 
User-agent: KDD-Explorer/0.1 
User-agent: KO_Yappo_Robot/1.0.4(http://yappo.com/info/robot.html) 
User-agent: LabelGrab/1.1 
User-agent: logo.gif crawler 
User-agent: Lycos/x.x 
User-agent: Lycos_Spider_(T-Rex)
User-agent: Magpie/1.0 
User-agent: MediaFox/x.y 
User-agent: MerzScope 
User-agent: NEC-MeshExplorer 
User-agent: MOMspider/1.00 libwww-perl/0.40 
User-agent: Monster/vX.X.X -$TYPE ($OSTYPE) 
User-agent: Motor/0.2 
User-agent: MuscatFerret 
User-agent: MwdSearch/0.1 
User-agent: NetCarta CyberPilot Pro 
User-agent: NetMechanic 
User-agent: NetScoop/1.0 libwww/5.0a 
User-agent: NHSEWalker/3.0 
User-agent: Nomad-V2.x 
User-agent: NorthStar 
User-agent: Occam/1.0 
User-agent: HKU WWW Robot, 
User-agent: Orbsearch/1.0 
User-agent: PackRat/1.0 
User-agent: Patric/0.01a 
User-agent: Peregrinator-Mathematics/0.7 
User-agent: Duppies 
User-agent: Pioneer 
User-agent: PGP-KA/1.2 
User-agent: Resume Robot 
User-agent: Road Runner: ImageScape Robot (lim@cs.leidenuniv.nl) 
User-agent: Robbie/0.1 
User-agent: ComputingSite Robi/1.0 (robi@computingsite.com) 
User-agent: Roverbot 
User-agent: SafetyNet Robot 0.1, 
User-agent: Scooter/1.0 
User-agent: Senrigan/xxxxxx 
User-agent: SG-Scout 
User-agent: Shai'Hulud 
User-agent: SimBot/1.0 
User-agent: Open Text Site Crawler V1.0 
User-agent: SiteTech-Rover 
User-agent: Slurp/2.0 
User-agent: ESISmartSpider/2.0 
User-agent: Snooper/b97_01 
User-agent: Solbot/1.0 LWP/5.07 
User-agent: Spanner/1.0 (Linux 2.0.27 i586) 
User-agent: Mozilla/3.0 
User-agent: Tarantula/1.0 
User-agent: tarspider 
User-agent: dlw3robot/x.y (in TclX by http://hplyot.obspm.fr/~dl/) 
User-agent: Templeton/ 
User-agent: TitIn/0.2 
User-agent: TITAN/0.1 
User-agent: UCSD-Crawler 
User-agent: urlck/1.2.3 
User-agent: Valkyrie/1.0 libwww-perl/0.40 
User-agent: Victoria/1.0 
User-agent: vision-search/3.0' 
User-agent: VWbot_K/4.2 
User-agent: W3M2/x.xxx 
User-agent: WWWWanderer v3.0 
User-agent: WebCopy/
User-agent: WebCrawler/3.0 Robot libwww/5.0a 
User-agent: WebFetcher/0.8, 
User-agent: weblayers/0.0 
User-agent: WebLinker/0.0 libwww-perl/0.1 
User-agent: WebMoose/0.0.0000 
User-agent: Digimarc WebReader/1.2 
User-agent: webs@recruit.co.jp 
User-agent: webvac/1.0 
User-agent: webwalk 
User-agent: WebWalker/1.10 
User-agent: WebWatch 
User-agent: Wget/1.4.0 
User-agent: w3mir 
User-agent: WWWC/0.25 (Win95) 
User-agent: XGET/0.7 
User-agent: Nederland.zoek 
User-agent: BizBot04 kirk.overleaf.com 
User-agent: HappyBot (gserver.kw.net) 
User-agent: CaliforniaBrownSpider 
User-agent: EI*Net/0.1 libwww/0.1 
User-agent: Ibot/1.0 libwww-perl/0.40 
User-agent: Merritt/1.0 
User-agent: StatFetcher/1.0 
User-agent: TeacherSoft/1.0 libwww/2.17 
User-agent: WWW Collector 
User-agent: processor/0.0ALPHA libwww-perl/0.20 
User-agent: wobot/1.0 from 206.214.202.45 
User-agent: Libertech-Rover www.libertech.com? 
User-agent: WhoWhere Robot 
User-agent: ITI Spider 
User-agent: w3index 
User-agent: MyCNNSpider 
User-agent: SummyCrawler 
User-agent: OGspider 
User-agent: linklooker 
User-agent: CyberSpyder (amant@www.cyberspyder.com) 
User-agent: SlowBot 
User-agent: Surfbot 
User-agent: Bizbot003 
User-agent: WebWalker 
User-agent: SandBot 
User-agent: EnigmaBot 
User-agent: spyder3.microsys.com 
User-agent: www.freeloader.com. 
User-agent: Googlebot
User-agent: METAGOPHER
User-agent: Seekbot/1.0 
User-agent: Mozilla/5.0
User-agent: curl/7.9.5 
User-agent: Avant Browser
User-agent: Ask Jeeves
User-agent: msnbot/0.3
User-agent: msnbot/1.0
User-agent: Yahoo! Slurp
User-agent: aconon Index
User-agent: Yahoo-MMCrawler/3.x
User-agent: ChangeDetection/1.0
User-agent: ia_archiver
User-agent: MJ12bot/v0.7.3
User-agent: psbot/0.1
User-agent: Gigabot/2.0
User-agent: Googlebot/2.1
User-agent: appie 1.1
User-agent: NaverBot-1.0
User-agent: InnerpriseBot/1.0
User-agent: Konqueror/3.0-rc5
User-agent: Konqueror/3.0-rc6
User-agent: Konqueror/3.2
User-agent: Toutatis 5-4.10
User-agent: Toutatis 5-4.11
User-agent: NPBot
User-agent: Python-urllib/1.16
User-agent: SunOS 5.4 sun4m
User-agent: SP Jeanie/3.3.2.609
User-agent: LinkWalker
User-agent: grub-client-2.3
User-agent: libwww-perl/5.801
User-agent: libwww-perl/5.800
User-agent: ZyBorg/1.0
User-agent: findlinks/0.89
User-agenT: OmniExplorer_Bot/1.07
User-agent: wish-la
User-agent: PHP version tracker
User-agent: KummHttp/1.1
User-agent: NPT 0.0 beta
User-agent: LWP::Simple/5.65
User-agent: LWP::Simple/5.76
User-agent: LWP::Simple/5.79
User-agent: VoilaBot BETA 1.2
User-agent: ConveraCrawler/0.5
User-agent: Lynx/2.8.3rel.1 libwww-FM/2.14
User-agent: htdig
User-agent: Marvin v0.3
User-agent: FDSE robot
User-agent: AsQuickAs FDSE robot
User-agent: DelphiKompendium
User-agent: FEHLSTART Superspider
User-agent: SpiderS2
User-agent: meet.de multisubmit 2.1
User-agent: lwp-trivial/1.40
User-agent: PrimeFind Spider
User-agent: Yoodel
User-agent: 2 OK KAMEL
User-agent: Suchmaschinebox
User-agent: tufee\xae Internetguide
User-agent: Java/1.4.1_04
User-agent: SEARCHBUSTER
User-agent: Zao-Crawler
User-agent: Googlebot-Image/1.0
User-agent: Mediapartners-Google/2.1
User-agent: Microsoft URL Control - 6.00.8862
User-agent: Teradex_Crawler
User-agent: Spider-for-w8.net
User-agent: ichiro/1.0
User-agent: Faxobot/1.0
User-agent: TurnitinBot/2.0
User-agent: libwww-perl/5.65
User-agent: obot
User-agent: BecomeBot/1.23
User-agent: NextGenSearchBot 1
User-agent: TheSuBot/0.1
User-agent: Mozilla/3.0 (compatible;miner;mailto:miner@miner.com.br)
User-agent: Crescent Internet ToolPak HTTP OLE Control v.1.0
User-agent: Explorer 
# User-Agent: SearchIt.Bot 1.62
# User-Agent: Crawler
# Disallow: /*.gif$
# Disallow: /*.jpg$
Disallow: /cgi-bin/
Disallow: /anzeigen/
Disallow: /archiv/
Disallow: /logs/
Disallow: /forum/
Disallow: /classifieds/
Disallow: /danke4.htm
Disallow: /flags/
Disallow: /images/
Disallow: /script/
Disallow: /werbe/
Disallow: /foto/
Disallow: /script/
Disallow: /bild1/
Disallow: /bild2/