Home About Music Links

webcrawler

Jac Timms

MJ12bot and blocking web crawlers with IIS 7

Our web servers were recently getting hit pretty hard by web crawler bots that simply ignore your Robots.txt

1 min read