ichi.co.uk

IIS 7.x “404 - File or directory not found” for .iso image files

If you get the following message when trying to download an .iso disc image from IIS 7/7.5…

404 - File or directory not found.

The resource you are looking for might have been removed, had its name changed, or is temporarily unavailable.

You just need to add the MIMI type. To do this, open IIS Manager, select the server on the left, open MIME Types…

Then on the top right click Add > File name extension: .iso
MIME Type: application/octet-stream

The file should now download properly. 

Virgin Media Super Hub Session Data Downloaded/Uploaded Bug

Virgin Media automatically updated the firmware on my Super Hub (Netgear VMDG480 router) recently. Since then I’ve noticed that the Session Data Downloaded value has been ridiculous, like they’re showing MB as GB or something…

28TB in 8 days, 3 of which my computer was off and I was away with my laptop…

2,560GB in 2 hours 17 minutes…

Or how about 16GB in 26s…

I’ve tweeted them about it, so I’ll update this post with their response if I get one.

Throttling internet bandwidth in OSX

This is useful for testing the effect of slow connections on mobile apps in the development emulators. 

It uses IPFirewall rules to limit connections from your computer to the web (on TCP port 80) to 15KB per second (or any number you choose). Fire up the terminal and enter…

sudo ipfw pipe 1 config bw 15KByte/s
sudo ipfw add 1 pipe 1 src-port 80

Just make sure you remember to remove the rule when you are done with it…

sudo ipfw delete 1

MJ12bot and blocking web crawlers with IIS 7

Our web servers were recently getting hit pretty hard by web crawler bots that simply ignore your Robots.txt file. We were getting hit especially hard by a bot called MJ12bot, which is supposed to be for a distributed search engine which they say lives in an alpha state at search.majestic12.co.uk (at the time of writing that is down). It seems they get people to run their SETI@Home like software that uses your power, CPU and bandwidth to help them build what they say is a distributed search engine. What they definitely are doing with the data at present is running a SEO business, that tells you how many back-links you have, etc. via what seems to be the huge index that they have built via people donating their computing resources. 

We use Windows 2008 R2, so the solution for us was to block these crawlers via their user agent (easily done with Apache as well). To do so, open your site in IIS 7 and select the URL Rewrite module (you may have to add this via the Add/Remove Windows Features tool). Select Add Rule (in the top right) and then select ‘Blank rule’. Set you’re new rule up like the image below shows…

And add a condition like the one below…

Our regex string is “MJ12bot|soso|baidu|youdao|NaverBot|Yeti|ichiro|moget|sogou|Speedy”, however you may want to include Yandex in that as well. I’ve just removed them as they emailed us asking us to allow them to index our site and said that they obey Robots.txt rules, such as crawl-rate and disallow. 

***UPDATE***

Steve from Majestic12 has posted a detailed response regarding MJ12bot below

Redirecting www.* subdomain to the root domain in IIS 7

Having your site available on www.yourdomain.com and yourdomain.com isn’t good for your SEO as it splits the domain authority. You can use the Google Webmaster Tools to specify which domain Google should use, but you should also redirect on the site itself. You can do this in your web.config directly, but IIS has a module to do it for you. Redirecting from a sub-domain in IIS 7 is really easy using the URL Rewrite feature. 

First select the URL Rewrite module for your domain:

Click Add Rule in the top right panel and select Canonical domain name.

Add the your domain name without the www.

You will see your rule has been created…

Now we need to make the redirect send the 301 HTTP header, which makes it a permanent redirect, so select your rule and hit “Edit…” in the right side bar, scroll to the bottom and change the Redirect type to Permanent (301).

This will make your web.config file something like this…

<?xml version="1.0" encoding="UTF-8"?>
<configuration>
    <system.webServer>
        <rewrite>
            <rules>
                <rule name="CanonicalHostNameRule1">
                    <match url="(.*)" />
                    <conditions>
                        <add input="{HTTP_HOST}" pattern="^scorch\.io$" negate="true" />
                    </conditions>
                    <action type="Redirect" url="http://scorch.io/{R:1}" redirectType="Permanent" />
                </rule>
            </rules>
        </rewrite>
    </system.webServer>
</configuration>