The Cost of Hot-Linking

It had never really occurred to me before, because I hadn't ever had trouble, but I was utterly shocked when I got my quarterly statement through from my web-space host. I was surprised as my bandwidth seemed to have been hovering around 800Mb per day (which seemed a lot) but on the 13th March had shot up to 1200Mb a day! This started costing me money. At first about £6 a day, and the bandwidth just went up and up, until it reached about 1600Mb a day yesterday costing me £9 that day alone!

Of course, the moment I realised I went on a mission to find out what the hell was going on. The statistics the web-host provided seemed to show nothing unusual. I run about 10 sites on my web-hosting and none of them seemed to be getting lots of hits (according to the generated statistics).

So, I downloaded the original web-access logs that the web-servers churn out day-in-day out. Opening them in Wordpad and I noticed an unusual number of hits to a photo that was on my personal site from me and my wife's honeymoon. Huh? In the first 100 lines, at least 80 were to this image, all HTTP 200 (i.e. transaction completed ok).

I downloaded a great program called Web Expert Lite to analyse the logs more effectively, and there I spotted it.



The image was being hit over 13,000 times a day (20,000 on one day) to be used for people's profile pages on myspace.com, a social networking website. It had been added to HotFreeLayouts.com by some guy called Jesse, and apparantly (according to the page about it), has been used on 24,728 myspace profiles.

Now, I'm half flattered that one of my wife's photos should be so popular, however, I'm not prepared to pay £10 a day for some spotty teen to have it as their website's backdrop. I mean, it's not even nicely done - just look at the background on that Jesse bloke's website!

So, the way you stop this 'hot linking' as it's known it to re-write the .htaccess file like so:


RewriteEngine On
RewriteCond %{HTTP_REFERER} ^http://(.+\.)?myspace\.com/ [NC,OR]
RewriteCond %{HTTP_REFERER} ^http://(.+\.)?blogspot\.com/ [NC,OR]
RewriteCond %{HTTP_REFERER} ^http://(.+\.)?altlab\.com/ [NC,OR]
RewriteCond %{HTTP_REFERER} ^http://(.+\.)?livejournal\.com/ [NC]
RewriteRule .*\.(jpe?g|gif|bmp|png)$ - [F]


Adding this to your .htaccess at the root of your domain will stop hot-linking from the listed addresses. Each line is a regular expression pattern match in the HTTP_REFERER. NC means ignore case when matching, and OR provides a logical OR with the next line. The last HTTP_REFERER pattern therefore doesn't have the OR.

If one of these match the last line matches an image file and the "- [F]" causes an HTTP 403 - Forbidden to be returned, blocking access to the image.

You might read this as:

if( myspace.com, blogspot.com, atlab.com OR livejournal.com ) then
    if( jpg, jpeg, gif, bmp OR png ) then
          return 403 Forbidden
    endif
endif

I found this information on the AltLab htaccess tutorial and that page allows you to test whether your htaccess is working (hence why altlab appears in one of the patterns!).

So, the upshot is that no-longer can people hot-link to my images and cost me lots of money in excess bandwidth costs!! The second upshot is that 24,728 myspace profile pages now look blank. Ha!

The moral of the story is that you should download a local copy and not hot-link.

Browser Accessibility

It's often overlooked that accessibility refers to a few other things alongside making websites usable by the disabled. Browser accessibility is one of the things that is very important, because you never know what browser is going to be used to view your site, so to maximise your visitor base you need to make sure that they can read your website whatever they are using.

Testing out your website in various browsers is a relatively easy affair - you simply need to download the browsers and try it. If you don't have a Macintosh, and don't know anyone who has one, you can use services like NetMechanic, or the thorough BrowserCam, which basically take screenshots of your site on various browsers.

There is a particular exception which isn't covered in the above, and a situation which will probably become more common, and that's people on mobile devices accessing the website. Mobiles phones are becomes more and more popular, particularly with youngsters who are most conversant with the web. Providing websites that are accessible to these devices (which generally have very small screens) is increasingly necessary.

Detecting browser (device screen) resolution can only be achieved with Javascript and that has problems all of its own. Server-side rewriting of the resulting page is always the most effective and accessible method of presenting different pages. It's possible to test the user agent string to detect mobile devices. A quick search on Google found the following PHP code, that does this detection (by regular expression matching on the user agent string) and sets the variable $mobile to 1 if the device is a mobile device. You might decide this should be a boolean.

<?php
        // From: http://www.vbulletin.com/forum/showthread.php?t=108641&page=2
        // PDA BROWSER DETECT
        $browsers = array(
                "Windows CE",
                "WebTV",
                "AvantGo",
                "Blazer",
                "PalmOS",
                "lynx",
                "Go.Web",
                "Elaine",
                "ProxiNet",
                "ChaiFarer",
                "Digital Paths",
                "UP.Browser",
                "Mazingo",
                "Mobile",
                "T68",
                "Syncalot",
                "NetFront",
                "Danger",
                "Symbian",
                "Nokia",
                "Xiino",
                "AU-MIC",
                "EPOC",
                "BlackBerry",
                "Wireless",
                "Handheld"
        );

        if(preg_match('/('.implode('|', $browsers).')/i', $_SERVER['HTTP_USER_AGENT'], $match))
        {
                $mobile=1;
        }
        else if (isset($_SERVER['HTTP_UA_OS']))
        {
                if (strstr($_SERVER['HTTP_UA_OS'],"POCKET PC") !== false)
                {
                                $mobile=1;
                }
        }
        else
        {
                $mobile=0;
        }
?>

To that end, I've used the above code to provide a PDA-based view onto this site. As there's no service to test this on multiple mobile devices, I can only test this on my Pocket PC, but it does seem to work. I'm just not sure how good the browsers are on other devices (the Nokias and Sonys), and whether they deal with stylesheets well.

Usability Exchange

Usability Exchange is a new website where disabled users are drafted in to test your website for you, for accessibility criteria. They are paid for each website they test, yet registration for users and organisations is apparantly free. I haven't yet used them, but I shall certainly sign up and use them if they really are free.

PAS 78

A new report (Publically Available Specification number 78) has been produced by the Disability Rights Commission that goes some way towards making accessibility in websites more widely accepted. Clearly the report is mainly aimed at accessibility for disabled users, but at least getting the word out may make people think more about the general idea of accessibility when it comes to making websites.

As part of the launch of this new document, they held a conference, but it was rather disappointing to see the lack of effort put into making that page accessibile. For example, their accessibility statement was a link to a Word document. It's all very well putting anchor titles that say it's a link to a Word document, but that won't make a jot of difference to someone who doesn't have Word installed!

There's an overview of the document at Bruce Lawson's website (link thanks to Blether).