Ctrl.blog

The hotlinked image cache

Constantly shrinking and compressing the same image would put an unnecessary strain on server resources. Every hotlinked image should thus be cached for some time so that subsequent requests can be served from the cache rather than reprocessed for every request.

Start by creating a folder for the image cache, and assign it to the apache user:

mkdir -m 755 "/var/www/cache/hotlinked-assets/"
chown apache:apache "/var/www/cache/hotlinked-assets/"

For a system protected by SELinux, you’ll need to set a label like ‘read-writable web content type’ on the caching folder to allow for image files to be created and read from the cache directory:

semanage fcontext -a -t httpd_sys_rw_content_t "/var/www/cache/hotlinked-assets(/.*)?"
restorecon -R -v -F "/var/www/cache/"

We’ll get back to how to prevent the image cache from consuming an enormous amount of disk space by cleaning out the files that aren’t getting requested a little later on.

With the cache in place, the next step is to set up the image processor that will receive requests for images, reduce them in size, and serve them back to the user.

On-demand image processor

This is a simplified version of what I use for this website. It should bee fully functional, however. I’m not going to explain the code itself in too great a detail, but the main logic of it’s as follows:

1. Determine the path to the image file on disk from the request path coming in from the server. Verify that the file exists.
   1. If the file doesn’t exist, return a placeholder image.
   2. If the file does exist:
      1. Combine an unique hash of the path to the file name with the remote origin (domain name) that requested the image. This allows for per-origin processing.
      2. Check to see if the combination of hash and origin is already in the image cache, and return the image from the cache if it exists.
      3. If it’s not in the cache:
         1. Shrink and compress the image to the given quality parameters. Save the resulting image to the cache.
         2. Return the original image as this is the first request for the image from the given referring host.

I’ll leave the obvious optimization of not reprocessing the same hash (image path) multiple times up to the reader as an exercise.

Create a file called /assets/hotlinker.php (relative to your DocumentRoot) and include the following script:

<?php

# SPDX-License-Identifier: CC0-1.0

function request_uri_to_path($img_address) {
  if (strpos($img_address, '..') !== false || strpos($img_address, '"') !== false  || strpos($img_address, "'") !== false  || strpos($img_address, '`') !== false) {
    return false;
  }

  $file_path = "/var/www/html${img_address}";
  if (file_exists($file_path)) {
    return $file_path;
  }

  return false;
}


function get_hotlinkable_image( $img_address ) {
  $img_path = request_uri_to_path(strtok($img_address, '?'));
  if ($img_path === false) {
    return false;
  }

  $referer_hostname = preg_replace("/[^a-z0-9]/", '_', parse_url($_SERVER['HTTP_REFERER'], PHP_URL_HOST));
  $img_hash = md5($img_path, false);
  $file_path = "/var/www/cache/hotlinked-assets/${img_hash}-${referer_hostname}.jpeg";
  if (file_exists($file_path)) {
    return $file_path;
  }

  # Maximum dimensions
  $width = 550;
  $height = 270;

  # Calculate new image ratio
  list($width_orig, $height_orig) = getimagesize($img_path);
  $ratio_orig = $width_orig / $height_orig;
  if ($width / $height > $ratio_orig) {
     $width = $height * $ratio_orig;
  } else {
     $height = $width / $ratio_orig;
  }

  # Read in the original image and compress it
  $image_p = imagecreatetruecolor($width, $height);
  $image = imagecreatefromstring(file_get_contents($img_path));
  imagecopyresampled($image_p, $image, 0, 0, 0, 0, $width, $height, $width_orig, $height_orig);
  imagedestroy($image);

  # Write out new cached image
  imagejpeg($image_p, $file_path, 30);
  imagedestroy($image_p);

  return $img_path;
}

# Serve an image
$image = get_hotlinkable_image($_SERVER['REQUEST_URI']);
if ($image === false || !isset($image)) {
  $image = '/var/www/html/assets/img/fallback-hotlinking-warning.png';
}

$fp = fopen($image, 'rb');

header('Vary: Referer');
header('Content-Type: image/jpeg');
header("Content-Length: " . filesize($image));
fpassthru($fp);

This code has obviously been minimized for the purpose of demonstrating how to implement the above logic. It will, however, perform it’s job adequately.

Rewriting externally referenced images in Apache

Include the below Apache configuration chunk in your VirtualHost or Server block for the domain you want to deploy the protection mechanism on.

# Hotlink protection
RewriteEngine On
RewriteCond %{REQUEST_URI} ^/images/(.*).(gif|jpe?g|png|webp)$ [nocase]
RewriteCond %{HTTP_REFERER} !^$
RewriteCond %{HTTP_REFERER} !^https?://(www.)?example.com/.*$ [nocase]
RewriteCond %{HTTP_REFERER} !^https?://((translate|www).)?(baiducontent|fanyi.baidu|googleusercontent|microsofttranslator).com/.*$ [nocase] [OR]
RewriteCond %{QUERY_STRING} !is= [nocase]
RewriteRule ^(.*)$ /assets/hotlinker.php?$1 [last]

The above consists of two sets of rewrite conditions. The first set starts by checking that the requested resource is in the /images/ path, and then checks to see that the referer (sic) header is either empty or matches your domain name.

The second set of domains checks against popular machine-translation services from the Baidu, Google, and Bing search engines. For the purposes of hotlink protection, these service’s domains should be considered to be synonymous with your domain.

Depending on your type of website and its contents, machine-translations will open up your websites to a much larger segment of the world’s 7.3 billion population. Note that these domains shouldn’t be considered as synonymous for other security related purposes.

You’ll realize why this last set of rules re important if you’ve ever come across an interesting blog post in a language you don’t understand – only to be met by aggressive accusations of hotlinking and bandwidth theft in the translated version of the page.

These translation services only swap out the text strings of your website with machine-translated variants in the visitor’s native language. Your ads and page are otherwise left unchanged.

The last part of the rewrite instruction is the rewrite rule. It says to pass any request for image resources that don’t meet the above criteria through the on-demand image processor that we set up before. Note that this address must be relative to the host root (the DocumentRoot) directory, and must not be an absolute URL.

Run the below command to perform an automatic test on your configuration before proceeding to spot any mistakes you may have made. If you’ve modified the regular expressions and run into problems, you might find Regex101 helpful in troubleshooting the problem.

apachectl configtest

If you didn’t run into any issues, then you should now be all set to start serving the reduced quality image on hotlinking sites. Restart Apache to apply your changes. Note that the “httpd” web server service may be called “apache” on some systems.

service httpd stop
service httpd start

Periodic clean-up

To make sure the cache stays lean, you should make sure to automatically clean up files when there’s no longer any interest in them.

Add a crontab to periodically delete generated images that haven’t been accessed in a more than two months. Run crontab -e and introduce the following weekly cleanup command:

@weekly nice -n 18 find "/var/www/cache/hotlinked-assets/" -name "*.jpeg" -atime 30 -delete

This runs the find program through the image asset cache and cleans out files with no recorded access in the last 30 days. The command is then run once every week through the nice program to run it at a low system priority with minimal impact on the system.

You can reduce or increase the last-access time check, but you shouldn’t run this any less often than once a week. If you put off the task for too long, your image cache can grow quite large depending on the number of hotlinking hosts and images.

Website modifications

The only modification required to your website is for images that you wish to exempt from quality degradation. This could include the website icon or similar. You can exclude them by putting them in another folder than your primary image folder or using a query parameter tag.

The following example shows how you can append the exclusion parameter on images intended to be embedded on external websites via the OpenGraph Protocol (OGP), or a Twitter Summary Card:

<meta property="og:image" content="https://www.example.com/images/wet-man-300x300?is=ogp">
<meta name="twitter:image" content="https://www.example.com/images/wet-man-300x300.jpeg?is=twitter">

It doesn’t matter what value or even if you include a value you give the parameter. It’s the fact that it’s set which makes it trigger the exception to the rewrite condition. Of course, this means that anyone can bypass your image hotlink protection system by appending the same parameter to any other image.

However, the likelihood that anyone would realize let alone do this is minuscule, and if it ever where to happen it would more likely be accidental than intentional.

You can take this a step further by counting and limiting how many times an image can be requested from the same external referral domain before you start degrading the image quality.

Apple News, Facebook, Feedly, Flipboard, Google+, Twitter, and other large platforms that consume OpenGraph Protocol data all deploy caching and serve your images from their own servers. An non-well-behaved OpenGraph data consumer would stand out by requesting the same image more than maybe two–five times per hour (or more likely days).