Recently added: X-SendFile support for Apache

By Dave|GeekISP, Tips

One of the recent additions we’ve had at GeekISP is mod_xsendfile support in Apache.  This module allows any Apache-based server-side language (e.g. PHP, any CGI program, etc) provide fine grained control and app logic over static files while still allowing Apache to do what it does best – serve static files quickly and efficiently.

First, some background.  It’s a fairly common web construct to want to provide some sort of extra control over a static file.  As a contrived example, consider a PDF your customer has paid for and now wants to download.  You want to make sure only paying customers can download the file, but at the end of the day, it’s just a regular PDF.  The standard PHP technique for this would be to use something like the ‘readfile()’ function, which works great for small files.  For large ones it’s not so great since PHP reads the entire file and then passes the entire buffer to the webserver for output.  That code would look something like this (adapted from the PHP readline docs):

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16

$file = "super_secret.pdf";
 
if (customer_has_access($file)) {
    header('Content-Description: File Transfer');
    header('Content-Type: application/octet-stream');
    header('Content-Disposition: attachment; filename='.basename($file));
    header('Content-Transfer-Encoding: binary');
    header('Expires: 0');
    header('Cache-Control: must-revalidate, post-check=0, pre-check=0');
    header('Pragma: public');
    header('Content-Length: ' . filesize($file));
    ob_clean();
    flush();
    readfile($file);
    exit;
}

The big problem with the above is that if your file is large (say, a 100 meg MP3) you’re going to need 100M of ram to hold that file in memory before the webserver gets a chance to write it out. That’s 100M per concurrent user. Typically to get around that problem, folks will read and write the file in chunks and thereby avoid the memory issues. That solution, however, means that your PHP process is tied up sending the file (something Apache can do on it’s own!) instead of handling logic in your app. If a client is downloading the file slowly, PHP is stuck waiting. This example demonstrates that problem:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26

$path = $_SERVER['DOCUMENT_ROOT']."/path2files/";
$fullPath = $path.$_GET['download_file'];
 
if ($fd = fopen ($fullPath, "r")) {
    $fsize = filesize($fullPath);
    $path_parts = pathinfo($fullPath);
    $ext = strtolower($path_parts["extension"]);
    switch ($ext) {
        case "pdf":
        header("Content-type: application/pdf");
        header("Content-Disposition: attachment; filename=\"".$path_parts["basename"]."\""); // use 'attachment' to force a download
        break;
        default;
        header("Content-type: application/octet-stream");
        header("Content-Disposition: filename=\"".$path_parts["basename"]."\"");
    }
    header("Content-length: $fsize");
    header("Cache-control: private");
    while(!feof($fd)) {
        $buffer = fread($fd, 2048);
        echo $buffer;
    }
}
fclose ($fd);
exit;
}

At GeekISP we are using PHP as in FastCGI mode, which means your site has a certain number of backend PHP processes running at all times. It also means there’s an upper limit on the number of processes for your site, and if your PHP processes are all wasting time feeding bits to Apache in spoonfuls then you’re going to hit that limit the moment you have a remotely popular file. X-SendFile let’s you avoid this and is also a memory-efficient solution. The X-SendFile solution looks like this:

1
2
3
4
5
6
7
8
9

$file = "super_secret.pdf";
 
if (customer_has_access($file))
{
    header("X-Sendfile: $file");
    header("Content-Type: application/octet-stream");
    header("Content-Disposition: attachment; filename=\"$file\"");
    exit;
}

When Apache sees the X-SendFile header, it’ll apply all the usual logic for sending the file, which means that you don’t have to specify any cache control headers at the PHP level – the defaults come from Apache and you can supplement these with a htaccess file if desired.