Improving Performance by Writing a Custom Static Asset Server in Rust

The ability to serve static assets is critical for many SaaS products, including Shopblocks. Many of these products will store the files to be served in a segmented way, perhaps across multiple drives, and in multiple folders.

At Shopblocks, our setup means that each customer has their assets stored on specific mounted drives, and in specific folders per customer. For example, a customer may have their images stored on a server STORAGE-001 and the file path for the customer might be /var/www/storage/STORAGE-001/files/customers/<id>/images/.../image.jpg.

This means there are three specific pieces of information required to locate the file and serve it:

1. The storage server required

2. The customer ID

3. The file path and filename

We will be ignoring how to obtain 1 because it may be different for every person. In the script below, this will simply be ignored as a concept.

Because of the dynamic manner in which the files are stored, many "ordinary" ways of serving static assets, such as using Apache or NGINX, are trickier, or impossible, without custom modules. At Shopblocks, we initially had our requests map through Apache with a custom virtual host per customer with the dynamic configuration hard coded per virtual host.

This didn't scale too well and made retroactive changes harder to manage (such as migrating a customer's data from one server to another).

A second attempt was made using Apache and a custom PHP script to serve the files. This was easy to make dynamic, but slow. All requests required a full load of Apache just for the serving of a simple image or text file. So we looked for another way.

A few attempts were made with htaccess files, and HAProxy with LUA scripting. These didn't pan out for us.

Eventually we settled on using Rust. We had used Rust for other small projects within the platform and were getting a little more confident in its ability to solve problems for us in an efficient way. One of the members of the development team set aside a couple of weeks to try to come up with a working prototype for serving static files quickly using Rust.

Below is the simplified version of that script.

If you are unfamiliar with Rust, then I would recommend reading The Rust Programming Language. You can install Rust from rustup.rs.

This script makes an assumption that there is a layer in front of the static server script that will assign a custom HTTP header containing the customer's ID (or identifier required to locate customer files on a disk). For testing purposes, I will fake this using Postman to manually set the header.

Our structure for this project will be:

.
├── src
|   └── main.rs
└── cargo.toml

cargo.toml

[package]
name = "static-server"
version = "1.0.0"
authors = ["Alex Bowers <alex@shopblocks.com>"]
[dependencies]
hyper = "0.10.12"
iron = "0.5.1"
regex = "0.2"
staticfile = "0.4.0"
mount = "0.3.0"
time = "0.1.36"

Once you have set your cargo file up with its dependencies, run cargo build and it will download and install those dependencies for you.

main.rs

extern crate iron;
#[macro_use]
extern crate hyper;
extern crate mount;
extern crate staticfile;
extern crate time;
extern crate regex;
use iron::prelude::{Iron, Response, Request, IronResult};
use iron::Handler;
use iron::status::Status;
use hyper::header::{
    AcceptRanges,
    RangeUnit,
    CacheControl,
    CacheDirective,
    Connection,
    Expires,
    HttpDate,
    LastModified,
    IfModifiedSince,
    AccessControlAllowOrigin
};
use time::{Duration};
use std::fs::{self};
use std::path::Path;
use staticfile::Static;
use mount::Mount;
use std::time::UNIX_EPOCH;
header! {(XCustomerId, "X-Customer-ID") => [usize]}
fn main() {
    fn files(request: &amp;mut Request) -> IronResult<Response> {
        if let Some(&amp;XCustomerId(id)) = request.headers.get::<XCustomerId>() {
            let id_num: usize = id;
          // For testing, use a local folder such as ~/static-server/{}/files and create some numbered folders with files within.
          // From there, you can use Postman to test the request by sending through the `X-Customer-ID` header with the file path and seeing the response.
            let storage_path = format!("/var/www/storage/STORAGE-001/{}/files", id_num);
            let mut mount = Mount::new();
            mount.mount("/", Static::new(Path::new(&amp;storage_path)));
            match mount.handle(request) {
                Ok(y) => {
                    let mut response = y;
                    response.headers.set(AcceptRanges(vec![RangeUnit::Bytes]));
                    response.headers.set(CacheControl(vec![CacheDirective::MaxAge(86400u32)]));
                    response.headers.set(Connection::keep_alive());
                    response.headers.set(Expires(HttpDate(time::now() + Duration::days(365))));
                    response.headers.set(AccessControlAllowOrigin::Any);
                    let new_path = request.url.path().as_slice().join("/");
                    let file_path = format!("{}/{}", storage_path, new_path);
                    match fs::metadata(file_path).ok()
                        .and_then(|md| md.modified().ok())
                        .and_then(|t| t.duration_since(UNIX_EPOCH).ok())
                        .map(|t| time::strptime(&amp;t.as_secs().to_string(), "%s").ok()) {
                        Some(Some(timestamp)) => {
                            response.headers.set(LastModified(HttpDate(timestamp)));
                            if request.headers.has::<IfModifiedSince>() {
                                if let Some(&amp;IfModifiedSince(HttpDate(if_modified_since))) = request.headers.get::<IfModifiedSince>() {
                                    if timestamp <= if_modified_since {
                                        return Ok(Response::with(Status::NotModified))
                                    }
                                }
                            }
                        },
                        _ => {}
                    }
                    Ok(response)
                },
                _ => {
                    Ok(Response::with(Status::NotFound))
                }
            }
        } else {
            Ok(Response::with(Status::BadRequest))
        }
    }
    println!("Starting static server on port 8080");
    // Change 0.0.0.0 to be your servers private IP address, or use UFW to restrict traffic appropriately.
    let _server = match Iron::new(files).http("0.0.0.0:8080") {
        Err(e) => {
            println!("Static server failed to initialise because: {}", e);
            std::process::exit(1);
        },
        _ => {}
    };
}

There is a fair amount to go over there, but it's remarkably simple when you delegate tasks to specific crates and break down the script.

The top bit imports and notifies Rust that we will be using external crates from crates.io. We then notify Rust about the parts of the crates we will be using in our script.

We add an expected HTTP header using the macro from hyper, header! {(XCustomerId, "X-Customer-ID") => [usize]}, to allow us to read an HTTP header called X-Customer-ID.

In our main function, two things happen.

First, we define a function called files that will eventually return with an HTTP response, either 200 OK, 304 Not Modified, 404 Not Found, or 400 Bad Request.

Second, we then set up the Iron server to listen on port 8080, passing all requests through to the files function. We set up mount to map any request starting from / as a route to be our storage path as defined using our customer ID. We let mount handle the requested asset, and if it is found Ok(y), then we modify a couple of response headers to set up the cache and access control allow origins. We also read some information from the disk about the file to determine when the file was last modified, to set that header, too.

If a request comes with an IfModifiedSince header, we compare the dates -- if the file has not been modified, return a 304 Not Modified response. Once all the response headers are set up, we send the asset to the browser using Ok(response).

And there we have a working static asset server.

!Sign up for a free Codeship Account

To run it yourself, run cargo build and then execute the created executable.

To test it, create a test file following the convention used in your script storage_path variable, and send a request using Postman.

The actual code we use at Shopblocks is a little more complex such as obtaining more information from customer configuration settings to help locate the files, but in principle, it is the same.

These are the numbers from some of our production traffic at Shopblocks. As you can see, the static server is able to locate and return the correct file to the user often in less than 35ms. This was a significant improvement over our previous setups.

By having this script custom coded, it adds the facility for expansion in functionality. For example, you could now quite easily expand this script to serve a missing image placeholder in the event that a specific image is not found, instead of returning a 404 Not Found.

You could on the fly convert and serve other content types than what's requested if the user's browser supports it. For example, a request for a JPEG could respond with a WebP image if the Accepts header indicates that WebP is supported.

You could serve up mobile optimized images (eg, lower quality JPEG, resized images) if you determine that to be suitable (such as a request originating from a mobile device).