Marshall Bowers

Conjurer of code. Devourer of art. Pursuer of æsthetics.

Reviving Drop Seven and a Half Years Later

Sunday, June 26, 2022
913 words
5 minute read

Back in December 2014 I spent a lot of time hanging out in IRC. While spending time in various channels, I found myself wanting to share images with other users.

Unlike Discord—which came on the scene the following year—IRC did not have the same support for easily sharing images inline; all images needed to be shared as links to somewhere on the internet.

This posed a problem for any image that wasn't sourced directly from a website, such as screenshots or locally-produced images. These would first need to be uploaded to a service, like Imgur, before they could be linked in an IRC channel.

Being a college student on winter break, my solution to this problem was to write my own image sharing service. What resulted was a program called drop.

Humble beginnings

drop (stylized as ) allows you to upload files—primarily images—and get back shareable links to them.

At the time, drop took the form of a small Express app. Files would be uploaded to local disk, saved under their MD5 hash with the original file extension, and could then be referred to by URLs that looked like this:

http://drop.elliott.codes/i/faff1d9410adf3161446d00fb0c4d107.jpg

Pardon the http://, Lets Encrypt was only a month into its public release at this point.

Uploading files was done via a built-in administrative UI protected by a username and password. These credentials were read from the program's configuration as a plaintext username and brypt-hashed password.

There were also some limited analytics built in, with each hit of a file getting recorded in MongoDB.

All of this was deployed to my Linode server with the rest of my projects.

The death of drop

Over time my usage of drop declined. I was hanging out in Slack and Discord communities instead of IRC, and I also moved away from hosting my own tools.

Unfortunately I don't have a good record of this decline, but the outcome speaks for itself: drop was dead.

But now, after seven and a half years, I decided to revive this project.

Revival

Over the weekend I rebuilt drop from the ground up, as what I have affectionately (and most originally) named "drop v2".

This time around I decided to use Rust. Underpinning everything is a Rocket server, powered by Rocket's brand new support for async Rust.

Rather than being stored on local disk, drops are uploaded to an S3 bucket. Each drop is assigned a ULID as its ID. This has some handy benefits, like being able to use the timestamp component to get the time the drop was uploaded.

There's also an administrative SQLite database for managing users and API keys for interacting with the drop API. Interactions with this database are done through the excellent sqlx crate.

sqlx allows you to write queries in raw SQL that are then checked at compile-time for syntactic and semantic correctness against the actual database schema.

The rewrite takes an API-first approach to drop management. For instance we can upload a file like so:

curl -X POST https://drop.maxdeviant.com/drops \
    -H "Authorization: Bearer <API_KEY>" \
    --data-binary "@some-image.png"

We can also upload plain text as a drop:

echo "A wild drop appeared" | \
    curl -X POST https://drop.maxdeviant.com/drops \
    -H "Authorization: Bearer <API_KEY>" \
    --data-binary @-

Either way, the end result is a URL to a drop:

https://drop.maxdeviant.com/drops/drop_01g6hevwpk80xyv4e3eqt1dncn

We can curl this drop to see what we get back:

λ curl -i https://drop.maxdeviant.com/drops/drop_01g6hevwpk80xyv4e3eqt1dncn
HTTP/2 200
content-type: text/plain
server: Fly/9ece5bcd (2022-06-21)
x-frame-options: SAMEORIGIN
x-content-type-options: nosniff
permissions-policy: interest-cohort=()
content-length: 21
date: Mon, 27 Jun 2022 03:30:17 GMT
via: 2 fly.io
fly-request-id: 01G6HKBE29WAAPZ71843J47V66-lga

A wild drop appeared

drop uses the magic_tree_mini crate to introspect the drop's binary contents to determine its MIME type. The MIME type is then used to set the Content-Type header on the response, which is what ultimately allows the browser to correctly display the drop.

This introspection is the same way that the file command works.

As you might have noticed from the response headers, all of this is running on Fly.

What's next?

My goal for this weekend was to have a live version of drop up and running again, and that's exactly what I did. The code is a bit of mess and I spent far longer debugging Dockerfiles than I care to admit, but I'm proud to say I shipped the dang thing!

I'm looking forward to start using drop and improving it as I go.

Some things I'd like to look at in the short term:

  • Clean up the code
  • Add API key scopes to help limit the impact of an API key leaking
  • Either go all-in on server-side SQLite or switch to Postgres
  • Figure out where interactions with S3 can be optimized (or even eliminated, through caching)

I'm building drop in the open, so have a gander at the repo if you'd like to follow along (again, apologies for the current state of the code).

If you want to check out the original code, see the v1.0.0 tag on GitHub.