How my site went from 20 hits a day to 5 MILLION in a week for no extra cost

by Samuel Attard

On the 6th of March 2016 I posted on reddit about my new release of my Google Play Music Desktop Player. A few hours later I logged back in and my inbox had over 600 replies, I rushed over to look at my website statistics and saw the server was getting absolutely hammered with requests.

Here began the 12 hours of "just keep the damm site up", the site was being hosted on a $5 a month Digital Ocean droplet and I was amazed it hadn't crashed already. I had set up no caching and it was serving all requests directly. This was partly due to my own laziness and the fact that until now the site had got maybe 20 hits a day...

My process was very simple, I am a student so I had no money, I needed to offload as much traffic and processing as possible to other FREE services. Here are the steps I went through

Setup Cloudflare

For those who don't know what cloudflare is, it's a great service that acts as a CDN for your entire site. It caches your content on it's servers around the world and then serves the content from it's network instead of hitting your server every time. According to their own stats, in the last week and a bit they saved my server over 200GB of traffic and over 5,000,000 requests.

Optimize the actual Site

The site itself shows some cool stats such as total downloads, number of open issues and direct download links for the latest GitHub releases. The problem was these values were being fetched from my own server which was then making API calls to GitHub. This put a LOT of strain on my own server. I moved this logic to client side JS code so each client would request the information from GitHub directly. This meant I no longer needed an API server running on my side as all processing was passed off to the client.

Make GitHub pick up my traffic

So on that picture above you guys might be able to see the sudden dip on the 11th of March. This was when the site was still failing randomly, throwing 500 errors and not responding to some requests. I realized I was serving updates through my Update Server by proxying the download from GitHub.

This means that I was first downloading the release to the server and then sending it to the client. This was beyond inefficient and again adding extra load on my minimum cost server. I instead reconfigured the update server to redirect requests to GitHub instead of proxying them. This is the sudden dip in requests you see in the picture.

What's next?

You can also probably see on that picture that traffic is steadily climbing it's way back up again and I am flat out of ideas. I'm interested if anyone has anything to add in the comments below as to how I can improve the efficiency of either the static site or the Update Server.