I am building 10MPage.com which captures the state of the internet in 2025. Every internet user is allowed to upload a small image of 64x64 pixels and contribute to this archive. That means you too! ;)

As a solo developer I want to spend as little money as possible in the early stages of this project which is why I host on a cheap VPS.
But I do want to be prepared for what comes next, when this project grows I need to have a plan on scaling the hosting of this application.

The application and the services it requires

The application is written in PHP using the Laravel framework and currently hosted on a single machine. The application heavily relies on background processes for handling the images, finding free places on the grid to place new images and sending e-mails.

The background processes are run through Laravel Horizon which is a queueing system that uses a long lived PHP process. This is running via supervisor to ensure that it is automatically restarted when it stops. Each deployment of the application the process will stop and be restarted by supervisor.

All jobs and caching are done with Redis and the data is stored in a MySQL database. Web requests are served through nginx using PHP-FPM.
The current setup looks like this:

I am aware that there are services like Laravel Forge but they require an extra subscription which is more that my single server costs. For this reason I am setting everything up myself on a simple VPS.

Scaling up

The first and most simple way to scale up is to upgrade the VPS to a bigger machine. This can be done within a few minutes and has some downtime. But it's a good way to buy some time if scaling is needed due to high load to setup other servers.

To scale even more I need seperate servers. I'd like to do this with as little downtime as possible so here's my plan in (count) steps:

1. Move Redis to a seperate server
2. Add a load balancer
3. Create new worker server(s)
4. Add webservers
5. Old webserver becomes the new database server

Step one, Redis

Downtime: None

I'd start with moving Redis to another server. The web appliction can be configured to temporarily use the local file system for caching and the job queue can be temporarily shut down. I'd setup a new server and add it to a private network. I am not bothering with clustering Redis as Laravel Horizon does not support Redis clusters.
After Redis is moved we can turn it off and uninstall it on our original server.

At this point I'd have two servers, one Redis server and one web/database/worker server handling the rest.

Step two, load balancer

Downtime: None

In order to load balance across multiple web server we first need a load balancer. I've got experience with HAProxy so that is what I'll be using. It has a little more options than nginx for load balancing such as active health checks and more load algorithms.
This server will also handle SSL termination.

For this second step I will still be running the web server on our first server. After this load balancer is setup I will change the DNS to point to this server.

At this point the setup has three servers, one Redis server, a load balancer and our first server still handling the web requests, workers and database.

Step three, seperate worker servers

Downtime: None

As I explained before, the background worker is essential for running this project. It needs to talk to Redis and the database to do it's work and can be turned of for a little while. The application is designed so that it will always start off where it was last left. This is done by using states, for example a pending tile can have the state 'place' so that the worker knows that it must place that tile. After placement the status will change.

Laravel Horizon is also designed to run on multiple servers, that means that I do not have to shut down the current worker when adding a new one. The process of seperating this to another server is straightforward. We create a new server, deploy and configure the application and use supervisor to run the worker.

After that we can shutdown the worker on the original server. Scaling workers is as easy as replicating a worker server.

Step four, multiple webservers

Downtime: None

We've already added a load balancer, to seperate the webservers we can follow the same steps as the worker server. But instead of running Horizon with supervisor the server will serve the application via HTTP using nginx and FPM. Also here, when we've got one server we can simply replicate it. The only configuration that has to be done is adding the webservers to the load balancer.

At this point we have a single redis server, a single database server, a single loadbalancer, seperate worker servers and seperate web servers.

Set five, single database server

Downtime: Few minutes

The original server now has only one function, the database. In order to keep it clean I will remove any other software that is not required on this server. Nginx, supervisor, redis and PHP can all go away. As the database server is one of the most important parts of the application I need it to have enough capacity. Clustering is an option but also a lot of work. I think that one big DB server is fine for this project but that does come with a cost. Scaling up will require a server reboot during which the application does not work.

And there we have it, all five steps of my scaling plan for 10MPage. Here are all te steps visualized:

Application deployments

Deployments have to be adjusted, initially there was only one place where the app had to go. But with multiple servers running the application (web and worker servers) it now must be deployed to all of them.

I am using git based deployments, I have a small script included in the repository that deploys the application. For a single setup it does all steps but with seperate servers it must do a few checks to know what to deploy. For example, I can run php artisan horizon:status to see if Horizon is running and only restrt it when it is. I can do the same for PHP-FPM.

The only adjustment is that it needs to run on multiple servers. I will create a simple script that triggers this on all of the servers.

Single point of failure

I am aware that this setup has multiple single points of failure.
The points of failure are:

• Single load balancer
• Single database server
• Single Redis server

The next step in this setup would be to eliminate some single points of failure. The load balancer can be setup redundant on two servers with a floating IP, so when one fails the floating IP can be switched to the other node.

The database and redis server are another story, if I need to scale those up I will write a follow up article on that. But I do not expect it happen.

A note on containers / clusters

I'm a big fan of running applications in containers and the scalability of them is great. But for smaller projects like this I believe that it is overkill to containerize the application and deploy it on a cluster. I'm keeping the hosting setup simple to that I can set it up as quickly as possible in the initial phase of the project. I do not see it needing things like autoscaling. The only reason for using containers in this project would be the ease of setup when scaling to multiple servers. But since this project does not require much I decided that I'd rather make snapshots and clone machines when I need to scale

Conclusion

Scaling 10MPage.com is an exciting challenge that I approach with a pragmatic mindset, keeping the infrastructure simple yet effective. Starting with a minimal setup on a single VPS allows for cost efficiency in the early stages, while the outlined scaling plan ensures a smooth transition as the project grows. Each step, from moving Redis to a separate server to implementing a load balancer, adding workers, and deploying multiple web servers, is designed to minimize downtime and maintain the application’s functionality.

By keeping the hosting straightforward and avoiding the complexities of containerization or clusters, I can focus on building the project and adapting as needs evolve. This deliberate approach balances preparation with simplicity, ensuring 10MPage.com remains robust, scalable, and aligned with its mission to capture the essence of the internet in 2025—one tile at a time.

Thank you for reading this artile, I hope you've learned something.
If you did, why not add your favorite programming language, crypto coin or your pet to the 10MPage? It is free!