How to set up a basic storage server

So, you are a developer and have a great idea that could potentially change the internet world. How do you go about implementing this great idea and what kind of infrastructure is needed? 

This blog post describes a simple solution for creating a storage server, a topic I have been asked about several times over the last few years. 

The required infrastructure for storage servers

In order to create a storage server, you will need the following basic infrastructure: 

2 x front-end servers acting as web and application servers 

1 x back-end server to store and serve the content via web servers 

1 x private network between the front-end and back-end server 

The drawing below gives an impression of such a setup. 

Each server is connected to both the internet and a secure Private Network. Remote Management of the servers goes via the Leaseweb Customer Portal and a secure VPN connection allows us to control the servers from the console. 

Now let’s get started and determine our specifications:

Ordering front-end servers

We‘ll need two dedicated servers to act as the front-end servers and each should have a minimum of 4 cores, 8GB of RAM, and a small disk for the OS. Network wise, the servers must have an internet link, a minimum of 100Mbps port speed, a public IP and Private Networking (1Gbps). Something like a HP DL20 G9 should do the trick. 

We’ll then need to do some simple load balancing and failover, so we have a couple of options for this: 

1. Use a Loadbalancer appliance. This option is probably the most expensive one, but it takes away some of the headaches of configuring load balancing software.
2. Do it yourself and create your own HA-failover solution using local heartbeat scripts and failover IP’s. The biggest advantage is that you then control what is happening and you can also use it to run both your server’s load balanced as active/active. The downside is that it requires a bit more technical know-how.
3. Have someone else do this for you. There are several DNS providers available that offer complementary services on top of their DNS that can help you with that. For example, DNS Made Easy provides health checks for your servers and can do the failover to a (standby) server.

Ordering the back-end server

In this hypothetical example, the storage server requires a minimum of 4TB of storage capacity and 8GB of RAM, and we’re not that concerned about the CPU power. The servers need to be connected to a private backend network, preferably with a minimum of 1Gbps so there is enough bandwidth between them to share data. 

Something like a DL380e Gen8 (12 LFF), which can be customized will fit the bill. 

The disks should be in a RAID-set. The RAID configuration depends on your requirements, but I prefer to use RAID-5 as that gives me better read than write performance and at the same time allows the array to recover data from a failed drive. When a user uploads content to my website, the writing is less interesting than serving content quickly.

If more performance is required, there is the option of using SSD drives or even NVMe SSD drives (although not in a RAID-set).

Deployment of a storage server

Now it’s time to start deploying our solution, so the following steps need to be taken: 

• Deploy the front-end dedicated servers with a Linux distribution of choice. 

• Customize, order and deploy a back-end storage server with enough capacity/disk performance, running a Linux distro of choice. 
• Interconnect all 3 servers on a back-end network. 
• Create NFS shares on the storage server and mount these shares on the webservers. 
• Configure your LAMP stack to host your designed website. 
• Configure DNS with round-robin using the 2 extra assigned public IP’s or as an alternative use a 3rd party for most advanced DNS features. 

I will not go into depth on how to configure each server, as there is plenty of information to be found on the internet on how to do this. See for examples on Ubuntu here: 

Setting up a LAMP Applications stack. 

Setting up NFS shares. 

Have fun with your new storage solution!

 

Editor’s note: This article was originally published on 14 November 2018 and was updated on 29 October 2022.