The articles in this series:

Before deploying home server, difference between GPU hiring and Colab or Kaggle

Selection of home server chassis and purchasing advice for server components

What Operating System should you choose for your home server and why

My initial purpose was to build an AI training server, but since the composition of servers is similar, this article is still very universally applicable. Anyone interested can draw inspiration from it, and I also hope my article can save you some detours.

Similarly, this series may mention brand names, but it does not contain any advertisements throughout, so feel free to consume.

Before purchasing a server, we need to understand why we need one. For me, I need a server to train AI models for extended periods. Of course, servers can do more than just AI tasks. Although the term “server” sounds much more advanced than a computer, in reality, a server is simply a computer designed for long-term operation.

How do you determine if you need a server? Basically, if you have services that need to run for a long time, then you need one. However, this article focuses on “high-performance” junk servers. If your server needs are for quiet, compact, low-power solutions, then this article might not be very relevant to you. I’ll also provide some solutions for low-power scenarios later on.

So, let’s get started.

Server Chassis and Accessories Selection

For consumers in China, Xianyu is actually a very good platform. Of course, the same goes for friends abroad. Domestic second-hand goods trading websites are the best places for us to choose the main accessories for servers. The reason is actually very simple. First of all, the price of buying a brand new server is very high. After all, servers themselves are mainly for businesses rather than individual consumers. For them, economies of scale are very obvious. Buying several hundred units at once can lower the price. So unless you have very abundant funds, it is recommended to start with second-hand platforms and purchase servers discarded by enterprises. Large enterprises update their equipment very quickly, so you often see servers that are one or two generations behind on second-hand platforms, but the price is half or even lower than when they were newly released.

About the Chassis

Make sure you choose the chassis before selecting other components, otherwise there may be compatibility issues later. Don’t ask me how I know, returning such heavy items like chassis back and forth is very time-consuming and costly.

Actually, server chassis are basically similar in quality, so the brand doesn’t matter much. As long as it meets your needs, brands like DELL, HP, Huawei, Inspur, and Lenovo are all worth considering. Below are some points to consider when purchasing:

• Supported CPU Models
  This can be said to be the most important point, as different CPUs will directly affect your costs. If you have high requirements for the CPU, you basically need to go for the new Intel Xeon platform, which will also increase the cost of your chassis. But if you don’t have specific requirements for the CPU, the E5 platform is still the best choice for us budget enthusiasts, after all, more cores are powerful. However, here’s a point to note: one pit I encountered at the beginning was that the E5 page has many types like V1, V2, V3, V4, so it’s important to clarify this. I bought two V4 CPUs at first and later found out they couldn’t be installed.The image below shows the E5 V4 series CPUs.
  The image below shows the new Intel Xeon series CPUs.
• Supported Graphics Cards
  This requirement may be less common because there are few instances where we install graphics cards in 2U chassis. However, I happen to be one of those peculiar users because 4U graphics card servers are much more expensive, and I only need to insert one graphics card, so choosing 4U would be a waste. Here I have to mention another pit I fell into: the first server I purchased couldn’t install the graphics card I bought because they were incompatible. It’s important to note that some servers are sold without expansion cards, but for most 2U servers, the graphics card is inserted on the expansion card. When comparing prices, don’t forget to check if the server comes with an expansion card.The image below shows 4U graphics card or storage servers.
  The image below shows a typical 2U server.

These are the recommendations I give for these two points. Before purchasing, be sure to check the CPU and graphics card models supported by the server’s official website. The official website of the server provides detailed information, which is not difficult to understand. Basically, checking the device support list can solve most compatibility issues.

• RAID Cards
  RAID cards may or may not come with the server you purchase, so when comparing prices, don’t forget to consider the value of RAID cards.
  RAID cards are generally divided into two types: SATA RAID cards and SAS RAID cards. Note that SATA RAID cards cannot connect to SAS hard drives, but SAS RAID cards are compatible with SATA. Currently, SAS hard drives on the market are generally slightly cheaper than SATA drives, but there’s not much difference in performance. My suggestion here is unless you’re certain you need SAS drives, go for SATA drives. It’s worth mentioning that solid-state drives are basically SATA interfaces. For most people, one RAID card is enough, as one RAID card can typically support 8 to 12 hard drives.Below are some photos of RAID cards, and there are many ways to install them. Some servers have reserved positions for RAID card installation, while others require installation on PCIe expansion cards.
• Hard Drives
  First, you need to decide between mechanical and solid-state drives, but regardless of which type you choose, I don’t recommend buying from second-hand platforms for data security reasons. Here are my personal recommendations:
  1. For those who don’t mind spending money, go for all solid-state drives, preferably with support for NVMe booting if possible. The performance difference between solid-state and mechanical drives is significant.
  2. For most people, I suggest preparing two solid-state drives: one with a larger capacity, ideally 1-2TB, for the system, and another with half the size of the first one for storing temporary data that requires quick access. Then, purchase mechanical drives as needed.
  3. If budget is a concern, still opt for a reputable platform to purchase a 512GB to 1TB solid-state drive for the system, and choose mechanical drives for other storage needs. If you’re not concerned about data loss or hard drive failure, you can try your luck on second-hand platforms.

Regardless of the chosen configuration, it’s crucial to have at least one SATA or NVMe solid-state drive as the system drive. Currently, using mechanical drives as system drives is not recommended. Also, I recommend checking if the server supports NVMe booting when purchasing. If it does, go for NVMe directly.

• Mechanical Drives
  Overall, mechanical drives offer significantly lower storage costs compared to solid-state drives, typically around one-fourth of the price. I don’t recommend choosing second-hand hard drives unless you’re really tight on budget or don’t care about the data stored on them.
• Solid-State Drives
  Solid-state drives come in various types: SATA, NVMe (PCIe 3.0, PCIe 4.0). Here’s a brief overview: SATA solid-state drives generally have read/write speeds around 500MB/s. PCIe 3.0 NVMe drives offer speeds of around 3000MB/s, while PCIe 4.0 can reach up to 7000MB/s. Since the price difference between solid-state drives is not significant, go for the fastest option supported by your server. Note that NVMe drives need to be plugged into PCIe expansion cards, so ensure the server’s expansion card supports the PCIe protocol. Using a PCIe 4.0 drive in a PCIe 3.0 slot would be a waste of resources.Below are images of NVMe and SATA solid-state drives:

  

  

• Memory Modules
  This is straightforward; just make sure your server supports either DDR4 or DDR3 memory modules. Avoid making the same mistake I did by trying to save a bit of money. I initially bought DDR3 memory for a server that supported DDR4, only to find out they were incompatible. Most servers come with ECC (Error-Correcting Code) memory by default, which may not be essential for regular users, but since it’s included, why not make use of it?
• Power Supply
  Power supply isn’t a big concern for servers as they typically come with high-quality, platinum-rated power supplies. You don’t need to worry much about their quality. Most servers support redundant power supplies, and I recommend going for this option. Used power supplies for retired servers are quite affordable, so having two spares wouldn’t hurt.

To be continued

Here basically a server is almost set up. In fact, you may not have noticed that there is not much difference between a server and a regular computer. This also confirms what I said before, the only difference between a server and a computer is that a server is designed to be left on for a long time without shutting down.

Next time, I will talk about the software configuration of the server, what operating system to install, and which system suits you.