Getting over 70 TB of storage
I'm a data hoarder; that's nothing new to people who know me. This year around, I drove my passion a little bit further and streamlined my setup. The following articles will be a multi-part series introducing various areas such as:
- Virtualisation using a PowerEdge R820
- Storage with a QNAP TVS-1271U-RP
- Networking with Ubiquiti
- Media streaming with Plex
- Download management with SABnzb and Transmission
- Service management with Sonarr, Radarr, Lidarr, Readarr, Bazarr and Mylar
- Additional tools like Calibre, Ubooquity, Ombi or Organizr
- Monitoring with Checkmk
Choice of storage
I was pleased with my Synology DS1815+ as my primary storage solution for years. It handled the automatic download, categorisation and storage of all my digital assets. It was reliable, and I didn’t have any hardware or disk failure so far. So why did I switch to a QNAP?
The goal was to get a 2U 12-Bay NAS for under 1’000 CHF, including all upgrades, without the storage drives. Thus, it didn’t need to be the newest model. Additionally, I wanted upgradability for additional bays, expandable SSD cache, more memory or better CPUs for the future. Sadly, most vendors are restricted by one or the other technology. However, Synology lagged significantly behind when comparing their models to QNAP as the performance to price ratio seemed to be much better with QNAP’s hardware.
After some further research, I found a QNAP TVS-1271U-RP for a relatively affordable price. It had a stock Dual-core Intel® Core™ i3-4150 CPU with 8 GB memory. I promptly ordered a Quad-core Intel® Core™ i7-4790S CPU and upgraded the RAM to 32 GB. I also ordered two cheap GM512GB mSATA drives that will provide the SSD cache and an M.2 NVMe SSD Express card to add my spare 1TB Crucial SSD as an additional resource for virtual machines. On top of that, I also added a 10 GB NIC 😁, ending up with the following specs:
- Intel® Core™ i7-4790S (4 Cores up to 2.30 GHz)
- 32 GB DDR3L 1600 MHz Memory (4x 8GB modules)
- 12 bay hard drive slots
- 2x 512 GB mSATA SSD
- 1x M.2 NVMe SSD express card with 1TB Crucial SSD
- 1 Gbps integrated quad-port NIC
- 10 Gbps X540 dual-port PCIe NIC
All that together kept me under the 1k range, by like a beer or two. Thanks to some cheap offerings from Tutti and the fantastic prices from AliExpress.
Choosing the drives
I used Western Digital Red for years, but the 2020 scandal with wrong labelling of SMR instead of CMR. I got two 6 TB SMR’s to fill up my Synology from 6 to 8 drives just a couple of years ago, and ever since, the RAID consistency checks take forever to complete. Digitec support did a better job offering me replacements than the WD support, even if they didn’t have to. Cudos to their support! 🥳
These issues led me to switch to 12 x 8 TB Seagate IronWolf (CMR) NAS drives. I didn’t want to save costs there as the data was essential to me. So I waited for a reasonable offer to come along and bought them new at around 180 bucks (the original price was 260 bucks a piece back then).
Setting up the NAS
The QNAP interface was similar to Synology’s “DiskStation Manager” (DSM). QNAP calls theirs “Turbo NAS Operating System” (QTS). Both operating systems offer similar features. The notable difference is that DSM tries to simplify processes and provide fewer options, while QTS allows for basically any modification leading to a cluttered UI. If you are an IT specialist – QTS is for you. Otherwise, go with DSM (and Synology, for that matter).
The setup wasn’t that hard though, choose a user, tweak some settings and be done with it.
Resolving some issues
Though sadly, there was an issue. Every now or then, I had to restart the NAS multiple times to recognise the discs. Sometimes only 8 LED were displayed instead of all 12, meaning that somewhat four brand new drives get lost, and the RAID fails.
QNAP’s support was excellent and responsive. They quickly discovered that I was running on a BIOS version with a disk problem known to the TVS-1271U-RP.
So I followed the provided information to upgrade the BIOS version to the latest, and everything was back in order.
Finally, I updated the firmware and operating system to the latest versions as well.
Setting up the storage
Now I had a working NAS without any issues and upgraded everything to the latest and greatest. Next was to set up the 12 drives, the cache, and the additional M.2 storage.
My storage pool consists of 12 NAS drives. I choose to go with a RAID 6 with two-disk failure tolerance. That gives me ample space and some breathing room. I have two volumes set up, one that holds my data and another one for surveillance recordings.
I also set up cache acceleration using the two 512 GB mSATA SSDs. It speeds up data transfer for the most requested files, which is nice. I never used cache acceleration, so I wanted to play around with it and see if it delivers real benefits. So far, it didn’t disappoint.
The M.2 drive is set up as an additional storage pool to host VMs or VM backups. I don’t use it yet, but it’s reserved for projects in a later stage.
Setting up backups
As most know, RAID is not a backup solution. If more than two disks fail, all data is gone. Even if just one disk fails and gets replaced, the rebuild can cause a cascading failure. To remedy that, I added two 4TB USB-Drives to the NAS for “not so important data but still important enough to be saved somewhere else” and a remote backup solution using Backblaze for my most treasured data.
QNAP offers an extensive interface to set up backups using “HBS 3 Hybrid Backup Sync”.
Accessing the volumes
After all that, the only thing left was tweaking some settings and figuring out how to connect to the drives.
Also, that was pretty simple. I headed over to “Control Panel” > “Network & File Services” and enabled “Apple Networking” and “NFS Services”. The latter will be used later for the VMs.
After that, I can connect to the drives through finder with specific users set up for network access.
I transferred the data on my Synology NAS via
rsync to a designated folder and then rearranged everything to match my taste.
So what’s next
Now, I have a running QNAP TVS-1271U-RP with the latest firmware and OS. All my data is migrated to the new NAS, and everything was working, fast and reliable.
The following section will deal with my networking setup using Ubiquiti hardware to connect all the data. And yeah, there are some bottlenecks 😭.
‘Till next time!