Networking with Ubiquiti

29. June, 2021 • 5 min read • Homelab

Upgrading to a 10GbE Network

I'm a data hoarder; that's nothing new to people who know me. This year around, I drove my passion a little bit further and streamlined my setup. The following articles will be a multi-part series introducing various areas such as:

Choosing the hardware

Who hasn’t struggled with their network setup at some point? From failed routing to dropped WiFi connections, networking done right is complex. I hope I finally learn how to set up everything correctly with proper monitoring to debug where the 🤬 package drops come from at school…

I started with the typical vendor-specific routers with an included WiFi module. The standard setup everyone gets when ordering an internet product from UPC Cablecom, Swisscom and the sorts.

However, once Fiber became available at my place, I immediately switched to Fiber7. It was not just cheaper but offered better uptime at higher bandwidth and excellent support. Though not everything was a 🌈. At the same time, my hate-love relationship started with the new Turris Omnia router. Open hardware is incredible, but that piece of 💩 had one of the worst WiFi modules installed I’ve ever come across. It just didn’t work most of the time. It did everything else flawlessly.

That was the moment where I decided to look for an alternative setup. I looked at AmpliFi, Linksys, TP-Link and the sorts settling on Ubiquiti. Why? Cause it’s simple, affordable, provides a unified experience, an extendable mesh network, and it just works.

The stack

The first components I bought were an UniFi nanoHD together with a Cloud Key. The UniFi nanoHD and Cloud Key directly connected to my Omnia Turris. I also purchased an Unifi PoE 8-Port Switch to allow all my other LAN capable devices to connect to my router (as I ran out of ports). So far, I was pleasantly surprised by how seamless everything worked.

I still got package losses here and there due to my Turris Omnia. I decided to replace the remaining infrastructure entirely with Ubiquiti hardware once I started my server setup, ending up with:

  • 1x UniFi nanoHD
  • 1x UniFi Switch PoE 8 (150W)
  • 1x UniFi Switch 48
  • 1x UniFi Switch XG 16 (10-gigabit networking)
  • 1x UniFi Security Gateway Pro 4 (new router)
  • 1x UniFi Cloud Key Gen2 with Rack Mount (upgrade)

The nanoHD and 8-Port PoE switch were the only components I bought brand new. I acquired the remaining devices either through Tutti or Ricardo.

The 48-port switch is slightly overkilled but ended up cheaper than the 24-port candidates ¯\_(ツ)_/¯.

The setup

Now my Fiber connection goes directly via SFP into the Security Gateway Pro. The second SFP connection is used as an uplink to the XG 16, which in turn has another SFP connection to the 48-port switch.

The XG 16 has two 10-gigabit connections to the storage server and another two 10-gigabit connections to the virtualisation server. That connectivity allows both to communicate super fast with each other. The additional NICs connect each via a 1-gigabit connection to the 48 port switch. The different links are mainly for custom VLAN configuration through VMware ESXi for some of the virtual machines.

I also plan to play around with custom VLANs from the Ubiquiti Administration Interface further to separate my internal LAN from the WAN where necessary.

There are also additional cables connected to the 48-port switch for general management, monitoring, power failure, etc.

The bottlenecks

One significant drawback in my whole setup is the connection from my server in the cellar to the network in the apartment.

I use two Zyxel PLA5456 Powerline connectors as there is no direct LAN cable. Even though they advertise as 1800MBit/s, the reality is around 1000MBit/s. Which is fine for my needs. Though I’m looking forward to proper cabling in the future 🥳

Checking the speeds

Performance tests were next in line. I tested the following scenarios, the “Client” being a computer inside the apartment:

  • Storage to virtualisation server and vice-versa
  • Client to the storage/virtualisation server and vice-versa
  • Client to another Client

I did that using iperf3 and speedtest.net. Here a rough outline of the procedure:

sudo apt install iperf3
sudo apt install speedtest-cli

Then start iperf3 on the target server and run the speed-tests on your source device:

# target server (e.g. 192.168.1.10)
sudo apt install iperf3
iperf3 -s -p 8088

# source device
speedtest-cli
iperf3 -c 192.168.1.10 -p 8088

So what’s next

With the network ideally set up, I’m ready to set up the software for managing my digital assets. We’ll cover this in the next section.

‘Till next time!