Tag Archives: benchmark

Not all routing is equal

Ran into an interesting issue with my Routerboard CRS226-24G-2S+ “Cloud Router Switch” which is basically a smart layer 3 capable switch running Mikrotik’s RouterOS.

Whilst it’s specs mean it’s intended for switching rather than routing, given it has the full Mikrotik RouterOS on it it’s entirely possible to drop out a port from the switching hardware and use it to route traffic, in my case, between the LAN and WAN connections.

Routerboard’s website rate it’s routing capabilities as between 95.9 and 279 Mbits, in my own iperf tests before putting it into action I was able to do around 200Mbits routing. With only 40/10 Mbits WAN performance, this would work fine for my needs until we finally get UFB (fibre-to-the-home) in 2017.

However between this test and putting it into production, it’s ended up with a lot more firewall rules including NAT and when doing some work on the switch, I noticed that the CPU was often hitting the 100% threshold – which is never good for your networking hardware.

I wondered how much impact that maxed out CPU could be having on my WAN connection, so I used the very non-scientific Ookla Speedtest with the CRS doing my routing:

4735498067

After stripping all the routing work from the CRS and moving it to a small Routerboard 750 ethernet router, I’ve gained a few additional Mbits of performance:

4735587010

The CRS and the Routerboard 750 both feature a MIPS 24Kc 400Mhz CPU, so there’s no effective difference between the devices, in fact the switch is arguably faster as it’s a newer generation chip and has twice the memory, yet it performs worse.

The CPU usage that was formerly pegging at 100% on the CRS dropped to around 30% on the 750 when running these tests, so there clearly something going on in the CRS which is giving it a handicap.

The overhead of switching should be minimal in theory since it’s handled by dedicated hardware, however I wonder if there’s something weird like the main CPU having to give up time to handle events/operations for the switching hardware.

So yeah, a bit annoying – it’s still an awesome managed switch, but it would be nice if they dropped the (terrible) “Cloud Router Switch” name and just sell it for what it is – a damn nice layer 3 capable managed switch, but not as a router (unless they give it some more CPU so it can get the job done as well!).

For now the dedicated 750 as the router will keep me covered, although it will cap out at 100Mbits, both in terms of wire speed and routing capabilities so I may need to get a higher specced router come UFB time.

Ubiquiti Unifi UAP-AC in-home performance

Having completed the most important move-in task of wiring the house for ethernet and installing the Ubiquiti Unfi UAP-AC access point, I decided to run some benchmarks to see what sort of performance I could actually get.

Unifi UAP-AC just chilling out on the roof.

Unifi UAP-AC just chilling out on the roof.

Manufacturers always quote theoretical figures which can make buying hard, but even when relying on public third party tests, a particular device will perform differently in your own specific situation in a house with walls, interference, crappy consumer devices and other causes for them not to hit the theoretical maximum performance.

Ubiquiti claim that the UAP-AC can do a theoretical maximum of 1300 Mbits in 5Ghz and 450 Mbits in 2.4Ghz. Unfortunately public benchmarks for my testing laptop (Macbook Pro Retina 2013) show it capping out at 615 Mbits, so it’s not going to possible to truly test what’s possible with the UAP-AC past that speed.

Rather than a benchmark of the absolute maximum speed possible consider this benchmark a general consumer use case test in a suburban setting to see what the real world performance is going to be like with a common consumer client device.

 

Environment

The testing was done with the following equipment:

  • iperf2 as the network performance tool.
  • Ubiquity Unfi UAP-AC access point (duh)
  • Apple Macbook Pro Retina 15″ late 2013 (Running MacOS Mavericks).
  • Lenovo Thinkpad x201i laptop running Linux and connected via GigE to run the iperf server.
  • Mikrotik CRS226-24G-2S+RM switch, acting as a dumb switch to connect the AP and the Lenovo Thinkpad running iperf2.
  • House

The latest firmware was applied to the access point, all other settings like power modes are on their default values.

The Macbook Pro should be pretty typical of most high end laptops currently available and supports 802.11ac and both 2.4Ghz and 5Ghz bands, sadly I didn’t have any other 802.11ac client devices that I could run tests with – would be very keen to test with a client device that can deliver closer to the theoretical 1300 Mbits, as well as being able to test with multiple devices concurrently.

Because of where I am, there were no other WiFI access points detected (yay for elderly neighbours) so the testing was free from interference by other networks. Other sources of potential interference such as microwaves and wireless audio streaming was powered down.

Whilst they were not particular active, or being used or testing, the following other devices were also connected to the access point during the tests and could of course influence how well the AP performs.

  • Apple Iphone 5s (802.11n 2.4Ghz and 5Ghz capable)
  • Samsung Galaxy Note 2 (802.11n 2.4Ghz and 5Ghz capable)

The house itself also requires note – it’s an old single-story Wellington home with walls containing about 20mm of solid hardwood plus a layer of plasterboard. They’re thick walls and utterly punishing to 5Ghz WiFI which generally doesn’t like solid objects at the best of times.

Finally the AP is roof mounted in the rough middle of the house in the hallway – the downside is that every room gets it’s signal after passing through at least one wall, but it gives me all of house (and some garden) coverage from a single AP.

 

Test Results

I did two distinct tests – one where the client device transmits, then receives and another where the client device transmits and receives concurrently. Both tests were done using UDP.

Respectively, the following were the commands used:

  • iperf -u -c SERVERIP –tradeoff -b 1000M
  • iperf -u -c SERVERIP –dualtest -b 1000M

I performed each test a total of 3 times and the results provided are the averages result of those 3 tests. I’ve also included a column showing the general connection type that was established for most of the testing, however in some rooms it’s possible the laptop swapped between 5Ghz and 2.4Ghz throughout testing which would mess with results.

6 different locations were used:

  1. Wired GigE connection (For control purposes to verify the iperf setup and switch were not impacting performance).
  2. Hallway – This test was holding the laptop and standing directly underneath the roof mounted AP.
  3. Master Bedroom – 1 wall
  4. Office – 1 wall, but signal potentially impacted by house details.
  5. Lounge – 1-2 walls depending where the signal goes.
  6. Dining Room – 3 walls between AP and room.

 

Individual Tests Dual Tests Reported Mode
Location Transmit Receive Transmit Receive
WiredGigE 745.00 803.00 null [1] null [1]
Hallway 565.67 570.33 120.43 379.00 802.11ac 5ghz 64%
Master Bedroom 127.00 394.00 84.40 92.47 802.11ac 5ghz 12%
Office 150.00 341.33 93.37 109.13 802.11ac 5ghz 15%
Lounge 67.47 153.00 32.87 62.30 Mixed [2]
Dining Room 55.33 116.00 33.10 45.30 802.11n 2.4Ghz 47%

The locations in the results are ordered closest-furthest, so these results make sense. The fact that receive performance is often better than transmit makes sense when considering that the AP probably has far better antennas and transmission power than the laptop.

Some specific notes:

  • [1] The dual test of Wired GigE just wouldn’t work – I think this was a bug in iperf2 and it wasn’t vital to solve for the testing, hence no metrics.
  • [2] The laptop’s reported mode when in the lounge would vary. On some tests the AP would report 802.11n 2.4Ghz 50% and other times it would report 802.11ac 5Ghz 0.0% (sic), so I think the laptop was struggling to maintain 5ghz, yet could see it strongly enough to try to switch to it.

 

 

Conclusion?

So am I happy with my purchase? The answer is yes – some other access points might have scored higher performance at range but considering the punishment the walls of the house are delivering to the AP, I’m pretty happy with the speeds I’m getting in the rooms.

Considering my WAN connection is 40mbits down / 10mbits up maximum, even the worst location is still able to max the connection and if I need more capacity to push to the file server, I’ll go use one of the stronger signal rooms or just plug into one of the ethernet ports in every room of the house.

I also intentionally brought Ubiquiti’s hardware since in future it’s likely that I’ll add additional access points around the house to expand the coverage in the areas we end up spending the most time in. The software provided Ubiquiti makes configuration and management of multiple access points very easy which is a big advantage.

At just under $500 NZD, these units aren’t cheap, but compared to the time/cost of running ethernet everywhere, it wouldn’t be a bad investment to install a 3-pack spread around the house to provide maximum coverage if you really want to get as fast as possible without the pain of installing ethernet (well you still have to run it through the attic… but it’s easier than running inside the walls). You’re not going to get GigE speeds, but you should be able to get 500-600 Mbits within the same room as the AP.