The RK3399 SoC has been used in a huge variety of devices. From Chromebooks to SBCs, routers, TV boxes, and even the upcoming Pinebook Pro. With its six cores, gigabit ethernet, USB 3.0, and PCIe support, it’s clearly an SBC powerhouse in the ARM world. I’ve often wondered what kind of benefit was actually gained from using an SSD with an ARM CPU, though. So, I put the Rock Pi 4 to the test.
We’ve covered the Rock Pi 4 before, which you can check out here. It’s a very solid SBC with the RK3399 CPU, which has been modeled in the image of the Raspberry Pi. It bears an almost identical footprint by using the known and beloved Raspberry Pi form factor. This becomes all the more impressive when considering that it comes with six cores, up to 4GB of RAM, PCIe x4 in the form of an M.2 connector, 2x USB 3.0, USB C, 2x USB 2.0, gigabit ethernet, 5GHz wireless, Bluetooth, and a Raspberry Pi compatible GPIO array. There’s more connectivity on this board than any project can sensibly make use of.
For a long time, people have struggled with using Raspberry Pis because they’re very slow. Running a system update for the first time in a while can tie up your Pi for an hour. Especially if you’ve neglected to get yourself a decent microSD card. Many microSD cards can maintain somewhat decent data rates now, even the cheaper ones can sustain about 10MB/s. That’s alright for smartphones and cameras that only need occasional access for small files. But when running an operating system, there are thousands of file operations that can happen during boot up. A single round of updates can easily include tens of thousands of file operations. This tends to be very burdensome on our poor microSD cards, so much so that many ARM-based Linux distributions will actually cache larger amounts of data before committing to the storage. This is in part to help reduce wear on the NAND flash but also serves to reduce bottlenecking.
I’ve always wondered how an ARM SBC would perform if the storage wasn’t the bottleneck. Many people have seen new life breathed into a laptop or desktop with an SSD (and if you haven’t, you should check it out, it’s a world of difference). So, I was curious if this was the same case, or if the ARM desk experience hadn’t breached this sluggish state yet. When I originally reviewed the Rock Pi 4, I was provided with an eMMC module. This was much faster than the microSD card when being imaged. But it was still a little slower than I had hoped.
I was contacted by Radxa, the manufacturer of the Rock Pi 4, to try out the M.2 extender board which provides proper mounting for NVMe SSDs given that there simply is not enough room on board to accommodate it. I jumped on the opportunity to test my concerns about the state of ARM SBCs.
So, I ordered myself a 1TB Intel 660p. The 660p line consists of QLC NAND and is certainly not as mature as TLC or MLC processes, but still nets you well over three times SATA speeds. Given the incredibly low price of $130CAD (less than $100USD), it’s an amazing deal, and the fact that it comes with a five-year warranty just adds to the value.
I assembled my kit, to find that I was missing a couple of parts, but I made do. I figured that I enjoyed using Manjaro on the Star Labs Labtop, so I gave Manjaro ARM a try. Well, that didn’t go as planned because there, apparently, isn’t PCIe support in the mainline Linux kernel for the Rock Pi 4 yet. As a result, the nvme0n1 device doesn’t show in /dev/. So, I downloaded the Debian image that’s provided by the Radxa team. Once booted, I was able to see the device.
This is where things get a little more technical. There are tons of utilities for flashing USB and SD card media from your computer. After this, you just plug it into your SBC and boot it. Because of this very well established paradigm, it’s very uncommon for ARM-based Linux distributions to include an installer of any kind. They’re intended to be run indefinitely from the initially imaged storage device. This is not an ideal system for NVMe drives, unless, maybe, you have one of those cool, open benchmarking cases that you see around online. Because of this, you have to write to the micro SD card and then after you’ve booted up into that, clone it over to your SSD. I recommend using dd for this (or dcfldd if it’s available on your system). It’s a very easy and incredibly powerful tool that is typically available on all UNIX and UNIX-like systems. This might appear daunting, but if you practice the command a few times, it becomes very easy. Keep in mind, though that this process can take a while, especially if you happen to have a larger capacity card.
After my clone completed, I issued the sync command (which clears all the write buffers to storage) then checked that the partition table on the SSD was as expected. With the partition table verified, I ran the reboot command, which wasn’t found… so I tried the shutdown command, which also wasn’t found (seriously, Debian?). Then I just used the reboot button in the application menu. The menu lingered for a few seconds, the display went black for like two seconds, and then the desktop was visible again. I was a bit annoyed. Without the reboot and shutdown commands and with having a faulty reboot button, how was I supposed to gracefully restart this SBC?
Well, when I opened up the application menu again, it was significantly faster. I thought there was no way the thing actually rebooted. So, I ran the good ol’ mount command and looked for the trusty root partition. Lo and behold, it was on the NVMe drive.
This is where things started to get interesting. I was able to install the Gnome Desktop Environment (over the existing LXDE) and it ran alright. Then I needed to benchmark the drive. This is where I ran into some trouble again. You can’t just benchmark a live partition in gnome-disk-utility, for whatever reason it requires the ability to unmount it (my guess is that it wants to remount it with specific options). In any case, it makes sense that you can’t just unmount your root partition (but you should be able to pass the remount option, which makes this more suspicious).
The Rock Pi 4 doesn’t appear to have any type of boot device selection, it just opts for the NVMe drive over anything else if it’s available. Now, I didn’t want to go and try hot-swapping my brand new drive, so I had to overwrite the GPT on it and then boot off the microSD card again to run the benchmark from there. So, using DD again I wrote a single megabyte to the beginning of the drive, effectively erasing the GPT.
Once fully booted off of the microSD I had to install gnome-disk-util and start it up. I really wasn’t sure what to expect in terms of numbers, but I figured it would top out somewhere short of SATA speeds. The 660p has both read and write speeds of up to 1,800MB/s. I was sure that the RK3399 would be far from saturating it, though, because that’s a lot of throughput.
With a benchmark setup of 100 samples of 1000MB each, the average read was 673MB/s and the average write speed was 789MB/s. The average access time across 1,000 samples was 0.06 milliseconds. These are incredibly impressive numbers. The RK3399 is a beast in terms of SBCs, but it still pales in comparison to most x86 CPUs. It might — just might — compare to some Atom CPUs.
In light of this, the (re-) boot time of about two seconds is stunning. Now, this isn’t all just CPU power, the Debian spin is very optimized for this board and has many unnecessary bits removed (which is a must when you’re running on an SD card). So, there’s more than just blazing storage going on here. It’s the combination of lightning-fast storage and, just as importantly, a well-optimized system.
After speaking with the Radxa rep, he told me I should be able to get more performance out of the Rock Pi 4. After following his instructions of turning on PCIe gen 2 mode, I re-ran the benchmark. I got a whopping 1.2GB/s read speeds and a staggering 1.4GB/s write speeds, with the access time of 0.06 milliseconds unchanged. The higher write speeds have to do with the 660p’s write cache, which treats a portion of the QLC as SLC.
If you can get your hands on an RK3399 based board, I highly recommend trying it out with a cheap NVMe drive since they’re getting more and more affordable. I opted out of getting an NVMe SSD last year when I built my new desktop because it was too pricy for me, this year I bought a 1TB NVMe SSD for under $100USD (you can go as small as 128GB or 256GB for significantly cheaper, too).
I’m floored to see the performance and responsiveness of this ARM SBC running on an SSD. It gives me hope for ARM laptops and desktops in the near future. Let us know if you’ve run your SBC(s) off of an SSD and how your performance was.
