I recently picked up a Drobo 5N to downsize from my current custom NAS solution (ZFS on Nexenta).
I ended up choosing a Drobo really for the easy of use/upgrade - and I understand that it will be at the expense of performance. I put in 2x Seagate 4TB NAS drives + 120GB mSATA SSD cache.
I mainly use it as a NAS for the computers at home (data, music, videos, backup…etc), but I do need it for running a few VMs running on a small ESXi server.
I was able to quickly install and get NFS setup and connect to ESXi (thanks to droboports!)
I ran some quick benchmarks using some VMware IOmeter configurations to just get an idea of how much performance I was giving up.
My first guess with the async NFS comparison, it looks like perhaps the delta in IOPS in both read and write may be just raw speed delta in the platform (CPU possibly - my NAS runs an Intel core i3 and ZFS does write caching) and the results are just about 1/2 in IOPS for both read and write.
I’m curious about what people’s thoughts on running a handful of VMs from ESXi off this Drobo in NFS async. It’ll be on a UPS, so hopefully no crashes before the write commits.
I’m somewhat sad to hear that the SSD cache on the Drobo is only a read cache (I can somewhat understand the possible complications of write caching - but would be nice to be given that option as a “hack” or advanced feature?)
Also - how would the Drobo handle the VMDK files (since they’re large single files)? Would any part or the whole VMDK file possibly be cached in the SSD?
To be honest with you I’m very pleased with the results that you got. The fact that an ARMv7 processor can achieve half the performance of a desktop-class Intel CPU is something amazing.
Benchmarking synchronous I/O is a little bit pointless, since you’re either not really doing synchronous writes, or you’re just benchmarking how fast the HDDs are.
As far as I remember, the SSD cache is a block cache, meaning that if your VM files do not change too often they should cache pretty well.
There is no write cache on the SSD because it would kind of defeat the point of the whole array behind it. I.e., what good is a redundant array of disks if your only copy of the data died with the SSD?
I admit that it’s probably somewhat comparing apples and oranges, but the net result is still not too bad. But you can really see the difference with sync writes.
I figured async was really the only way to work around the fact there was not write cache (which ZFS has).
I have other benchmark numbers that really show the max read/write throughputs for both Nexenta and the Drobo is actually fairly similar (ie, limited basically by the HDD’s)
That would be great - assuming only part of the VMDK file would be cached (because some of the VMDK files are large).
Is there some stats that we can peak at to find out what the mSATA cache is doing? perhaps something in /proc? it’d be great if we could find out what the cache hit/miss %'s
Agreed - I guess I would have like to see some “smarter” way to do this - ie, the write-back caching really only hold a small amount of data, and only allow for short cache lifetime. That would help with IOPS, but by-pass the cache for large files and go straight to disk.
Not that I know of. The whole disk subsystem is hidden away under a pseudo-SCSI device. In reality, the whole storage subsystem runs under a completely different OS which interacts with the Linux OS using shared memory.
I know that the SCSI interface has a channel for control commands, but what those commands are is proprietary.
There is a pretty straightforward way of doing this. A simple RAID 1 (mirror) of two mSATA SSDs would provide enough protection for such a write buffer. At least in theory it would provide much faster writes than traditional HDDs.
After the initial burst the data could be placed on the HDDs. Keep a close eye on the health of the SSDs, and bypass them as soon as weird behavior is detected. To make things even safer include some kind of checksum on writes.
Yeah I think most really robust write cache solutions would add significant cost.
Here are some of the max throughput comparison benchmarks in ESXi (these now are in MB/s). In this case, probably maxing out the network (except for the Drobo write case)
Thanks for the benchmark data. I think it settles some of the doubts that people have around real-world performance of a 5N.
There is one scenario that you haven’t mentioned, and that I believe is the true Achilles’ heel of the 5N. Did you benchmark performance under several concurrent accesses?
My experience so far with the 5N is that it performs very well under a single client. Once you put a few concurrent clients (say 3 to 5), then the performance drops quite significantly.
I updated my 1st table with the MB/s numbers that correspond to the IOPS values.
I haven’t had a chance to take a look at concurrent sessions, since this was off of only 1 ESXi server - and the benchmark numbers are actually running IN a virtual machine from VMDK’s stored and access via NFS, so there are some overhead involved there. But at least this was the actual use conditions I wanted to look at.
I started looking at comparing performance for actual end-user clients (win/mac/linux) vs (smb/afp/nfs), and I’ll publish what I see for those when I get a time to finish those.
But extrapolating the data above, if limiting the performance is possibly CPU (and the overhead that Drobo has in their underlying storage layer), then I have to imagine you can quickly divide those numbers by the # of concurrent hits (assuming it’s not the network layer or something else) => and those numbers can “look” fairly dismal.