My first guess with the async NFS comparison, it looks like perhaps the delta in IOPS in both read and write may be just raw speed delta in the platform (CPU possibly - my NAS runs an Intel core i3 and ZFS does write caching) and the results are just about 1/2 in IOPS for both read and write.
I’m curious about what people’s thoughts on running a handful of VMs from ESXi off this Drobo in NFS async. It’ll be on a UPS, so hopefully no crashes before the write commits.
I’m somewhat sad to hear that the SSD cache on the Drobo is only a read cache (I can somewhat understand the possible complications of write caching - but would be nice to be given that option as a “hack” or advanced feature?)
Also - how would the Drobo handle the VMDK files (since they’re large single files)? Would any part or the whole VMDK file possibly be cached in the SSD?
To be honest with you I’m very pleased with the results that you got. The fact that an ARMv7 processor can achieve half the performance of a desktop-class Intel CPU is something amazing.
Benchmarking synchronous I/O is a little bit pointless, since you’re either not really doing synchronous writes, or you’re just benchmarking how fast the HDDs are.
As far as I remember, the SSD cache is a block cache, meaning that if your VM files do not change too often they should cache pretty well.
There is no write cache on the SSD because it would kind of defeat the point of the whole array behind it. I.e., what good is a redundant array of disks if your only copy of the data died with the SSD?
I admit that it’s probably somewhat comparing apples and oranges, but the net result is still not too bad. But you can really see the difference with sync writes.
I figured async was really the only way to work around the fact there was not write cache (which ZFS has).
I have other benchmark numbers that really show the max read/write throughputs for both Nexenta and the Drobo is actually fairly similar (ie, limited basically by the HDD’s)
That would be great - assuming only part of the VMDK file would be cached (because some of the VMDK files are large).
Is there some stats that we can peak at to find out what the mSATA cache is doing? perhaps something in /proc? it’d be great if we could find out what the cache hit/miss %'s
Agreed - I guess I would have like to see some “smarter” way to do this - ie, the write-back caching really only hold a small amount of data, and only allow for short cache lifetime. That would help with IOPS, but by-pass the cache for large files and go straight to disk.
Not that I know of. The whole disk subsystem is hidden away under a pseudo-SCSI device. In reality, the whole storage subsystem runs under a completely different OS which interacts with the Linux OS using shared memory.
I know that the SCSI interface has a channel for control commands, but what those commands are is proprietary.
There is a pretty straightforward way of doing this. A simple RAID 1 (mirror) of two mSATA SSDs would provide enough protection for such a write buffer. At least in theory it would provide much faster writes than traditional HDDs.
After the initial burst the data could be placed on the HDDs. Keep a close eye on the health of the SSDs, and bypass them as soon as weird behavior is detected. To make things even safer include some kind of checksum on writes.
I updated my 1st table with the MB/s numbers that correspond to the IOPS values.
I haven’t had a chance to take a look at concurrent sessions, since this was off of only 1 ESXi server - and the benchmark numbers are actually running IN a virtual machine from VMDK’s stored and access via NFS, so there are some overhead involved there. But at least this was the actual use conditions I wanted to look at.
I started looking at comparing performance for actual end-user clients (win/mac/linux) vs (smb/afp/nfs), and I’ll publish what I see for those when I get a time to finish those.
But extrapolating the data above, if limiting the performance is possibly CPU (and the overhead that Drobo has in their underlying storage layer), then I have to imagine you can quickly divide those numbers by the # of concurrent hits (assuming it’s not the network layer or something else) => and those numbers can “look” fairly dismal.