Old files transfer fast, while new files transfer painfully slow

I have been struggling with performance issues for about a month now. I already have a case open with support but frankly they haven’t been much help. I’ll start from the beginning. I have a MacMini with 10.6.4.My Drobo has two 1tb WD green drives (EADS) and two 500gb Samsung Spinpoint 7200rpm drives connected via FW800. I have had the Drobo since December of 2009. When I first built the array I copied a bunch of TV shows, music, pictures and documents to the device immediately filling up almost 1tb. Everything was fine the device was fast enough for my needs. I then began storing Time Machine backups from two computers to the Drobo using sparsebundles. Everything still running fine and fast enough for my needs. Over the summer things started to slow down as I reached 75% capacity, OK that is to be expected. Recently though transfers have been painful, talking like 2-4 mb/s read/writes. Not fast enough to stream SD avi files without hitches and most certainly not fast enough to watch AVCHD files I recorded even after converting to H.264.
Tech support has not been able to help me so I conducted some of my own troubleshooting. Today I realized files I added to the Drobo early on in its life still have great transfer rates (30-40 mb/s) but files added within the past few months I am lucky if I hit 5 mb/s. I found this to be very strange and is really stumping me.
I forwarded that information on to tech support but I still haven’t heard back from them so I though I would try my luck here. Thanks!
EDIT Forgot to mention the Drobo firmware is the latest 1.3.7

And have you run repair disk on your drobo? ON all your volumes?

Yes I only have one volume and have ran repair disk from OSX and the install disk. I also ran disk warrior several times.

Tech support just suggested I buy a Drobo S. Really?

Your problem is consistent with my “fragmentation hypothesis”, described in this thread : when your Drobo was young and virginal, the space allocation between all disk units was optimal.
Then, when you increased space usage and Drobo number of files, it became more fragmented for newly allocated files, and thus slower when accessing those.
Some people reported such a slowdown when Drobo usage reached 50%, well before the Data Robotics advertised 95% threshold.
It is probably made worse in your case (compared to mine) because your 4 disks have different capacities. Also, the huge number of files created by Time Machine likely makes fragmentation worse.
If you could download all your Drobo data externally and reformat your Drobo then upload it, it would probably improve your access times; but if you are like me, it is not an option since the bulk of your storage is your Drobo, and you don’t have a spare one… :frowning:

I opened too a case on my performance issues with Data Robotics and got nowhere.
If this dire “wear down” property is intrinsic to the Beyond Raid algorithm, they are not likely to confirm it…
However, some thorough “defragmenting utility” should be able to improve things, but it would have to be run by the Drobo firmware itself (there is already one built-in, running in the background when Drobo is idle, but obviously it does not do a perfect/complete job).

I would agree with the fragmentation theory. You mentioned you were using sparsebundles for your time machine backups. Sparesebundles allocate space as it is needed, so although you may have made a 1TB sparsebundle when you created it, it grows ‘dynamically’ and I am sure the Drobo is fragmenting the heck out of it as each hourly TM backup runs.

Tech Support really needs to provide a tool that forces a ‘defrag’ on the Drobo.

Not so sure a “forced” defrag will make a difference. Why? Because supposedly BeyondRAID defrags on the fly. I’d agree that it does that given how sometimes my Drobo drives are clunking away when there’s no activity. The problem as I see it is two-fold, the way sparse bundles are built with many small bands and the CPU on Drobo.

A sparse bundle that isn’t mounted can be traversed like a directory from Terminal (or if stored on a *nix host) showing literally dozens of files and folders (the bundle bands). Since I don’t think Drobo understands what a sparse bundle is it probably drops the bands (which it sees as files and directories) all over the place. When you (or TimeMachine) mounts the bundle again it needs to touch many of those bands, causing a lot of I/O on Drobo. It’s like writing a bunch of small-ish files that really are one big file

Add to that the CPU in Drobo. IIRC the Drobo second gen has a 500Mhz Marvell (ARM) system on a chip with not a lot of DRAM. It’s not exactly a speed demon to begin with and then when you add having to re-assemble and optimize dozens (hundreds?) of files every hour I can see why it’d be slow.

Essentially with sparse bundles you have a file system on top of a file system (HFS+) that is thin provisioned onto a third file system (BeyondRAID).

If I were to do it again I’d either not use a sparse bundle and pre-allocate all of the storage I need (can’t recall if TimeMachine can use a regular disk image) or see about setting up multiple partitions (one for each computer) without the bundle.

It was much the same on my *nix based NAS box. The main difference was that the NAS has a 1.6Ghz CPU and 1GB of DRAM so it’s not resource bound. I just deleted a TimeMachine sparse bundle of 700GB and it took about an hour from the command line. Doing it from the Mac (where the share is mounted via AFP) would’ve been even slower.