Two drives failed - rebuilding taking time

[antstore400]

Hello… I have a Drobo 5N with all 5 drives populated and in use. Got dual-drive redundancy turned on, and last week I had the pleasure of having two drives fail almost simultaneously.

I did the best I could - replaced both drives with new ones of similar sizes. (Actually one 1TB bigger, the other 1TB smaller.)

All five drive bays show flashing green and yellow lights, which is, I guess, to be expected - but it has been like that for six days now. Perhaps worse, the Drobo isn’t showing up in the Dashboard software (either on PC or Mac).

Mac Finder reports being unable to connect. Windows XP report that the device cannot connect due to an I/O error. Windows 7 shows three shares, but the ‘public’ one where my data is (was!) appears to have nothing inside it except a ‘lost+found’ directory, which appears empty.

I’m trying not to panic but I have a feeling that this isn’t good. In the middle of all this I did power off the Drobo (using the switch at the back - and it appeared to shut down safely) but obviously I’m now wondering if I’ve done something silly.

The Drobo doesn’t seem to be in a reboot loop, and the sound of the drives suggest that it is indeed just working through the data protection process. But not being able to see the Drobo in the dashboard or any files on the network share is a little concerning. The only reassurance is that Windows 7 does seem to show the right kind of numbers in My Computer (e.g. 6.04TB free of 15.9TB - suggesting 9.9TB in use, which is about right), and the blue lights on the front do seem to be showing about 80% of the drive being full, which again is correct and - I’m hoping - suggests that the files are still safe and accounted for.

My instinct is to try powering off to see if that brings the device back to being visible on the network again - but having now had time to read the web pages I see that it does say that green and yellow lights should not be interrupted in this way. I did also see another page which suggested that in some situations a Drobo might appear not to mount until it’s done with the data protection process. Fair enough.

In the few hours before the Drobo dropped off the network, I did see it saying ‘Data protection in progress - 71 hours remaining’ (long since passed) and at one point that changed to something like 881 hours, but I’m hoping it won’t take that long. [Although it’s OK if it does, of course, although it might take me past the end of my warranty!]

Just wanted to share my sorry tale here and see if anyone had any thoughts. Thanks for listening!

[johnm]

Hello… I have a Drobo 5N with all 5 drives populated and in use. Got dual-drive redundancy turned on, and last week I had the pleasure of having two drives fail almost simultaneously.

It happens. You were unlucky, but as hard drives get bigger and their reliability doesn’t improve more and more people will join you.

I did the best I could - replaced both drives with new ones of similar sizes. (Actually one 1TB bigger, the other 1TB smaller.)

You say nothing about your previous configuration but your replacement disks may well not provide sufficient storage for your needs. Suppose your configuration was five 4 TB disks and you replaced two that failed with one 5 TB and one 3 TB. The extra capacity on the larger disk would be unavailable for use (marked as “reserved for expansion”) so your overall capacity would be 1 TB less than you started with. I’m not sure what Drobo does in the circumstances but I would hope that if it found itself unable to provide the dual disk redundancy you asked of it then it would at least provide single disk redundancy and then warn you with a solid red light to replace that smaller disk.

All five drive bays show flashing green and yellow lights, which is, I guess, to be expected - but it has been like that for six days now. Perhaps worse, the Drobo isn’t showing up in the Dashboard software (either on PC or Mac).

Yes, all five bays flashing green/yellow is exactly what you’d expect, but it should still show up in the Dashboard. It has a lot of data to move around and not a lot of free space to do it in so it’s going to take a long time.

Mac Finder reports being unable to connect. Windows XP report that the device cannot connect due to an I/O error. Windows 7 shows three shares, but the ‘public’ one where my data is (was!) appears to have nothing inside it except a ‘lost+found’ directory, which appears empty.

That doesn’t sound good but not really cause for panic. Just let it get on with its job. Is Windows XP still supported, BTW?

I’m trying not to panic but I have a feeling that this isn’t good. In the middle of all this I did power off the Drobo (using the switch at the back - and it appeared to shut down safely) but obviously I’m now wondering if I’ve done something silly.

Why on earth did you do that? Yes, you might well have done something very silly. You say “in the middle of all this” - do you mean “in the middle of all this flashing green/yellow data protection activity”? You were unlucky to lose two drives near simultaneously but you might be lucky if your Drobo shut down gracefully. It might be able to continue where it left off. Otherwise you’ll have to restore your data from a backup.

The Drobo doesn’t seem to be in a reboot loop, and the sound of the drives suggest that it is indeed just working through the data protection process. But not being able to see the Drobo in the dashboard or any files on the network share is a little concerning. The only reassurance is that Windows 7 does seem to show the right kind of numbers in My Computer (e.g. 6.04TB free of 15.9TB - suggesting 9.9TB in use, which is about right), and the blue lights on the front do seem to be showing about 80% of the drive being full, which again is correct and - I’m hoping - suggests that the files are still safe and accounted for.

The difference between a reboot loop and data protection is very obvious. From what you say you might be lucky, if you give it a chance to do its job.

My instinct is to try powering off to see if that brings the device back to being visible on the network again - but having now had time to read the web pages I see that it does say that green and yellow lights should not be interrupted in this way. I did also see another page which suggested that in some situations a Drobo might appear not to mount until it’s done with the data protection process. Fair enough.

You’ve already tried powering off and did it help? No! I’m glad you had the opportunity to read the manual, which would have been my first instinct (but then, I’m old school). The user interface for the Drobo is pretty simple and the legend on the back of the front cover is a big clue. You really ought to understand the basics before committing so much data to it. I can understand the shares becoming unmountable for a while because, while it’s fighting to protect your data the last thing it wants is you making changes to the filesystem. However the failure of the Dashboard is something for which I would criticise Drobo heavily. Too many people are experiencing a failure to connect at the moment and though I personally haven’t, it does take a couple of minutes to find my 5N, which seems an inordinately long time. What version of Dashboard and firmware are you using?

If you really want to power it off again then why not abandon your data, reinitialise your disk pack and restore your data from a backup? It might be quicker in the long run.

In the few hours before the Drobo dropped off the network, I did see it saying ‘Data protection in progress - 71 hours remaining’ (long since passed) and at one point that changed to something like 881 hours, but I’m hoping it won’t take that long. [Although it’s OK if it does, of course, although it might take me past the end of my warranty!]

The estimate of how long it’s going to take is notoriously unreliable. It’s probably a difficult thing to calculate but it presumably gets more accurate as time progresses. Again, the lack of Dashboard connectivity is desperately inconvenient. You might want to raise a support request while you’re still within warranty.

Just wanted to share my sorry tale here and see if anyone had any thoughts. Thanks for listening!

You’re welcome. Please report back.

[Docchris]

Short observations:

1. sometimes drobo’s dont repsond while rebuilding… but then pop back online when its finished, its annoying, but not TOO much to worry about

2. rebuilding two sets of redundancy at once will take quite a while… (the second set is mathematically much more demanding than the first, so it will take more than twice as long as rebuilding one set)

3. using slightly differnet sized disks probably means its relaying out at the same time as rebuilding - which may also make it take longer

4. do not power off while its rebuilding - it may even cause it to start over

5. be patient, VERY patient, and it will probably be ok

wait for it to be all green - or give you an actual error, before doing anything drastic…[hr]
I should have added

i would guess your rebuild may take a week… but i wouldn’t be worried if it took up to two weeks…

[antstore400]

John, Chris, thank you both for your level-headed and sensible replies, much appreciated.

Pleased to say that after nine days, the flashing yellow/green lights eventually went green. A reboot was still needed to restore the Dashboard, but in the end it was all fine and everything was still there.

I’ll write up a fuller account once the dust has settled. Thanks again to you both.

[Docchris]

glad it all worked out ok (and i was right… hahaha, woop/phew!)[hr]
even down to my 1-2 week estimate