5Dt stalled, please help!

Hi all, hope someone would help me.
The situation is this: I have a 5Dt drobo with 5 6TB drives. About 18.5TB of space used. A few days ago I received the report that the disk space of the first slot was about to run out of space and it was better to increase the capacity with a new one. After a few days of inactivity, when turning the drobo back on, all LEDs of the disks were off and only the LED of the first disk was on, continuously red. I tried swapping it for a new 8TB drive and nothing happened. Always continuous red light. The dashboard said there was no disks in the drobo. Then I tried to remove the first disc and at that point the other 4 discs were also present. The dashboard was telling me occupancy was 100% and the discs started flashing yellow and green. The first empty slot, on the other hand, remains with a continuous red light.
Now In this first phase the data was accessible, so I started transferring important and urgent files. At some point the transfer stopped and the disk was no longer visible from the dashboard and from the computer, so I restarted the computer. On the other hand, I have not restarted the drobo, which continues to flash green and yellow, with a single continuous red light where the disk is missing. Although I connect and disconnect the cable with the computer (imac), the system does not see the drobo. How do you suggest me to continue? I don’t know if the data is fixing itself or if the drobo is waiting for a new disk in the empty bay. In the indications of the site it is written not to insert or remove discs during the yellow / green flashing phase. The situation is this for about 10h now.
Thanks
Fabio

I add an information: I have the old 6tb disk that was inserted previously, and a new 8tb disk not yet formatted

It may be that the new disk is either faulty or incompatible. How long did you leave it in there? What make & model is the new disk?

The length of time it’s taking to rebuild (flashing green & yellow) isn’t unusual you can figure a minimum of 1hr per TB of used space & often significantly more, nor is disappearing from or randomly reappearing in dashboard during the process, it shouldn’t happen but often does, at least when attached to a Windows PC, I can’t give you any tips or tricks when it comes to a Mac…

What is happening is that the red lit disk/drive is not recognised (for whatever reason) and your 4 other disks are now readjusting all your data across them as a 4-disk RAID array (instead of 5-disk). They will blink yellow/green until that process is complete, after which they will turn green and you will have a functional RAID array.

Given the time already passed I would let the process complete itself. During that time, you can access your data, but note that this will significantly lengthen the time for the 4-disk array to reallocate and duplicate your data across them. I would be recommending a full external backup is made of your data after it is complete to be safe.

Once all 4 drives are green, you can then look to introduce a replacement HDD (CMR not SMR) for the red lit drive, and assuming it is recognised, the Drobo will then repeat the process (all drives blinking yellow/green) as it readjusts you data across the 5 drives in your array.

A word of note: If your Drobo advised that it is running out of space prompting this upsizing of a drive, you will likely need 2x new upsized disks to extend your capacity as data duplication implies a replication of available data space - though it is not as simple as a 1 for 1. Using the Drobo Capacity Calculator, you can visually see how your data is allocated when you upsize a drive. Note that ‘reserved for expansion’ is not usable space.

EDIT: If you are expanding your capacity, you will need to let each new HDD be accepted in the array (ie all green) before replacing another drive with the next HDD. Do NOT insert 2 new drives at the same time.

Thank you both for the quick reply.
There are currently 4 6TB Seagate Ironwolf drives in the drobo, and out there I have the 6TB one that was full, and a new 8Tb Ironwolf. The whole Ironwolf series (specific for NAS) is CMR.

My doubt is that I do not understand why there is the red light in the empty housing. As if it’s waiting for a new drive. I wouldn’t want it stalled waiting for that.
Is it dangerous if I restart the drobo at this stage? (so I can try to connect to the computer)

The red light is perfectly normal for a full or almost full Drobo with an empty bay, it’s simply requesting extra capacity be added urgently, that won’t stall it doing the recovery, though being low on space may slow it considerably.

It’s safe to connect the Drobo to your computer running (without shutting it down), shutting it down ought to be safe, but connecting it while running & not interrupting the rebuild process is safer.

Normally I’d say that adding a new drive in the empty bay during the rebuild would also be safe, only since your new drive wasn’t recognised the first time you put it there I wouldn’t right now.

Personally I’d connect that drive to a PC & do a full test of it (Is Seagate’s Seatools test suite available for Mac? If so then a Mac will do.) I’d run the long self-test to ensure you’ve not been supplied a dud & that being the reason it wasn’t initially recognised… it does occasionally happen just because it’s new out of the packet isn’t a 100% guarantee it’s also working.

the fact is that the disks are quite silent; the are hot now, but when you write or read in then they use to be much more noisy. It’s like nothing is happening. Now they are more than 24 hours. I’ve tried to connect to another mac, and the computer can’t see the drobo

You’re worrying too soon. the fact the disks are getting “quite hot” is actually reassuring, if you’ve ever felt how a disk gets during continuous writes over most or all of it’s space (e.g. during a secure erase) it’s completely normal & expected.

Dropping in or out of Dashboard finding it is also quite common during rebuilds… as is dashboard taking ages (potentially hours) to find it during the process, Drobos have a weak processor, the rebuild process effectively pegs it at 100% doing the parity calculations & communicating with dashboard gets unreliable.

I wouldn’t even begin to worry it might have hung with an effectively full Drobo until the rebuild process is running over 3 times as many hours as there are TB of data on it. My last rebuild on my 5C (10TB drives ~18TB used) took 3 days to complete, about half of that time dashboard couldn’t find it.

I want to offer another opinion.

If the disks are ‘silent’ and there is no vibration signifying data reads and writes, then I am postulating that the red light is asking for a new disk as the 4 disk array cannot provide the data capacity needed to complete the full data realignment.

@cyber_beardy what is your opinion of adding a new drive (and not the previous failed new drive) now? My view is that the Drobo will accept it and commence (ie restart) reordering data across a new 5 disk array. I say this because I had an previous experience when replacing a HDD was delayed and my 5D started to reformat into a 4 disk array, until the new HDD was finally seated and the process recommenced again as a 5 disk array.

Further to Fabio’s earlier comment, I don’t think there is a problem in powering down his Drobo in the middle of a data reorder [EDIT: if there is no disk activity] and then restarting it, as it will only pickup where it left off. Your thoughts? Is this specifically prohibited? The data in theory remains secure - just there is no completed replication.

Sorry Fabio, I just wanted to offer these comments for others to comment on ahead of you doing anything that might compromise your data.

Personally I’d be tempted to secure erase the old removed drive & reinstall it, that’d get me back to where I started with a known working drive (without any data to potentially confuse the issue on it) & space for the rebuild to definitely complete, even if it was asking for more capacity… Meanwhile I’d conduct the testing I previously suggested on the “new” drive which appears problematic… it saves the immediate purchase of a new extra drive, even if one is going to be needed to actually increase capacity… With a Drobo that has 5 matched drives actually increasing capacity requires two bigger drives (three with dual redundancy enabled). As playing with the capacity calculator will confirm if @fabiocattabiani actually does so.

I wouldn’t hesitate from dashboard, I’m slightly less sanguine about powering down using the power switch, just in case there actually is activity & the forced shutdown interrupted a write I’d absolutely avoid doing so by say interrupting the power from the wall.

I’ve tried to turn off the drobo, turned it back on, and the situation returned where there was only the red LED lit in the empty housing. I tried doing a hot fill with the old disk. Nothing happened. I tried to turn it off and on and it recognizes the drobo in the dashboard, but as if it had no drives (as it happened before, at the beginning), with the fix red light on the first bay with the old drive. I turned off the system, took out the disk and turned it back on. Now I have a new state never seen before: all the LEDs are yellow, the power light off, all the small blu led lights of space lit up. I’ve been waiting for a few minutes… more and more worried

how about removing all the discs and checking if the drobo is in the correct operating state? could it be a problem with the drobo and not the discs?

Worth trying, daft though it seems if that shows a working Drobo, consider the PSU suspect.

This sort of consequence being possible was why I was advocating leaving it alone while it appeared to be rebuilding until there was no doubt it had actually stalled / hung… Given my experience of just how long rebuilds can take, I didn’t think it’d had long enough to declare that & try more extreme measures like forcing restarts or feeding it drives yet.

I’ve tried to remove all the disks, the situation is the same with the disks inserted, all the yellow LEDs.
At first I also suspected a power problem, so much so that I had ordered a new power supply. Not the original one that is nowhere to be found, but this:

seems to have the same specs.

Update:
I bought the power supply above, by changing the power plug. I inserted the 5 discs that have always been from the beginning (5 x 6TB Ironwolf).
The drobo started, recognized the 6 disks, saw the actual data occupation (18 TB) and started in data recovery, flashing yellow and green on all disks.
After about 30 seconds it has disappeared from the dashboard, but is still blinking (yellow and green).
I tried from the beginning to access the data, but it doesn’t load directories. Even if the computer sees it.
Now waiting…

Look the same situation it was at my first post :frowning:

I had a vaguely similar situation with a 5D3. I would not follow the procedure I’m about to post from Drobo support unless you feel you are out of options. It did allow me to get the Drobo up and running long enough to copy what I needed to off. In the end, I ended up doing a factory reset and reformating all the drives. I did have one drive that failed, but it was not the one Drobo dashboard had been complaining about.

Again, the procedure below should probably only be attempted if you feel you have exhausted all other options.

From Drobo support:
David Stempnakowski,

In order to help resolve the issue, we will need to do a couple of things. The first step is to get a diagnostic file in read-only mode (procedure below), this will determine next steps.

Please follow this procedure to put the Drobo into Read Only mode:

  1. Put your Drobo into Standby/Shutdown via the Drobo Dashboard (if it’s not already) and remove the power. With the Drobo completely powered off eject your drives at least 1 to 2 inches.

How do I safely shut down my Drobo device? https://myproducts.drobo.com/article/AA-01686

  1. Power up the empty Drobo.

  2. Once the Drobo connects to the Drobo Dashboard, double-click the Drobo icon and press keystroke: CTRL-ALT-SHIFT-R in Windows or CTRL-OPTION-SHIFT-R if you are running Macintosh.

  3. You should get two prompts, one right away, and one a few minutes later. Please click to reboot the Drobo at that time (if you don’t get the second prompt for 5 minutes standby the unit using Drobo Dashboard and then cycle the power).

  1. Once the Drobo connects to the Drobo Dashboard please check that the Drobo is in read-only mode (RO) by looking at the name of the Drobo in the Status or Capacity screens and verifying that there is an (RO) after the name. If it is not in read only mode please start over at step 4.

  1. Once verified Drobo is in Read Only (RO) Mode, put Drobo into Standby. Once in Standby/Shutdown disconnect power. It is Critical that step 5 above is confirmed. Data loss can occur if the Drobo is not in read only mode.

  2. Once the drives are securely seated in the Drobo, power On the Drobo. The Drobo should connect to the Dashboard. Once connected to Dashboard please generate a diagnostic file and upload it to the case (procedure below)

How to Generate a Diagnotic File

After the Drobo boots are you able to access your data? If so please begin copying the data to another location.

If the Drobo does not connect to Dashboard with drives installed please:

  • Properly Shutdown the Drobo.

  • With Drobo powered off eject the drives.

  • Power up the Empty Drobo and generate a diagnostic and upload it to the case.

  • Properly Shutdown Drobo and wait for next steps.

Kind Regards,
Sydney K
Drobo Support Team

Hi David,
thank you very much for your suggestion.
Just today I’ve started to try this path: I bought a box that allows me to read the 5 discs individually. I’m doing a scan with UFS RAID recovery, which is a data recovery software that supports Beyond RAID.
It appears to be rebuilding the file system correctly, but it takes time.

If I had known about your solution before, I probably would have tried that route. If I find it difficult to continue with what I am doing now, I will try it.
Thanks again!

P.S. Sorry for my English, I’m from Italy

here I am again, a question for David:

Did your system and dashboard recognize the Drobo before doing this procedure? In my case even the mac disk utility didn’t see the connected disks, only for few seconds.

It will take about 1 month to rebuild the File System :face_exhaling:

I’ve asked a data recovery company and they asked me 8000 Euro (same as $ now) to recover the data, I can wait one month with my method