Drobo

[SOLVED] Broken Drobo FS

Hello

Last night I noticed that mysql on my FS would not respond to queries. I could however still browse the filesystem on the drobo, so I assumed this was just a problem with mysql crashing for some reason.

I then rebooted the drobo (from the control panel), and soon discovered that something was terribly wrong.

The first thing I noticed was that all bay lights were green (I am a little low on disk space, so I was expecting one to be yellow, as it has been for a few days). Going into the control panel again, I saw the message “Internal error - contact drobo support”, and under advanced controls, the drobo claimed I was only using 20 mb and had about 8 TB of free space. I tried “hard rebooting” via the switch on the back, this took quite a while, but I still got the same message - however the yellow light was back on.

Searching on support.drobo.com got me this:
http://support.drobo.com/app/answers/detail/a_id/500/~/i-powered-up-my-drobo-fs-and-now-the-drobo-dashboard-displays-a-'internal

I did the file system check as described, this took about 8 hours. The drobo then said “File system repair complete. Please contact drobo support.” Under advanced controls, I could now also see that used/free space was back to normal (about 7.4 TB used, 740 mb free). Encouraged by this message, I did a new reboot, and got a new message, “Mount failed - drobo detected an internal error. contact drobo support” and used/free had now changed to 1.5 TB / 6.5 TB).

I did the repair procedure once more (this time it ran for just a few minutes), and the numbers changed back to normal, however the “Mount failed” message persists (did not change to repair complete this time). I have not touched the drobo since.

The drobo responds to ping.
It does not respond to SMB or SSH (OpenSSH).

Dashboard version 1.7.3
Firmware 1.1.1
Recent changes: swapped 2x1.5 TB drives for 2x3 TB drives (WD Green) about two weeks ago.

My clients are windows and ios-based.

Edit: I should add that I have two Drobo FS’s, bought at the same time, and with the same firmware and droboapps installed. The other one is working just fine, however, it does not have any 3 TB disks.

Any advice?

So, I took the plunge. Upgraded dashboard to v1.8.4, then drobo FW to v1.1.2 and after the obligatory reboot after FW update, everything is back to normal.

No messages, nothing. SSH back up, MySQL back up, file system back up.

While I am happy my files seem to be safe for the moment, I am still wondering why the drobo failed in the first place, and I have to say I am not very trusting of the Drobo to keep my files safe for the future.

Best regards,
Tusse.

I have the feeling that your problems are due to the fact that you are using 3TB drives with the firmware that does not support them. I’m running 1.1.1, but my largest disk is only 2TB.

I can’t remember any major complaints about 1.1.2 around the forum.

That being said, it is very likely that you’ll have data loss at this point. If you have so little free space left, then you are definitely over the 2TB address space and who knows what happened to the data being saved there…

According to Data Robotics, the Drobo FS shipped with firmware that can handle 3 TB disks. That said, the release notes for v.1.1.1 includes this:

Previous Fixes from 1.0.1, 1.0.2, 1.0.3, 1.0.4, 1.0.5 and 1.1.0

Support for Large Capacity Drives (>2TB)
(… misc other stuff …)

(I.e. support for large capacity drives was introduced in either 1.0.1, 1.0.2, 1.0.3, 1.0.4, 1.0.5 or 1.1.0)

I was running v1.1.1 previously, so I am fairly confident that >2TB support was present when I started using 3 TB drives. There have been no fixes regarding >2TB drives in 1.1.1, 1.1.2 or 1.2.0 (according to the release notes, anyway).

Good to hear.

Any tips on how to test this? Check recently copied files?

Tusse.

My bad, I thought 1.1.1 did not have support for 3TB drives. That’s why I mentioned data loss. They probably fixed some bug in the 3TB drive support. In any case you should check the recently changed files to see if everything is ok.

It seems like the only change between 1.1.1 and 1.1.2 is “Solve rare issue related to the file system repair capabilities”, and that that rare issue was the one I was having. Lucky me.

Will check some files and report back if I find any errors, but I expect it to work.

Now, wouldn’t it be great if the Drobo actually told us exactly what was preventing it from mounting / repairing the filesystem? I guess I will never really know what the problem was - unless there is an internal log file somewhere we can access…? Outside the regular filesystem obviously.

My Drobo FS is back to normal, except for Time Machine, but I downgraded to 1.1.2. from 1.2. That caused problems so I went back further to 1.1.1. It then wouldn’t mount the file system so I did a repair which fixed things. I then moved all 3TB’s of data off the Drobo to an iOmega NAS which I bought in desperation. Fortunately I didn’t lose anything on the Drobo (thought I had lost everything ). I then reset the Drobo and set it up as though it were new. AFP and SMB shares worked and I could even log in via smb which I couldn’t before. I then moved the 3TB stuff back and all is working great. Of course I won’t ever keep anything on there which isn’t duplicated elsewhere so I have to figure out an automated way to keep the FS in sync with the iOmega ( which supports Time Machine under Lion and does it just fine). The iOmega 4TB NAS came with upgraded AFP support, NFS and cifs. It has rsync, FTP, automated backups over the network and so on … Much faster than my FS, but the FS also has its strengths. EMC which owns iOmega has been in the Mac backup business for a long time. It’s not a Drobo but it saved me from the 1.2 firmware nightmare I have had with my FS.

Philip

The same Bernoulli/Zip/Jaz/“click of death” Iomega? Interesting…
Mind you, I loved my Bernoulli and had few problems with it. Zip and Jaz, on the other hand… :\