Drobo FS disappear after minute.

Drobo - End of Sale Drobo FS
1

Hello everybody.

I have problem with my drobo fs. I have 4 drives 3TB and 1 drive 1TB in it. It is starting and during start up I can connect and see box itself in the network for one minute exactly. The blue lights goes lights up to second or sometimes third and then box disappearing from network, blue leds lights up to seventh and that’s it. I have checked HDDs separately and they are phisycally fine. Box itself is also fine ( its starting without hdds). I did FW update to latest one and same story.
I suspecting filesystem failure, but how to fix it. I don’t have SSH option ( it would help). I’ve checked what kind of file system is it and turns out its a EXT3/EXT4. I have connected all hdds to computer and used a software to check what RAID is it. Can’t find RAID configuration. I don’t want to lose the data.
Any ideas my friends?

Thank in advance for any ideas.

2

Do you have an extra drive that you can use for testing?

It’s a good idea to eject each of your drives and store them as a disk pack for now outside the Drobo.

I would suggest:

  1. power up the drobo with no drives installed.
  2. insert a single blank test drive. (personally I have a bunch of spare 80g drives I use for this)

In my testing (I did this), the Drobo FS complained that the drive was compatible with my firmware… I used Dashboard to Reset the Drobo (make sure your main disk pack is safely tucked away somewhere outside the Drobo!), and I let it setup the test drive as a new disk pack.

  1. setup the test drive as a new disk pack. It will complain that it can’t protect your data without a second drive… that’s ok, it still runs
  2. Reconfigure the name of the drobo and network settings while the test drive is in…
  3. turn on DroboApps support
  4. connect to the box from a client machine, and copy Dropbear SSH tool into droboapps folder, and restart the drobo.
  5. after it comes back up, check the droboapps folder… the Dropbear package should now be a Dropbear folder…
  6. SSH into the Drobo. Check logs. ( I think they are in /var/logs if I remember correctly). If the logs are blank due to resetting the drobo, then do the following:

power down the drobo, remove the test drive(while power is off). reinsert your main disk pack(while power is off). power on the drobo. let it boot until it fails. then power down the drobo using dashboard if you can, otherwise use power button on the back. then remove your main disk pack (while power is off) and put the test drive/single disk pack back in(while power is off). power on the drobo and let it start up. Now, SSH back into the drobo and go check again for logs.

Note that my testing shows: Once you have two separate valid disk packs, you can swap them back and forth pretty freely as long as you power down the unit between swaps. My testing also shows that the log files are not stored on the disk pack, but rather in the flash memory(?) portion of the Drobo itself, so it’s still there after a swap, but I suspect the logs might get deleted during a reset. (I didn’t test that part)

Hopefully you can find something in the logs that either explains what’s going on, or maybe you’ll find something you can share with us, so we can help further!

3

hi, a quick question too…
when you mentioned that the drobo disappears from your network, can i check if the dashboard program is still able to see the drobo?

4

Quick question: Where the RAID configuration is kept? If I remove my set of drives with data and connect new HDD will I lost configuration of RAID for original drives?
I know I can solder terminal to PCB and connect to OS on flash this way?
If Iinstall SSH with test drive where SSH port is really installed? On the chip or on the hard drives?
Regards

IS it possible to start dropbear from local folder, without using any HDD?

5

The BeyondRAID configuration seems to be stored on the disk pack. You don’t really need to think about the raid configuration yourself, as long as you keep the disk pack together.

Here’s my experience: I have two Drobo FS chassis. One has 5 2tb drives and it wouldn’t start due to file system corruption (the disks were fine, the data was fine, but the Linux file structure got damaged due to power-outage).

My second Drobo FS chassis I bought on eBay expecting to want to use it for additional expansion… but I’m not actually using it to store anything yet, so I used it for experiments. I created a 2 disk “disk pack” first. that worked fine. Then, I pulled those out (powered down) , powered up the drobo, and inserted a third test disk. The drobo complained, so I did a reset, and it created a 1 disk “disk pack” with that drive. After that, I was able to switch back and forth (powered down) those two “Disk packs” freely, and the data changed back and forth as I expected… So, one Drobo FS was able to seamlessly swap between two disk packs without loosing or corrupting any data. Then, I took my 2 disk “disk pack” and I put it into my main drobo (after removing my big disk pack and setting it aside)… and powered up… now the 2 disk “disk pack” was running in my main drobo as it did while in the test chassis. So, this proves that one disk pack can swap back and forth between two drobo units without any issues. The shares config and user config seemed to follow the disk pack, but the name of the drobo and the network configuration stayed with the chassis.

At no point during any of this did I need to touch, configure, or even think about the RAID configuration. Just treat the disks as a pack, and NEVER insert or remove any drive while the power is on unless you intend to erase that drive’s data.

I didn’t have to do any soldering… this was all software work.

if you install the dropbear droboapp, the SSH server is on port 22. I used PuTTY on my windows machine to connect using the IP of the drobo and port 22.

When using Dropbear, the SSH is installed on the disk pack… so if you remove the disk pack, it removes the SSH server as well. There’s a method of moving the SSH startup commands from the disk pack to the flash memory area of the Drobo, but it’s tricky, and requires manually editing some of the config files in the flash memory area.

I had to the procedure in order to use dropbear to fix my corrupt disk pack because the disk pack would not mount - and as such, I couldn’t put dropbear onto that disk pack. I had to use a test disk pack, copy to the settings from the disk pack down to the flash memory, then swap the packs, then I was able to SSH into the drobo even when the disk pack wouldn’t start.

I hope that information helps to answer some of your questions.

6

Hi. This is exactly what I need :). This is exact situation I have now. FS is currupted.I even started writing script to start SSH from chip itself, but you see if I will make any mistake I won’t be able to start the machine at all and I have only this one. Can you share the info about config file you have changed to start dropbear from chip?

Regards

7

For my problem, I posted the thread here: http://www.drobospace.com/forums/showthread.php?tid=144784

I started using the steps mentioned here: http://myproducts.drobo.com/article/AA-01333 … but it didn’t fix the problem, so I had to continue to research.

In the end, it turns out I had the dreaded “multiply-claimed blocks” problem.

Most of what I learned during that work came from here: http://www.drobospace.com/forums/showthread.php?tid=141627&page=1 … that thread includes the instructions on how to get dropbear working from the flash memory instead of the disk pack, and how to get NBD working so that a desktop machine running linux can actually do the fix.

Step 1, however, is proving to yourself whether or not you have the dreaded problem. You don’t need any magic to prove that, you just need a separate test disk that you can insert as a working 1 disk “disk pack” (you might need to reset the Drobo to initialize it (*while your precious disk pack is safely stored outside the Drobo (**which you took out with the power OFF))). Use the 1 disk disk pack to install Dropbear Droboapp, then login and go check your logs. If you performed the steps mentioned in the link here (http://myproducts.drobo.com/article/AA-01333), there will be a repair.log in /var/logs , and you can clearly see if you have the multiply-claimed block problem.

If you do, you’ll need to reread the links above.

,* lol - Around here we need to include disclaimers every time…

8

Hi. I’m able to ssh now to box even without the drives in it.
I have another problem. To fix damaged file system I have umount the /dev/sda1.
I do that without any errors, but when I run e2fsck -p -y -v /dev/sda1 it is telling me the file system is in use. I have shutdown all the services and after a mount command I see the /dev/sda1 isn’t mounted. Any idea for that?

Best Regards

9

Arg! No, for my problem sda1 wouldn’t mount at all, so I didn’t have to deal with unmounting it…

I think Ricardo mentions in the thread I linked above that some processes would need to be killed to guarantee sda1 was not in use… off the top of my head, I don’t remember which… (nasm? something like that?)

I think there may also be a trick where you can force fsck on next reboot by simply placing a specifically named file in the root folder…

cd /
touch forcefsck

I didn’t try that, but if it works, it would probably be easier and safer than killing processes… (assuming the box doesn’t get stuck at a prompt asking you if it should repair? I dunno…)

10

Hi again.
I have backed up all data now with smb protocol.
Next step is to start nbe-server. I’m using Ricardo tutorial, but nbd-server binary link is faulty ( https://copy.com/aprnpwwU9GbX ).
Is there any chance you or someone else could support me with this file?
I would very much appreciated it.

Best Regards

11

hi heres some links that might help?

old ricardo page:
https://sites.google.com/a/droboports.com/www/app-repository/nbd-3-7

sourceforge site:
http://sourceforge.net/projects/nbd/

info site:
http://nbd.sourceforge.net/

12

I used the pre-compiled version on my Drobo FS from here: https://sites.google.com/a/droboports.com/www/app-repository/nbd-3-7/drobofs … but between that page and the original thread (http://www.drobospace.com/forums/showthread.php?tid=141627&pid=180042#pid180042), there is a little confusion as to exactly where to put the files in the firmware area.

I don’t remember off the top of my head all the steps I took, but I did get it working. The package at the link is designed to be a DroboApp, which means it installs onto the Disk Pack, meaning it goes away when you swap the bad disk pack in… so you have to manually copy a few files down into the firmware area before you power down and swap disk packs again…

13

Hi Paul. Drobo doesn’t disapear from the network. It does that from dashboard. Ping to the device shows it is alive.

14

ah ok thanks lmk

15

Hello guys again. I have access to data onlu with smbd and nmbd started. Dashboard will see the box itself only for about minute. I have fully fixed file system and I can see and open all files.
To make box working for dashboard I have to type in console #nasd start.
This is log from console:
Sun Oct 11 14:37:18 2015: Waiting for command connection…
Sun Oct 11 14:37:18 2015: didn’t get command connection
Sun Oct 11 14:37:18 2015: Waiting for command connection…
Sun Oct 11 14:37:18 2015: Shutting down listener sockets- command 3: 0, discovery 4: 0, notify 6: 0

Any ideas?

16

hi, can i just check if this means that you have a way currently, to access and copy and paste the files somewhere else?
(even if dashboard only sees the drobo for a minute)

if so, i would recommend that you copy and paste and verify the data files while you can, before trying any more fixes (just in case a fix causes more harm before fixing)

17

Hi Paul ;).

I have access to data with smb. The data looks fine and file system is fine. To me it looks like a busybox problem. Nasd is starting and the suddenly stops with this result. I can copy/paste more, but best of all would be give you a file with log. Saturday I have started full and dashboard would see it. After reboot it. I’m even thinking it happend after firmware update which wasn’t done by me. The guy had access to data before flashing it.

Regards.

18

hi lmk :slight_smile:
ok hang in there, as another user might find a way to fix that hanging service.

19

Hi. I see other people have the same problem, but not every one knows linux that good to check which part of OS is damaged. It’s a problem with nasd itself I think and this app is corresponding to dashboard connection. It is just my idea, but i’m not sure.

20

there was a thread here which mentioned something about a nas service log that was preventing a drobo from starting up, but am not sure if that fix actually applies in your case:
http://www.drobospace.com/forums/showthread.php?tid=142911&highlight=unbrick