Drobo fills up free space by itself

This is a weird one, I’ve searched and haven’t found anyone with the same problem. I have the original Drobo and I have 4 1 TB Drives installed. I’ve had it for 5 years and this problem only started in the last 9 months. Normally the Drobo runs with about 10-15% free space. Randomly I will all of a sudden get a Red alert saying there is no space available - 0%. I know the space is there, I can delete some some files but then the new free space will fill up rapidly by itself. Though tinkering I discovered that if I re-seat all the drives (when powered down) that this would put the Drobo into rebuild mode and after about 5-7 days the Drobo would rediscover the lost space, after the rebuild and I would be back to 10-15% free space. After the rebuild the Drobo would run normally for months or week then it would happen again and I’d have to force the rebuild. Now it is happening more frequently and I don’t really want to be re-seating the drives constantly. Any ideas? Thanks!

Are you getting a red alert that there’s really no space available, or not enough space available to re-establish data protection?

Given what you’ve said it sounds more like the latter, and that can be caused by a drive “dropping out” of the disk pack. It could be a drive going bad, or a problem with the backplane.

To see if it’s a backplane issue, try powering everything down, then change the position of the drives. If the “lost” drive follows the a specific drive, there’s your culprit. If the “lost” drive stays with the specific bay, then it’s probably the backplane.

Also be sure your Drobo isn’t on a surface that is prone to movement or vibration and that it is oriented in the standard position on its feet, not on its side, face, or hanging vertically.

bhiga

Thank you for your fast and very helpful reply. I am getting the no space available red alert and when I look at the drive there is 0% available. I was suspecting maybe a drive going bad, I was trying to rebuild the drive with Disk Warrior and the program was reporting drive errors, unfortunately it didn’t say which drive had the errors.

That was a great tip about swapping the drives, I didn’t know that you could do that. I tried that and got all red then tried re-seating the top drive and sure enough the red light flowed the drive that I moved from the first bay to the second bay. Shortly afterwards it when into recovery mode and my 11% free space re-appeared. So I guess that drive is going bad, strange because it’s the newest.

Thank you also for the tip about the surface, it was on a table with a copier so it did get some movement, it’s now on a stable surface that never moves.

I’ve ordered a new drive and I will swap out the one the second bay and see what happens. I hope that it isn’t the backplane, I don’t think that’s repairable.

Thanks again for your expert help!

hi, i was going to add something about running a chkdsk and some windows ideas, but when i saw you mention DiskWarrior, i realised you were on a mac :slight_smile:

i think bhiga might have found the culprit, but here are a few other things to bear in mind which might help in future…

vibration:

  • putting the drobo unit onto a heavy (solid table) or surface etc, can help reduce vibration
  • putting something (with a bit of weight) on top of the drobo can sometimes help too. (preferably not a glass of water) :smiley:
  • putting some mini drinks coasters (eg cork) just below the drobo’s rubber feet, helped me a great deal.

space getting eaten up:

  • if some programs are installed onto the drobo’s drive letter, or if a program is configured to use the drobo for temporary working space etc, that can also cause the drobo’s free space value to suddenly change or drop.
    (some programs might automatically select the largest volume, or the volume with the larget amount of free space - which on a drobo, 10% of 3TB is about 300GB which is quite large - so you might need to look at some program configs to see.)

  • filesystem checks are usually good to do as well.

  • time machine can cause problems if not configured correctly, but as in your case above, it might not be time machine.

5 years worth of use so far isnt that bad :slight_smile: and luckily the new 4-slot drobo v3 (gen3) looks like it allows an upgrade path from older models, so it might be worth investigating.

See how the drive replacement goes, and hopefully you can catch that culprit. :slight_smile:

(if you have spare drives/space somewhere else, and time, you might want to backup your data, eg just in case a different drive happens to go bad during the replacement’s rebuild, though if you’ve been doing rebuilds quite often, then your drobo seems fairly robust so far but its worth considering).

Paul,

Thanks for the tips, I’ll let you know the drive swap goes when it arrives this weekend (I should have one on hand anyway).

I do have my eye on a Drobo Mini, this one has been a great workhorse.

ok good luck,

(the mini has pro’s and cons… being smaller, it can be more portable than the others, but can also only give limited drive redundancy and use only drives up to a certain size)

Did you ever resolve this issue? I’ve started having the same (or a similar) problem with my Drobo S, where I have about 12% of 8TB free, but every once in a while it will completely fill itself up (and the dashboard doesn’t say any drive has failed; just that the drobo is now full).

hi martin,
overall, i think the general userbase case is that a large majority of users will usually only post when they need help to overcome a problem or question etc (rather than just to post about all the good experiences) which is understandable :slight_smile: - some do though, or sometimes it isnt as big a problem than other things, and ends up with less priority for the user. (eg i might like to get a ups, and start looking into things there, but then had a hard drive fail and time ended up needing to go there and the ups is on hold for now) :slight_smile:

but for your case, is there a way that you can run a search of all files updated or created recently, and then to sort by filesize or date or path, and to try and find some examples of such unexpected files?

that might help to pinpoint which files and where, which might lead to more info to help pinpoint what program is doing it etc

I have had some drives going bad. I noticed only because I had set my drobo to send emails. (I had enough space so the drobo would have been able to protect all data with only 3 drives.) After what I supposed was seconds or minutes, the drive recovered and continued to work normally. By the time I read the email and looked at the Drobo, all lights were green again.

I extracted the log from the Drobo and sent it to Support. They told me what drive had caused the alert. I removed that drive and run a deep scan that took hours using the appropriate software from the manufacturer of the drive. The drive was bad and I got a warranty replacement. It was not enough to do a quick test in a few minutes.

I suppose I would get your symptoms if I had been as low on space as you.

Thanks. That sounds like it might be related to the issue I’m having, as there aren’t actually any new files being created to fill up the space (that I can see – even checking for hidden/system files). I’ll try checking the logs and see if there’s anything I can make out.

[quote=“Paul, post:8, topic:139538”]but for your case, is there a way that you can run a search of all files updated or created recently, and then to sort by filesize or date or path, and to try and find some examples of such unexpected files?

that might help to pinpoint which files and where, which might lead to more info to help pinpoint what program is doing it etc
[/quote]

That’s the weird thing. The last time this happened (when I was at my computer – often times it happens overnight), I tried looking for the source of the used space. I even checked the hidden/system files, and I couldn’t find it. I’m thinking that the drobo or one of the drives in it might be faulty. Unfortunately, the support team doesn’t look like they’re going to help, because the unit is 3 years old and therefore no longer supported.

btw which operating system and filesystem are you using?

if you are on windows using ntfs on the drobo, then running a chkdsk is usually a good idea even when youre not having problems (from a preventative point of view)

it might also be that some filesystem corruption took place, and ntfs might have been having lost clusters or similar, and trying to save those (but i dont think they would show up as visible files until chkdsk processes them)

eg if you have more than 1 volume, run it only on 1 volume at a time.

to run it, you can try 2 methods (assuming “d” as your drobo drive letter)
chkdsk d:
(that will run in info mode)

chkdsk d: /x
(that will actually run in full fix mode)
its worth repeating it until you see a message about “finding no problems”.

i think the most i ever had to run a chkdsk was 3 or 4 times in a row via the /x command, before getting no problems reported, but that was in times of multiple power cuts and other shenanigans :D[hr]
edit: only run one instance of chkdsk at a time though and let it complete and finish :slight_smile:

It’s NTFS on Windows 7. I’ll give that a try – thanks.

Hmm. It was a good thought, but that isn’t it. I’ve run chkdsk a few times on the volume and it hasn’t found any errors.

Another factor could be if you are using Windows Backup and Restore on an older model Drobo.If this is the case you may need to copy your data off and reformat.

[quote=“DroboMod, post:15, topic:139538”]
Another factor could be if you are using Windows Backup and Restore on an older model Drobo.If this is the case you may need to copy your data off and reformat.[/quote]

I had actually done this a long time ago (shortly after buying the Drobo 3+ years ago) before realizing my mistake and fixing it. I was never able to reformat, due to not having enough free hard drive space to copy everything off my Drobo, but I hadn’t suffered any problems until a few months ago. I don’t have windows backup running any more, but it’s possible that something wonky is still stuck in the file system and that’s causing this problem? Either way, I don’t have anywhere to move 7TB of data to try and reformat the Drobo.

ah i didnt know about windows backup,

martin maybe you could download sysinternals process explorer and to run it as an admin, and then if your drobo volume is called “d:”

without quotes, you can run a process/handle/dll search for “d:”
( if it finds too much you can try “d:” )

and the same for each volume letter.

to see what results you get… that might show some windows remnants
(please dont force close any handles etc, but rather to see what results it finds?)

[quote=“Paul, post:17, topic:139538”]
without quotes, you can run a process/handle/dll search for “d:”
( if it finds too much you can try “d:” )[/quote]

This is what I got (there’s only one volume):

thanks for the info martin,
ive seen similar entries before and cant see anything untowards…

was hoping to see something like g:\programfiles\apple\livecatalogue or somethig similar :slight_smile:

some of the entries seem part of ntfs transactions, and even on a local drive are also there.
i think the logs just reach a certain amount and then are supposed to purge or trim themselves.

there is an fsutil command that can show the current size of the log (though ive only been able to try it on a local drive and not a drobo yet) = fsutil resource info D:\ but i dont thats the culprit and its best not to mess with those manually.

7tb is quite a lot of data, but maybe you could select a bunch of root-level folders in a few groups, and to run a right-click > properties on them to get the actual used space info.

eg:
1a) if you made any new folders within g:\ you could try that
(but exclude any that you didnt make, such as hidden/system folders like recycling bin and volume info etc)

1b)
then invert the selelection (of folders only) to get the properties for the system folders

and then to select the same for any root-level files.
2a) own files
2b) system files.

this might show actual usage reported by nfts and we can cross reference it with the drobo leds and dashboard.

then we can try to find mismatches, where it just might simply be that youve pasted or moved or something created a set of folders into a location that is already heavily nested, meaning that if you go beyond a filepath or max folder path, you will not get the actual size reported correctly.

some tools exist which can find the deepest nested path, (for further manual inspection), possibly a free version of treesize or a pro demo, or there might be another way to find out.

depending on the sheer number of files that you have, a tool might be useful called Cathy (by r vasicek) to create an offline but searchable and context-useable catalogue of your drobo g:\ volume, and you might be able to spot things when sorted by largest folder (periodically checking for changes) - it can take quite a bit of time to create though.
(for me its about 20 mins on one of my 2tb volumes, and over an hour for the other)

[quote=“riogator, post:1, topic:139538”]
Normally the Drobo runs with about 10-15% free space. Randomly I will all of a sudden get a Red alert saying there is no space available - 0%.[/quote]

The community has some great tips above but I wanted to ask a couple questions.

When the space disappears, what are the lights on the front of the Drobo? Bay lights and Cap lights.

It’s easy for a Drobo to see a drive issue (like a drive timeout) drop into data protection, see the drive come back and do a quick degraded re-layout to bring it back into the pack, even before you might notice that it was in re-layout. The dashboard will leave a message up even after the Data Protection process has completed if a Data Protection requires all space on the array. If you look at previous messages do you see anything that might indicate a re-layout?

If this scenario is correct it is concerning that the reported space available does not change when the Data Protection is over. That’s why I was curious about the lights. Have you tried restarting the Drobo when all bay lights are solid green and the space is at 0%?

One thing to keep in mind is that the Dashboard is a great tool but, the lights on the front are 99 time out of a hundred the most accurate method of knowing what the Drobo is doing/not doing.