OSX TimeMachine volume limit

Forum History General Discussion
1

Assuming I have the real physical size of my drives equal to the volume size I set when I first setup my Drobo (Gen2 4-bay), does TimeMachine/OSX execute the same “trim-off-oldest-first” procedures, as it does for a “normal” drive? I say this upon remembering the TimeMachine “Preferences” window shows the usual “X GBytes of Y GBytes available” for each TM drive I have, but the “Y” shown for the Drobo box is equal to the size of the volume (real hard-drive space), as opposed to the much less “total usable” space that the Drobo Dashboard shows.

For example, if 16TB = Volume Size = Hard-drive space, the total usable space might be 11TB (just a guess). In this case, Time Machine Preference window will show “X GB of 16 TB available” message.

Will TimeMachine, upon exceeding my 11TB limit, cause the drives to be corrupted (as it has for me before)? Or will Time Machine “do the right thing”?

2

Short answer: Maybe (regarding corruption). No (regarding the “right thing”).

TimeMachine (and all other Applications) only see the “Volume Size” of 16GB. So the OS is told that there is lots of free space and when the data is written it will fail. That is hard to handle for any application / backup software. The artificial slowdown of file transfers has only one reason: get the attention of the user and make the device painfully slow that the limit will not be hit.

Additionally: I don’t know if it’s related to OS X or Drobo but when you delete a lot of (small) files…your free space will not immediately start to increase. The opposite will be the case. This bit me when migrating my Gen2 4-Bay Drobo to a 5N with only a single spare drive. I was adventurous and thought I could just select all files and move them to the new device. From time to time I would yank a drive from the old Drobo and add it to the new one. In the beginning I was optimistic that this could work and made a plan in which order I would have to move the drives not to lose any precious drive space as “reserved for expansion”. After some hours I began to think about the lowest free space I would tolerate. When I finally cancelled the operation I was terrified as I realized that the old Drobo was going to hit 100% capacity half an hour after I stopped all file transfers. Old Drobo did run out of space and totally freaked out. That was fun. :slight_smile:

Drobo is not a passive dumb storage device. Because of it’s “magic” BeyondRaid and the housekeeping it does and the capacity that can literally change any second (drive fails so the rebuild kicks in) it bluffs and reports the volume size instead. You must not hit the 100% storage limit at all costs.

Keeping a TimeMachine sparse bundle from filling the whole drive is tedious. Using the TimeMachine App to delete old backups is not what I would call user friendly. As far as I know limiting the size of the sparse bundle does not work as this is reverted every time TimeMachine writes a backup to the sparse bundle. I am happy that the Drobo 5N offers to set up a dedicated TimeMachine share with a fixed maximum size. My Gen2 Drobo has always been much to slow to consider using it as a backup volume.

3

Time Machine is designed to fill a volume. Only then does it’s internal pruning mechanisms kick in.

I have never been comfortable with using a fixed-size sparse bundle to ‘fool’ Time Machine. It’s demonstrated time and again that it’s fragile.

What I have had error free results with is using Dashboard to create a volume that is smaller than the physical space in the Drobo.

I have a Gen2 with 4x3TB configured with 2 volumes via Dashboard. 8TB and the rest of the space.

It avoids two problems. The dreaded Drobo ‘slowdown’ and Time Machine trying to overrun the logical volume.

YMMV

4

hi, i wouldnt trust time machine, especially if you had that problem before, so it’s probably best that you limit the available space that timemachine can use, from the start.

if you have a lot of space available, then you could either create a new (fixed) volume, of a certain size, which you know is smaller than the real drobo free space, (leaving behind some space to avoid major slowdown), similar to mgriffin,
or alternatively you could make a container (similar to a volume), and map that as a virtual drive.

either way, it would instantly take up that amount of space as used, but it sounds safer than giving timemachine a free reign :slight_smile:

5

Thanks everyone. It’s all confirmed the sinking feeling that I do not have the solution for my problem, and that is, having Time Machine “owned” drive that has a usable space of greater-than 4TB. Guess I didn’t really think this through when I started using Drobo a while back. I was drawn to Drobo in the beginning because of it’s expandability well into multiple TBytes, not necessarily for the redundancy (although, it’s the redundancy that allows for the expansions).

6

hi zomo, just to check something please…

while i have to admit that i do not have a mac or know too much about how time machine works, if you want to have a drive (eg a drive letter / volume) which is bigger than 4tb, and if you want to use that on time machine, then it should be possible. and if you prefer you can use a container file, and map it to a virtual drive letter, and then use time machine on it.

(eg, assuming that the operating system and filesystem supports 4tb or more, and that you have enough overall space within your whole drobos real capacity, then it all you need is to make a volume, of say 5tb and use that for time machine)

If you later upgrade your drives and have more free space, with more space for timemachine, then yes, you might need to do some additional manual work to prepare for that, but it might not be as bad as you think :slight_smile:

what if an actual single timemachine snapshot only takes up say 50gb.
and when you later increase your drobo space by another 4tb
can you not make a separate timemachine snapshot, with a 1tb allocation.
and then once reached, can you not then zap the initial 4tb timemachine volume
and recreate it as a 7tb snapshot, and then use that going forward, until the next major expansion?

(that would still give you the original 4tb of data, plus several ongoing backups up to the 1tb of the 2nd volume.)
and then you’d have that 2nd volume full of recent data of backups, while you recreate the 1st volume as a larger one
and when that 1st larger volume reaches the 7tb, you probably wont need the 2nd smaller 1tb volume anymore)

or was your goal, to turn the whole of the drobos free space, into a self-growing-and-expandable-time-machine-volume-which-never-reaches-the-full-space-limit? (there might be a way to do that with drivebender and another device, but maybe an email to apple would be easier, to ask them to include a checkbox saying “never use more than Xtb of space” within Time Machine configs itself, and you simply edit that value to something greater as you expand) :slight_smile:

7

Paul, Thanks for the continued details …

I’ll think a little bit on your idea for an expansion … but first, it seems like I’d have to figure out if I can shrink the size of the volume created on my Mac’s drive. It’s currently set to 16TB, the maximum imposed at Drobo setup time.

And, yes, my original idea was “turn the whole of the drobos free space, into a self-growing-and-expandable-time-machine-volume-which-never-reaches-the-full-space-limit”. I like that phrasing. I’ve been having trouble expressing it succinctly, but that expression nails it. Alas, I may well not be able to do it at all, but I will look at other devices that might help me.

Stepping up a level, I find myself asking myself how bad I want an indefinitely-long Time Machine history.

8

This whole thread is very informational however it makes me nervous that I don’t understand things or that I am just making assumptions.

I have a Drobo5N with
4x3tb WD Reds
1x1TB Samsung
6.3TB Usable Storage

I have set up 5 shares for TM BU’s at the following LIMITS
4 at 500G’s for MBP’s
1 at 1TB for an iMac

All Macs are backing up fine. No-where near the limit yet. It normally will take me at 6 months to start hitting the limits set.

My Time Machine Preferences show the Drive/Volume size equal to the DD TM BU limits specified above.

So my thought is that TM space for Backup Matches the DD setup and therefor I should be fine when I start to hit the limits set above TM will do what it does and start removing the older BU’s.

Am I nieve?
Will this work like its supposed to?

Reading this I am now concerned.
However I am not worried about changing my TM or DD TM Share size later on. If I do I will just delete the BU and remove the share and create a new share at a larger size. Yes it will have to completely BU to a new sparse bundle. And that will take time to replicate that much data. but I don’t care and actually think there is benefits to a new backup anyway.

9

Hmmm. Interesting. I wonder why I have never thought of setting up a TM share for each mac individually.

So far I have not hit the limit of my (single, huge) TM share on my 5N, yet. I would say your setup should work fine as long as you actually have the physical capacity for all your TM shares available. If this does not work then the whole fancy TimeMaschine Share / Limit stuff in DroboDashboard would be futile. :slight_smile:

The OP asked about his 2nd Gen 4Bay Drobo which lacks the the whole “shares” concept with TimeMachine support of the 5N.

10

thats true, if it does not work then it has no point :slight_smile:
btw sekuehl are you able to post a few screenshots of these Limit’s in the preferences, maybe to imgur or something?

would be good to see them

11

Not sure if you want a screen shot of the drobo window or the time machine preferences window.

Here is the drobo window showing the 500G limitation for one of my TM shares.

https://www.dropbox.com/s/1txjrtk2cn2c5l6/Screen%20Shot%202014-06-14%20at%208.22.04%20PM.png

12

I’ve been setting up a new Drobo800FS to use as a network back up drive (like a Time Capsule) for our networks of 30+ Macs.

I’ve had mixed results with Drobo’s in the past. Recently I’ve been having good luck with Drobo 5n’s via thunderbolt. I’ve tried the DroboFS unit in the past and had terrible experiences with it, in particular when it comes to speed. I decided to try the 800fs now though, because it’s been a couple of years, and in light of the recent good experience with the 5d’s I thought things might be better. Honestly, If there was an 8 bay thunderbolt drobo I would have bought that, plugged it into our server, and let Mac OSX Server do the file sharing. It does a wonderful job.

As has been pointed out, since the Drobo reports the file system to be much larger than the actual capacity, you simply can’t let time machine try to use the whole thing because it will crash and burn. Since I’m backing up to a network device, time machine creates sparsebundle’s for each machine. It’s been pointed out that you can try to change the sparse bundle size, but time machine changes it right back next time it runs to the max capacity of the volume.

That would be doing via something like this which in this case is setting the size to 950GB
hdiutil resize -size 950g /Volumes/Data/Niunia.sparsebundle

I tried this, and yes in fact you can see from the logs that right away Time Machine tries to change the size.
I watch what time machine is doing in the logs by doing something like this.

tail -n 1000 -F /var/logs/system.log | grep backupd

Anyway, you can google around and see that the workaround for this is to lock the certain files in the sparsebundle like this.
hflags uchg /Volumes/Data/Niunia.sparsebundle/Info.*

I tried this and it works, you can see it by watching the logs, when Time machine tries to resize the sparsebundle, it reports a failure doing so, but continues on with the backup.

If you ever need to resize the sparsebundle later you would unlock it like this.
chflags nouchg /Volumes/Data/Niunia.sparsebundle/Info.*

I kept digging and found what seems to be a more elegant approach. You are supposed to be able to limit the size of a Time Machine backup by changing its defaults in terminal. The command looks like this. (in this example changing to 300GB 300x1024)

sudo defaults write /Library/Preferences/com.apple.TimeMachine MaxSize 307200

I’m trying this now to see if it works, but I’m pointing it out to see if anyone has tried is approach, and what your experience has been.

I’m also using TimeMachineEditor to change the default backup schedule. In my case I’m just setting it to a time after 5pm, and I’m staggering the times of all the computers by half an hour to avoid too many computer trying to back up to the drobo at the same time. I’ve already found that when this happens, if the backups are large it crashes the Drobo.

I’ve also noticed that once the backup exceeds about 100GB the speeds drop way down.
You can see from these log entries that
Jan 7 00:39:20 Fitzgerald.local com.apple.backupd[64653]: Copied 50.91 GB of 168.38 GB, 1439652 of 1586364 items
Jan 7 01:39:40 Fitzgerald.local com.apple.backupd[64653]: Copied 62.12 GB of 168.38 GB, 1448584 of 1586364 items
Jan 7 02:39:45 Fitzgerald.local com.apple.backupd[64653]: Copied 74.46 GB of 168.38 GB, 1459979 of 1586364 items
Jan 7 03:39:53 Fitzgerald.local com.apple.backupd[64653]: Copied 82.13 GB of 168.38 GB, 1464567 of 1586364 items
Jan 7 04:40:03 Fitzgerald.local com.apple.backupd[64653]: Copied 92.03 GB of 168.38 GB, 1465255 of 1586364 items
Jan 7 05:40:04 Fitzgerald.local com.apple.backupd[64653]: Copied 101.16 GB of 168.38 GB, 1465588 of 1586364 items
Jan 7 06:40:12 Fitzgerald.local com.apple.backupd[64653]: Copied 109.03 GB of 168.38 GB, 1466017 of 1586364 items

So the actual data transfer (in the middle of the night, with nothing else on the network) is about 11.6GB/hr. Pretty slow to me.

13

Did the

sudo defaults write /Library/Preferences/com.apple.TimeMachine MaxSize 307200

strategy work to limit the size of the sparse bundle? I believe it should work for OS X 10.8 Mountain Lion onwards, as mentioned here, but it would be nice to have it confirmed.