Drobo

DroboPro/Drobo for Time Machine

Over the weekend, my 1 TB Time Capsule seems to have failed, after only 16 months or so – the power light won’t even come on, and I suspect a power supply failure.

This is forcing me to reevaluate my data storage and backup strategy.

First of all, I will never, ever, again buy a disk storage unit that does not have an easily removable disk drive, so that if the enclosure fails, I can easily move the data to another device. As it stands, if I take the Time Capsule into Apple for maintenance, their employees will have access to both my public and private data, even if it was just a power supply failure.

Now, I don’t have anything that is sensitive on that drive, much less anything illegal or even inflammatory, unless you consider my tax records for 1998 particularly salacious. It just violates my sense of privacy. Nonetheless, I really don’t like the thought of someone browsing through my data, looking for possible porn or other gems. And now, if I decide I want to prevent such snooping, I have to drill holes right through the Time Capsule, ruining the enclosure, as well as the disk drive.

I believe that I need three different forms of data recovery.

The first is a form of journaling, that permits me to undo an “oops” if I screw up and delete a file that I really needed, or want to revert back to a previous state.

The second is a backup mechanism that allows me to recover quickly if my four disk RAID 0 drive fails, as it certainly will, sooner or later. But that is primarily an issue of convenience , as the operating system and applications could be restored after some effort, assuming the important stuff, my data, can be recovered.

The third is an archive mechanism, which can record a snapshot of what existed as of a particular point in time, in case I need to prove who knew what, when, or recover data from several years ago.

Obviously, the solution to most of these problems is a Drobo, but which one, and how?

I’m not particularly concerned about the performance issues. Time Machine runs once an hour, in the background, and I don’t really care how fast it does its job – it certainly will exceed my typing speed.

One option would be to carve out a volume on the DroboPro, and use it for Time Machine. that would give me the advantage of RAID 6, which I consider invaluable.

Another option would be upgrade the drives in one of my Drobos, and use that for Time Machine. ?that would be a relatively inexpensive solution.

Now, there was a lot of discussion on this issue on the old forum, but that data is lost. So now I need some advice.

Should I carve out a partition on the DroboPro for Time Machine, and in addition use another partition for the archive and other functions, taking advantage of the RAID 6 capability? Or should I use either Drobo or DroboPro for Time Machine, and another drive for everything else?

I could probably sell several of my four Drobos and buy a second DroboPro, which would provide much better reliability, but might diminish the overall redundancy, considering the possibility of equipment failure.

Decisions, decisions. Advice is welcome.

One piece of advice. Don’t trust Time Machine as a reliable backup and certainly don’t use it as your only backup. I’ve seen it fail far too often to trust it for anything except a quick and dirty way to keep incremental versions of say a Documents folder. I personally use SuperDuper to do nightly clones to disk images (sparse bundles, not sparse images). For my mission critical data (i.e. my boot/applications/documents drive) I changed my internal raid setup so that I’m using 2 drives mirrored to hold this … my other 2 drives are striped for size/speed. If one of the mirrored drives fail I lose nothing, just get a notification from the OS and replace the failed drive at my convenience and it re-mirrors without any intervention on my part.

I also not only do nightly updated disk image backups but I rsync any critical data to two other places, one on another machine on my LAN and one offsite.

As for storing critical backups on the drobo or drobo pro I can honestly say at this point I’ve lost faith in the true reliability of these units (which is a shame) so keep this rule in mind … anything you have on your drobo/drobopro should also be backed up somewhere else.

I have a separate volume on the Pro for Timemachine. Works fine. But I lost this twice because of the current problems (you know the threads).

I also use Crashplan Pro. I can only recommend this. I store offline and on 2 separate local locations (readyNAS ans LaCie Biggest at office).
It is like TimeMachine on steroids. HIGHLY RECOMMENDED

Let’s put aside the various problems and performance issues regarding the Drobo or DroboPro, and arrive at an understanding as to whether Time Machine itself is problematic. Gerk, what kind of problems have you seen, and can you distinguish between a problem with Time Machine and the underlying hardware? Anyone else – have you had problems with Time Machine itself?

After talking to the folks at the Apple store, even if my Time Capsule is covered under the Apple Protection Plan (TBD – I have to dig up the receipts), they won’t remove the drive from the device – they will send the entire drive back to Apple, so I won’t get my data back, and can’t even zeroize or scrub it. Vice versa, if I take the drive apart, I can’t return it, even though it is only about 14 months old.

So lesson learned – never buy a data storage device without a removable drive! I’m going to take it apart, recover the internal drive, and then transfer that Time Machine image to one of my Drobos or the DroboPro. The question is, which one?

As I see it, I have the following needs:

  1. An on-going, hourly backup of my documents and other data, to guard against the “oops” factor. Functionally, Time Machine seems very well suited for this purpose, but I’ve seldom had to use it to restore any data, so I can’t talk about its reliability. In the past, I haven’t backed up my music or photographs to Time Machine (not enough space), but I certainly could if I use DroboPro for that purpose.
  2. A bootable image of a stable version of the operating system, in case of a hard disk crash, or an OS screw-up. Presently, I’m using a 2TB WD MyBook in RAID 0 for this purpose (to have enough capacity), but I might switch to using one of the Drobos instead.
  3. An on-site periodic archive of all documents and other data, but not necessarily the OS and applications. My intent is to find and use a program that will perform a hash comparison between my RAID 0 hard drive and the data in the archive, and alert me if anything has gone wrong. I do NOT recommend simply cloning the hard drive with SuperDuper, as that would merely spread any undetected errors that might occur on the primary drive. The old forum had an extensive discussion of this problem, and a real life example. In addition to hardware errors, it is also necessary to worry about malware and virus contamination.
  4. To date, I have been freezing and locking the old archive and creating a new set of archives about every six months or so, just in case a (probably human) error occurs. Eventually, I will need to do some data de-duplication (or kep buing more disk drives.)
  5. I have had two sets of those archives on site, plus the Time Capsule, plus the WD MyBook, plus yet another complete set of archives stored offsite and formatted in NTFS, and maintained in XP using SyncBackPro.

As should be apparent, keeping all of these up to date and synchronized across multiple computers and operating systems, yet without spreading any errors or contamination, is a complex task.

manfredell, I really appreciate your mentioning CrashPlan Pro. I’ve heard other people mention it, but I thought it was only for “cloud” backup to their servers – I didn’t realize I could install it on my own server. I’m probably going to buy it, as it looks like it will solve my cross-platform requirements.

Putting aside the performance problems of the DroboPro, which I consider an annoyance but not a show-stopper, I haven’t had any failures or other operational difficulties, and I really appreciate the dual-disk redundancy feature. Without it, rebuilding or replacing a drive is simply too dangerous, because a single failure would take you down completely.

Unless or until I buy a second DroboPro, I’m thinking of the following distribution:

  1. Set up Time Machine to go to the DroboPro with dual disk redundancy, attached via iSCSI to the Mac Pro, is only powered on when I am actively using it. Back up all user data except (possibly) VM snapshots to Time Machine.
  2. Set up a Drobo V2 with four 1 TB drives connected via FS800 to my Mac Pro, and use SuperDuper once a week to create bootable backup of the RAID 0 drive, complete with the current working set of data. Although the RAID 0 is 4 TB in size, it isn’t full, so it will fit on the Drobo easily.
  3. Install a CrashPlan server on my Mac Mini, together with a Drobo V2 with four 2TB drives, connected via FW400. Put a complete copy of the four different archive files on that Drobo, with three of them locked and protected against modification. Use CrashPlan to keep that archive synched with the master files on the RAID 0 drive.
  4. Duplicate the four archive folders on the DroboPro. (Normally, I wouldn’t want to have those files on the Time Machine files on the same device, but since I have another copy on the DroboV2, I am less concerned, and this gives me dual disk redundancy as well.
  5. Duplicate and synchronize the current working archive, plus the older three archives, with a Drobo V1 in my office (offsite) connected to my XP machine, using CrashPlan.
  6. Periodically create a bootable backup of the Mac Mini server to my fourth Drobo, V1 with four 500GB drives, using SuperDuper.
  7. Reconfigure the WD MyBook to be a 1 TB RAID 1 drive with mirroring, and use that to support Time Machine on my Mac Book when I am in the office.
  8. Reconfigure a 750 GB Maxtor to HFS, and use it to support a SuperDuper bootable backup for the MacBook at the office.
  9. Run an hourly backup without hash verification of all of My Documents from my XP machine to the Office Drobo using CrashPlan or SyncBackPro, and a once a day backup with hash verification.
  10. Write up everything, and put it in a safe, so that I don’t become the single point of failure and my descendants will be able to access it all!

Whew!

Does anyone see any flaws, or have any other suggestions? With this setup, I could survive SIX simultaneous disk crashes, or FOUR simultaneous hardware failures, or TWO backup software failures, and still not lose the current archive set!

(I think I just created a new benchmark for paranoid prudence!)

The issues I’ve seen are time machine related for sure. I’ve not even bothered to do much with TM and my drobopro. If you google Time Machine problems you get over 91 million hits – that should tell you something about the issues. The WORST thing that can happen is your backups are not accessible – when you assume you are safe because you have backed up religiously and the software hasn’t reported any errors. You try to actually restore from them after a hardware failure or a system update gone bad and POOF … they don’t work. I’ve seen this happen too often with TM for my liking, among other issues. If you have important data do not trust it to TM. Use TM as a secondary backup … when it works it’s very useful for getting back previous revisions of files … but that’s honestly about the only use I have for it.

I have 3 volumes on my Pro (had the same setup on my V2):
one for data
one for TimeMachine
one for SupeDuper clone

This works well and I never had problems with TM. Booting from the clone only via FW.

I also as commented use Crashplan Pro as my failsafest (!!!) backup. I’m running the Vmware VM on our ESX at the office, so I have my own cloud backup. I also backup when I’m at the office to a LaCie Quadra and at home to my ReadyNAS Pro.
So with CP I have 3 backups on different locations. All set up once and forget. I DO think this will solve most of your issues!!![hr]
P.S: Also CP is VERY space saving. TM backups full files, CP only the difference. I feel completely safe with this, regarding to my data.[hr]
P.P.S. Their support if TOP NOTCH!!!

Manfredell, CrashPlan Pro is rather expensive, at $350 for the minimum configuration (five computers), whereas CrashPlan+ gets rid of the ads, uses 448-bit Blowfish encryption (I would have preferred AES-256) and some unspecified kind of public key encryption if you choose to use it. It is also (apparently) somewhat more trouble to set up and administer – not surprising, if it is intended for a large enterprise. It costs $59.95, but it isn’t clear to me if that means that a separate license is need for both the source and the destination, but I assume it does.

Any further comments? I wish they had a FAQ on CrashPlan+ that was at the same level of depth as that for CrashPlan Pro. I guess I can download the simple version for free, and try it out.

Yes the Pro http://www1.crashplan.com/business/index.html is for enterprises and if you want to setup your OWN backup server, which you must. I wouldn’t have my data on other peoples server.

There is the free version http://www1.crashplan.com/consumer/index.html where you backup to local folders or other computers AND to their servers.

The technology is the same! It’s just a choice of which cloud you want to use, your own or theirs.
Try out the free version and have a look. This is what we did and in the end we set up our own server.

P.S: regarding the setup of Pro. It is very easy, really. You have to run your own server for it (not exclusively). Again we just downloaded the Vmware image, which is based on Linux installed a license key, defined where to store the backups (can be also multiple locations), installed the clients on the laptops and that was it.

I bought CrashPlan+, and tried it last night. However, rather reluctantly, I’ve decided it is not for me.

Several years ago, I decided that proprietary recovery schemes (like Retrospect) were “evil”. If that company failed, or the system just stopped working, then I’m screwed, and have no way of recovering the data.

CrashPlan does something right, at least from one perspective – they encrypt any data that is to be uploaded to a “cloud” computer, or perhaps a friend’s computer, so that no one else can access it, or even derive anything at all from even the file name. On the other hand, that almost guarantees that my grandkids won’t be able to access my photos and other memorabilia, which is the reason why I am so careful about backing things up.

However, from my perspective, having multiple GB or TB of files with completely incomprehensible file names leaves me completely at sea, much less my descendants. For that reason, if I feel the need to encrypt my storage drives, I’ll apply some kind of Full Disk Encryption, then at least I can still see it, once I log on.

I am not that concerned about the security of my photos (sorry, no porn!), or most of my documents. If my 1998 tax records excite anyone, well, what can I say! I will hardware encrypt on a file-by-file basis any proprietary documents from work.

On a more technical, geek basis, although I respect Bruce Schneirer as a very competent cryptographer, his Blowfish algorithm was submitted as a candidate for the Advanced Encryption Standard competition, but didn’t win.

Moreover, the ONLY independent laboratory validation of cryptographic modules is performed in conjunction with the Cryptographic Module Validation Program carried out jointly by NIST and the Canadian Security Establishment (CSE). Without such a validation, there is virtually NO assurance that the cryptographic functions are carried out competently, and without any apparent flaws. Certainly most consumers won’t have access to the source code, and would n’t know what to look for in any case. It is therefore hard to understand why a company would deliberately choose to forego sales to Federal or many state organizations, or even many private enterprises that require FIPS 140-2 validation.

Although in my personal case I am not particularly concerned about the possibility of my data being compromised, it is just one more down side, and something that raises my level of concern as to their competency and level of understanding.

So I won’t be using CrashPlan+, and I am now trying the latest version (4.04) of ChronoSync, even though it doesn’t support my XP platform. I can continue to use SyncBackPro for that function, although it would certainly be nice to have one program that would work reliably across platforms.

One more thing – CrashPlan was running at about 16 MB/S, whereas ChronoSync is running at around 44 MBPS, with a peak of around 56 MBPS. On the other hand, to be fair, CrashPlan apparently supports incremental, byte-by-byte backup synchronization, and some degree of data de-duplication.

CrashPlan is maddeningly vague about what kind of public key algorithms they use, and their key strength, and both CrashPlan and ChronoSync are equally vague about what kind of hash algorithms they use (if any) to determine whether two files are identical, and/or whether one or the other should be updated. But without that basic information, I really have a difficult time trying to make an informed decision.

Suite B,

it’s obviously your choice and you have your reasoning but from what you tell I have the following comments:

  • a backup is just a backup, your original data is untouched. If the company goes bellyup you have your data and can backup to somewhere else.
  • CP shuld be just ANOTHER backup solution not the only one, you can still clone locally but if things go wrong you have another backup somewhere!
  • the initial speed is slower but once all the data is there only the incremental bytes are transferred and then highly compressed!
  • you get as in TimeMachine as many file revisions as you want, for ALL operating systems!!
  • as for the longevity: who is guaranteeing you that in 10,20,50 years you have the software to read your todays data?

From ALL the backup solutions I have tried over the years, this was the best!
And if the company tomorrow is gone, I’m running the software on my own server!

Just my 2 cents

Here’s my advice from a person who knows a lot about hardware but not too much about software but I thought it might help you have different options.

Get any old hardware & some decent case that can grow with you later and a sata raid card ( preferably Adaptec ) & install any version of windows server but not WHS.

Here what you get :
Depending on your choice of raid, if you get a hard drive failure just replace it.
If another hardware failure then swap the board & since its a add on sata raid card it will retain its settings on any board.

If windows fail ( just incase) then reinstall on another drive & ur seetings, files are the same.

You can even share all that or make a separate partition for use with time machine.

Problem is not everyone is up to getting his hands “wet” in assembling hardware… But obviously if you can than go for it.
Speaking of this, anyone has tried this one: http://www.lime-technology.com/joomla/home

Seems VERY promising

naser, and everyone, I appreciate the input.

I’m sure many people would think I am paranoid crazy, but I’ve been bitten too many times by hardware, software, and communications problems to be complacent.

Although I don’t really need it for my personal work, for a business it is important to differentiate between a BACKUP and an ARCHIVE. So I am using my personal system to explore and emulate what I would recommend for our company, and others.

A simple backup is easy, but an archive that can provide forensic-level evidence, if necessary, as to who knew or sad what, and when, is much more difficult. Some attorneys understand this issue; most IT departments and even CIOs do not.

On the old forum, I had an article called Life-Long Data Protection which discussed some of these issues. As time permits, I’m updating that to reflect the new Drobo and DroboPro capabilities.

In that case you should also consider (tape backups or DVD, BR) as they r usually for “Life-Long” data storage, i honestly havent even read about them (TB’s) in ages but back then it was for that particular purpose.

manfredell, that software is very promising, windows home server does exactly the same its just more expensive & needs to be installed so yes its definatly a great option.

Having grown up with tape recorders, don’t get me started on the long-term reliability of tape. Nowadays, a 1 TB tape cartridge costs more than a 1 TB disk drive, offers significantly poorer performance, can’t be searched except linearly, etc. Yuck!

RAID disk drives, on the other hand, offer significantly better reliability, and a scheme like Drobo’s allows you to conveniently upgrade and migrate those drives to larger ones ever few years.

Yes, eventually the drive enclosure will fail, at which point you can by another one (assuming the company is still in business), or, if you were sufficiently prudent and had a spare or a duplicate (like a second DroboPro), you can pull the drives out and pop them into the spare, and then copy everything over to whatever new device you choose.