Drobo

Can an FS be used with iMovie?

Hi everyone.

I already have an FS and I love it - I use it for storing all my photo’s and movies. Both my PC’s and my MacBook Air can transfer files to and from it.

My Macbook Air only has a 256 GB hard drive though - not enough.

I discovered that using an external drive with iMovie would not work unless the drive was Mac formatted. If the drive is not Mac formatted then iMovie simply cannot see it - very frustrating.
I purchased a Mac shop sold 3TB G-drive and iMovie can use this fine.

Now I have ordered my second FS and have suddenly realised that it might not work with iMovie - up until now I mostly use PC’s for Photo and Movie editing.

I don’t want to reformat my existing Drobo, but can my new one be Mac formatted?

I have just thought of this and am now quite concerned!

Thanks for any answers or help.

Conan.

Well, a bit of a complex answer. For one thing, you cannot format a DroboFS. For local storage (such as a direct-attach Drobo or Drobo S) you can format it to your heart’s content and treat it like any other local disk. However, network storage is different. After all, you’re not mounting it directly on your desktop, you’re accessing it via a network protocol - AFP, SMB/CIFS, NFS, etc. Each one of these will have its own quirks of how it presents the underlying storage. For instance, while the Drobo FS is formatted as EXT3 (a Linux filesystem, and something that cannot be changed), accessing it via AFP will make it look like a Mac disk, and you can do Mac-like things with it (custom icons, resource forks, extended attributes). It’s then the responsibility of the software providing the AFP layer (netatalkd) to figure out how to store these foreign attributes on the underlying Linux system.

There are occasionally specific filesystem features that are required by programs - or at least heavily recommended. Some programs need sparse file support, some need particular case sensitivity, some need long file names.

My personal hunch is that iMovie will work fine with files stored on a Drobo FS. But since you have one sitting there, why not just try it and see?

Thanks for your reply.

However sadly I don’t fully understand some of your terminology :slight_smile:

My existing Drobo FS will not work with iMovie - so I cannot use it to store anything that I want to access with iMovie.
I find this very frustrating as the Macbook can access it to transfer files etc.

But as the Macbook has such a small hard drive I wanted to store all of the iMovie projects etc on the Drobo - for the drive redundancy etc.

You talked about AFP and allso Netatalkd, can you explain them further?

As I want to use the new Drobo purely with the Macbook, I am now wondering if there is a way of making the Mac think that it is mac formatted - using the network protocol?

However I really have no idea.

Or maybe I would be better suited with a different model of Drobo - can the other models can be formatted as a Mac partition?

Thanks again :slight_smile:

Conan.

I’ll answer this first as it’s a whole lot simpler. Yes, any of the “direct attach” Drobo’s would work for you, so the original USB Drobo, 2nd-gen Firewire800 Drobo, or the new Drobo S would all work. They’re just large block-storage devices and the host can do what it will with them - including format as HFS+.

If you’re on a MacBook, you’re going to use USB 2.0 or Firewire 800 to connect to such a Drobo, so I’d recommend the 2nd-gen Firewire800 Drobo. You can typically find this for ~$300 online. The newer Drobo S does provide you with an additional drive bay which is useful, but it also provides newer connectivity options that you can’t easily use on a Mac - the eSATA and USB 3.0 won’t do you any good.

[quote=“ConanButcher, post:3, topic:6016”]
However sadly I don’t fully understand some of your terminology :-)[/quote]

No problem - I’ve been doing enterprise storage for a decade or so, so don’t sweat it. :slight_smile:

Block Storage

Let’s start with disks. They’re what’s called “block storage”, because when you format them, all they are is a huge long list of blocks to write data to. This is actually a bit of an abstraction layer in that it sits on top of the even larger list of simple 1’s and 0’s that make up binary storage - but it would suck to deal with that many addresses, so we group it into chunks called blocks. The block size is set when the drive is formatted - typically 512 bytes, 1K, 2K, or 4K. There’s still no filesystem, no data - just a lot of blocks of a fixed size.

Every local disk - your internal hard disk, external firewire or USB disks (including USB keys, inserted SD cards, etc) are block devices. The computer can at any point change data at the block level, so for instance you can write changes to a 2GB file without reading in the whole file, then writing it out again.

RAID systems are also typically block-level, so all the magic is done by a RAID controller with how to lay out the blocks. RAID is a perfect case of abstraction - higher layers (i.e., your computer) still see a big block device that can be formatted as needed, and the controller takes care of all of the magic. Drobo’s BeyondRAID is a little different, but it’s still a good way to think about it.

Filesystems

Once you have a concept of block storage as a base abstraction, you can layer another on top of it - the filesystem. That includes all manner of conventions for how to lay out the data and what you can do with it. How long can a filename be? What characters can be used? How many folders can be created? What’s the largest size of a file? How big a disk (or how many blocks) can we address? Do we have permissions? Etc. Everything that an OS needs is generally encapsulated in the filesystem, which is why every OS tends to have its own - Macs use HFS+, Windows used to use FAT32 and these days prefers NTFS, Linux typically uses EXT3.

Networked devices like the Drobo FS have an interesting problem. They have to manage the data on disk, so they’re doing all kinds of proprietary magic at the block layout level. Then they need a filesystem so they can actually store their OS and your data. Pretty much all consumer NAS devices use Linux, and so use EXT3 for the filesystem internally.

Networking

Networking is then another layer of abstraction. When you connect to the DroboFS, you’re not getting direct access to the blocks (well, not without iSCSI, but that’s not for this discussion), you’re just getting access to the files. And it’s indirect - you’re using a network protocol to ask the DroboFS to do certain things - list the files in a directory, get this file, write that file, change the name. It’s a fairly limited subset of commands, reflecting the fact that it doesn’t know what the remote device is using for a filesystem. Some things just can’t be done across this type of abstraction.

Just like filesystems, each OS has tended to create its own network protocol. Apple wanted really easy discoverability and support for their own proprietary things, so they created AFP (Apple Filing Protocol). Microsoft needed different workgroup and sharing features, and created CIFS (Common Internet File System). Unix and Linux needed robust permissions and the ability to seamlessly link network storage into their filesystems, and use NFS (Network File System). All of these work in different ways and provide different feature sets. One of the Drobo FS’s biggest jobs is translating all of the features they need onto the EXT3 filesystem that lives underneath. And that can be tricky.

For instance, when Mac users are having problems with AFP or Time Machine, what’s really likely at fault is something called “netatalkd” - that’s the software that runs on the Drobo and provides the AFP network protocol that Mac OS X uses to connect. It’s responsible for making Mac features like resource forks, custom icons, hard links, permissions, etc work on a foreign filesystem like EXT3 that doesn’t understand those concepts, or uses them in a different way. Similarly for Windows, “samba” handles the CIFS connectivity for Windows clients, so if they’re seeing a problem you’d typically look there.

Hey, what about my question?

So what does all of this have to do with iMovie? Well, as you’ve seen, using block storage or a filesystem directly has some different capabilities than a network drive like the Drobo FS. With block storage you can made edits without reading entire files, and with direct filesystem access you can store and manipulate all manner of data unique to that filesystem. One of these (or both) is why iMovie is insisting on a local HFS+ disk. Over the network, all it can work with is AFP, which just may not be fine grained enough to iMovie to do the stuff it needs to.

If you’d been lucky, iMovie would have just been requiring HFS+ because it needed something unique to it - case-preserving yet insensitive file naming, resource forks, or extensible metadata. Most of these can be emulated across AFP reasonably well. Unfortunately, it sounds more likely that it needed those and direct block access, so it can quickly edit and manipulate large files. And for that, you just have to have a local disk formatted as HFS+.

Didn’t you say something about iSCSI?

Yes. iSCSI is an interesting method of providing block storage - just like your hard disk or USB key - across the network to a device like a Drobo FS. It’s kind of cool in that you can keep all of your storage across the network and centralized, but still access it exactly like a local disk, including formatting it however you want, and typically with outstanding performance (as you’ve removed some bottlenecking layers of abstraction). Unfortunately there are a lot of reasons this isn’t included on the DroboFS and similar products. iSCSI (and this approach in general) has significant limitations that make it unsuitable for most people and most situations - most importantly, only one system at a time can connect and “own” the storage. Since the Drobo FS and most similar products are designed for multiple computers on your network to connect to the same storage, iSCSI hasn’t been a significant feature.

So - unfortunately it sounds like iMovie needs features only available from a local HFS+ drive - probably direct access to blocks instead of files, and some of the more esoteric filesystem features HFS+ offers. Unfortunately, this means sticking with your external hard drive.

Wow, no iMovie with the FS.

Return coming…

Well, no iMovie with any NAS - it just needs things only a local disk can provide, and that’s not the DroboFS’s fault.

Now, that said, there has been some very alpha-level progress on iSCSI support on the DroboFS - which as mentioned above would alleviate these issues. However, more work still needs to be done on the DroboPort, and there are some significant downsides (a big one being you have to permanently allocate a chunk of storage for it - which doesn’t shrink even if it’s empty).

diamondsw is 100% right.

Video editing on a NAS (or even server) using a standard file-sharing protocols is ill-advised. The file-sharing protocols (CIFS aka SMB and AFP) are not friendly for video editing use.

Direct-connection allows direct access to blocks, as does iSCSI, which acts more like a direct connection that happens to go over Ethernet.

It can work, but you must be patient, and to be honest, most people aren’t that patient, and most software applications expect the video assets to be on high-speed local storage.