Drobo

iscsi target port for Drobo FS [it works! HOWTO included]

Hello everyone,

This is my very first post on this forum!
I’ve been looking for iscsi target support on the Drobo FS (something like http://stgt.sourceforge.net/ or http://sourceforge.net/projects/iscsitarget/files/), but couldn’t find an already compiled DroboApp…
I suppose it would be difficult to take the raw disk devices and share them via iscsi, but creating a -big- flat file and exporting that as an iscsi device could be easier… How come people managed to compile huge packages with lots of dependencies (eg MySQL), but not anything related to iscsi? Is there a certain firmware (OS, kenrel, etc) constraint? Or maybe Data Robotics opposes to certain features being developed due to marketing issues?
I’ve never cross-compiled for ARM, and (although it seems that following the docs, it can be done by a newbie), it would be really nice if people with the know-how and the proper development framework setup took some time to compile the iscsi target source code…

Thanks in advance!
Theodor

I can verify that i have managed to compile and install stgt in the DroboApps folder!

The package required perl (+ libconfig-general-perl module [which also required gnu make]).
A custom tgtd was created for DroboFS file structure, and finally i placed the target configuration under DroboApps/tgt/

I then created a 10G flat file with dd, created a test iscsi target and mounted it from a Windows 2008 server… And guess what! IT WORKED!

Having said that, right now I don’t feel confident enough to provide a .tgz file, since this is my very-first droboapp and the installation is a bit messy…

To be honest with you, I was not aware of STGT. My impression was always that iSCSI would require a kernel module. If you want, I can review what you did and try to package it as a DroboApp.

Ok, here is a simple iscsi-on-DroboFS HOWTO:

---- DISCLAIMER ---- USE AT YOUR OWN RISK! ----

  1. Install perl DroboApp from http://www.droboports.com/app-repository/

  2. Download the latest gnu make from http://ftp.gnu.org/gnu/make/ and compile it in the Drobo dev VM. It will produce ‘make’ for ARM architecture (I have the executable already available)

  3. Download Config::General from http://www.daemon.de/config-general/ from http://www.daemon.de/config-general/ (using ‘perl Makefile.pl’ and ‘make install’ inside Drobo, so that the appropriate module will go into the perl directory).
    These 3 steps are only needed for some perl scripts of TGTD.

  4. Download tgtd from http://stgt.sourceforge.net/ and place it inside the ./code directory of your Drobo dev VM

  5. Edit the Makefile and change the PREFIX to where you want the binaries (ie PREFIX=/root).

  6. Compile it by issuing ‘make programs’, ‘make install-programs’ in the dev machine. (This way you skip the manuals and extra docs).
    This should produce tgt-admin, tgt-setup-lun, tgtadm, tgtd and tgtimg in the /root/bin directory.

  7. Create a folder (ie tgt) in the Drobo FS (mkdir /mnt/DroboFS/Shares/DroboApps/tgt). One can also create and use ./bin ./var and ./etc folders inside tgt directory.

  8. Place the above executables inside tgt (or tgt/bin). Add appropriate path to $PATH if you need to test things.

  9. Create a service.sh that starts/stops the tgtd daemon (I have a working sample, but it could use some cleanup)

  10. Create an appropriate file targets.conf inside tgt (or tgt/etc) (I also have the default sample config that explains all the options). You may take some ideas from this excellent guide: http://www.linuxjournal.com/content/creating-software-backed-iscsi-targets-red-hat-enterprise-linux-6 to create a FILE-BASED image using dd (keep in mind that several modifications are needed for DroboFS, but the general idea is there!). WARNING: due to ext3 filesystem being used in Drobo, the MAX filesize for each image, is 2TB!

  11. Change the first line in tgt-admin to where you have perl installed (ie #!/mnt/DroboFS/Shares/DroboApps/perl/bin/perl. This is needed because even if you put a symbolic link to /bin/perl, a firmware update will delete it and then the tgt-admin will stop functioning until you recreate the symbolic link.

  12. Also, change $configfile in line 14 of tgt-admin so that it points to the file created in step 10. (ie my $configfile = “/mnt/DroboFS/Shares/DroboApps/tgt/targets.conf”:wink:

  13. start the service using ‘service.sh start’ and enjoy!

I got a ~27Mbytes/sec average READ performance with HDTune and ~21Mbytes/sec in practice, when copying (READ) many 1Mb files. Drobo was directly attached to a Win2008 server with a crossover cable and Microsoft iSCSI initiator was used. I’m using Drobo Firmware 1.2.1. Not bad!
Of course it’s not as easy and fun to setup/use/administer as the iSCSI available in the DroboPRO, but, well… it comes at no additional cost!

UPDATE: top indicates a 74% MEM usage for tgt daemon and WRITE performance is not consistent! Most of the times I’m getting ~12Mbytes/sec while there are rare cases where transfer rate drops to ~5Mbytes/sec… All these, with write-cache disabled in my targets.conf.
With write-cache enabled, I’m getting a constant ~21Mbytes/sec for the same set of files (5300 files/2.45Gb total size), but one risks data loss in case of power/network failure…

Theodoropoulos Theodoros

ps. I have already packaged tgtd for DroboFS (tgtd-1.0.22.tgz along with the sample targets.conf file, but I cannot upload files to this forum). Keep in mind that you still need steps 1-3 so that tgt-admin perl script works properly.

The app was sent to ricardo, was reviewed/recompiled/repackaged and is now hosted in: http://www.droboports.com/app-repository/stgt-1-0-22 (along with detailed instructions!).

Thank you, but keep in mind that the package hosted on DroboPorts is not exactly the same as yours. I have recreated the compilation and packaging steps from scratch.

Because of that, there is no need for any steps on the Drobo before the actual deployment. The perl module ships with the app, and only if you want to use tgt-admin directly you have to set an specific PERL5LIB variable.

So very interested in this. However, the dependency on Perl worries me as that seems to be a bit of a funky App. The dependency on OpenSSH, dynamic linking, and lack of CPAN are all things that make me nervous (but I definitely feel your pain in building Perl - I’ve tried before and gave up).

Let me clarify on the dependency on perl: the version currently on DroboPorts ships with all the extra perl dependencies that are needed by STGT to run. You only need to have the perl package installed.

In other words, the procedure goes like this:
[list=1]
[]install the perl package
[
]install stgt, and then stop it (using service.sh)
[]create the file containers
[]manually edit targets.conf to point to those file containers
[
]start stgt again (using service.sh)
[/list]
Service.sh contains the environment variables necessary for perl to find the needed modules in the stgt folder. Therefore, unless you need to use tgt-admin (a perl script), you’ll never have to worry about PERL5LIB.

*) Theoretically speaking we could repartition the FS and have most, if not all, of the storage area as an iSCSI container. I have only tested the deployment of a file based container.

Something is amiss, as when I start stgt after all setup, I get the following:

bash-4.1# ./stgt/service.sh start Starting target framework daemon ./stgt/service.sh: line 154: /mnt/DroboFS/Shares/DroboApps/stgt/sbin/tgt-admin: not found

It doesn’t appear that tgt-admin is what’s missing - the perl script exists, but all I can think is it’s in turn missing something it needs. I downloaded “Config:General” from http://search.cpan.org/~tlinden/Config-General-2.50/General.pm , but after creating the makefile I can’t actually install it as “make” doesn’t exist by default:

bash-4.1# make install bash: make: command not found

I’m rather surprised as I thought “make” would already be installed.

One other thing to note is you should log out of your SSH session between installing Perl and STGT, as otherwise your session profile won’t be updated with the paths to the perl binaries. Even so, I’m still seeing:

bash-4.1# export PERL5LIB="/mnt/DroboFS/Shares/DroboApps/stgt/lib/perl5/site_perl:$PERL5LIB" bash-4.1# ~/Shares/DroboApps/stgt/sbin/tgt-admin bash: /mnt/DroboFS/Shares/DroboApps/stgt/sbin/tgt-admin: /usr/bin/perl: bad interpreter: No such file or directory

I could always just make a symlink to perl, but I’m guessing there’s more to it than that - not to mention, I hate making any changes to the underlying Drobo system.

Nope, that didn’t work either:

bash-4.1# ~/Shares/DroboApps/stgt/service.sh restart Stopping target framework daemon Starting target framework daemon Command: tgtadm -C 0 --lld iscsi --op new --mode target --tid 1 -T iqn.2011-12.drobofs:image exited with code: 16777215.

Although I think I’m very, very close. Here’s my configuration file:

[code]bash-4.1# cat /mnt/DroboFS/Shares/DroboApps/stgt/etc/tgt/targets.conf

Empty targets configuration file – please see the package

documentation directory for an example.

backing-store /mnt/DroboFS/Shares/Archive/iSCSI/image [/code]

Hello diamondsw, although Ricardo uses a slightly different configuration, you may get some hints from the following comments:

You have to add the tgt-admin directory to the PATH environmental variable. Alternatively, specify the directory of the tgt-admin inside the service.sh script. See the HOWTO.

See step 2 from my HOWTO. Use the full path when executing make (eg /mnt/DroboFS/Shares/DroboApps/make if you have installed make inside that folder)

See step 11 from the HOWTO.

This will go away once you’re able to execute properly the perl script and you have the tgt directory in your PATH.

Hmm… It’s pretty minimal, but it should work.

I’ll have to check my profile script - I know it’s set up to add bin and lib directories for all apps; it’s possible I hadn’t set it up for sbin.
EDIT: Shame on me - it wasn’t auto configuring that…

I consider that a bug in the DroboPort. The whole point of being downloading an app is that I don’t need to set up a development VM. Looks like make should be its own port and listed as a dependency for stgt.

Needs to be fixed out-of-box for the DroboPort.
EDIT: Fixed - but needs to be packaged in an out-of-box working state.

Oddly, although I still don’t have make and thus can’t install the Perl module, restarting the service after these two changes worked - or at least appears to have. Now I have to figure out what to do client-side to connect to the iSCSI target (as Drobo’s Mac OS X initiator appears to be locked down to only work with iSCSI-supporting Drobo devices).

My fault there. There should be no need for setting PATH (it’s in service.sh) or to have to compile “make” (the DroboApp contains the perl module). Bujam1 and myself were wondering whether to update the tgt-admin script and hardcode it to my perl DroboApp or to keep it as it is and update the perl DroboApp to create the symlink.

I noticed there is an official version of perl for the FS. If they do create a symlink, I’ll probably do the same. Otherwise I’ll just hardcode the DroboPort version of perl.

FYI, even without make and the associated perl module, everything appears fine here.

So in retrospect, it seems the only changes needed are:

  1. Create a “make” DroboPort and list it as a dependency to allow installing the Perl module
  2. Handle the location of Perl - whether symlink, hardcoding, determining it from the user’s path, or just an instruction to the user - but out-of-box working would be nice. :slight_smile:

Now I just have to find a decent and free iSCSI initiator for Mac OS X (why Apple hasn’t built this into the OS is beyond me). Sadly the GlobalSAN initiator went from free to $89 when Lion came out, and the older version hangs the underlying I/O subsystems - which effectively hangs the OS. ATTO is excellent but even more expensive at $195. Drobo Dashboard licensed the ATTO driver, but it’s locked down to only work with Drobo’s own iSCSI equipment. Long and short of it is there doesn’t appear to be any free way to connect to iSCSI under Mac OS X.

EDIT: Well… with some tinkering I actually managed to cajole the Drobo Dashboard into connecting to the iSCSI volume on Mac OS X. I never got that Perl module installed, but iSCSI is up and humming just fine. Now to come up with a use for it.

After rebooting my DroboFS (for silly reasons), iSCSI did not start properly. I logged in and tried to manually stop and start the service, but this failed:

bash-4.1# ~/Shares/DroboApps/stgt/service.sh stop Stopping target framework daemon bash-4.1# ~/Shares/DroboApps/stgt/service.sh stop Stopping target framework daemon tgtd is not running bash-4.1# /mnt/DroboFS/Shares/DroboApps/stgt/sbin/tgtd --foreground (null): iscsi_tcp_init_portal(227) unable to bind server socket, Address already in use (null): iscsi_add_portal(275) failed to create/bind to portal (null):3260 can't open /proc/sys/fs/nr_open, No such file or directory tgtd: work_timer_start(150) use signal based scheduler tgtd: bs_init(319) use pthread notification

At first I thought that “(null):3260” meant it had issues getting the current IP address of the DroboFS and couldn’t initialize properly as a result. However, it looks more likely that tgtd failed for some reason on boot, and then couldn’t be shut down.

Despite the assurances of “service.sh stop”, I ended up with many, many instances of tgtd running - including a pair with a much lower PID (presumably the ones that ran on boot). After killing all of them by hand with “kill -9”, starting the service worked. It looks like the service script cannot shut down tgtd properly without the assistance of the tgt-admin script - which doesn’t work on my system because I can’t get the Config.pm perl module installed (no make binary).

So, something kept it from running properly on boot. While I can of course fix this manually as needed, I’d much rather it run as expected.

Bah, humbug! :slight_smile:

Note to self: service.sh needs a serious rework.