Drobo

Introducing DroboLibs + zlib example

Since I wrote the post about setting up a VM to cross-compile for the FS, I have been trying to make a nice how-to post on how to create your own DroboApps.

So, after fighting it out with OpenSSH, here is the first post on what you have to do not only to cross-compile, but also to be able to get a droboapp out of it. The biggest problem was to figure out an elegant way to handle the libraries required by most programs.

This is where the concept of DroboLibs comes in: DroboLibs are packaged like DroboApps, but they do nothing but provide the files that other libraries or apps require.

You might be asking, wouldn’t this make a mess out of the droboapp folder? In a way, yes. As we move on to more complicated applications and dependency chains, it is expected to have several folders on the droboapps share whose only purpose is to provide the libraries that other apps need.

In other words, it will make the droboapp folder more populated, but on the other hand it will make it easier to upgrade dependencies as we compile them in the future.

Please notice that the mere existence of more folders won’t necessarily cause a longer start-up time for the FS, since the FS just does something if there actually is a service.sh (or install.sh) file in the folder. In fact, I think we can make the distinction between DroboApps (i.e., software that will run on the FS, and therefore may cause a performance impact) and DroboLibs (i.e., software that is just there to support other software, and whose impact is only on disk space).

To give you an example of a DroboLib, let’s make a DroboLib out of zlib.

Pre-requisites

Make sure you got your VM set up as indicated previously.

What has changed from the original setup?

Not much. The major difference is that we no longer throw everything in a single /usr/arm folder. From now on, we compile everything (including libraries) as separate droboapps.

In particular, the content of crosscompile.sh must be changed to this:

export PATH=/usr/local/arm-2007q1/bin:$PATH
export CFLAGS="-march=armv5te"
export CPPFLAGS=${CFLAGS}
export CXXFLAGS=${CFLAGS}
export LDFLAGS=""
export CC=arm-none-linux-gnueabi-gcc
export CXX=arm-none-linux-gnueabi-g++
export AR=arm-none-linux-gnueabi-ar
export RANLIB=arm-none-linux-gnueabi-ranlib

The differences are on CFLAGS and LDFLAGS. Before:

export CFLAGS="-I/usr/arm/include -march=armv5te"
export LDFLAGS="-L/usr/arm/lib"

After:

export CFLAGS="-march=armv5te"
export LDFLAGS=""

Preparations

Make sure you are in the cross-compile virtual machine. As previously indicated, we have chosen /root/code as the place to host all the files, so make sure that you are in that folder before starting.

Compilation summary

The following sequence of commands takes you from zero to compiled library. Explanation below.

wget http://zlib.net/zlib-1.2.5.tar.gz
tar zxf zlib-1.2.5.tar.gz
cd zlib-1.2.5
./configure --prefix=/mnt/DroboFS/Shares/DroboApps/zlib
make
mkdir -p /mnt/DroboFS/Shares/DroboApps/zlib
make install

It is the usual fare until the ‘configure’. Notice that at that point we are defining the destination as a folder inside the DroboApps share. That is the main difference for zlib. Notice also that after the ‘make’, we make sure that the folder given on the configure step actually exists.

Packaging summary

The following sequence of commands takes you from compiled library to drobolib. Explanation and content of files below.

cd /mnt/DroboFS/Shares/DroboApps/zlib
nano README.txt
# copy and paste the content of README.txt from below
nano install.sh
# copy and paste the content of install.sh from below
chmod a+x install.sh
tar czfv ../zlib-1.2.5.tgz *

Content README.txt:

Name: zlib Description: zlib compression library Version: 1.2.5 Requires: n/a

Content install.sh:

[code]#!/bin/sh

name=“zlib” # library name
lib_dir=dirname \realpath $0``
droboapps="/mnt/DroboFS/Shares/DroboApps" # droboapps folder

ln -fs ${lib_dir} ${droboapps}/${name}
[/code]
(Note: this install.sh can be reused for all drobolibs. Just make sure you edit the name variable and you’re good to go. If you leave zlib there you are going to make a lot of people unhappy.)

The README.txt file provides useful information once the drobolib has been installed. The install.sh file makes sure that there is a symlink from the current version (1.2.5) to a ‘default’ name (‘zlib’). That is necessary to simplify the compilation of app that depend on zlib, since we will be able to reference the default name instead of a specific version of the library.

zlib-1.2.5.tgz can now be distributed and simply be placed in the DroboApps share for usage.

Recommended conventions and final words

I think it is necessary to name the drobolib tgz file with the whole version information (e.g., zlib-1.2.5.tgz is ok, zlib.tgz is not ok). That way it is easier for everyone to find out which version a given tgz contains.

I think also that it is ok to name the ‘default’ folder for the drobolib without the version information (e.g., DroboApps/zlib/). This way makes it easier to upgrade, since libraries can be updated without messing the path for all the dependent apps. People cross-compiling apps that require specific versions of the libraries could then make specific versions of these libs using the complete name (e.g., DroboApps/zlib-1.2.1/) for their specific app.

A compiled version of zlib 1.2.5 as a drobolib can be found here. As usual, comments and feedback are welcome.

P.S.: For reference, the contents of the zlib droboapp folder are:

[code]root@ubuntu:/mnt/DroboFS/Shares/DroboApps/zlib# ls -laR
.:
total 28
drwxr-xr-x 5 root root 4096 Feb 7 01:28 .
drwxr-xr-x 8 root root 4096 Feb 7 01:28 …
-rw-r–r-- 1 root root 78 Feb 7 01:00 README.txt
drwxr-xr-x 2 root root 4096 Feb 4 12:41 include
-rwxr-xr-x 1 root root 211 Feb 7 01:14 install.sh
drwxr-xr-x 3 root root 4096 Feb 4 12:41 lib
drwxr-xr-x 3 root root 4096 Feb 4 12:41 share

./include:
total 104
drwxr-xr-x 2 root root 4096 Feb 4 12:41 .
drwxr-xr-x 5 root root 4096 Feb 7 01:28 …
-rw-r–r-- 1 root root 13357 Feb 4 12:41 zconf.h
-rw-r–r-- 1 root root 79564 Feb 4 12:41 zlib.h

./lib:
total 304
drwxr-xr-x 3 root root 4096 Feb 4 12:41 .
drwxr-xr-x 5 root root 4096 Feb 7 01:28 …
-rw-r–r-- 1 root root 149322 Feb 4 12:41 libz.a
lrwxrwxrwx 1 root root 13 Feb 4 12:58 libz.so -> libz.so.1.2.5
lrwxrwxrwx 1 root root 13 Feb 4 12:58 libz.so.1 -> libz.so.1.2.5
-rwxr-xr-x 1 root root 146912 Feb 4 12:41 libz.so.1.2.5
drwxr-xr-x 2 root root 4096 Feb 4 12:41 pkgconfig

./lib/pkgconfig:
total 12
drwxr-xr-x 2 root root 4096 Feb 4 12:41 .
drwxr-xr-x 3 root root 4096 Feb 4 12:41 …
-rw-r–r-- 1 root root 282 Feb 4 12:41 zlib.pc

./share:
total 12
drwxr-xr-x 3 root root 4096 Feb 4 12:41 .
drwxr-xr-x 5 root root 4096 Feb 7 01:28 …
drwxr-xr-x 3 root root 4096 Feb 4 12:41 man

./share/man:
total 12
drwxr-xr-x 3 root root 4096 Feb 4 12:41 .
drwxr-xr-x 3 root root 4096 Feb 4 12:41 …
drwxr-xr-x 2 root root 4096 Feb 4 12:41 man3

./share/man/man3:
total 16
drwxr-xr-x 2 root root 4096 Feb 4 12:41 .
drwxr-xr-x 3 root root 4096 Feb 4 12:41 …
-rw-r–r-- 1 root root 4235 Feb 4 12:41 zlib.3
[/code]

One could delete the share subfolder, since there is no ‘man’ on the FS. A minor space gain, in my opinion.