App submission for DroboPorts.com?

Forum History DroboApps
1

I’m going to have to go a bit wider with this message than I intended, but Ricardo’s PM inbox is full :).

I’m interested in trying to get the transmission package I’ve been updating on-and-off for the past year or so posted onto DroboPorts.com.

My update schedule is too few and far between for the liking of the community, I’m sure. The one caveat is that I don’t actually own a DroboFS appliance, but I’ve had enough information contributed from other forum members to put together a package that works.

Is there a documented submission process?

2

its probably me with my lengthy pms with ricardo :slight_smile:

3

Sorry about that. I don’t know why exactly this happened, but it seems that my recent ban somehow filled my inbox.

Don’t worry about update schedules. I update stuff only when I have enough free time and nothing better to do. And if anyone is bothered enough to get the latest version, they can always go to the site and compile it themselves. All the info is there.

None, really, since the only one submitting stuff to DroboPorts is me. Since I tend to agree with myself pretty often, I never had to ask myself for a submission process.

But if I were to give myself a submission process it would be this:

  1. A little description of the app
  2. A list of all the dependencies (libraries, interpreters, other apps)
  3. How you compiled it: statically or shared libs? (Static linking is preferred)
  4. A complete set of commands to go from source to compiled app using the “standard VM”, including downloading the source. This is the most important part. Anyone should be able to just copy and paste the commands and it should work.
  5. A complete set of steps to package the app, which includes any extra configuration files, service.sh script, admin.url, avahi profiles, etc.
  6. [Optional] Some final words about how to get the app configured, if any extra steps are required.
  7. For server apps, indicate if it’ll run as root or if it runs as an unprivileged user account, and what other measures would be needed to secure the server, if any.

I do have some style guidelines, as well. If APP is the app base folder (e.g., /mnt/DroboFS/Shares/DroboApps/APP):

  1. NEVER create files outside the APP folder, in particular anywhere outside /mnt/DroboFS. The root folder is on a flash device, which has less than a few MB free. If the flash memory gets filled, the DroboFS won’t boot, and a firmware reflash will be necessary.
  2. NEVER change system files. If you absolutely need to change system files, do it in a way that:
    2.a) won’t hurt the system if your app is suddenly deleted, and;
    2.b) can be easily undone by the end-user (provide an uninstall script or web interface).
  3. All executables have to be placed either in APP/bin, APP/libexec, or APP/sbin. APP/sbin is for reserved for executables that should be run only by ‘root’.
  4. All shared libraries have to go in APP/lib.
  5. All configuration files have to be placed in APP/etc.
  6. All logs should go in APP/var/log
  7. The pidfile goes in APP/var/run
  8. All temporary files go in APP/tmp
  9. If the app has a web interface, those files should be under APP/www
  10. Manpages are optional (the FS has no ‘man’ command), but if provided, they should be in APP/share/man
  11. As a rule of thumb, the only files in APP/ should be admin.url and service.sh.

The reasons for guidelines (3) and (4) are presented here: http://www.droboports.com/using-command-line-apps
In general these guidelines aim to standardize locations of files so it is easier to understand a new app, even if we’re not familiar with it.

Of course, these guidelines are not necessarily applicable when repackaging apps (Oracle JVM comes to mind). In that case, you should try your best to conform to these guidelines without breaking the app, especially the one about not creating files outside of the app folder.

For other apps, the point is that in the end I should be able to compile it myself if I wanted to. The whole point of DroboPorts is not only to host TGZ files – you can do that in the official DRI repository – but to provide a source of information to help build other DroboApps, by sharing the information on how each DroboApp was cross-compiled.

4

Excellent information. I don’t see any reason why the application wouldn’t conform to just about all of those guidelines.

I’ve already grabbed a copy of the dev VM and I’ll be experimenting with it shortly. Thanks again for the info.

5

What follows is my submission of Transmission, version 2.73, for the DroboPorts website. I tried to follow the general outline of the existing guides posted there. The configuration file and service.sh file are sandwiched between BEGIN/END tags, with the idea that you can pull the relevant lines and make them downloadable files, with wget lines inserted in the packaging phase as done in guides such as with lighttpd.

Let me know if there are any gaps I need to fill in, or anything else for that matter. I welcome all comments.

I can also email you, or make available from my G-Drive, the files of this build including the resulting tgz built on the cross-compiler VM.

Thanks.

[code]transmission-2.73

Overview…
Transmission is a lean, small footprint BitTorrent client with support for protocol encryption, peer exchange, magnetic links, uPnP and NAT-PMP port forwarding, watch directories, global and time based network speed limits, and comes bundled with an easy to use web interface. Transmission requires openssl, libcurl, libevent, and zlib. Gzip is required to build gunzip, itself used to un-gzip a client block list which is downloaded when Transmission is started via the service.sh script. All libraries are cross-compiled statically for easier deployment.

Preparation…
Make sure you have a cross-compiling virtual machine properly setup.

Compilation…
export DEST=/mnt/DroboFS/Shares/DroboApps/transmission
mkdir -p $DEST/bin
cd ~/code

wget http://zlib.net/zlib-1.2.7.tar.gz
tar zxf zlib-1.2.7.tar.gz
cd zlib-1.2.7
ZLIB=pwd
CFLAGS="$CFLAGS -O3" ./configure --prefix=$ZLIB --static
make clean && make && make install
cd …

wget http://www.openssl.org/source/openssl-1.0.1c.tar.gz
tar xzf openssl-1.0.1c.tar.gz
cd openssl-1.0.1c
OPENSSL=pwd
./Configure linux-generic32 -DL_ENDIAN --prefix=$OPENSSL no-shared no-zlib-dynamic --with-zlib-include=$ZLIB --with-zlib-lib=$ZLIB
sed -i -e “s/CFLAG= /CFLAG=${CFLAGS} /g” Makefile
make clean && make
cd …

wget http://curl.haxx.se/download/curl-7.28.0.tar.gz
tar zxf curl-7.28.0.tar.gz
cd curl-7.28.0
CURL=pwd
CFLAGS="$CFLAGS -O3 -I$ZLIB -I$OPENSSL/include" LDFLAGS="$LDFLAGS -L$ZLIB -L$OPENSSL" LIBS="-ldl" ./configure --host=arm-none-linux-gnueabi --prefix=$CURL --disable-shared --with-ssl --with-zlib --with-random --with-ca-bundle=$DEST/etc/ssl/certs/ca-certificates.crt
make clean && make
cd …

wget https://github.com/downloads/libevent/libevent/libevent-2.0.20-stable.tar.gz
tar zxf libevent-2.0.20-stable.tar.gz
cd libevent-2.0.20-stable
LIBEVENT=pwd
./configure --prefix=$LIBEVENT --host=arm-none-linux-gnueabi --disable-shared --enable-static
make clean && make
cd …

wget http://ftp.gnu.org/gnu/gzip/gzip-1.5.tar.gz
tar zxf gzip-1.5.tar.gz
cd gzip-1.5
./configure --host=arm-none-linux-gnueabi
make clean && make
cp gunzip $DEST/bin
cd …

wget http://download.transmissionbt.com/files/transmission-2.73.tar.bz2
tar jxf transmission-2.73.tar.bz2
cd transmission-2.73
LIBCURL_CFLAGS="-I$CURL/include" LIBCURL_LIBS="-L$CURL/lib -lcurl" LIBEVENT_CFLAGS="-I$LIBEVENT/include" LIBEVENT_LIBS="-L$LIBEVENT -levent" OPENSSL_CFLAGS="-I$OPENSSL/include" OPENSSL_LIBS="-L$OPENSSL -lssl -lcrypto" ./configure --prefix=$DEST --host=arm-none-linux-gnueabi --disable-shared --enable-static --enable-cli --enable-daemon --disable-mac --disable-nls --with-zlib=$ZLIB
make clean && make && make install

Packaging…
cd $DEST
mkdir -p etc var/log var/run
-make settings.json.default file-

{
“alt-speed-down”: 50,
“alt-speed-enabled”: false,
“alt-speed-time-begin”: 420,
“alt-speed-time-day”: 127,
“alt-speed-time-enabled”: false,
“alt-speed-time-end”: 1260,
“alt-speed-up”: 5,
“bind-address-ipv4”: “0.0.0.0”,
“bind-address-ipv6”: “::”,
“blocklist-enabled”: true,
“blocklist-url”: “http://www.bluetack.co.uk/config/level1.gz”,
“cache-size-mb”: 4,
“dht-enabled”: true,
“download-dir”: “/mnt/DroboFS/Shares/Media/Torrents/Downloads”,
“download-queue-enabled”: true,
“download-queue-size”: 5,
“encryption”: 1,
“idle-seeding-limit”: 30,
“idle-seeding-limit-enabled”: false,
“incomplete-dir”: “/mnt/DroboFS/Shares/Media/Torrents/Incomplete”,
“incomplete-dir-enabled”: false,
“lazy-bitfield-enabled”: true,
“lpd-enabled”: true,
“message-level”: 2,
“open-file-limit”: 50,
“peer-congestion-algorithm”: “”,
“peer-limit-global”: 300,
“peer-limit-per-torrent”: 60,
“peer-port”: 51413,
“peer-port-random-high”: 65535,
“peer-port-random-low”: 49152,
“peer-port-random-on-start”: false,
“peer-socket-tos”: “default”,
“pex-enabled”: true,
“port-forwarding-enabled”: true,
“preallocation”: 1,
“prefetch-enabled”: 1,
“proxy”: “”,
“proxy-auth-enabled”: false,
“proxy-auth-password”: “”,
“proxy-auth-username”: “”,
“proxy-enabled”: false,
“proxy-port”: 80,
“proxy-type”: 0,
“queue-stalled-enabled”: true,
“queue-stalled-minutes”: 30,
“ratio-limit”: 2,
“ratio-limit-enabled”: true,
“rename-partial-files”: true,
“rpc-authentication-required”: false,
“rpc-bind-address”: “0.0.0.0”,
“rpc-enabled”: true,
“rpc-password”: “{85ff11839c1acdc800cd80026f1dee2ceff144dd6d7UDkX3”,
“rpc-port”: 9091,
“rpc-url”: “/transmission/”,
“rpc-username”: “”,
“rpc-whitelist”: “192.168.1.*”,
“rpc-whitelist-enabled”: false,
“scrape-paused-torrents-enabled”: true,
“script-torrent-done-enabled”: false,
“script-torrent-done-filename”: “”,
“seed-queue-enabled”: false,
“seed-queue-size”: 10,
“speed-limit-down”: 100,
“speed-limit-down-enabled”: false,
“speed-limit-up”: 10,
“speed-limit-up-enabled”: false,
“start-added-torrents”: true,
“trash-original-torrent-files”: false,
“umask”: 18,
“upload-slots-per-torrent”: 10,
“utp-enabled”: true,
“watch-dir”: “/mnt/DroboFS/Shares/Media/Torrents”,
“watch-dir-enabled”: true
}

-make services.sh file-

#!/bin/sh

transmission

Load the DroboApps service functions

. /etc/service.subr

Required DroboApps variables

prog_dir=dirname \realpath $0``
name=“transmission”
version=“2.73”
pidfile=${prog_dir}/var/run/transmission.pid
logfile=${prog_dir}/var/log/transmission.log
conffile=${prog_dir}/etc/settings.json

export TRANSMISSION_WEB_HOME=${prog_dir}/share/transmission/web

This is needed to avoid segfault with eventlib

This information was found at https://trac.transmissionbt.com/wiki/NAS

EVENT_NOEPOLL=1
export EVENT_NOEPOLL

start(){
# make sure that the config exists
if [ ! -f $conffile ]; then
cp ${conffile}.default ${conffile} > /dev/null 2>&1
fi

# Download a fresh blocklist if there isn't one or current one is over four days old
if [ ! -d ${prog_dir}/etc/blocklists ]; then
	mkdir ${prog_dir}/etc/blocklists
fi

find ${prog_dir}/etc/blocklists -mtime +4 -type f -name level1 -exec rm {} \;

if [ ! -f ${prog_dir}/etc/blocklists/level1 ]; then
	wget -q -O ${prog_dir}/etc/blocklists/level1.gz http://www.bluetack.co.uk/config/level1.gz
	if [ -f ${prog_dir}/etc/blocklists/level1.gz ]; then

${prog_dir}/bin/gunzip ${prog_dir}/etc/blocklists/level1.gz
if [ $? -eq 0 ]; then
chmod go+r ${prog_dir}/etc/blocklists/level1
else
rm -f ${prog_dir}/etc/blocklists/level1*
fi
fi
fi

mv ${logfile} ${logfile}.last > /dev/null 2>&1

# Start Transmission
${prog_dir}/bin/transmission-daemon -g ${prog_dir}/etc -x ${pidfile} -e ${logfile} >> ${logfile} 2>&1

}

case “$1” in
start)
start_service
;;
stop)
stop_service
;;
restart)
stop_service
sleep 3
start_service
;;
status)
status
;;
*)
echo “Usage: $0 [start|stop|restart|status]”
exit 1
;;
esac

chmod a+x service.sh
~/bin/package.sh

Installation…
Download the TGZ file below and place it in your DroboApps share. If you do not have SSH access, reboot your Drobo. Otherwise, SSH in and type:

/usr/bin/DroboApps.sh install

After installation, Transmission can be accessed with a web browser at…
http://:9091

Alternately, you can use a cross platform remote GUI such as…
http://code.google.com/p/transmisson-remote-gui/

Final remarks…
Upon installation and startup, Transmission will begin listening for non-authenticated connections on port 9091 from any IP. There are obvious security implications if your DroboFS is not behind a firewall. If you’re going to be operating your DroboFS on an unsecured network, you can use the rpc-whitelist setting in the configuration file etc/settings.json to restrict network access to a set of IPs or subnets.

Those truly worried about snooping can leverage Apache or lighttpd with mod_proxy, mod_ssl, and mod_auth to place Transmission behind an HTTPS portal with authentication.

Not all of the settings in the etc/settings.json file are configurable from the web interface or remote GUI client. Information on these settings and their options can be found at https://trac.transmissionbt.com/wiki/EditConfigFiles.

Note: The Transmission daemon should be halted by calling “./service.sh stop” before editing the contents of etc/settings.json, otherwise your changes may be overwritten.

Upgrade Advice…
The etc folder contains configuration data, session statistics, and information on any torrents currently registered in Transmission. While these files and directories shouldn’t be overwritten during an upgrade, it is advisable to make a backup of this directory prior to applying any updates.
[/code][hr]

Given that the compile steps are the same for DroboShare as they are for DroboFS, I wonder if we(I) should include a set of start/stop scripts for those that are still, myself included, putzing around with DroboShares.

6

Quick question: As far as I can tell, the FS ships with a version of gunzip. Is there any particular reason you need to compile a new one?

7

No. My primary platform is the DroboShare which doesn’t come with it. If gunzip is there on the DroboFS and is in the path, then that part can be dropped with appropriate edits to service.sh, and the user would only have to do a “mkdir $DEST”.

My instructions could probably use a bit of peer review. I used the existing guides for packages the included the same material that transmission requires, such as openssl, modifying them where it made sense to. That also means that I may have left in material that doesn’t need to be there, such as the --with-ca-bundle configure option. But it may also not make that much of a difference.

In the case of zlib, I do a make install on the source directory to create include/ and lib/ subdirectories since the configure script for transmission seems to be wired to look for them under the location specified by --with-zlib.

8

It is online: http://www.droboports.com/app-repository/transmission-2-73

First of all, let me say congratulations on the excellent work. It was really easy to follow, and worked pretty much on the first try.

I hope you don’t mind that I did some changes and added some functionality. Quick summary of the changes I made so far:

  1. In settings.json.default:
    1.a) rpc-whitelist = 127.0.0.1,192.168.. (Many home networks are not 192.168.1.* anymore)
    1.b) prefetch-enabled = false (According to the transmission forums setting this to ‘true’ causes a very high load)
    1.c) cache-size-mb = 2 (This is recommended when using resource-constrained devices, such as NAS devices)

  2. I added some compiler flag magic to the compilation script. The resulting binaries are smaller and should be faster as well. These extra flags give a much bigger gain from stripping the executables. They lose about 80% of their size (from 10MB to less than 2MB).

  3. I moved the web interface from share/transmission/web to www. Service.sh updated accordingly.

  4. I added avahi support. This means you can find the web interface without even knowing the FS’s name or IP addess. This works very well with Safari, for example.

  5. Partly because of (4), I reorganized service.sh into functions, and linked it to my service.functions library.

By the way, if you have a website that you want to promote, please let me know and I’ll link it in the “Acknowledgements” section.

9

One quick note - if you’re going to add private networks to the whitelist, you’ll likely want 10.* as well (I don’t know anyone that uses 172.16.0.0 - 172.31.255.255).

10

No, no website. I keep a low profile. The name is enough for me.

I think I’ll go recompile for my own so I can see those space and performance savings too.

One other thing, the link to the forums is current pointed to the app announcement of Unison.

11

Duly noted, and already added to the default settings.

Oops. Fixed that as well.

A new version is on the DroboPorts page.

12

These are really excellent guidelines. I would like to suggest that you feature them in in a more prominent location, perhaps as an addition to one of your many excellent articles on droboports :slight_smile:

13

Well, at least some of these guidelines have been deprecated by SDK 2.0. For example, the pidfile now has a very specific location. But you are right, I should probably update them and publish in a more visible place.