13 December 2012

Raspberry Pi Usenet downloads with NZBGet

So what's this about?

Previously, I've set up my Raspberry Pi to be a torrent box. I decided to extend this functionality so that it can also download NZBs from news servers. The main reason for this is that unless you subscribe to a premium service you will probably be subject to throttling. In my case, the throttling is the worst during the day, so it's a good idea to let the downloads run overnight. Unless you have a very quiet PC, this can get annoying, so the fan-less Pi is a good choice.

I looked around for a NZB grabber that has support for a Web UI, similar to what we set up for remote managing torrents. I found one called NZBGet. On their main page, they say:

NZBGet is a cross-platform binary newsgrabber for nzb files, written in C++. It supports client/server mode, automatic par-check/-repair and web-interface. NZBGet requires low system resources and runs great on routers, NAS-devices and media players.

That seems like exactly what we're looking for here. Kind of like last time, this might seem like a pretty long and complex process, but I have just gone into a lot of detail with step-by-step instructions to get this set up.

NZBGet Web UI
Neat remote management of NZB downloads with NZBGet

Installing NZBGet on Arch Linux

We will be adding NZBGet onto the same Arch Linux ARM base as last time. The latest version with a built-in Web UI is available in the Arch User Repository as nzbget-svn. The version available in the official repository is out of date. For more information on the Arch User Repository (AUR) and how it works, see the relevant page on the Arch Wiki: Arch User Repository.

Step 1: Grabbing the PKGBUILD

A PKGBUILD file is a simple user-made script that describes how to download and install the software. This script could contain any malicious commands so it is highly recommended to run the install as a normal user (not root). However, the script can obviously be checked by the end user, so it is a fully transparent process.

First of all, we need a centralized place to do the fetching and building. I decided to do it in a directory called "~/builds". Make sure you are logged in as a normal user, not root, and set up a build space:

$ mkdir ~/builds
$ cd ~/builds/

Now we can download and extract the PKGBUILD archive:

$ wget "https://aur.archlinux.org/packages/nz/nzbget-svn/nzbget-svn.tar.gz"
$ tar -xzvf nzbget-svn.tar.gz
$ cd nzbget-svn/

There should be a single file, PKGBUILD, in this directory. It has to be slightly modified to work on the ARM architecture. The specific version that was pulled from SVN was also missing some files for install, so they have to be removed from the install process.

Edit the PKGBUILD:

$ nano PKGBUILD

Find the line describing the architecture, and modify it to use "armv6h":

...
arch=('armv6h')
...

At the bottom of the file there is a section for installing "Other files". This should have the part commented out that tries to install missing files, because it causes the install to fail:

...
# Other files
install -d ${pkgdir}/usr/share/$pkgname || return 1

# The rest should be commented out like so:
#install -m644 -t ${pkgdir}/usr/share/$pkgname \
#AUTHORS postprocess-example.sh postprocess-example.conf \
#ChangeLog nzbget.conf.example README || return 1
...

If the PKGBUILD is ever updated this would be an incorrect example, but I will post my resulting PKGBUILD here for reference: PKGBUILD.

Step 2: Building and installing the package

Before building anything from the AUR, one needs to install a set of basic build tools. The package also has dependencies, so to avoid setting up Sudo, install the required packages dependencies manually as well:

# pacman -S base-devel
# pacman -S gnutls libpar2 libxml2 ncurses svn autoconf automake

Then, to build a package file as a normal user, simply run

$ makepkg

This will fetch all the required files and build a package file, "nzbget-svn-518-1-armv6h.pkg.tar.xz". This can be installed (as root), quite simply:

# pacman -U nzbget-svn-518-1-armv6h.pkg.tar.xz

You can check if the install was successful by locating the nzbget binary:

$ which nzbget
/usr/bin/nzbget

Configuring NZBGet

The idea is to set up NZBGet to run as a system service that automatically runs on startup. The example configuration file, nzbget.conf.example, is not downloaded as a part of this install. However, there is one available on SourceForge: nzbget.conf.example.

As root, you can download the example config file to its correct directory, or write the relevant sections out manually (I recommend downloading):

# wget "http://nzbget.svn.sourceforge.net/viewvc/nzbget/tags/0.6.0/nzbget.conf.example?revision=294&pathrev=294" -O /etc/nzbget.conf
# nano /etc/nzbget.conf

The relevant sections to modify to your liking are pretty well set out in the file, but for reference, you should probably change the generic sections such as NEWS SERVERS:

...
Server1.Host=my1.newsserver.com
Server1.Port=119
Server1.Username=user
Server1.Password=pass
...

Raspberry Pi specific settings

There are a couple of sections in /etc/nzbget.conf that need to be modified for the Pi. If you have a USB storage unit set up as specified for the torrent box, you should use this as a base path for NZBGet. Under the PATHS, section modify the NZBGet root:

...
$MAINDIR=/mnt/usb
...

The daemon should also run as its own user for security purposes. In the PERMISSIONS section, modify like so:

...
DaemonUserName=nzbget
...

This user needs to be created, so when you are done editing, make a user for the daemon with the "useradd" command:

# useradd nzbget

Setting up NZBGet to run as a daemon

Because Arch Linux uses systemd to start services, we are going to write a relevant service descriptor for nzbget. This is the first time that I have done this so it might not be an ideal service file, but it worked for me and didn't seem to have any problems. Create the service file as root:

# nano /usr/lib/systemd/system/nzbget.service

NZBGet has a built in daemon mode that can be started with the "-D" flag. Accordingly, this is the service file that I set up (remember the nzbget binary path from earlier?):

EDIT: this has been updated after a helpful comment by Tolga Cakir, thanks for your input! I have left the original comments in the relevant sections for explanation, but I am not sure if the systemd script format supports comments at all and cannot test this at the moment.

[Unit]
Description=NZBGet
# This will make sure, that nzbget is loaded AFTER setting up the network
DaemonAfter=network.target

[Service]
# Even if you set up the Daemon User in nzbget.conf, you will still
# need to set that name here. Otherwise, nzbget will be run as root
User=nzbget
ExecStart=/usr/bin/nzbget -D
ExecStop=/usr/bin/nzbget -Q
ExecReload=/bin/kill -HUP $MAINPID
KillMode=process

[Install]
# Needed to run nzbget at boot
WantedBy=multi-user.target

Now it can simply be enabled and started like a normal service:

# systemctl enable nzbget.service
# systemctl start nzbget.service

You can check if it is running (even as a normal user) by checking the current daemon status:

$ nzbget -L
Request sent
Queue List
-----------------------------------
No matches founds
-----------------------------------
Files: 0
Remaining size: 0.00 MB
Current download rate: 0.0 KB/s
Session download rate: 160.3 KB/s
Up time: 166:08:50
Download time: 07:04:07
Downloaded: 3983.16 MB
Threads running: 5
Server state: Stand-By

Setting up the Web UI

If everything is working so far, the built in Web UI for NZBGet can now be enabled. First, the running daemon needs to be stopped:

# systemctl stop nzbget.service

There are only a few changes to be made to /etc/nzbget.conf. The following section can be added to the bottom:

# Web UI
WebDir=/usr/share/nzbget/webui
ServerIP=0.0.0.0
ServerPort=6789
ServerPassword=password

Obviously, you should replace "password" with the password you intend using. The username is, by default, "nzbget". When you've saved the config file, you can start up the daemon again:

# systemctl start nzbget.service

The Web UI should be located at http://192.168.1.123:6789, or whatever you chose for your Pi's IP address and the server port. You can log in with the username "nzbget" and your password.

Future improvements

The only thing that I can think of that I still haven't gotten working (but should be possible) is SSL support for the Web UI. If I ever have the energy to set it up I will update this post.

blog comments powered by Disqus