flu0r1ne.net/logs/packaging-nebula-for-debianLast Updated: 2021-07-19

Packaging Nebula for Debian

I am close to concluding a multi-week endevor to package Nebula, a VPN-style network mesh networking overlay. If all goes well, it will be uploaded to debian/experimental within the next few days. This would also mean the package would be pulled into Ubuntu during the next merge window.

Timeline

Unfortunately, Debian does not adhere to a constant release cycle. This means the timeline is uncertain. It will likely be uploaded to experimental within a few days. See the new queue. It will stay in experimental for the next three months or so until the next release occurs. (It is incompatible with the version of protobuf in unstable. This prevents it from moving into unstable until the next version release.)

{upload queue} -> [experimental] -> [unstable (sid)] -> [testing] -> [next release]

Preemptively, I'm going to write up a set of install instructions specific to debian derivatives and briefly a few of the decisions made during the packaging process.

Installation

Step one will currently fail. See installing from experimental

For the sake of simplicity, I'm going to assume that you're setting up a network with two nodes -- one lighthouse node and a node on your laptop. Once you understand the process, it easily scales to as many nodes as you wish. Pick your favorite virtualization provider in order to set up the lighthouse. The lighthouse requires minimal resources because it functions as a mutually-reachable node which synchronizes the address mappings. You could use a home server provided that you have a static ip (unlikely) or setup dynamic DNS. The latter may introduce some instability. I'm also assuming both clients are debian derivatives and have access to apt.

If this is not the case, please consult the upstream instructions which will guide you through the processing of installing the binaries directly.

1. Install Nebula through Aptitude

You'll need to install Nebula on both endpoints.

sudo apt install nebula

2. Creating a certificate authority

The certificate authority is to "root of trust" for a Nebula network. Compromising the certificate authority's key file would compromise the integrity and security of the entire network. The upstream instructions recommend that you store the key file in a location with strong encryption [^1].

You can generate a ca.key and ca.cert file with the following command:

nebula-cert ca -name "Myorganization, Inc"

You will copy the ca.crt file to all the hosts. The ca.key file should remain secret.

3. Nebula host keys and certificates generated from that certificate authority

With your ca.key file in hand, generate keys for each node.

nebula-cert sign -name "lighthouse" -ip "192.168.100.1/24"
nebula-cert sign -name "laptop" -ip "192.168.100.2/24"

Repeate this process for each node. It is important that each is issued a unique internal ip. The IPs are specified in CIDR notation [^2]. This internal ip will be used to configure Nebula later.

4. Copy the configuration files to each host

Each host requires the host.key, host.crt, and ca.crt files to be present on the system. By convention, these are located in the /etc/nebula directory. Make sure to copy them into this directory.

For example, to copy the credentials to a lighthouse with ip 203.0.113.11 as user you may use sftp and ssh as follows:

sftp user@203.0.113.11 <<EOF
put lighthouse.key
put lighthouse.crt
put ca.crt
EOF
ssh user@203.0.113.11
sudo install -m 600 -o root lighthouse.{key,crt} /etc/nebula
sudo install -m 600 -o root ca.crt /etc/nebula
rm ca.crt lighthouse.{key,crt}

5. Configure your network

The upstream recommends that you start from an example configuration file:

cp /usr/share/doc/nebula/examples/config.yml /etc/nebula/my_network.yml

  • On your lighthouse, you'll want to change the cert and key sections to the paths /etc/nebula/lighthouse.crt and /etc/nebula/ligthouse.key. Change am_lighthoue: true. Remove the lighthouse ip from the hosts section under lighthouse.

  • On the host, change the cert and key sections to the paths /etc/nebula/laptop.crt and /etc/nebula/laptop.key. Ensure the lighthouse is added to the static_host_map and the hosts section.

Once you're done, you can test whether your configuration is valid with nebula-service -test -config /etc/nebula/my_network.yml.

6. Bringing up the tunnel

To start the tunnel, you can use the templated systemd service packaged alongside Nebula [^3][^4].

sudo systemctl start nebula@my_network

There is also a means by which a Nebula lighthouse can be run by a unprivileged user but further configuration is required [^5].

Once both ends of the tunnel have been started, you should be able to ping the lighthouse from the laptop node and vice versa.

ping 192.168.100.1
PING 192.168.100.1 (192.168.100.1) 56(84) bytes of data.
64 bytes from 192.168.100.1: icmp_seq=1 ttl=64 time=5.67 ms

7. Additional Configuration

Nebula has built-in default deny firewall. The default configuration file allows network traffic outbound. (That is, any node is permitted to initiate a connection.) In order for a node to provide services, the port mapping needs to be added to the inbound section. For instance, to permit ssh to the lighthouse:

inbound:
   - port: 22
     proto: tcp
     host: lighthouse

Now, an ssh connection should be able to be initiated via Nebula's internal ip:

ssh 192.168.100.1

Once you're happy with the setup, you can automatically start Nebula when the laptop / server boots:

sudo systemctl enable nebula@my_network

For more information on usage and configuration, you may want to take a look at nebula.yml(5), nebula(1), and nebula-cert(1).

Enjoy.

  *  . . *    * .          . *    .      *   .   .   * .
. * .     .  *      .   *    .     *   .      *    .    *
.    .   *   .    *   .      *    .     *    .      *   .
. *   *   .   *   .   *   .    .   *   .     .  *    *   .
  .     .  *   .         *    .      *   .  *    *  .   .
 *   . *      .   *   .  *   . *    .    *   .   .    *

Installation Footnotes

[^1] If you're in need of a technology to provide strong encryption, LUKS is a popular choice on Linux. Veracrypt is a venerable cross-platform encryption application. Some password managers, like KeePassXC, also allow you to attach files.

[^2] Effectively, this means all nodes will receive an ip in the form "192.168.y.x". The y part is a value in the range [0, 255] and is specific to the network. (Thus, all nodes should have the same "y" value.) "x" should be a unique ip for each node and be in the range [0, 244].

[^3] The launcher I wrote will detect the my_network.yml and my_network.yaml files. Do not specify the extension when launching the service. The launcher has no way to discriminate between my_network.yml and my_network.yaml extension so pick a distinct name for each network.

[^4] If Nebula is misconfigured, the service will fail without warning. You can check the status of the unit with systemctl status nebula@my_network. Nebula can also be started within the terminal using nebula -conifg /etc/nebula/my_network.yml.

[^5] The systemd unit that is packaged with Nebula runs the interface as root. This is what I expect most users will want. If the lighthouse doesn't need to be connected to the network, you can sudo systemd edit nebula@.serivce and simply change the User section to the user you wish to use to launch Nebula. The user will also need read access to the configuration file, key, and cert files.

Packaging Notes

I am going to create a brief summary of the changes made while packaging. I suspect other distros might benefit from some of the work done to package Debian [^6].

The Debian package differs from the packaging done on Arch. There's also a package created for NixOS but Nix is its own beast.

Templating the Unit File

If we were to use the unit file provided by the upstream project, it would fail without warning until the user fully setup the service because (1) the path of to Nebula configuration was hard coded as /etc/nebula/config.yml and (2) the user needed to change the configuration file in order for Nebula to function.

To make the relationship between the user configuration and the systemd unit clear, the systemd unit was templated. This also means that there is a clear and simple way to connect one machine to multiple Nebula networks. To accomplish this and support both .yml and .yaml extension, the systemd file executes a shell script under /usr/lib/nebula/bin/nebula-systemd-launcher passing the "instance variable" as the first argument. This script then identifies the proper configuration and launches Nebula with this configuration. The script was installed user /usr/lib so that it would not autocomplete in the user's shell.

Doc and examples

  • Man pages were generated from the nebula help flag to create nebula(1) and nebula-cert(1).
  • nebula.yml(5) man page was created to describe the configuration process. It was derived from the comments in the example configuration.
  • The config.yml example configuration was installed under /usr/share/doc/nebula/examples/ so users could copy it into /etc/nebula if they wished to use it as a starting point.

Patching for Go 1.13

Debian packages all go dependencies to maintain tight control over the versions used while building go binaries. It also packages go itself. Currently, go 1.16 is not in the debian repos [^7]. The following patch was applied since net.ErrClosed is not available in older versions of go.

--- nebula.orig/sshd/server.go
+++ nebula/sshd/server.go
@@ -1,7 +1,7 @@
package sshd
import (
-   "errors"
+   "strings"
   "fmt"
   "net"
   "sync"
@@ -116,7 +116,8 @@ func (s *SSHServer) run() {
   for {
       c, err := s.listener.Accept()
       if err != nil {
-           if !errors.Is(err, net.ErrClosed) {
+           str := err.Error()
+           if !strings.Contains(str, "use of closed network connection"){
               s.l.WithError(err).Warn("Error in listener, shutting down")
           }
           return

Dependencies

  • I packaged golang-github-nbrownus-go-metrics-prometheus since changes were made to go-metrics-prometheus that were not backwards-compatible
  • I also packaged golang-github-flynn-noise since it was not in the debian repositories

Packaging footnotes

[^6] All files used to create the package are located in Salsa (Debian's VCS). All external configuration and build rules are located in the debian directory.

[^7] Actually, it is packaged individually but not under the golang-go moniker. I initially compiled it by preloading the PATH with go 1.16 to force dh-golang to use those build tools. Thus caused dh-golang to misbehave and not harden or strip the binaries. Since the changes required to adapt Nebula to go 1.13 were minimal, I opted to create a patch.

Installing from experimental

This is a temporary aside. As mentioned above, the package is currently bouncing around Debian's packaging infrastructure. I'm assuming at the time of reading that it is in experimental. This is an internal Debian repository which allows maintainers, developers, or the curious to test the newest version of software before it enters the next Debian unstable.

If you are running buster, you cannot install it directly using apt. If you would like to test the package while it is experimental, I will offer some instructions here. All the usual disclaimers apply. This is fairly safe since Nebula is a binary package (and doesn't have any runtime dependencies other than glibc).

There is a remote chance it will segfault due to binary incompatibilities with the version of glibc. If so, run sudo apt purge nebula and try installing from source. Building it from sources would require you to pull in a plethora of experimental build dependencies.

1. Add experimental to your sources.list file

sudo sh -c "
sudo cat >/etc/apt/sources.list.d/99-tmp-nebula-overrides.list <<EOF
# Temporary pull in packages from the experimental distribution
 
deb https://deb.debian.org/debian experimental main
EOF
"

2. Demote experimental in your apt preferences

sudo sh -c "
sudo cat >/etc/apt/preferences.d/99-tmp-nebula-prefer-stable <<EOF
Package: *
Pin: release o=Debian,a=experimental
Pin-Priority: -10
"

3. Update

sudo apt update

If they above steps succeed, you should see:

All packages are up to date.

3. Force APT to install the package from experimental

sudo apt install -t experimental nebula

After installing, you can continue to creating a certificate authority. Just ensure to remove nebula when you're finished testing.

4. When you're done testing

sudo rm /etc/apt/sources.list.d/99-tmp-nebula-overrides.list \
       /etc/apt/preferences.d/99-tmp-nebula-prefer-stable
sudo apt purge nebula