Overview
The EdgeRouter runs EdgeOS, which is a Debian based platform produced by Ubiquiti. Its CPU is a Cavium OCTEON Plus, which is a 64 bit MIPS dual-core processor.
To build ProL2TP for any platform, we need a build environment: C compiler, system headers and system libraries that match the target platform. If we are building ProL2TP for a customer, our customer will supply an SDK including these. For the Ubiquiti ER-Lite platform, we download the Cavium Octeon SDK via the Cavium forums (requires registration).
The build dependencies of ProL2TP are: libmnl, libcrypto, zlib. Ubiquiti provide source code of the GPL software components that they use in their Edgerouter so we download these and build them in order that we have a set of headers of the build dependencies which match the software versions installed on the target.
Cross-compiling ProL2TP
ProL2TP source code is built using a typical GNU autotools environment. Cross-compiling is therefore straightforward - we simply point configure at the cross-compiler tools and use the familiar, three-step configure, make, install build steps.
./configure --host=mips64-octeon-linux-gnu --prefix=/usr --sysconfdir=/etc --libexecdir=/usr/lib
make -j9
make install DESTDIR=/tmp/prol2tp
tar -C /tmp/prol2tp -zcpf /tmp/prol2tp-er3-install.tar.gz .
Building L2TP kernel components
The Ubiquiti GPL release tarball includes a tarball of the kernel sources, along with the configuration options used. If kernel features required by L2TP are not enabled, the missing features must be enabled. The kernel config options enabled by Ubiquiti can be found in the .config file at the top level of the kernel source tree.
$ grep L2TP kernel/.config
CONFIG_L2TP=m
CONFIG_L2TP_V3=y
CONFIG_L2TP_IP=m
CONFIG_L2TP_ETH=m
CONFIG_PPPOL2TP=m
# CONFIG_RA_HW_NAT_PPTP_L2TP is not set
In this case, the L2TP kernel features are already enabled. We can ignore the hwnat option because this feature is not supported by the ER-Lite hardware.
If the ProL2TP Access Concentrator is required, our alternative l2tp-drivers kernel modules must be used instead of the standard drivers. These are compiled as out-of-tree kernel modules against the Ubiquiti kernel.
Reconfigure the Ubiquiti kernel and prepare it for building external modules:
$ cd kernel
$ make silentoldconfig ARCH=mips
$ make prepare scripts ARCH=mips
Build l2tp-drivers:
$ export L2TP_KCONFIG="CONFIG_L2TP=m \
CONFIG_L2TP_V3=y \
CONFIG_IPV6= \
CONFIG_L2TP_IP=m \
CONFIG_L2TP_ETH=m \
CONFIG_PPPOL2TP=m \
CONFIG_L2TP_DEBUGFS=n"
$ export L2TP_KCONFIG_DEFINES=`echo $L2TP_KCONFIG | sed 's/CONFIG_/-DCONFIG_/g'`
$ make M=../l2tp-drivers/src EXTRA_CFLAGS="-DL2TP_EXTERNAL -I../l2tp-drivers/src/include \
$L2TP_KCONFIG_DEFINES" $L2TP_KCONFIG modules
There's a fair bit going on here, so it warrants some explanation:
- The source directory for the external kernel modules is passed to kbuild (the kernel build system) with the M= parameter.
- The kernel configuration settings we need to enable building of the L2TP drivers aren't present in the Ubiquiti kernel config. So here we're passing them in as both make environment variables (stored temporarily in the variable L2TP_KCONFIG), and we're also passing them as defines via EXTRA_CFLAGS (stored in the variable L2TP_KCONFIG_DEFINES, and generated from the contents of L2TP_KCONFIG).
Finally, we install the drivers to a temporary location from which we make an archive to transfer to the EdgeRouter.
$ make M=../l2tp-drivers/src INSTALL_MOD_PATH=/tmp/kernel $L2TP_KCONFIG modules_install
$ tar -C /tmp/kernel -zcpf /tmp/l2tp-drivers-er3-install.tar.gz lib/
Delivery
The build outputs are installed on the target by extracting the tarball into the rootfs.
When a customer buys a custom build, we maintain it as a part of our normal Continuous Integration process. Each time ProL2TP is released, we rebuild binaries for the plaform.