ProL2TP performance and scalability
It's difficult to give a universal answer to the question of "what server do I need for my deployment?", because requirements vary depending on the specifics of the application.
This said, it is useful to have ballpark figures to work from when planning a deployment. This article takes a look at such guideline figures first, before going on to discuss the factors which dictate ProL2TP scalability and performance.
ProL2TP scalability: an example using EC2
| EC2 Instance | CPU | RAM | Max L2TPv3/PPP session count |
|---|---|---|---|
| t2.micro | 1 x Intel(R) Xeon(R) @ 2.40GHz | 1GB | 4000 |
| t2.medium | 2 x Intel(R) Xeon(R) @ 2.40GHz | 4GB | 20000 |
| t2.large | 2 x Intel(R) Xeon(R) @ 2.40GHz | 8GB | 46000 |
These example figures were for an Amazon EC2 setup instantiating L2TPv3/PPP sessions, and give a conservative estimate of the number of sessions which could be practically deployed on a system of this specification.
The test setup used ProL2TP as LAC and LNS, and created multiple PPP pseudowires inside a single L2TPv3 tunnel.
The operating system used was Ubuntu Bionic, with some modifications to the base system configuration to disable components which became swamped when many network interfaces were created.
For each test population we created sessions until either the LAC or LNS ran out of memory, as indicated by the Linux Out-Of-Memory handler running.
What limits ProL2TP scalability?
When testing ProL2TP at scale there are a range of factors which come into play.
We can break these factors down into those relating to scalability of the data plane (i.e. user traffic over L2TP pseudowires); and those relating to the control plane (i.e. the L2TP protocol messages associated with establishing and maintaing tunnels and sessions).
Dataplane scalability
By default ProL2TP uses the Linux kernel dataplane. Each session is represented as a Linux network interface.
Interface name limits
When allocating a new network interface, Linux has the ability to autogenerate a name for that interface using a pattern specified by the kernel driver for the specific pseudowire type, e.g. ppp0, ppp1, ppp2, l2tpeth0, l2tpeth1, l2tpeth2, etc.
The algorithm used for autogenerating these names uses a page of memory to generate these names. For typical Linux page size this imposes a limit of 32767 autogenerated interface names.
If this is likely to be a restriction for your deployment, it is possible to work around by specifying the interface name on a per-session basis in ProL2TP configuration.
Protocol overhead
Each data packet passed over an L2TP session will include an L2TP header, in addition to any pseudowire-specific encapsulation (e.g. ppp).
The size of an L2TPv3 data packet header may vary depending on ProL2TP configuration. Firstly, ProL2TP supports both IP and UDP encapsulation: IP is slightly more efficient than UDP. Secondly, for either encapsulation type an optional cookie of up to 64 bits may be included. ProL2TP doesn't set a cookie by default.
IPSec
IPSec encrypts IP traffic between two hosts, and can be used with L2TP deployments, for example in some VPN implementations.
Where IPSec is used, the two peers must encrypt and decrypt IP traffic on the fly, which adds computational overhead.
MTU misconfiguration
Because L2TP encapsulates packets with an L2TP header, the L2TP link MTU is effectively reduced by the size of that header.
It is important therefore to propagate the correct MTU setting to clients in order to fully optimise the L2TP link. If the client MTU is larger than the L2TP link MTU, the L2TP node (LAC or LNS) will have to fragment the packet in order to pass it over the link. This adds overhead to the dataplane.
By contrast, if the client configures an MTU which is much smaller than the L2TP link allows, the link will not be used efficiently. Although this doesn't add computational overhead per-se it will lead to sub-optimal use of the L2TP link bandwidth.
System software
Many Linux distributions have system components which listen for network interfaces being created in order to perform integration functions such as updating DNS settings, or executing user scripts.
These components are often designed with a typical desktop or server use case in mind, where a system might have at most a handful of network interfaces to deal with.
Adding thousands of network interfaces can cause these components to consume excessive CPU or memory resources, which can become a practical scalability limit for a ProL2TP system.
We typically find that once a problem component has been identified it's possible to work around using appropriate configuration, or even by disabling the component altogether if it isn't required for the deployment use case.