Freedom Internet Fibre optic link configuration
Introduction
This document describes using a Linux computer instead of a FritzBox as internet router.
Note: The default configuration files and scripts that come with most
software packages assume that there already is a internet link up and
running; In other words a router somewhere on the LAN.
This however, is about a DIY router. A setup that initiates and then
maintains an internet link. This requires some tweaking of config files
and scripts. Depending on the desired setup, you may or may not want to
run some or all of the scripts in '/etc/ppp/ip-up.d/',
'/etc/ppp/ip-down.d/' and /etc/wide-dhcpv6/, modify them, replace
them or add your own.
Setup
Me
I have various Debian GNU Linux PCs hooked up to a gigabit Ethernet network.
In my setup, everything is static. There is no DHCP (at least not for NIC IP
address configuration purposes) or routing daemon. Everything is configured by
means of static config files; The resolvconf package is not installed:
I run my own nameserver so /etc/resolv.conf points to my own server! And
network-manager isn't installed either.
Freedom Internet emailed me my IPv4 address and IPv6 /48 prefix (both are
static), which I then cut and paste to the various config files (convert upper
case hex digits to lower case first).
I don't have systemd; I use a classic SysV boot (actually insserv).
The Ethernet interface for my LAN is 'eth0'. The link to my NTU
(fibre media converter) uses 'eth1'. Both are gigabit Ethernet;
┌────────┐
┌─────┐ eth1 │ Linux │ eth0
FttH <─>──────────────┤ NTU ├────────────────┤ Server ├────────────────<─> LAN
Gbit └─────┘ Gbit │ │ Gbit Other computers,
Fibre Optic Ethernet └────────┘ Ethernet SIP phone
Cable and printer
I do not use the 'FritzBox';
The NTU (fibre media converter) is directly connected to the Linux server!
The Linux server acts as a internet router; It contains a
firewall, a VoIP PBX, webserver, nameserver,
mailserver, timeserver and a web proxy server.
Only a part of the IP addresses in the IPv6 /48 are used;
I use some the first 256 addresses (/120) in the 2nd /64 range.
In other to avoid NDP exhaustion attacks, the rest are firewalled;
Results here.
Note: Newer systems may use
different ethernet interface names such as enp3s0 and enp5s0. You may need
to modify the stuff below accordingly.
Freedom Internet
For fibre optic- and VDSL(2) links, Freedom Internet uses
PPPoE
in IEEE 802.1Q
VLAN 6.
Baby
jumbo / rfc4638 is
supported. So you can use an IP MTU of 1500, provided your NIC supports a MTU
> 1500;
┌───────────────────────┐ │ +VLAN hdr 1512 │ │ ┌───────────────────┐ │ │ │ PPPoE MTU 1508 │ │ │ │ ┌───────────────┐ │ │ │ │ │ IP MTU 1500 │ │ │ │ │ └───────────────┘ │ │ │ └───────────────────┘ │ └───────────────────────┘
Otherwise you need to reduce your IP MTU;
┌───────────────────────┐ │ +VLAN hdr 1500 │ │ ┌───────────────────┐ │ │ │ PPPoE MTU 1496 │ │ │ │ ┌───────────────┐ │ │ │ │ │ IP MTU 1488 │ │ │ │ │ └───────────────┘ │ │ │ └───────────────────┘ │ └───────────────────────┘
To get Freedom Internet to route the IPv6 /48 to you, you need to do a DHCPv6 Prefix Delegation request. To this end the Debian package 'Wide-DHCPv6-Client' is installed.
Required packages
- ppp
- vlan (or iproute2)
- wide-dhcpv6-client
'Quirks'
PPPD and IPv6
- For IPv6 the PPPD uses IPv6 link-local addresses for the PPP link ends. Which actually makes sense, but may seem a bit odd at first.
- These addresses may be different each time the PPP link comes up.
- It does not necessarily set an IPv6 route the the remote end.
- It does not necessarily set an IPv6 default route, even if the config file says it should!
Wide-DHCPv6-Client
- dhcp6c won't start if the PPP link isn't up.
- dhcp6c is hard to kill.
- You need to remove the DUID file (/var/lib/dhcpv6/dhcp6c_duid) before a (re)start.
- dhcp6c doesn't necessarily know where to send config requests to. You need an IPv6 route to the outside world (IPv6 default route) for dhcp6c to work. Without this it will complain: 'client6_send: transmit failed: Network is unreachable'.
The DUID (DHCP Unique Identifier) contains the media type, a mac address and a timestamp in seconds since 00:00:00 Jan 1st 2000 UTC.
Config
Please note that I don't use systemd, which requires a different configuration!
Ethernet
Speed
The network card connected to the NTU needs to support 1 Gbps.
Modules
You may need to load the '8021q' module for VLAN. Just put it in /etc/modules or /etc/modules-load.d/local.conf;
8021q
This will load the module on boot.
You can also load the module manually with modeprobe;
~# modprobe 8021q
/etc/network/interfaces;
Note: This is pre Debian Stretch (9).
Newer stuff below.
auto lo iface lo inet loopback auto eth0 iface eth0 inet static address 192.168.1.1 netmask 255.255.255.0 network 192.168.1.0 broadcast 192.168.1.255 # IPv6 iface eth0 inet6 static address 2001:db8:1234:1::1 netmask 64 # Fibre optic link auto eth1 iface eth1 inet manual mtu 1508 # VLAN 6 # VLAN number is derived from interface name auto eth1.6 iface eth1.6 inet manual pre-down ( poff provider ) & up ip link set eth1.6 up mtu 1508 down ifconfig eth1.6 down post-up ( pon provider ) &
If you stick to the 'ethx.y' naming scheme, where 'x' and 'y' are decimal
numbers, you don't need a 'vlan-raw-device ethx' statement.
'manual' means that it doesn't get configured (IP address assigned to it)
right now (or ever).
You need the 'up ip link set eth1.6 up' statement to actually make the
interface go up. Otherwise it just gets configured without going up.
If you want to check this: 'ifconfig -a' shows all interfaces, 'ifconfig'
just the ones that are actually up.
'post-up ( pon provider ) &' actually starts the PPPD.
Note: One might think that the eth1 MTU needs to be 1512 (1500 + 8 + 4).
However, the eight extra bytes for PPPoE are part of the Ethernet payload
and therefore get added to the MTU (1500 + 8 = 1508).
The four extra bytes for VLAN on the other hand, are part if the Ethernet
header, not the payload (See
IEEE 802.1Q).
Debian Stretch (9) and newer (2017+)
The processing of '/etc/network/interfaces' has become real fussy in Stretch,
to the extent that I consider it broken. For instance, editing the file and
then doing a '( ifdown eth0 ; sleep 1 ; ifup eth0 ) &' doesn't work
anymore. What does work is changing things on the commandline or putting
stuff in a script.
The setup above doesn't work on Debian Stretch either. Below a fix:
auto lo iface lo inet loopback # The primary network interface auto eth0 iface eth0 inet static address 192.168.1.1 netmask 255.255.255.0 network 192.168.1.0 broadcast 192.168.1.255 # IPv6 iface eth0 inet6 static address 2001:db8:1234:1::1 netmask 64 pre-down /usr/local/sbin/wan-link stop post-up ( /usr/local/sbin/wan-link start )
Which calls '/usr/local/sbin/wan-link';
#!/bin/bash
check_syslog()
{
CNT=0
while [ ${CNT} -lt 10 ]
do
# Wait for syslogd
if ( netstat -an | grep ":514" > /dev/null )
then
break
fi
sleep 1
let CNT+=1
done
}
case "$1" in
start)
check_syslog
ifconfig eth1 mtu 1508 up
vconfig add eth1 6
ifconfig eth1.6 mtu 1508 up
pon provider
;;
stop)
poff provider
ifconfig eth1.6 down
ifconfig eth1 down
;;
*)
echo "usage: wan-link {start|stop}"
;;
esac
To make sure everything is logged, the scripts waits for the syslogdaemon.
Note: the 'check_syslog()' function
expects a syslog daemon listening on port 514 (syslog)! If your syslogd
doesn't, you have to check in some other way.
A '( netstat -anp | grep syslog | grep "dev/log" > /dev/null )' might work.
Use IP
If you want to use 'ip' instead of 'ifconfig' and 'vconfig', the following will probably work;
ip link set dev eth1 mtu 1508 ip link set dev eth1 up ip link add link eth1 name eth1.6 type vlan id 6 ip link set dev eth1.6 mtu 1508 ip link set dev eth1.6 up
Note: This hasn't been tested!
PPP
/etc/ppp/peers/provider
ifname wan noipdefault +ipv6 ipv6cp-use-ipaddr defaultroute connect /bin/true noauth persist lcp-echo-interval 10 maxfail 0 debug holdoff 10 mtu 1500 mru 1500 noaccomp default-asyncmap plugin rp-pppoe.so eth1.6 user "fake@freedom.nl"
'ifname wan' provides the PPP interface with an unique name.
If your network card doesn't support an MTU > 1500, you need to set the mtu
and mru in 'provider' to 1488;
mtu 1488 mru 1488
You need the appropriate entries in /etc/ppp/pap-secrets as well.
DHCPv6
/etc/default/wide-dhcpv6-client
# Defaults for dhcpv6 client initscript # Used by /etc/init.d/wide-dhcpv6-client # Interfaces on which the client should send DHCPv6 requests and listen to # answers. If empty, the client is deactivated. INTERFACES="wan"
Interface 'wan' only exists after the link had gone up. The boot process might complain about that.
/etc/wide-dhcpv6/dhcp6c.conf
profile default
{
script "/etc/wide-dhcpv6/dhcp6c-script";
};
interface wan {
send ia-pd 0;
};
id-assoc pd 0 {
prefix-interface eth1 {
sla-len 16;
sla-id 0;
ifid 1;
};
};
You can replace 'dhcp6c-script' with a script which doesn't do anything at all;
#!/bin/bash exit 0
Note: The original 'dhcp6c-script' messes with '/etc/resolv.conf', which you don't want if you run your own nameserver!
'sla' means 'Site-Level Aggregation identifier'. In a prefix
'2001:db8:1234::/48' it's the '1234' bit.
These are four hex digits, so it's 16 bits long (sla-len).
It insists on assigning an
IP address to an interface.
I have no idea how to get rid of this!
So I have it assign an address to eth1 (which is mostly firewalled).
Scripts
/etc/ppp/ipv6-up.d/add-rem-rt
#!/bin/sh
# Sets route to remote IPv6 address of PPP link
if ! ( ip -6 route | grep -q "${PPP_REMOTE}" )
then
ip -6 route add "${PPP_REMOTE}" dev "${PPP_IFACE}"
fi
# Set IPv6 default route
if ! ( ip -6 route | grep -q default )
then
ip -6 route add default dev wan
fi
Called from /etc/ppp/ip-up
I don't use the regular scripts in '/etc/ppp/ip-up.d/' and '/etc/ppp/ip-down.d/'. I modified '/etc/ppp/ip-up' instead. You may need to modify the stuff below to suit your needs.
# Take care of the (default) route(s)
case $PPP_LOCAL in
"Your_Freedom_Internet_IPv4_address")
# Freedom iface
# Store remote IPv4 address if you want to; '/run/ppp/' needs to exist.
echo $5 > "/run/ppp/wan-rem-ip"
chmod a+r "/run/ppp/wan-rem-ip"
# Restart NTPD
/etc/init.d/ntp restart 2>&1 | mail -s "NTPD restarted" root@Your_Domain
# IPv6 stuff
# Make sure link works
ping -c5 $PPP_REMOTE
# Default route
if ! ( ip -6 route | grep -q default )
then
ip -6 route add default dev wan
fi
# Restart DHCPv6
( /etc/init.d/wide-dhcpv6-client stop ) &
sleep 2
killall dhcp6c
sleep 2
killall -9 dhcp6c
sleep 2
rm /var/lib/dhcpv6/dhcp6c_duid
sleep 2
/etc/init.d/wide-dhcpv6-client start
;;
esac
Notes
IPv6 uses IPv6 link-local (fe80::/10) - and IPv6 multicast addresses
(ff02::/16) for internal housekeeping and configuration. If you firewall these
too heavy, some things will not work.
It can be handy to enable debugging. In '/etc/init.d/wide-dhcpv6-client' add
'-D' (shown red);
case "$1" in
start)
log_daemon_msg "Starting $DESC" "$NAME"
start-stop-daemon --start --quiet --pidfile $DHCP6CPID \
--oknodo --exec $DHCP6CBIN -- -D -Pdefault $INTERFACES
sleep 2
With Debian Stretch you can set the debug level in /etc/default/wide-dhcpv6-client (shown red);
# Defaults for dhcpv6 client initscript
# Used by /etc/init.d/wide-dhcpv6-client
# Interfaces on which the client should send DHCPv6 requests and listen to
# answers. If empty, the client is deactivated.
INTERFACES="wan"
# Verbose level for syslog. Default is 0 (0: minimal; 1: info; 2: debug)
VERBOSE=2
If things don't work, have a look at '/var/log/syslog'.
Check the MTU with ping
With these you can check your MTU.
'-s' sets the payload size.
The '-M do' bit disables fragmentation.
IPv4
An IPv4 ping header is 28 bytes (1472 + 28 = 1500);
~$ ping4 -s 1472 -M do Some_Host
IPv6
An IPv6 ping header is 48 bytes (1452 + 48 = 1500);
~$ ping -s 1452 -M do Some_Host
Additional notes on booting
A boot might 'hang' with considerable timeouts on the following;
- All interfaces need to be up.
- All NFS volumes need to be mounted.
The 1st I fixed by calling the PPP-link from /etc/network/interfaces instead of putting the PPP interface directly in /etc/network/interfaces. If the PPP-link doesn't come up, all the services are still started. Alternatively you might want to try 'allow-hotplug' instead of auto for the wan interface (This is based on experiences from the 1990-ies. Things might be different now).
The 2nd I fixed by using IP addresses instead of hostnames for NFS mounts in /etc/fstab: If the nameserver is down, the remote discs are still mounted. I also mount in the background (bg option in /etc/fstab). This way if a remote disc can't be mounted, the boot still proceeds.
Debug examples
I my current setip, 'enp5s0' (was eth0) is the LAN interface, 'enp3s0' (was eth1) the link to the NTU. 'enp3s0.6' is the VLAN 6 interface.
Tcpdump
This shows the beginning of the PPoE negotiation. The first line is my system sending data to the ethernet broadcast address, 'ff:ff:ff:ff:ff:ff'.
root@sput:~# tcpdump -n -e -vvv -s1500 -i enp3s0.6 -U tcpdump: listening on enp3s0.6, link-type EN10MB (Ethernet), capture size 1500 bytes 13:49:00.367689 Local_Ethrn_Addrs > ff:ff:ff:ff:ff:ff, ethertype PPPoE D (0x8863), length 30: PPPoE PADI [Service-Name] [PPP-Max-Payload 0x05DC] 13:49:00.376422 Remote_Ethr_Addrs > Local_Ethrn_Addrs, ethertype PPPoE D (0x8863), length 63: PPPoE PADO [PPP-Max-Payload 0x05DC] [AC-Name "asd-dc2-rtr1.cambrium.net"] [Service-Name] [EOL] 13:49:00.376559 Local_Ethrn_Addrs > Remote_Ethr_Addrs, ethertype PPPoE D (0x8863), length 30: PPPoE PADR [Service-Name] [PPP-Max-Payload 0x05DC] 13:49:00.396705 Remote_Ethr_Addrs > Local_Ethrn_Addrs, ethertype PPPoE D (0x8863), length 63: PPPoE PADS [ses 0x29] [Service-Name] [PPP-Max-Payload 0x05DC] [AC-Name "asd-dc2-rtr1.cambrium.net"] [EOL] 13:49:00.411982 Local_Ethrn_Addrs > Remote_Ethr_Addrs, ethertype PPPoE S (0x8864), length 32: PPPoE [ses 0x29] LCP (0xc021), length 12: LCP, Conf-Request (0x01), id 1, length 12
Debug in syslog
This is from /var/log/syslog with PPP debug on. It shows the first bit of the PPPoE negotiation. The rest is no different from any other PPP link setup.
Nov 21 13:47:40 sput pppd[12799]: pppd 2.4.7 started by root, uid 0 Nov 21 13:47:40 sput pppd[12799]: Send PPPOE Discovery V1T1 PADI session 0x0 length 10 Nov 21 13:47:40 sput pppd[12799]: dst ff:ff:ff:ff:ff:ff src Local_Ethrn_Addrs Nov 21 13:47:40 sput pppd[12799]: [service-name] [PPP-max-payload 05 dc] Nov 21 13:47:40 sput pppd[12799]: Recv PPPOE Discovery V1T1 PADO session 0x0 length 43 Nov 21 13:47:40 sput pppd[12799]: dst Local_Ethrn_Addrs src Remote_Ethr_Addrs Nov 21 13:47:40 sput pppd[12799]: [PPP-max-payload 05 dc] [AC-name asd-dc2-rtr1.cambrium.net] [service-name] [end-of-list] Nov 21 13:47:40 sput pppd[12799]: Send PPPOE Discovery V1T1 PADR session 0x0 length 10 Nov 21 13:47:40 sput pppd[12799]: dst Remote_Ethr_Addrs src Local_Ethrn_Addrs Nov 21 13:47:40 sput pppd[12799]: [service-name] [PPP-max-payload 05 dc] Nov 21 13:47:40 sput pppd[12799]: Recv PPPOE Discovery V1T1 PADS session 0x2008 length 43 Nov 21 13:47:40 sput pppd[12799]: dst Local_Ethrn_Addrs src Remote_Ethr_Addrs Nov 21 13:47:40 sput pppd[12799]: [service-name] [PPP-max-payload 05 dc] [AC-name asd-dc2-rtr1.cambrium.net] [end-of-list] Nov 21 13:47:40 sput pppd[12799]: PADS: Service-Name: '' Nov 21 13:47:40 sput pppd[12799]: PPP session is 8200 Nov 21 13:47:40 sput pppd[12799]: Connected to Remote_Ethr_Addrs via interface enp3s0.6 Nov 21 13:47:40 sput pppd[12799]: using channel 5 Nov 21 13:47:40 sput kernel: [4829498.873711] wan: renamed from ppp0 Nov 21 13:47:40 sput pppd[12799]: Renamed interface ppp0 to wan Nov 21 13:47:40 sput pppd[12799]: Using interface wan Nov 21 13:47:40 sput pppd[12799]: Connect: wan <--> enp3s0.6