OSPF and MTU issues with Proxmox

At Freifunk Aachen, I recently had an issue with our OSPF routing layer due to a different MTU on one link. OSPF requires to have the same MTU on all links. This is needed so that the packets are not thrown away due to a too big MTU when routed in any direction with a smaller link MTU. In theory this should only affect the link with the differing MTU (this is the case with FRR). However, when using bird and even bird2 this results in a stop of other neighbors as well.

This means, that some neighbors are detected correctly - typically the ones nearer on the link - but neighbors further away are not discoverd, as soon as any single neighbor on a link has a mismatching MTU.

This happens, when there are multiple peers on the same link (as the case on a VXLAN SDN network in proxmox). It seems that bird is not inteded for such use cases, as it is assumes to have only one neighbor per link.

This is especially stupid if the change is introduced partially by changing only one host after the other to switch the MTU size for performance reasons.

Workaround

The workaround is to create a second virtual interface in which the different MTU is spoken, so that all the migrated VMs are in the other interface, while you move the VMs one by one without breaking the existing OSPF links.

How to detect problems

One can see MTU problems on your link using ping -s <size> <target>. By pinging with an MTU of 1500 one can check if the traffic goes from one side to the other. Reducing the size to something like 1280 discovers that the issue is in fact MTU.