PPPoE, MTU packet size: A5, A5c, C5x, etc... (Not backhaul radios)

So, I seem to be having some issues with PPPoE on my Mimosa driven wireless network.

I would like some input on this as I have been researching what MTU’s are supported on Mimosa AP gear and seem to be having a hard time.

  1. All my CPE clients connect via a VLAN enabled SSID.
  2. All my CPE C5x’s have a management VLAN assigned via radius, and my AP’s are on another VLAN for management.
  3. All my clients use PPPoE on their mikrotik routers to connect with a static IP (private) assigned to them via PPPoE (Radius), PPPoE servers do rate limiting dynamically.
  4. All my Switches (Netonix) Routers, (Mikrotik) have support for larger MTU values to support: VPLS, MPLS and VLANs.

Here is where I am kinda stuck / may be missing something that perhaps someone has the answer to:

When I try to set my PPPoE clients to true full 1500 MTU they will connect - but I seem to have issues uploading and other strange packet / VOIP issues etc.
If I lower the MTU to say 1470 then all seems to work - but the default PPPoE MTU of 1492 does not seem to work as well. ( I think that is industry standard)

If I do some math ( and I may be doing it wrong - so please speak up if you see something) and compute the packet size I should have with a 1470 MTU the following:
1470 = data + IP = 20 + VLAN = 4 + PPPoE = 8
Total = 1502 flowing from CPE router through CPE Radio to AP to switch.

So by this logic the C5x’s only push 1500 MTU through them and I am over - it should not work at 1470 - but it does.
If I ping with “Do not fragment” from one of my client devices to the PPPoE server - I get stopped up about 1497 - 1496 does work.
That feels like the VLAN (4 bytes) being added / passing through the Mimosa gear.
I will admit there are gaps in my knowledge - but the rest of my network is able to handle everything else - just the CPE routers seem to have this problem.

For reference I have at each point in my network a L2 MTU of 1580 and an MTU of 1550.

As I see it this should be the max packet size anywhere in the network:
Data = 1480 + IP =20 + MPLS = 4 + VLAN = 4 + Eth = 14 + PPPoE = 8
Total = 1530

I guess what I really need to know here - is if you are setting up your AP’s with VLANS for customers are you adding 4 bytes to each packet giving you only 1496 over the Mimosa CPE and AP?
Or are you still at 1500 MTU to be able to pass through each CPE / AP.
If 1496 then you have to add PPPoE on that in my case ( So 1488 should be max?)

Now, I do see that the Mimosa AP’s have the ability to use Jumbo frames - but I do not see any mention in the CPE’s of MTU.

So, I need to know:

  1. What max MTU do the Mimosa C5x’s (CPE radios) support? Is it changeable?
  2. Does the Mimosa A5(x) (C in this case) include VLAN’s in the 1500 MTU by default - or do you have to subtract 4 for that?

2.5 Oh - and do management VLANS work into all of this as well for the C5X and the A5c’s?

Thanks - and I hope the answers we come up with here will help others in the future.

Oh boy, looks like I have to eat some crow here, and I did find a small issue on my network stopping things up. I made the adjustment and things are working as I want them too.

I can now pass full 1500 byte MTU all the way from my customers out to the net.

However, I still want to understand / have others understand how I am able to pass larger than 1500 byte MTU through a Mimosa AP and C5x CPE radio with PPPoE on the client router.

I say “larger” using the following logic:

Packet size based on:
Data = 1480 - Standard?
IP = 20 - Standard?
VLAN = 4 - From the SSID VLAN generated by Mimosa AP
Ethernet = 14 - Standard?
PPPoE = 8 - PPPoE Generated from client router.
Total = 1526

Keep in mind that this does not include backhaul stuff: MPLS / VPLS, more VLANs etc…
Researching, I do find that PPPoE is considered a later 2.5 protocol - so not sure if that is added to a packet that the Mimosa gear would see.
Wiki about PPPoE

So lets call it 1518.
If we remove 4 for the VLAN then we are at 1514 - I am just assuming here that Mimosa includes the VLAN and possibly the Ethernet.
if we remove 14 for Ethernet then we are back at 1500!

So, the assumption is that the Mimosa 1500 byte MTU includes:
Vlan and Ethernet headers.

Why someone may ask do you want to offer 1500 byte MTU to customers and not 1492 like most other PPPoE Servives?
rfc2516 and rfc4638 come into play here.
Most all of the Internet is 1500 byte MTU and to remain standardized it is best in my option to try to offer 1500 byte MTU to my wireless customers.

This should prevent packet fragmentation and mis-ordering of packets when they are re-assembled on the the “far” side of the Internet that you are sending your data to.
Things like speed tests using UDP, VOIP and others seem to do better when things are standard.

I have some more testing to do in my lab - but still looking for the answer to the question:
Does the 1500 byte MTU that Mimosa says is present by default include the Ethernet header?

Learning as I go here, but want to nail this down for myself and others.

Thanks!

1 Like

I also think that I should share this link as it has all that is useful about MTU and other packet protocols that may be involved in a wireless network.
(Wish I had found that before I posted most of this above!)

https://www.packetstreams.net/2018/07/the-secrets-of-mtu-l2-mtu-vs-l3-mtu.html

1 Like

@Rob_MCN kept posting and updating before I could finish a message.

I would highly recommend reading through his entire post because he brings up a lot of significant issues that effect anyone running a tunneling technology in their network.

IIRC, Mimosa’s MTU is dynamic depending on the MCS. There was a spreadsheet floating around that showed how MCS effected the maximum MTU of Mimosa links, but I don’t have it anymore and I don’t know how accurate it was. @DustinS might be able to comment further on that.

I do know that for my Mimosa PTMP network I can pass a 1500 byte IPv4 packet from a customer router (including IP Header and all the other stuff) all the way back to my edge routers. 1501 breaks. (I hide all of my VLAN overhead from my customers and run L2 MTUs of +1500 everywhere while the VLANs themselves are 1500 exactly. (I run a VLAN to my customers and isolate them from eachother that way and from the rest of my network. I also run many of my backhaul links over VLANs to isolate customer traffic as well.)

It all really depends on you Layer 2 tech, whatever the smallest link in the chain is is going to be your limiting factor. Then you are just adding on overhead. Here is a pretty good calculator that I have used for reference a couple of times. Visual packet size calculator — Daniil Baturin