Monday 23 July 2018

enable jumbo frame on cisco 3850

conf t
system mtu 9198
wr
reload

Taken from cisco forums
The limitations of the platform in question would make no sense if system mtu routing would apply to routed ports, but at the same time would not apply to SVIs. The point here seems to be that the model line cannot fragment in hardware.  So to avoid that, it just makes sure every L3 interface (routed port  and SVI alike) has the same IP MTU, so there is never any need to  fragment a packet.
I've tested this in the lab using two 3560E chassis, trivially set up with the default VLAN, connected through Gigabit (so system mtu jumbo will apply) and then pinging each other's SVIs. Now let's configure the  following (using the maximum frame size the platform supports, there's  not much sense in limiting it here):
system mtu jumbo 9198
After rebooting, all physical ports running 1000Base or better will allow payloads of up to 9198 bytes to be encapsulated. The routing mtu will still be 1500 at this point. Try to ping one switch from the other like
ping 10.1.1.2 size 1500 df-bit
and it will succeed, but increasing size to 1501 will fail.
Now let's add:
system mtu routing 9000
to both switches and try again (no reboots needed). You will see that
ping 10.1.1.2 size 6000 df-bit
will suddenly work now, and the interface counters  will make clear that no fragmentation happens - it's really a single  6000 byte IP packet bouncing forth and back between the switches. That  works up to 9000, and starts failing at 9001, exactly as you would  expect.
Why is there a rumour that system mtu routing doesn't apply to SVIs? Probably because show interface of an SVI will show you an MTU of 1500 (or whatever your system mtu is), while the same command applied to a routed port will show 9000.  This seems to be a glitch, as so often with interface MTU in show  commands. More specifically, the show interface MTU is  supposed to be the potential payload MTU of the underlying physical  interface of that routed interface, and there are other cases where it  displays rubbish. One should always compare to the IP MTU as given by show ip interface. Et voila: The IP MTU of our SVI (as given through show ip int vl1) displays as 9000. So the succeeding ping is not a mystery and system mtu routing does exactly what it intuitively states: Change the IP MTU of every L3 interface of the platform.
I know this won't help in cases like the one discussed  here, where supposedly two L3 interfaces running at different MTUs are  needed. In such cases, one should first reassure that what's needed is  really that and there's no way to redesign the setup to avoid that (by  placing L3 and L2 boundaries appropriately). If there is no way around  that, the 3560 will likely have to go for something that has  per-interface IP MTU, like the 49xx or 4500X platforms.
Discussions about MTU often mix up different problems  and lead to chaos. IMO this is because two things are often not regarded  to the necessary extent by the participants:
  1. There is not one MTU. When talking about MTU, always define  the layer you consider. That's often hard because you actually have to  think about a layer boundary, so two layers are involved. The mythical  1500 for instance is the L3 MTU on top of a classic L2 of the Ethernet  family. Things fundamentally change when you discuss L2 MTUs with regard  to some underlying L1 (but things are easier here, as L1+L2 are often  developed together as one technology, while the boundary between  technology layers and the network layer has more degees of freedom).
  2. MTU  doesn't exist as such, but is an emerging concept. In other words, MTU  is what everybody in a system of communicating nodes (typically a  broadcast domain) thinks it is, with the emphasis on everybody.  In that sense it's like the IP network that lives on top of a broadcast  domain - it doesn't exist per se, but by convention of everybody using  that broadcast domain as a bearer for IP packets sourced from  non-colliding adresses from the same visibility range (aka prefix, aka  network and netmask, in ancient times aka subnet/subnetmask). MTU is a  convention as well, a single node just changing its MTU (up or down from  the convention, that is, what everybody else uses) is a recipe for  disaster. That's why you don't do it except you know exactly what you  are doing, and have the might to change it everywhere in a broadcast  domain (every involved intermediate network device and every end node).  It's viral. Luckily it's not as widespread as the viral issue of people  configuring ports full duplex because that's what they always did,  introducing duplex mismatches up and down and then telling you that as  obviously auto negotiation doesn't work, they will continue this  practice. You see the common scheme: Breaking a convention is a bad  idea, unless you are the sole dictator.

No comments:

Post a Comment