Wireshark's weird ESP dissection

I recently observed Wireshark telling me obviously false information about the contents of ESP payloads. While the fix to that was trivial, the information learned in the process was worth noting down.

Wireshark was parsing many ESP payloads in the pcap and trying to make sense of the data therein. The result was columns of nonsensical frames. Antique protocols interspersed with more recognisable ones.

ESP, like AH, encapsulates the data between hosts communicating over an IPsec connection. There is no way Wireshark could have known what the contents were because the Security Associations were established to use encryption. You can tell Wireshark the keys behind SPIs as long as the ciphers matched a supported set.

The reason that this was happening was due to this setting being enabled:

Edit -> Preferences -> Protocols -> ESP -> "Attempt to detect/decode NULL encrypted ESP payloads"

It's off by default so apparently I'd enabled this long ago and completely forgotten.

The "Personal configuration" config file behind user preferences can also be easily seen by going to:

Help -> About Wireshark -> Folders

Specifically this, for the setting under discussion this should be the default setting:

# This is done only if the Decoding is not SET or the packet does not belong to a SA. Assumes a 12 byte auth (HMAC-SHA1-96/HMAC-MD5-96/AES-XCBC-MAC-96) and attempts decode based on the ethertype 13 bytes from packet end
# TRUE or FALSE (case-insensitive)
#esp.enable_null_encryption_decode_heuristic: FALSE

This is what I had:

esp.enable_null_encryption_decode_heuristic: TRUE

One fascinating aspect is, as I infer from the comment in the prefs file, that if the packet capture doesn't have the ESP negotiation (IKE phase 2) then Wireshark assumes that the ESP is using NULL encryption. If the first bytes of the ESP payload then matches a protocol, then the invoked protocol dissector will valiantly pick through an ESP stream-of-consciousness and will (often) throw up its hands, declaring the payload of the triggered protocol as invalid. Sometimes, ESP just appears as ESP because the payload matches no protocol known to Wireshark.

I think this would work much better as a Wireshark right-click option in the "packet list" pane to 'Decode ESP stream as NULL encryption'. This would reduce pointlessly attempting to decode every ESP packet in a packet capture and as a result speed up load times when opening a pcap with this user preference in place.

With Wireshark, beautiful product that it is, you get what you're given!