Spot collector disagreements

When you need this

“gNMI says this MAC is on Eth7. SNMP says Eth12. Which is right?”
“Is our SNMP collector silently lagging?”
“Where are the collectors actively disagreeing right now?”

A disagreement is a row in mac_observation where, for the same (mac, vlan, device) at the same wire-time, two or more sources point at different ports. The system allows this state (we don’t want to drop data because of disagreement) but surfaces it loudly.

In the TUI

Press Ctrl+O for OPS. The middle pane is the disagreements table.

OPS screen with FDB tree, disagreements pane, and quarantine log tail

Columns:

mac — the MAC that’s disagreed-upon
vendor — manufacturer from the OUI registry (e.g. VMware, Inc.) — a fast proxy for “is this a printer, a VM, a phone?” when investigating
vlan
device — which switch
sources→ports — comma-separated source:port pairs, e.g. gnmi:Eth1, snmp:Eth12
count — how many distinct ports the sources point at (always ≥ 2 for a disagreement)

The pane auto-refreshes every 10 seconds. If the table is empty, no collector is actively disagreeing right now — which usually means either sources agree, or you only have one source running.

Which source do you trust?

l2trace doesn’t pick one. It records all sources and lets you decide. As a starting heuristic:

gNMI is usually fastest and most trustworthy if the device supports it.
SNMP can lag by up to one poll interval (SNMP_POLL_INTERVAL_SECONDS, default 60s). A persistent gNMI/SNMP disagreement that resolves after one poll interval is just SNMP catching up.
SSH snapshots are slowest and most prone to staleness, but most resilient to telemetry bugs in vendor firmware.

The mac_observation_resolved view applies a priority order (gnmi > snmp > ssh > netconf > reconciler) and gives you a single “winning” row per key. The disagreement table is the raw truth before that resolution — useful for “is something wrong?” rather than “what’s the consensus?”.

In SQL

SELECT mac, vlan, device_hostname,
       sources_and_ports, port_disagreement_count
FROM mac_observation_disagreement
ORDER BY port_disagreement_count DESC, mac;

The view lives in alembic revision 0002. It groups rows by (mac, vlan, device) filtered to current beliefs and currently-open valid_during, returns one row per key where count(distinct port) >= 2.

When disagreement is a real bug

Pattern	Likely cause
gNMI on Eth1, SNMP on Eth12, persistent	Vendor firmware bug — file a ticket
gNMI on Eth1, SNMP on Eth12, resolves in <60s	Normal SNMP poll lag
gNMI alone on Eth1, gNMI again on Eth12	Move that hasn’t been fully reconciled — check for queue-dwell quarantine events
All sources on different ports	Topology / VLAN config drift — check LLDP adjacency