Skip to content

Catch transients with the syslog listener

  • “A phone moved desks, came up on the new port, and moved back — all inside one 300s SNMP window. We never saw it.”
  • “STP reconverged at 02:14; the CAM table flushed and we polled the stale one for four more minutes.”
  • “We already ship every switch’s syslog to a collector. Use it.”

SNMP polling gives you the MAC table as it looked at poll time. Anything that happens and reverts between two polls is invisible — the shorter the MAC aging timer relative to the poll interval, the more you miss. The syslog listener closes that gap: switches log topology changes and MAC moves the instant they happen, so l2trace hears about a transient at event time instead of at the next poll (or never).

The listener is one shared socket for the whole fabric, not a per-device collector. It resolves each message to a device by source IP (the allowlist is built from device.mgmt_ip), parses two message classes, and emits the same EventEnvelopes onto the same NATS pipeline everything else uses — the reconciler needs no syslog-specific code.

Message classWhat it does
Topology change (%SPANTREE, %STP, “topology change”, root-change traps)Retracts l2trace’s belief for that bridge — the exact same path the polled dot1dStpTopChanges counter drives, but in real time
Definitive MAC move (one mac + vlan + port)Emits a MAC_LEARNED at the named location

Vendor dispatch is by device.vendor: Cisco, Arista, and Juniper have their own regex sets; everything else falls back to cross-vendor patterns. A two-port flap notification (%SW_MATM … flapping between port A and port B) names two ports, so it has no single location — the listener recognizes it but deliberately does not guess a MAC location from it.

Syslog is unauthenticated and spoofable. Messages from a source IP that isn’t a registered device are dropped and counted, never ingested.

Two steps: turn the listener on, then publish its ports.

  1. Set SYSLOG_ENABLED=true in .env. When false, the reconcile stack binds no syslog socket at all, so existing deployments are unaffected.

  2. Bring the stack up with the syslog overlay, which maps host 514 → the container’s unprivileged 1514 (the container runs as non-root and can’t bind 514 directly):

    Terminal window
    docker compose -f docker-compose.yml -f docker-compose.syslog.yml up -d

The listener runs as a co-running service inside l2trace reconcile, alongside the reconciler / compactor / orchestrator. A listener crash is isolated — it doesn’t take down ingestion.

VariableDefaultPurpose
SYSLOG_ENABLEDfalseMaster switch
SYSLOG_UDP_PORT1514In-container UDP bind (overlay maps host 514)
SYSLOG_TCP_PORT1514In-container TCP bind (RFC 6587 newline framing)
SYSLOG_HOST_PORT514Host port the overlay publishes
SYSLOG_DEVICE_REFRESH_SECONDS60How often the source-IP allowlist is rebuilt from device.mgmt_ip

Both UDP and TCP bind at once. If you run l2trace on the host directly (no container), set the ports to 514 and skip the overlay.

On each switch, send logging to the collector host on port 514 and make sure the source IP the switch sends from matches the mgmt_ip you registered it with. Cisco IOS / IOS-XE example:

logging host <collector-ip>
logging trap notifications
logging source-interface <mgmt-interface>

Then register each device with that mgmt IP so its logs are accepted — the same registration that gives the SNMP collector a target also puts the device on the syslog allowlist. The listener rebuilds the allowlist from device.mgmt_ip every SYSLOG_DEVICE_REFRESH_SECONDS, so a newly-registered switch is picked up without a restart:

Terminal window
docker compose run --rm reconciler l2trace device add \
--hostname sw-access-7 --mgmt-ip 10.0.0.7 --source snmp --community public

An unregistered source is logged once per process (to avoid a spoofing-flood log storm) and its rejects are counted in the periodic stats line.

Auto-registering trusted sources (optional)

Section titled “Auto-registering trusted sources (optional)”

By default the allowlist is strict — only known mgmt_ips are accepted. If you’d rather accept any device whose reverse-DNS (PTR) name lives in a trusted domain, set SYSLOG_AUTOREGISTER_DOMAINS to a comma-separated suffix list. An unknown source IP that reverse-resolves into one of those domains is registered as a visibility='unknown' placeholder and accepted; a spoofer with no PTR or a foreign PTR is still dropped.

VariableDefaultPurpose
SYSLOG_AUTOREGISTER_DOMAINS“ (strict)CSV of trusted PTR suffixes, e.g. corp.example
SYSLOG_AUTOREGISTER_VENDOR“ (generic)Vendor stamped on auto-registered sources, selecting their parser regexes
SYSLOG_AUTOREGISTER_ATA_ROLE“ (no role)Role for sources whose PTR looks like a Cisco ATA/SPA voice adapter (in-path 3-port mini-switches)

If the network already aggregates syslog into a Grafana Loki instance, l2trace can read from there instead of binding a port itself. Set SYSLOG_SOURCE=loki and use the Loki overlay in place of the syslog one:

Terminal window
docker compose -f docker-compose.yml -f docker-compose.loki.yml up -d

The parse / allowlist / emit path is identical — only the transport differs. The Loki source keys the allowlist on a stream label (SYSLOG_LOKI_SOURCE_LABEL, default source_ip) that the ingest pipeline must set to the sending device’s IP, and anchors both bitemporal axes on the Vector receive time so replay/backfill latency lands on the queue-dwell axis, not device-clock skew. Selector and poll cadence are tunable via SYSLOG_LOKI_SELECTOR (default {job="syslog"}), SYSLOG_LOKI_POLL_SECONDS, and SYSLOG_LOKI_LIMIT.

Syslog is a supplement, not a replacement. Keep SNMP polling — and specifically keep dot1dStpTopChanges polling — enabled on the switches. SNMP gives you the periodic full CAM snapshot that reconciles the ground truth; syslog fills the gap between snapshots with the events that would otherwise age out unseen. A syslog topology change drives the same belief retraction the polled STP counter does, so the two agree by construction; syslog just gets there first.

  • The listener source: src/l2trace/collectors/syslog.py
  • The parser + vendor patterns: src/l2trace/collectors/syslog_patterns.py
  • Collect from an SNMP-only device — the periodic snapshot the syslog listener supplements
  • Why bitemporal? — how a late or replayed syslog line is handled as belief revision, not a silent overwrite