Skip to content

[slmon2] Fix monitoring bugs and improve the UI - #11

Open
comoglu wants to merge 4 commits into
SeisComP:mainfrom
comoglu:fix/slmon2-monitoring-bugs
Open

[slmon2] Fix monitoring bugs and improve the UI#11
comoglu wants to merge 4 commits into
SeisComP:mainfrom
comoglu:fix/slmon2-monitoring-bugs

Conversation

@comoglu

@comoglu comoglu commented Aug 4, 2026

Copy link
Copy Markdown
Contributor

Three related changes to slmon2, from actually running it against a
1000+ station production feed for a while.

Bug fixes (silent data loss / hangs)

  • A KeyError on any station/channel not present in the startup
    slinktool -Q snapshot was silently swallowed, so anything down at
    boot (or newly started since) never appeared on the dashboard again
    until a full restart.
  • No network timeout was passed through to the SeedLink client, so a
    stalled connection could block the process forever with no
    self-recovery.
  • HTML/JSON regeneration was gated on receiving a new SeedLink record,
    so a station (or the whole feed) going completely silent — the exact
    outage this tool exists to detect — never triggered a re-render. A
    background timer now regenerates on a fixed interval regardless.
  • FDSNWS coordinate lookups had no timeout either.

UI

  • Unified three independently hand-maintained status-color palettes
    (CSS vars, JS, Python's getColor()) into one source of truth. They
    had drifted apart — the same status could render a different color
    in the table's Status cell vs. its Latency cell vs. the map — and
    some severity colors weren't monotonic (a "stale for over an hour"
    status rendered green).
  • Station detail is now an inline slide-over panel instead of
    navigating to a separate page, so table/grid filters and scroll
    position survive opening it. Old NET_STA.html links still work as
    redirect stubs.
  • liveurl config values now also support a {netsta} token
    ("NET.STA"), alongside the existing %s (station code only) —
    purely additive.

Map fix

  • Stations near the antimeridian (e.g. Samoa/Tonga, lon ≈ -170) were
    rendering ~300° from an Australia/Pacific-centered map — numerically
    valid, but visually on the wrong side of the world. Longitude is now
    normalized to the map's reference point for marker placement only;
    popups/labels still show the real coordinate.

Tested end-to-end against a real ~1000-station config in -g
(generate-only) mode, plus rendered screenshots of all three views and
the detail panel.

comoglu added 3 commits August 4, 2026 11:47
- main(): a KeyError on an unseen 'net.sta.loc.cha.type' key was
  silently swallowed, so any station/channel not present in the
  startup slinktool -Q snapshot (down at boot, or newly started since)
  never appeared on the dashboard again until a full restart. Now such
  channels are registered on first sight instead of dropped.
- main(): pass a network timeout through to slclient.Input (-nt for
  slinktool) so a stalled SeedLink connection can't block the process
  forever with no self-recovery, matching the preexec_fn/SIGKILL
  cleanup already added in slclient.py.
- main(): HTML/JSON regeneration used to be gated on receiving a new
  SeedLink record, so a station (or the whole feed) going completely
  silent -- the exact outage this tool exists to detect -- never
  triggered a re-render. A background timer now regenerates on a fixed
  interval regardless of record arrival, guarded by a lock so
  overlapping runs can't race on the output files.
- load_station_coordinates(): add a timeout to the FDSNWS urlopen()
  call; an unreachable/slow server no longer stalls startup once per
  configured station.
- generate_station_html(): don't paint 'n/a' cells with a stale
  leftover background color when feed/data latency are blanked out.
- Remove a dead duplicate setActiveView() left over from an earlier
  edit; only the later, more complete copy was ever executing.
- Single source of truth for status colors (STATUS_STYLES): the CSS
  (:root vars + .station-* classes), the browser-side JS
  (getStatusColor/addMapLegend/cluster coloring), and Python's
  getColor() were three independently hand-maintained palettes that had
  drifted apart -- the same status could render a different color in
  the table's Status cell vs its Latency cell vs the map. Also fixes
  severity colors that weren't monotonic (hour-delayed rendered green,
  reading as "healthy" despite being past most other delayed states)
  and legends that silently covered only 6-8 of 12 statuses.

- Station detail is now an inline slide-over panel (720px, no-wrap
  datetimes) instead of navigating to a separate NET_STA.html page, so
  table/grid filters, scroll position, and map view survive opening
  it. Old links keep working: NET_STA.html is now a lightweight
  redirect stub to index.html?station=NET_STA, which also opens the
  panel directly on load for bookmarks/shared links.

- liveurl templates support an additional {netsta} substitution token
  (resolves to "NET.STA"), alongside the existing %s (station code
  only). Purely additive/config-driven -- no behavior change for
  existing %s-based liveurl values.
…r the antimeridian

Leaflet plots raw longitude on a single, unwrapped copy of the world, so
a station at e.g. -169 (Samoa/Tonga area) was rendered ~306 degrees from
the AU/Pacific-centered map (lon ~134) -- numerically valid, but visually
appearing near the Americas instead of next to Fiji/NZ across the date
line, ~54 degrees away.

Add normalizeLongitude()/mapRefLon(), applied only to the marker's
plotted position and the auto-zoom bounds -- popups/labels keep showing
the real (unwrapped) coordinate value so they stay factually correct.
@cla-bot cla-bot Bot added the cla-signed The CLA has been signed by all contributors label Aug 4, 2026
…ver UTC

renderDetailBody() computed Data/Feed Latency from Date.now() on the
browser, so a skewed local clock on the viewer's machine threw off the
displayed numbers. Use the server-computed ch.latency instead, and derive
feed latency from it (dataLat - diff) so it never depends on "now" either.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

cla-signed The CLA has been signed by all contributors

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant