Commit graph

15 commits

Author SHA1 Message Date
Andrew Pimenov
a64022b12a detect ECMP load balancing via N-run traceroute diff (--ecmp)
Shipped half of the original design on purpose: generalizing BSD's alt:
same-TTL annotation across backends is impossible - mtr's JSON aggregates
cycles and discards per-cycle next-hop variance, icmp_win/icmplib send one
probe per TTL by construction, and the Linux system-traceroute parser
doesn't model multi-IP hops even though the text can contain them. A
finding that only ever fires on macOS/Linux-system-binary would be worse
than not having it.

New probes/ecmp.py::detect_ecmp() takes N independently-run TraceResults
for the same target and diffs which IP showed up at each TTL - pure and
backend-agnostic, since it never touches capability detection. Flags
path.ecmp (info, multiple next hops exist - normal on a load-balanced
network) and path.ecmp_asymmetric (warn, branches differ by >20ms - the
diagnostic case, since one degraded branch of a load-balanced pair causes
intermittent slowness a single traceroute would never catch).

--ecmp is opt-in and deliberately excluded from --full: it multiplies
traceroute traffic against the same targets by ecmp.runs (capped at 5)
across ecmp.max_targets (capped at 3), sequential and sharing one
semaphore, only after the main fan-out has finished.

Live-tested against 2 real targets x 3 runs each - ran cleanly, correctly
reported no divergence on this network.

Part of plans/zesty-zooming-wilkinson.md (gitignored).
2026-08-12 22:42:38 +03:00
Andrew Pimenov
e27ccdfab5 give --watch a per-tick verdict and optional webhook alerts
watch.py never imported interpret and had no notion of a verdict at all -
summarize_cycle() now runs the same latency_findings/speed_findings/
overall_verdict pipeline the one-shot report uses, so every tick carries
status/score/finding_ids and the dashboard caption shows a verdict line.
Real improvement on its own, independent of whether a webhook exists.

New alerting.py carries the whole notification contract as pure functions:
should_fire() only fires on an actual transition, into a state the user
opted into via watch.webhook_on, debounced by
watch.webhook_min_interval_seconds (floored at 30 in a validator) so a
flapping link can't spam an endpoint every tick. Recovery is its own
opt-in "recovered" token, separate from configuring "crit", since wanting
outage alerts doesn't imply wanting a second ping when they clear.
build_payload() is flat JSON, not shaped for Slack/Discord - that's the
receiver's problem. post_webhook() swallows every failure and returns a
bool; a dead endpoint must never interrupt the loop watching it.

New --webhook URL flag overrides watch.webhook_url per run.

Verified the tick sequence end-to-end outside the Rich Live loop -
sending real OS signals to test --watch's Ctrl-C handling in this shell
wasn't reliable, so exercised should_fire/build_payload/render_dashboard/
to_report in the same order run_watch calls them.

Part of plans/zesty-zooming-wilkinson.md (gitignored).
2026-08-12 22:26:31 +03:00
Andrew Pimenov
245533f904 add Prometheus/CSV export, auto-diff vs previous run, and --trend
New metrics.py: collect_metrics() does the per-section extraction a flat
dict walk can't (latency needs label/host/method, TLS label/host, path hops
target/ttl/ip/asn), producing a flat Metric list that render_prometheus()
and render_csv() each format in about 15 lines. Simpler than treating them
as two separate renderers. FORMAT_EXTENSIONS gained "prom"/"csv" as the
single wiring point both parse_formats() and Output._normalize_formats()
already key off of.

New history.py finds a previous JSON report for the same ASN by reusing
the exact filename convention exporter.report_filename() already writes,
so the lookup key can't drift from what's on disk. Wired into diagnose()
before write_report() - a compact vs-previous-run summary
(compare.py::render_diff_brief(), new) prints automatically whenever this
run writes JSON and an earlier one exists for the same network; silent on
every other case (first run, prior run wrote no JSON prints one note
instead of guessing). New --no-auto-diff escapes it per run,
history.auto_diff in config disables it entirely.

New trend.py aggregates N saved JSON reports into per-label sparkline
series via --trend N - read-only, and it doesn't touch the network to find
"this network's" history either: history.latest_key() just reads it off
the newest report filename already on disk.

Growing FORMAT_EXTENSIONS to 5 entries broke three pre-existing tests that
hardcoded the old 3-format set; fixed in this commit.

Live-tested: prom/csv artifacts render correctly, auto-diff fired on a
second run, --trend 3 produced readable output, --compare unaffected.

Part of plans/zesty-zooming-wilkinson.md (gitignored).
2026-08-12 22:10:21 +03:00
Andrew Pimenov
99876103f8 verify TLS certificates and fingerprint them for MITM detection
tls_context() now defaults to real certificate verification instead of the
tool's original always-CERT_NONE posture; measure_tls() only falls back to
unverified on ssl.SSLCertVerificationError, retiming the retry since the
failed attempt's duration isn't representative. This is also the only way
to get subject/issuer/notAfter at all - getpeercert() returns {} for a
chain that never validated.

Every TLS result now carries cert_sha256 (SHA-256 of the full DER cert, not
the SPKI - that would need the cryptography package, a dependency this
doesn't need), cert_issuer, cert_days_remaining, and the long-dead
cert_verified field finally gets written. New tls.pinned_fingerprints
config (host -> expected hash, capped and hex-validated) drives three new
findings: a pin mismatch is crit, an unpinned validation failure is warn,
an expiring cert is warn.

Live-tested with --tls against real Cloudflare/Google endpoints - verified
handshake, correct issuer and expiry both times.

Part of plans/zesty-zooming-wilkinson.md (gitignored).
2026-08-12 21:48:36 +03:00
Andrew Pimenov
84c3dffbce detect captive portals up front, flag off-country speedtest servers
New Phase 0 in diagnose(): three generate_204-style probes run before every
other module, using detect-and-tag rather than short-circuit — a portal is
rarely total (DNS/ICMP often pass while only HTTP is intercepted), and an
aborted run would look identical to a dead link. On detection every later
phase still runs, tagged with a crit finding and a banner at the top of the
report.

Caught live before it shipped: detectportal.firefox.com/success.txt (one of
the original three default check URLs) returns 200 "success" on a genuinely
clean connection, which the generate_204-shaped classifier misread as a
portal redirect. Swapped for a second real generate_204 endpoint instead of
teaching the classifier a second success shape.

Also redesigned the speedtest server geo-sanity check after checking the
bundled Ookla CLI's actual JSON output: its server object never carries
lat/lon (documented schema, not a version gap), so speed.server_other_country
compares server_country against the client's own country instead of a
haversine distance that no tier can actually supply.

Part of plans/zesty-zooming-wilkinson.md (gitignored).
2026-08-12 21:36:54 +03:00
Andrew Pimenov
8d4c8bee8b enrich traceroute hops with ASN, AS name, country and PTR fallback
TraceHop.asn/as_name existed in the model since the start but nothing ever
populated them. New probes/hop_asn.py fires one Team Cymru DNS query pair
per unique hop IP (in-run deduped, no rate limit unlike the cached_json()
providers) after the latency/path/TLS fan-out, plus a PTR lookup for hops
whose reverse_dns is still empty because the backend that produced them
doesn't resolve hostnames itself (icmp_win/icmplib/mtr, unlike system
traceroute text).

New path.detour_country finding flags a route that spends two or more
consecutive hops in a country that's neither the client's nor the
destination's. Wires up providers.cymru_asn_zone, which was configured but
referenced nowhere, and adds cymru_origin6_zone for the IPv6 branch Phase 8
will need.

Part of plans/zesty-zooming-wilkinson.md (gitignored).
2026-08-12 21:17:00 +03:00
Andrew Pimenov
76cfd4605c detect CGNAT, NAT64, and first-hop-vs-path issues separately
- net.cgnat: three evidence paths (local IP, gateway, traceroute hop) for
  RFC 6598 100.64.0.0/10, since the local address alone misses the common
  home-router case
- net.nat64: RFC 7050 ipv4only.arpa AAAA probe (new probes/nat64.py) instead
  of checking the client's own address against 64:ff9b::/96, which never
  fires since that prefix is a synthesis target, not something the client's
  egress address falls into
- path.first_hop_loss/slow/unexpected: path_findings() takes an optional
  LocalNet/Thresholds so loss or latency at hop 1 is attributed to the
  user's own router/Wi-Fi instead of blending into the ISP-facing mid-path
  logic

All four findings are always-on additions to existing report sections, no
new CLI flags. Part of the larger extension batch tracked in
plans/zesty-zooming-wilkinson.md (gitignored, local planning doc).
2026-08-12 21:06:01 +03:00
Andrew Pimenov
3a0f95c793 document L4-L7 diagnostic modules and the python 3.14 bump 2026-08-10 21:26:46 +03:00
Andrew Pimenov
fd72604240 add --interface flag to force outbound probing through a specific adapter
Every probe used to follow whatever the OS default route picked, with no way
to point the tool at a specific adapter for comparison (VPN vs raw) or to
confirm which one actually carries traffic on a multi-adapter machine.

New iface.py resolves an adapter name or IP to a BindTarget (pure, tested):
exact/case-insensitive/unique-substring name match, rejects APIPA-only,
link-local-only and down adapters. Wired through every httpx provider client,
the Ookla binary tier, ndt7, and the whole ping/traceroute cascade (icmplib,
mtr, unix traceroute, TCP connect, scapy) plus --watch. Windows icmp_win now
uses IcmpSendEcho2Ex (adds SourceAddress) only when a bind is forced, so the
unforced path is unchanged. Windows tracert has no IPv4 source-address option
(verified from its own usage text) so that tier is dropped when forced,
noted in the report rather than silently measuring the wrong path.

An unresolvable/down adapter aborts with every adapter listed; a resolvable
adapter with no route to the internet degrades gracefully instead of
aborting - that's a real diagnostic answer, not an error. netinfo's
observation of the OS's actual default route stays untouched, so the report
can show the forced adapter and the OS default side by side.

Verified live: forcing Tailscale (no internet route on this box) triggered
the preflight warning and degraded correctly; forcing Ethernet ran clean
end-to-end including a full speedtest+MTR cycle.

Also noted, not fixed: probes/traceroute.py's icmplib tier imports
async_traceroute, which doesn't exist in icmplib 3.0.4 - that tier has
always raised ImportError and silently fallen through. Found while
verifying the icmplib source-binding wiring live.
2026-08-09 13:36:06 +03:00
Andrew Pimenov
4ffc39906b add --format/--ru/--json flags to control which report artifacts get written
Every run used to write all three files (en.md, ru.md, json) unconditionally.
Default is now English Markdown only, sourced from output.formats in
config.yaml; --format/--ru/--json pick exactly what a run writes, and any
explicit --format replaces the default rather than adding to it (all/none
supported). write_report() moved from a fixed 3-positional-arg signature to
a {format: text} mapping so writes are conditional instead of always-three.
The en/ru Markdown cross-link header is now only set when both languages are
actually being written.
2026-08-09 13:00:13 +03:00
Andrew Pimenov
031f85df5e netsleuth in docs, license, env example and release workflow 2026-08-09 11:43:03 +03:00
Andrew Pimenov
b4c541aecf docs: sync readme, architecture and tasks with the russian report feature 2026-08-09 11:14:56 +03:00
Andrew Pimenov
4497e2a871 docs: sync readme and architecture with the final module and flag set
readme's sample console output showed a multi-line summary table
(connection/egress/vpn/latency) the tool has never printed; the real
stdout is three lines (banner, verdict, file paths). also documented:
the tcptrace/ndt7 extras install command, bare-asn --target syntax,
ndt7's interactive consent prompt, tcp-trace's cascade fallback, and
that --watch writes one artifact per session rather than per tick.

architecture's storage section had the peeringdb cache path wrong
(.cache/peeringdb/*.json vs the real .cache/pdb-net-*.json) and never
mentioned the watch_*.json artifact from task 39. the file-size
exceptions note only covered exporter.py; interpret.py, cli.py,
models.py, bgp.py, ip_geo.py, netinfo.py, speed.py, reputation.py and
probes/traceroute.py all cleared ~200 lines too and are now listed
with the reason each stayed one file. test count updated from ~120
(written before a single module existed) to the real ~385.
2026-08-09 03:58:54 +03:00
Andrew Pimenov
879b974e57 exporter: markdown rendering with sparklines and hop bars 2026-08-09 02:44:43 +03:00
Andrew Pimenov
2a3020cdb9 bilingual readme and architecture docs 2026-08-08 21:21:29 +03:00