netsleuth/docs/superpowers/specs/2026-08-08-netcheck-design.md

# netcheck — design spec

Date: 2026-08-08
Status: approved for planning

## 1. Purpose

A cross-platform (Windows / Linux / macOS) Python 3.10+ CLI for deep network diagnostics: current-connection analysis (ISP/VPN/ASN identity, latency, path, bandwidth) and target-mode lookups (a given ASN/IP/domain). Produces a human-readable Markdown report and a complete machine-readable JSON dump per run. Personal/open-source tool, not commercial — this gates which free-tier APIs are usable (see §7).

## 2. Modes and flags

- **Auto mode (default):** diagnoses the device's active connection.
- **Target mode:** `--target AS<n>` or `--target <ip|domain>` — investigates a specific ASN/host instead of the local connection. Speedtest is skipped in target mode by default (measures the local line, not the target) unless explicitly requested.
- `--full` / `--quick` — full run (speedtest + MTR, ~60–120s) vs express (~10s: reference-host pings only, `mtr_cycles=1`, no speedtest).
- `--target-host <ip|domain>` — extra host added to ping/traceroute fan-out.
- `--speedtest-server <id|url>` — explicit speedtest server override.
- `--watch` — continuous monitoring: periodic re-run (lightweight probes; full speedtest only at a configurable interval), live Rich dashboard in-terminal. For catching intermittent issues (VPN drops in the evening, etc).
- `--compare <report_a.json> <report_b.json>` — diffs two prior JSON reports from `./logs/`, prints what changed (before/after a VPN switch, before/after an ISP call).
- `--dnsbl` — opt-in classic DNSBL reputation check (off by default, see §6).
- `--ndt7` — opt-in M-Lab NDT7 bandwidth tier in the speedtest cascade, with an explicit one-time consent notice (every NDT7 measurement is published as open CC0 data including the client IP — see §5).
- `--tcp-trace` — opt-in scapy-based TCP-SYN traceroute for probing through ICMP-filtering middleboxes (needs Npcap on Windows / root elsewhere). Not part of the default path.

## 3. Module layout

Public surface named in the brief (`cli.py`, `ip_geo.py`, `bgp.py`, `prober.py`, `speed.py`, `exporter.py`) plus supporting modules, split to respect the project convention of ~200 lines/file and one responsibility per module:

```
netcheck/
  pyproject.toml            # uv-based, source of truth
  uv.lock
  requirements.txt          # generated, for non-uv users
  config.yaml
  .env.example
  .gitignore                 # must include logs/, .claude/, .venv/
  README.md
  ARCHITECTURE.md
  src/netcheck/
    __init__.py  __main__.py
    cli.py                   # Typer app, flag parsing, phase orchestration, Rich progress — no business logic
    config.py                # pydantic-settings, YamlConfigSettingsSource, env > .env > yaml > default
    models.py                # ModuleResult[T], ProbeError, Finding, Signal — shared shapes
    netinfo.py               # local iface/gateway/MTU/DNS servers, 3-way OS branch
    ip_geo.py                # multi-provider identity/geo/IP-type + VPN signal gathering
    bgp.py                   # RIPEstat + CAIDA ASRank + Team Cymru + PeeringDB
    reputation.py            # FireHOL cached netsets + Shodan InternetDB + optional DNSBL/AbuseIPDB
    probes/
      __init__.py
      latency.py             # ping/jitter fan-out (icmplib / IcmpSendEcho2 / TCP-timing / cfL4)
      traceroute.py           # mtr → per-OS unprivileged ICMP → system-binary cascade
      icmp_win.py             # ctypes IcmpSendEcho2/Icmp6SendEcho2 engine (Windows only)
      dns_leak.py              # per-adapter resolver enumeration + echo probes + ECS detection
    traceparse.py             # pure text→TraceHop parsers: Windows/Linux/macOS
    speed.py                  # cascading speedtest: Ookla bin → Cloudflare → fast.com → [--ndt7]
    interpret.py              # pure verdict engine: thresholds → Finding[], VPN confidence scoring
    exporter.py               # JSON + Markdown rendering, atomic writes to ./logs/
    compare.py                # --compare: diff two JSON reports
    watch.py                  # --watch: periodic re-run loop + live Rich dashboard
  tests/
    conftest.py
    test_traceparse.py  test_interpret.py  test_exporter.py
    test_ip_geo_normalize.py  test_speed_math.py  test_prober_stats.py
    test_dnsbl_decode.py  test_compare.py
    fixtures/traceroute/{windows,linux,darwin}/*.txt
    fixtures/api/*.json
  logs/                      # gitignored — reports contain external IP, city, coords
```

## 4. Core data shapes

```
ModuleResult[T]:  name, status(ok|partial|failed|skipped), data: T|None,
                  errors: list[ProbeError], warnings: list[str],
                  started_at, duration_ms
ProbeError:       source, kind(timeout|http_error|rate_limited|blocked|
                  parse_error|unavailable|no_privilege|not_applicable), message, retryable
Finding:          id, severity(ok|info|warn|crit), title, detail, metric, value, threshold, advice
Signal:           name, observed, weight: float, direction(vpn|clean), note

Capabilities:     os_name('Windows'|'Linux'|'Darwin'), is_elevated,
                  icmp_dgram, icmp_raw, icmp_win_api,
                  mtr_binary, traceroute_binary,
                  chosen_latency_backend, chosen_trace_backend, notes

LocalNet:         iface_name, local_ipv4, local_ipv6, iface_mtu,
                  default_gateway_v4/v6, dns_servers_per_adapter: dict[str, list[str]],
                  is_dual_stack

IpGeo:            ip, ip_version, reverse_dns, asn, as_name, org,
                  country, country_code, city, lat, lon, timezone,
                  ip_type(residential|mobile|hosting|business|unknown), sources: dict[str,str]

VpnAssessment:    verdict(none|likely|confirmed), confidence, signals: list[Signal],
                  tunnel_iface, dns_leak: DnsLeak|None

DnsLeak:          per_adapter: list[AdapterLeakResult], ecs_leaked: bool, note
AdapterLeakResult: adapter, configured_resolvers: list[str],
                  echoed_ip, echoed_asn, matches_egress_asn

BgpIntel:         asn, holder, registry, allocated_at,
                  upstreams, peers, downstreams,
                  announced_prefixes, prefix_count_v4/v6,
                  flaps: list[BgpEvent], stability(stable|unstable|unknown),
                  ixps: list[IxpPresence], pdb_info_type, pdb_traffic,
                  asrank, cone_asns, cone_prefixes            # from CAIDA ASRank

Reputation:       internetdb: InternetDbResult|None,
                  firehol_hits: list[str],                    # matched blocklist names
                  dnsbl_hits: list[DnsblHit] | None,           # only if --dnsbl
                  dnsbl_query_blocked: bool,                    # true if resolver returned 127.255.255.254
                  abuseipdb_score, abuseipdb_reports,           # only if keyed
                  captcha_risk(low|medium|high), rationale

PingResult:       label, host, resolved_ip, method(icmp_win|icmp_dgram|icmp_raw|tcp|cfL4),
                  sent, received, loss_pct, min_ms, avg_ms, max_ms, mdev_ms, jitter_ms, samples

TraceHop:         ttl, ip, reverse_dns, asn, as_name, probes: list[float|None],
                  loss_pct, min_ms, avg_ms, max_ms, jitter_ms, annotations

TraceResult:      target, resolved_ip, backend(mtr_json|icmp_win|icmplib|system_traceroute|tcp_trace|none),
                  hops: list[TraceHop], cycles, completed, max_hops_reached

SpeedResult:      method(ookla_bin|cloudflare|fastcom|ndt7|none), tier_attempts: list[TierAttempt],
                  download_mbps, upload_mbps|None, server,
                  idle_rtt_ms, loaded_rtt_down_ms, loaded_rtt_up_ms,
                  bufferbloat_down_ms, bufferbloat_up_ms, bufferbloat_grade,
                  cfL4_stats: CfL4Stats|None, netflix_oca_onnet: bool|None
```

## 5. Speedtest cascade

1. **Ookla native binary** if on PATH (`speedtest --format=json --accept-license --accept-gdpr`). `speedtest-cli` (pip) is archived/unmaintained since 2021 — not used.
2. **Cloudflare** — raw httpx against `speed.cloudflare.com/__down`/`__up`, ramping transfer sizes, 90th-percentile throughput; harvest `cdn-cgi/trace` (`warp=`/`gateway=`/`rbi=` — also feeds VPN signals, §6) and the `Server-Timing: cfL4` header (kernel TCP RTT/loss/retransmit — used as an ICMP-free latency fallback everywhere, not just here).
3. **fast.com (Netflix)** — `api.fast.com/netflix/speedtest/v2` public token, download-only, download from returned OCA target URLs; the OCA hostname reveals on-net-vs-off-net Netflix appliance placement, a real ISP-peering-quality signal RIPEstat/PeeringDB can't provide directly.
4. **`--ndt7` (opt-in)** — M-Lab NDT7 over WebSocket via `locate.measurementlab.net`. Best TCP-level fidelity (BBR/retransmit detail). Requires one-time consent notice: results are published publicly as CC0 open data including the client IP. Not in the default cascade.

Bufferbloat measured inside `speed.py`: a concurrent latency probe loop during saturation, compared to the idle RTT baseline from `probes/latency.py`, graded A–F, download and upload reported separately.

Fails gracefully — cascade exhaustion yields `status=failed` for `speed`, never an exception; rest of the report unaffected.

## 6. VPN / IP-type detection

**Provider chain (parallel, merged by field):**
- `cdn-cgi/trace` (Cloudflare) — egress IP, colo, country, `warp`/`gateway`/`rbi` flags
- `ip-api.com` — geo/ASN + `mobile`/`proxy`/`hosting` booleans (HTTP-only free tier)
- `freeipapi.com` — HTTPS keyless equivalent, used in parallel so an ip-api block/rate-limit doesn't blank the classification
- `ipinfo.io` — opportunistic enrichment if a token is configured (keyless free tier 429s in practice; `privacy` field is paid-only)
- `ipwho.is` — geo/ASN backup
- RIPEstat `network-info` — authoritative prefix/ASN

Both IPv4 and IPv6 egress resolved and compared — providers have been observed to disagree (dual-stack egress can differ per path); mismatch is reported, not silently resolved.

**VPN confidence signals** (scored in `interpret.py`, gathered elsewhere):
private-gateway-vs-egress mismatch; tunnel interface present (`tun*`/`tap*`/`wg*`/`utun*`/`ppp*`, Windows `WireGuard Tunnel`/`TAP-Windows`); MTU anomaly (WireGuard ~1420, IPsec ~1400); `hosting`/`proxy`/`mobile` flags from the provider chain; Cloudflare `warp=on`; PeeringDB `info_type` indicating NSP/hosting rather than eyeball ISP; DNS-resolver-ASN ≠ egress-ASN (§ DNS leak below); OS timezone vs geo country mismatch.

**DNS leak test:** enumerate DNS servers per network adapter (not just "the system resolver" — catches the case where the VPN's tunnel adapter is clean but a Wi-Fi adapter still holds the ISP resolver). For each configured resolver, send echo probes to `o-o.myaddr.l.google.com` (TXT) and `whoami.ds.akahelp.net` (TXT — also surfaces EDNS Client Subnet leakage, a separate finding from ASN mismatch), map the echoed IP to an ASN via Team Cymru DNS, compare against egress ASN. Reported per-adapter. Report language is explicit that this detects OS-level resolver leaks only — a browser using DoH bypasses it.

## 7. BGP / ASN intelligence

- **RIPEstat** (`stat.ripe.net`, keyless) — `as-overview`, `asn-neighbours` (upstreams/peers/downstreams), `announced-prefixes`, `routing-status`, `bgp-updates` (flap history), `whois`. Verified global (no ARIN/APNIC-region gap). `routing-history`/similar bulk endpoints **must** be called with bounded `max_rows`/timeframe — unbounded calls on large ISPs return 10MB+ payloads.
- **CAIDA ASRank** (`api.asrank.caida.org`, keyless) — rank, customer-cone size (ASNs/prefixes/addresses), degree (customer/peer/provider) — the "how big is this network" metric RIPEstat doesn't have.
- **Team Cymru** (`<ip>.origin.asn.cymru.com` TXT, keyless DNS) — IP→ASN/prefix/RIR/alloc-date fallback for when HTTPS is unavailable; also used to resolve DNS-leak echo results (§6).
- **PeeringDB** (`www.peeringdb.com/api`, keyless REST) — IXP presence, `info_type`/`info_traffic`/`policy_general`. Anonymous limits are tight (20 req/min, large responses throttled to 1/hour) — results cached to disk, never re-fetched within a run.
- BGPView.io is defunct (shut down 2025-11-26) — not used. bgp.he.net is HTML-only — not scraped. bgp.tools has no REST API (whois:43 + static table dumps only, mandatory identifying User-Agent, HTML scraping bans) — not used in v1, candidate for a later optional enrichment.

## 8. Reputation

**Default (keyless, no per-query disclosure to a third party of "which IP am I checking"):**
- **FireHOL / community IP blocklists** — cached netset download (~2–5MB, refreshed periodically per config), local prefix-trie lookup. No runtime query, no rate limit, no privacy exposure.
- **Shodan InternetDB** (`internetdb.shodan.io`, keyless) — open ports/hostnames/tags on the egress IP. More actionable for a home user than a mail-blocklist hit (exposed router, forgotten port-forward). Free for non-commercial use, matching netcheck's own status (§ project status).

**Opt-in:**
- `--dnsbl` — classic Spamhaus ZEN / SpamCop / Barracuda / DroneBL zone queries. **Must** decode Spamhaus's `127.255.255.x` error range correctly: `.254` means "you queried via a public resolver" (hits a large share of real users on 1.1.1.1/8.8.8.8) and `.255` means rate-limited — neither is a listing, and a naive "any 127.x.x.x = listed" implementation produces false-positive red flags for most users. Both are surfaced as "DNSBL result unavailable" rather than a finding.
- AbuseIPDB / IPQualityScore — only when a key is configured in `.env`.

## 9. Traceroute / MTR

Cascade, no privilege ever required for the default path:

1. `mtr --json`/`mtr --report --json` binary if present (best data: per-hop loss% and stddev for free).
2. **Windows:** ctypes engine (`icmp_win.py`) around `IcmpSendEcho2`/`Icmp6SendEcho2` (`Iphlpapi.dll`) — the same unprivileged API `tracert.exe` itself uses internally. No Administrator, no Npcap, stdlib-only (`ctypes`). Per-hop RTT, TTL-expired detection, proper timeout handling — strictly better data than parsing `tracert.exe`'s text output (3 fixed probes, localized headers, no loss%/jitter).
3. **Linux/macOS:** `icmplib` unprivileged mode (`SOCK_DGRAM` ICMP) — works out of the box on macOS, on Linux subject to `net.ipv4.ping_group_range` (true by default on most desktop distros, may need adjustment in containers); falls back to `icmplib` privileged (raw socket, needs root) when available.
4. Text-parsing the system binary (`tracert.exe` / GNU `traceroute` / BSD `traceroute`) as the last resort, via `traceparse.py` — three separate parsers, since Windows/Linux/macOS output formats genuinely differ (BSD emits multi-IP-per-hop lines and `!H`/`!N`/`!X` annotations Linux doesn't; Windows output is locale- and code-page-dependent — cp866 on Russian Windows, not UTF-8 — so parsing is by line *shape* (leading hop integer, RTT tokens, IP regex), never by matching header/footer text).
5. `--tcp-trace` (opt-in, not default): scapy TCP-SYN traceroute for getting past ICMP-filtering middleboxes. Needs Npcap (Windows) / root (Unix) — kept out of the default dependency set and default path entirely.

Latency probing (`probes/latency.py`) never parses `ping`'s own text output (would mean three more locale-dependent parsers for no benefit): uses `icmp_win`/`icmplib` when available, else TCP-connect timing to port 443, else the Cloudflare `cfL4` header. Every result is tagged with the method used — TCP "loss" (connection-failure rate) and ICMP loss are different metrics and must never be silently conflated in the report.

## 10. Capability detection

Attempt-based, never OS-assumption-based; cached once per run into `Capabilities`, included in JSON `meta`.

- Elevation: `ctypes.windll.shell32.IsUserAnAdmin()` (Windows) / `os.geteuid() == 0` (Linux, macOS).
- ICMP: try unprivileged `SOCK_DGRAM` ICMP → raw `SOCK_RAW` ICMP → (Windows) `IcmpSendEcho2` availability (always available, no privilege check needed) → none.
- Never require elevation for the default path — worst case degrades to TCP-connect timing + system-binary traceroute + `cfL4` header, and the report states the degradation once, plainly, with the one-line remedy.

## 11. Error handling

Single wrapper `async def run_module(name, coro, *, timeout) -> ModuleResult` — times the call, enforces a per-module timeout, catches `BaseException` except `CancelledError`, classifies into `ProbeError.kind`, always returns a well-formed envelope. `asyncio.gather(..., return_exceptions=True)` as a second net at the orchestration layer. Failure is a value, not control flow — no single API/probe failure aborts the run.

Four status levels: `ok` → `partial` (some sub-sources failed) → `failed` (nothing usable) → `skipped` (not applicable / missing optional key).

Both outputs surface every failure: JSON carries per-module `errors[]` plus a flattened top-level `errors[]` union; Markdown has a mandatory "Run diagnostics" section (module/status/duration/error table) plus inline placeholders in the body instead of silently omitting sections (a report missing its speed section must not be indistinguishable from a report where speed was fine).

`exporter.py` uses `json.dumps(..., allow_nan=False)`, coercing `inf`/`NaN` (possible from zero-duration timing math) to `null` before serialization — otherwise "strictly valid JSON" breaks on exactly the failure cases that matter most.

## 12. Concurrency model

Single shared `httpx.AsyncClient`; per-host semaphores for rate-limited APIs (ip-api 45/min, PeeringDB 20/min). Blocking calls (`speedtest` binary via subprocess is fine natively; `icmplib`/ctypes send/receive; `psutil` enumeration) via `asyncio.to_thread` where genuinely blocking. Subprocess traceroute via `asyncio.create_subprocess_exec` (native on all three OSes, no thread wrapper needed).

Two hard constraints:
1. **Measurement isolation** — speedtest never overlaps traceroute/ping/API calls (an exclusive phase lock); the one intentional exception is the bufferbloat probe, which overlaps saturation *by design* inside `speed.py`.
2. **Bounded traceroute fan-out** — parallel traceroutes to different targets share early hops and inflate each other's latency; bound concurrent traceroutes with a semaphore (2–3), never unbounded `gather`.

Phase order: `netinfo` + `ip_geo` provider chain (blocking — everything downstream needs the ASN) → `bgp` ∥ `reputation` ∥ `dns_leak` (parallel) → `latency` ∥ `traceroute` fan-out (parallel, bounded) → `speed` (exclusive) → `interpret` → `exporter`. `--quick` skips speed and reduces the probe fan-out to reference hosts with `mtr_cycles=1`.

## 13. Config and secrets

`config.py` via `pydantic-settings` + `YamlConfigSettingsSource`, precedence env > `.env` > `config.yaml` > default. All three optional API keys (`IPINFO_TOKEN`, `PEERINGDB_API_KEY`, `ABUSEIPDB_API_KEY`) are `SecretStr | None = None` — a diagnostics tool must run with zero configuration; a missing key downgrades the dependent enrichment to `skipped` with a warning, never fails the run.

`.env` — secrets only (the three keys above). `config.yaml` — timeouts, `probing` (reference/service hosts, ping count, mtr cycles, max hops), `speedtest` (enabled tiers, transfer sizes, CDN fallback URLs), `providers` (base URLs, FireHOL refresh interval), `thresholds` (latency/jitter/loss/bufferbloat good/warn bands), `output` (logs dir, emoji on/off), `watch` (interval, dashboard refresh rate).

## 14. Output

**Filenames:** `report_<ASN>_<YYYYMMDDTHHMMSSZ>.{md,json}` (compact ISO — Windows forbids `:` in filenames), falling back to `report_unknown_…` when ASN lookup fails entirely. Written via temp-file + atomic rename into `./logs/` (gitignored — reports contain external IP, city, coordinates, ISP name).

**JSON top-level keys:** `schema_version, meta{run_id, started_at, finished_at, mode, target, flags, host_os, capabilities}, connection, ip_geo, vpn_assessment, bgp, reputation, latency, path, speed, interpretation{overall_status, overall_score, summary_text, findings}, errors, raw{<source>: <verbatim payload>, ...}`. `raw` stores every provider response verbatim under its source key — this is what actually satisfies "100% of collected data, no truncation," since the typed layer above it can drop fields the raw payload still has.

**Markdown sections:** Header (mode/target/timestamp/overall verdict) → TL;DR (top findings) → Connection & identity → VPN/proxy assessment (verdict + signals table) → ASN & BGP intelligence (upstreams, prefixes, stability, IXPs, ASRank) → Reputation (InternetDB, FireHOL hits, DNSBL if run) → Latency (per-host table + sparkline) → Path/MTR (hop table + ASCII bar, first-loss-jump highlighted) → Speed (down/up, cascade tier used, bufferbloat with plain-language consequence) → Problems & recommendations → Run diagnostics (module status table). ASCII graphs inside fenced code blocks so `rich.markdown.Markdown` doesn't reflow them; emoji (🟢/🟡/🔴) gated behind `output.emoji` config.

## 15. `--compare` and `--watch`

- **`--compare a.json b.json`** (`compare.py`): loads two prior JSON reports, diffs the typed fields (egress IP/ASN, VPN verdict, latency deltas per host, speed deltas, new/resolved Findings), prints a compact before/after table. Read-only, no new probing.
- **`--watch`** (`watch.py`): loop of lightweight runs (reference-host ping/jitter + a reduced traceroute) at a configurable interval, full speedtest only every N cycles (configurable), live-updating Rich table/dashboard in the terminal. Each cycle's summary is appended to the run's JSON log (not a full separate report file per tick) so a `--watch` session produces one time-series artifact, not hundreds of report files.

## 16. Testing strategy

Per project convention — meaningful behavior only, glue untested.

**Test:** `traceparse.py` (all three OS formats incl. BSD multi-IP hops, `!H`/`!N` annotations, cp866-localized Windows output, timeouts, truncated/max-hops-without-arrival); `interpret.py` (threshold→severity mapping, VPN confidence across signal combinations including combos that should *not* fire, bufferbloat grading); `ip_geo.py` normalizers (each provider's shape → common `IpGeo`, from captured fixtures); reputation `127.255.255.x` decoding (regression-critical — this is the false-positive bug found during design, §8); `speed.py` math (bytes+interval→Mbps, `cfL4` header parsing); `probes/latency.py` statistics (jitter/loss/mdev, all-timeout and single-sample edge cases); `exporter.py` (JSON validity with `allow_nan=False`/inf-NaN regression, all-modules-failed run still produces a complete report, filename sanitization); `compare.py` diff logic; cascade ordering (tier1 fails → tier2 used → ... → all-fail yields `status=failed` not an exception, via `pytest-httpx`).

**Skip:** live HTTP, real ICMP, Typer wiring, config loading, `psutil`/`ctypes` calls, `--watch` timing loop.

## 17. Dependencies

```
typer>=0.12          rich>=13.7           httpx[http2]>=0.28
pydantic>=2.7         pydantic-settings>=2.3
pyyaml>=6.0           dnspython>=2.6       psutil>=5.9
[optional] tcptrace = scapy>=2.5      # --tcp-trace only
[optional] ndt7     = websockets>=12  # --ndt7 only
[dev] pytest>=8  pytest-asyncio>=0.23  pytest-httpx>=0.30
```
`icmplib` is Unix-only per its own docs (ignored on Windows) — included as a base dependency, used conditionally by OS. No Ookla pip package dependency (archived); the native binary is an optional runtime dependency the tool detects, never installs.

## 18. Project status

Personal/open-source tool, not commercial, not sold or bundled into a paid product. This is a hard input to §6/§8's provider choice — Shodan InternetDB and Spamhaus's free mirrors are non-commercial-use-only, and this status is what makes them usable by default. If that status ever changes, §8's default reputation sources need re-review before shipping.