mirror of
https://github.com/Horizontal42/netsleuth.git
synced 2026-11-11 01:02:10 +03:00
enrich traceroute hops with ASN, AS name, country and PTR fallback
TraceHop.asn/as_name existed in the model since the start but nothing ever populated them. New probes/hop_asn.py fires one Team Cymru DNS query pair per unique hop IP (in-run deduped, no rate limit unlike the cached_json() providers) after the latency/path/TLS fan-out, plus a PTR lookup for hops whose reverse_dns is still empty because the backend that produced them doesn't resolve hostnames itself (icmp_win/icmplib/mtr, unlike system traceroute text). New path.detour_country finding flags a route that spends two or more consecutive hops in a country that's neither the client's nor the destination's. Wires up providers.cymru_asn_zone, which was configured but referenced nowhere, and adds cymru_origin6_zone for the IPv6 branch Phase 8 will need. Part of plans/zesty-zooming-wilkinson.md (gitignored).
This commit is contained in:
parent
76cfd4605c
commit
8d4c8bee8b
12 changed files with 318 additions and 5 deletions
|
|
@ -36,6 +36,7 @@ src/netsleuth/
|
|||
prefix_benchmark.py --prefix-bench: ping the first host of a capped sample of the AS's announced prefixes.
|
||||
dpi_check.py --my-server: single-host TCP RST/filtering self-diagnostic. Not a scanner — see Key decisions.
|
||||
nat64.py One RFC 7050 AAAA query (ipv4only.arpa) to detect NAT64/464XLAT translation.
|
||||
hop_asn.py Per-hop ASN/AS-name/country/PTR enrichment via Team Cymru DNS, in-run deduped.
|
||||
```
|
||||
|
||||
**File-size exceptions.** The ~200-line guideline (Global Constraints) held for most modules, but these grew past it during implementation and were kept as single files rather than split, because each one is a single cohesive responsibility that splitting would only scatter across more files with a thinner reason to draw the line anywhere in particular:
|
||||
|
|
@ -131,6 +132,8 @@ The AS prefix benchmark (`probes/prefix_benchmark.py`) gets the same treatment f
|
|||
|
||||
**The first traceroute hop is diagnosed separately from the rest of the path.** Loss or latency at hop 1 means the user's own router or Wi-Fi, not the ISP or anything downstream — `path_findings()` takes an optional `LocalNet` and, when supplied, compares the first hop's IP against the OS-observed default gateway before running the existing mid-path loss-jump logic, so the two questions ("is my LAN fine" vs "where does the wider path degrade") stay structurally separate rather than one drowning out the other.
|
||||
|
||||
**Per-hop ASN enrichment is a separate phase after the trace completes, not folded into traceroute itself.** `TraceHop.asn`/`as_name` existed in the model since the original design but were never populated by any writer — `probes/hop_asn.py::enrich_hops()` fills them (plus a new `country` field and a PTR fallback for backends that don't resolve hostnames themselves) by collecting the unique set of hop IPs across every trace and firing one Team Cymru DNS query pair per IP, deduped in-run. This runs as its own `run_module("path_enrich", ...)` step in `cli.py` after the latency/path/TLS fan-out and before speed, and mutates the already-collected `TraceHop` objects in place; a lookup failure folds into that step's warnings only; it can never downgrade the trace's own `ModuleResult.status`. Team Cymru's DNS service has no rate limit (unlike the `cached_json()`-wrapped RIPEstat/CAIDA calls), so no disk cache is needed — only in-run dedup of repeated hop IPs.
|
||||
|
||||
**Python 3.14 is the floor, not 3.10.** The project moved off `>=3.10` when this batch of probes was added; none of the new code actually depends on a 3.11+-only construct (`ssl.SSLObject.start_tls()`, `asyncio.timeout()`, `datetime.UTC`) — the two-sequential-connection TLS design above was chosen independently of what the interpreter allows. The version floor moved because there was no longer a reason to hold it back, not because a specific new feature required it.
|
||||
|
||||
## Storage
|
||||
|
|
@ -149,4 +152,4 @@ Five sections back the L4-L7 probes, all opt-in at the CLI level and all with a
|
|||
|
||||
## Tests
|
||||
|
||||
`uv run pytest -q`. Roughly 723 tests covering the parsers, normalizers, verdict engine, DNSBL decoding, throughput math, ping statistics, jitter percentiles, serialization and the report diff. Glue is deliberately untested: live HTTP, real ICMP, Typer wiring, `psutil`/`ctypes` calls and the `--watch` timing loop. Traceroute fixtures live in `tests/fixtures/traceroute/{windows,linux,darwin}/` and include a cp866-encoded Russian Windows sample, because that encoding is exactly what breaks naive parsers. DoH response fixtures (RFC 8484 JSON form) live in `tests/fixtures/api/doh_*.json` alongside the other provider fixtures.
|
||||
`uv run pytest -q`. Roughly 740 tests covering the parsers, normalizers, verdict engine, DNSBL decoding, throughput math, ping statistics, jitter percentiles, serialization and the report diff. Glue is deliberately untested: live HTTP, real ICMP, Typer wiring, `psutil`/`ctypes` calls and the `--watch` timing loop. Traceroute fixtures live in `tests/fixtures/traceroute/{windows,linux,darwin}/` and include a cp866-encoded Russian Windows sample, because that encoding is exactly what breaks naive parsers. DoH response fixtures (RFC 8484 JSON form) live in `tests/fixtures/api/doh_*.json` alongside the other provider fixtures.
|
||||
|
|
|
|||
|
|
@ -36,6 +36,7 @@ src/netsleuth/
|
|||
prefix_benchmark.py --prefix-bench: пинг первого хоста ограниченной выборки анонсируемых префиксов AS.
|
||||
dpi_check.py --my-server: одиночная самопроверка TCP RST/фильтрации. Не сканер — см. Ключевые решения.
|
||||
nat64.py Один AAAA-запрос по RFC 7050 (ipv4only.arpa) для детекта трансляции NAT64/464XLAT.
|
||||
hop_asn.py Обогащение хопов ASN/именем AS/страной/PTR через DNS Team Cymru, дедуп в рамках прогона.
|
||||
```
|
||||
|
||||
**Исключения по размеру файла.** Лимит ~200 строк (Общие ограничения) выдержан для большинства модулей, но эти выросли за него в процессе реализации и остались одним файлом, а не были разбиты, потому что каждый — единая цельная ответственность, и разбиение только раскидало бы её по файлам без чёткой причины проводить границу где-то конкретно:
|
||||
|
|
@ -131,6 +132,8 @@ src/netsleuth/
|
|||
|
||||
**Первый хоп трассировки диагностируется отдельно от остального маршрута.** Потери или задержка на хопе 1 означают проблему в собственном роутере или Wi-Fi пользователя, а не у провайдера или где-то дальше по маршруту — `path_findings()` принимает опциональный `LocalNet` и, если он передан, сравнивает IP первого хопа с шлюзом по умолчанию, который видит ОС, перед тем как запускать существующую логику потерь в середине маршрута — так два вопроса («в порядке ли моя локальная сеть» и «где деградирует остальной путь») остаются структурно разделены, а не тонут один в другом.
|
||||
|
||||
**Обогащение хопов по ASN — отдельная фаза после завершения трассировки, а не часть самого traceroute.** Поля `TraceHop.asn`/`as_name` существовали в модели с самого начала, но ни один писатель их не заполнял — `probes/hop_asn.py::enrich_hops()` заполняет их (плюс новое поле `country` и PTR-фоллбэк для бэкендов, которые сами не резолвят имена), собирая уникальный набор IP хопов по всем трассировкам и делая по одной паре DNS-запросов Team Cymru на IP, с дедупом в рамках прогона. Это отдельный шаг `run_module("path_enrich", ...)` в `cli.py` после веера latency/path/TLS и перед speed, мутирующий уже собранные объекты `TraceHop` на месте; ошибка поиска попадает только в предупреждения этого шага и никогда не понижает `ModuleResult.status` самой трассировки. У DNS-сервиса Team Cymru нет лимита запросов (в отличие от обёрнутых в `cached_json()` вызовов RIPEstat/CAIDA), поэтому дисковый кэш не нужен — только дедуп повторяющихся IP хопов в рамках прогона.
|
||||
|
||||
**Python 3.14 — минимальная версия, не 3.10.** Проект ушёл с `>=3.10` одновременно с этим набором проб; ни один новый код на деле не зависит от конструкций только для 3.11+ (`ssl.SSLObject.start_tls()`, `asyncio.timeout()`, `datetime.UTC`) — схема TLS с двумя последовательными соединениями выше выбрана независимо от того, что позволяет интерпретатор. Планка версии сдвинулась потому, что не осталось причин её удерживать, а не потому, что это потребовалось для конкретной новой функции.
|
||||
|
||||
## Хранение
|
||||
|
|
@ -149,4 +152,4 @@ src/netsleuth/
|
|||
|
||||
## Тесты
|
||||
|
||||
`uv run pytest -q`. Около 723 тестов: парсеры, нормализаторы, движок вердиктов, декодирование DNSBL, математика пропускной способности, статистика пингов, перцентили джиттера, сериализация и сравнение отчётов. Клей намеренно не тестируется: живой HTTP, настоящий ICMP, обвязка Typer, вызовы `psutil`/`ctypes` и цикл `--watch`. Фикстуры трассировок лежат в `tests/fixtures/traceroute/{windows,linux,darwin}/` и включают образец русской Windows в cp866 — именно эта кодировка ломает наивные парсеры. Фикстуры DoH-ответов (формат RFC 8484 JSON) лежат в `tests/fixtures/api/doh_*.json` рядом с фикстурами других провайдеров.
|
||||
`uv run pytest -q`. Около 740 тестов: парсеры, нормализаторы, движок вердиктов, декодирование DNSBL, математика пропускной способности, статистика пингов, перцентили джиттера, сериализация и сравнение отчётов. Клей намеренно не тестируется: живой HTTP, настоящий ICMP, обвязка Typer, вызовы `psutil`/`ctypes` и цикл `--watch`. Фикстуры трассировок лежат в `tests/fixtures/traceroute/{windows,linux,darwin}/` и включают образец русской Windows в cp866 — именно эта кодировка ломает наивные парсеры. Фикстуры DoH-ответов (формат RFC 8484 JSON) лежат в `tests/fixtures/api/doh_*.json` рядом с фикстурами других провайдеров.
|
||||
|
|
|
|||
|
|
@ -62,7 +62,7 @@ None of this is required — netsleuth degrades gracefully without any of it —
|
|||
- **DNS leak test** — enumerates resolvers *per network adapter*, so it catches the classic case where the tunnel adapter is clean but the Wi-Fi adapter still holds the ISP resolver. Also detects EDNS Client Subnet leakage.
|
||||
- **BGP intelligence** — upstreams, peers, downstreams, announced prefixes, route stability, IXP presence, and CAIDA ASRank customer-cone size.
|
||||
- **Reputation** — locally cached FireHOL blocklists (no query leaves your machine), Shodan InternetDB exposure, and optional classic DNSBL zones.
|
||||
- **Latency & path** — per-host ping statistics with real jitter/loss/mdev/p95/p99, and a traceroute that never needs administrator rights. The first hop is diagnosed separately from the rest of the route, so loss or slowness there is correctly attributed to your own router/Wi-Fi rather than the ISP.
|
||||
- **Latency & path** — per-host ping statistics with real jitter/loss/mdev/p95/p99, and a traceroute that never needs administrator rights. The first hop is diagnosed separately from the rest of the route, so loss or slowness there is correctly attributed to your own router/Wi-Fi rather than the ISP. Every hop is annotated with its ASN, AS name and country, and flags a route that detours through a third country on the way to the destination.
|
||||
- **Bandwidth** — a cascading speedtest (Ookla binary → Cloudflare → fast.com → optional M-Lab NDT7) with separate download and upload bufferbloat grades.
|
||||
- **TLS handshake RTT** (`--tls`) — TCP RTT, TLS 1.3 handshake time and time-to-first-byte to reference services, flagging servers whose handshake is disproportionately slow relative to the network (CPU-bound TLS termination, not a network problem).
|
||||
- **DNS: system vs DoH** (`--dns-advanced`) — compares your system resolver's answers and latency against Cloudflare/Google/Quad9 DoH, and probes a bogus resolver IP to detect a transparent DNS proxy intercepting port 53.
|
||||
|
|
|
|||
|
|
@ -62,7 +62,7 @@ netsleuth
|
|||
- **Тест утечки DNS** — перечисляет резолверы *по каждому сетевому адаптеру*, поэтому ловит классический случай, когда туннельный адаптер чистый, а на Wi-Fi-адаптере остался резолвер провайдера. Также определяет утечку EDNS Client Subnet.
|
||||
- **BGP-разведка** — аплинки, пиры, клиенты, анонсируемые префиксы, стабильность маршрутов, присутствие на IX и размер клиентского конуса по CAIDA ASRank.
|
||||
- **Репутация** — локально кэшируемые списки FireHOL (ни один запрос не покидает машину), экспозиция по Shodan InternetDB и опционально классические зоны DNSBL.
|
||||
- **Задержки и маршрут** — статистика пингов по каждому хосту с настоящими джиттером/потерями/mdev/p95/p99 и трассировка, которой никогда не нужны права администратора. Первый хоп диагностируется отдельно от остального маршрута, поэтому потери или медлительность там правильно относятся к вашему роутеру/Wi-Fi, а не к провайдеру.
|
||||
- **Задержки и маршрут** — статистика пингов по каждому хосту с настоящими джиттером/потерями/mdev/p95/p99 и трассировка, которой никогда не нужны права администратора. Первый хоп диагностируется отдельно от остального маршрута, поэтому потери или медлительность там правильно относятся к вашему роутеру/Wi-Fi, а не к провайдеру. Каждый хоп размечен своим ASN, именем AS и страной, а маршрут, уходящий крюком через третью страну, отмечается отдельно.
|
||||
- **Скорость** — каскадный замер (бинарник Ookla → Cloudflare → fast.com → опционально M-Lab NDT7) с раздельными оценками bufferbloat на приём и передачу.
|
||||
- **TLS handshake RTT** (`--tls`) — TCP RTT, время TLS 1.3-рукопожатия и время до первого байта к эталонным сервисам; отдельно отмечает сервер, чьё рукопожатие непропорционально медленнее сети (упирается в CPU TLS-терминатора, а не в сеть).
|
||||
- **DNS: системный vs DoH** (`--dns-advanced`) — сравнивает ответы и задержку системного резолвера с Cloudflare/Google/Quad9 DoH, а также проверяет фиктивный IP резолвера на признаки прозрачного DNS-прокси, перехватывающего 53 порт.
|
||||
|
|
|
|||
|
|
@ -44,6 +44,7 @@ providers:
|
|||
internetdb_url: "https://internetdb.shodan.io/"
|
||||
abuseipdb_url: "https://api.abuseipdb.com/api/v2/check"
|
||||
cymru_origin_zone: "origin.asn.cymru.com"
|
||||
cymru_origin6_zone: "origin6.asn.cymru.com"
|
||||
cymru_asn_zone: "asn.cymru.com"
|
||||
ripestat_max_rows: 200
|
||||
ripestat_timeframe_days: 14
|
||||
|
|
|
|||
|
|
@ -28,6 +28,7 @@ from netsleuth.interpret import (
|
|||
dual_stack_findings,
|
||||
gather_vpn_signals,
|
||||
latency_findings,
|
||||
path_asn_findings,
|
||||
path_diversity_findings,
|
||||
path_findings,
|
||||
prefix_findings,
|
||||
|
|
@ -39,6 +40,7 @@ from netsleuth.models import BindTarget, Finding, IpGeo, LocalNet, ModuleResult,
|
|||
from netsleuth.netinfo import collect_local_net, detect_capabilities, iface_for_ip, is_tunnel_iface, primary_interface_ip
|
||||
from netsleuth.orchestration import gather_modules, run_module, utc_now_iso
|
||||
from netsleuth.probes.dns_leak import collect_dns_leak
|
||||
from netsleuth.probes.hop_asn import enrich_hops
|
||||
from netsleuth.probes.latency import ping_fanout, tcp_connect_rtt
|
||||
from netsleuth.probes.nat64 import detect_nat64
|
||||
from netsleuth.probes.traceroute import filter_trace_tiers, tier_order, traceroute
|
||||
|
|
@ -574,6 +576,22 @@ async def diagnose(settings: Settings, options: Options) -> tuple[dict, list[Pat
|
|||
"dpi_check", _dpi_section(settings, options), timeout=timeouts.module_seconds
|
||||
)
|
||||
|
||||
# Phase 3c — per-hop ASN/AS-name/country enrichment. Mutates the trace hops
|
||||
# in place; a lookup failure must never downgrade the trace itself, so only
|
||||
# its warnings (not its status) get folded into modules["path"].
|
||||
enrich_result = await run_module(
|
||||
"path_enrich",
|
||||
enrich_hops(
|
||||
modules["path"].data or [],
|
||||
origin_zone=settings.providers.cymru_origin_zone,
|
||||
origin6_zone=settings.providers.cymru_origin6_zone,
|
||||
asn_zone=settings.providers.cymru_asn_zone,
|
||||
timeout=timeouts.dns_seconds,
|
||||
),
|
||||
timeout=timeouts.module_seconds,
|
||||
)
|
||||
modules["path"].warnings.extend(enrich_result.warnings)
|
||||
|
||||
# Phase 4 — speed, exclusive: nothing else is in flight at this point.
|
||||
pings = modules["latency"].data or []
|
||||
idle_rtt = min((p.avg_ms for p in pings if p.avg_ms is not None), default=None)
|
||||
|
|
@ -604,6 +622,7 @@ async def diagnose(settings: Settings, options: Options) -> tuple[dict, list[Pat
|
|||
findings = _dedupe(
|
||||
latency_findings(pings, settings.thresholds)
|
||||
+ [f for trace in traces for f in path_findings(trace, local, settings.thresholds)]
|
||||
+ [f for trace in traces for f in path_asn_findings(trace, geo.country_code)]
|
||||
+ speed_findings(speed, settings.thresholds.bufferbloat_ms)
|
||||
+ l7_findings
|
||||
+ cgnat_findings(local, traces)
|
||||
|
|
|
|||
|
|
@ -77,6 +77,7 @@ class Providers(BaseModel):
|
|||
internetdb_url: str = "https://internetdb.shodan.io/"
|
||||
abuseipdb_url: str = "https://api.abuseipdb.com/api/v2/check"
|
||||
cymru_origin_zone: str = "origin.asn.cymru.com"
|
||||
cymru_origin6_zone: str = "origin6.asn.cymru.com"
|
||||
cymru_asn_zone: str = "asn.cymru.com"
|
||||
ripestat_max_rows: int = 200
|
||||
ripestat_timeframe_days: int = 14
|
||||
|
|
|
|||
|
|
@ -609,9 +609,12 @@ def _path(report: dict, lang: str = "en") -> list[str]:
|
|||
]
|
||||
for hop in hops:
|
||||
marker = " <<" if jump is not None and hop.get("ttl") == jump else ""
|
||||
asn = hop.get("asn") or ""
|
||||
country = hop.get("country") or ""
|
||||
lines.append(
|
||||
f"{hop.get('ttl', 0):>3} {(hop.get('ip') or '*'):<39} "
|
||||
f"{_num(hop.get('avg_ms')):>8} ms {hop.get('loss_pct', 0):>5}% {_bar(hop.get('avg_ms'))}{marker}"
|
||||
f"{_num(hop.get('avg_ms')):>8} ms {hop.get('loss_pct', 0):>5}% "
|
||||
f"{asn:<8} {country:<2} {_bar(hop.get('avg_ms'))}{marker}"
|
||||
)
|
||||
lines += ["```", ""]
|
||||
return lines
|
||||
|
|
|
|||
|
|
@ -284,6 +284,38 @@ def dual_stack_findings(nat64_prefix: str | None, local: LocalNet) -> list[Findi
|
|||
]
|
||||
|
||||
|
||||
def path_asn_findings(trace: TraceResult, client_country: str | None) -> list[Finding]:
|
||||
if not client_country or len(trace.hops) < 2:
|
||||
return []
|
||||
final_country = trace.hops[-1].country
|
||||
run: list[TraceHop] = []
|
||||
for hop in trace.hops:
|
||||
detour = bool(hop.country) and hop.country != client_country and hop.country != final_country
|
||||
if not detour:
|
||||
run = []
|
||||
continue
|
||||
run.append(hop)
|
||||
if len(run) < 2:
|
||||
continue
|
||||
countries = ", ".join(dict.fromkeys(h.country for h in run if h.country))
|
||||
first, last = run[0], run[-1]
|
||||
return [
|
||||
Finding(
|
||||
id="path.detour_country",
|
||||
severity="info",
|
||||
title=f"Route to {trace.target} detours through {countries}",
|
||||
detail=f"Hops {first.ttl}-{last.ttl} are in {countries}, neither your country ({client_country}) nor the destination's.",
|
||||
metric="detour_countries",
|
||||
value=countries,
|
||||
advice="Not necessarily a problem, but it explains extra latency if the detour is geographically far.",
|
||||
title_ru=f"Маршрут до {trace.target} идёт через {countries}",
|
||||
detail_ru=f"Хопы {first.ttl}-{last.ttl} находятся в {countries} — не в вашей стране ({client_country}) и не в стране назначения.",
|
||||
advice_ru="Не обязательно проблема, но объясняет дополнительную задержку, если крюк географически большой.",
|
||||
)
|
||||
]
|
||||
return []
|
||||
|
||||
|
||||
SIGNAL_WEIGHTS: dict[str, float] = {
|
||||
"tunnel_iface": 0.35,
|
||||
"cf_warp": 0.50,
|
||||
|
|
|
|||
|
|
@ -292,6 +292,7 @@ class TraceHop:
|
|||
reverse_dns: str | None = None
|
||||
asn: str | None = None
|
||||
as_name: str | None = None
|
||||
country: str | None = None
|
||||
probes: list[float | None] = field(default_factory=list)
|
||||
loss_pct: float = 0.0
|
||||
min_ms: float | None = None
|
||||
|
|
|
|||
141
src/netsleuth/probes/hop_asn.py
Normal file
141
src/netsleuth/probes/hop_asn.py
Normal file
|
|
@ -0,0 +1,141 @@
|
|||
from __future__ import annotations
|
||||
|
||||
import asyncio
|
||||
import ipaddress
|
||||
|
||||
import dns.asyncresolver
|
||||
import dns.exception
|
||||
import dns.reversename
|
||||
|
||||
from netsleuth.bgp import parse_cymru_origin
|
||||
from netsleuth.models import TraceResult
|
||||
|
||||
_NON_ROUTABLE = (
|
||||
ipaddress.ip_network("10.0.0.0/8"),
|
||||
ipaddress.ip_network("172.16.0.0/12"),
|
||||
ipaddress.ip_network("192.168.0.0/16"),
|
||||
ipaddress.ip_network("100.64.0.0/10"),
|
||||
ipaddress.ip_network("127.0.0.0/8"),
|
||||
ipaddress.ip_network("169.254.0.0/16"),
|
||||
ipaddress.ip_network("::1/128"),
|
||||
ipaddress.ip_network("fe80::/10"),
|
||||
ipaddress.ip_network("fc00::/7"),
|
||||
)
|
||||
|
||||
|
||||
def _is_public(addr: ipaddress.IPv4Address | ipaddress.IPv6Address) -> bool:
|
||||
return not any(addr in net for net in _NON_ROUTABLE)
|
||||
|
||||
|
||||
def cymru_query_name(ip: str | None, origin_zone: str, origin6_zone: str) -> str | None:
|
||||
if not ip:
|
||||
return None
|
||||
try:
|
||||
addr = ipaddress.ip_address(ip)
|
||||
except ValueError:
|
||||
return None
|
||||
if not _is_public(addr):
|
||||
return None
|
||||
if addr.version == 4:
|
||||
reversed_octets = ".".join(reversed(ip.split(".")))
|
||||
return f"{reversed_octets}.{origin_zone}"
|
||||
nibbles = addr.exploded.replace(":", "")
|
||||
reversed_nibbles = ".".join(reversed(nibbles))
|
||||
return f"{reversed_nibbles}.{origin6_zone}"
|
||||
|
||||
|
||||
def parse_cymru_asname(txt: str) -> tuple[str | None, str | None]:
|
||||
parts = [p.strip() for p in txt.split("|")]
|
||||
if len(parts) < 5:
|
||||
return None, None
|
||||
return (parts[4] or None), (parts[1] or None)
|
||||
|
||||
|
||||
async def _txt(resolver: dns.asyncresolver.Resolver, name: str) -> list[str]:
|
||||
try:
|
||||
answer = await resolver.resolve(name, "TXT")
|
||||
except dns.exception.DNSException:
|
||||
return []
|
||||
return [b" ".join(record.strings).decode("utf-8", "replace") for record in answer]
|
||||
|
||||
|
||||
async def _ptr(resolver: dns.asyncresolver.Resolver, ip: str) -> str | None:
|
||||
try:
|
||||
answer = await resolver.resolve(dns.reversename.from_address(ip), "PTR")
|
||||
except (dns.exception.DNSException, ValueError):
|
||||
return None
|
||||
return str(answer[0]).rstrip(".") if len(answer) else None
|
||||
|
||||
|
||||
async def lookup_hop(
|
||||
ip: str | None,
|
||||
*,
|
||||
origin_zone: str,
|
||||
origin6_zone: str,
|
||||
asn_zone: str,
|
||||
timeout: float,
|
||||
) -> dict[str, str | None]:
|
||||
query_name = cymru_query_name(ip, origin_zone, origin6_zone)
|
||||
if query_name is None:
|
||||
return {}
|
||||
resolver = dns.asyncresolver.Resolver()
|
||||
resolver.lifetime = timeout
|
||||
origin_records, ptr_name = await asyncio.gather(_txt(resolver, query_name), _ptr(resolver, ip))
|
||||
result: dict[str, str | None] = {}
|
||||
if ptr_name:
|
||||
result["reverse_dns"] = ptr_name
|
||||
origin: dict[str, str] = {}
|
||||
for record in origin_records:
|
||||
parsed = parse_cymru_origin(record)
|
||||
if parsed.get("asn"):
|
||||
origin = parsed
|
||||
break
|
||||
if not origin.get("asn"):
|
||||
return result
|
||||
asn_number = origin["asn"].removeprefix("AS")
|
||||
as_name: str | None = None
|
||||
country = origin.get("country")
|
||||
for record in await _txt(resolver, f"AS{asn_number}.{asn_zone}"):
|
||||
name, name_country = parse_cymru_asname(record)
|
||||
if name:
|
||||
as_name, country = name, (name_country or country)
|
||||
break
|
||||
result.update({"asn": origin["asn"], "as_name": as_name, "country": country})
|
||||
return result
|
||||
|
||||
|
||||
async def enrich_hops(
|
||||
traces: list[TraceResult],
|
||||
*,
|
||||
origin_zone: str,
|
||||
origin6_zone: str,
|
||||
asn_zone: str,
|
||||
timeout: float,
|
||||
concurrency: int = 8,
|
||||
) -> None:
|
||||
hops_by_ip: dict[str, list] = {}
|
||||
for trace in traces:
|
||||
for hop in trace.hops:
|
||||
if hop.ip:
|
||||
hops_by_ip.setdefault(hop.ip, []).append(hop)
|
||||
if not hops_by_ip:
|
||||
return
|
||||
semaphore = asyncio.Semaphore(concurrency)
|
||||
|
||||
async def _bounded(ip: str) -> tuple[str, dict[str, str | None]]:
|
||||
async with semaphore:
|
||||
return ip, await lookup_hop(
|
||||
ip, origin_zone=origin_zone, origin6_zone=origin6_zone, asn_zone=asn_zone, timeout=timeout
|
||||
)
|
||||
|
||||
results = await asyncio.gather(*(_bounded(ip) for ip in hops_by_ip))
|
||||
for ip, info in results:
|
||||
if not info:
|
||||
continue
|
||||
for hop in hops_by_ip[ip]:
|
||||
if info.get("asn"):
|
||||
hop.asn = info["asn"]
|
||||
hop.as_name = info.get("as_name")
|
||||
hop.country = info.get("country")
|
||||
if info.get("reverse_dns") and not hop.reverse_dns:
|
||||
hop.reverse_dns = info["reverse_dns"]
|
||||
109
tests/test_hop_asn.py
Normal file
109
tests/test_hop_asn.py
Normal file
|
|
@ -0,0 +1,109 @@
|
|||
from __future__ import annotations
|
||||
|
||||
import pytest
|
||||
|
||||
from netsleuth.config import Thresholds
|
||||
from netsleuth.interpret import path_asn_findings
|
||||
from netsleuth.models import TraceHop, TraceResult
|
||||
from netsleuth.probes.hop_asn import cymru_query_name, enrich_hops, parse_cymru_asname
|
||||
|
||||
ORIGIN_ZONE = "origin.asn.cymru.com"
|
||||
ORIGIN6_ZONE = "origin6.asn.cymru.com"
|
||||
|
||||
|
||||
def test_cymru_query_name_reverses_ipv4_octets():
|
||||
assert cymru_query_name("1.2.3.4", ORIGIN_ZONE, ORIGIN6_ZONE) == f"4.3.2.1.{ORIGIN_ZONE}"
|
||||
|
||||
|
||||
def test_cymru_query_name_reverses_ipv6_nibbles_against_a_hand_expanded_literal():
|
||||
# 2001:db8::1 expands to 2001:0db8:0000:...:0001 -> 32 hex nibbles, reversed and
|
||||
# dot-joined. Verified independently via `ipaddress` before hardcoding here.
|
||||
expected = "1" + ".0" * 23 + ".8.b.d.0.1.0.0.2"
|
||||
assert cymru_query_name("2001:db8::1", ORIGIN_ZONE, ORIGIN6_ZONE) == f"{expected}.{ORIGIN6_ZONE}"
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
"ip",
|
||||
["10.0.0.1", "192.168.1.1", "100.64.0.1", "127.0.0.1", "fe80::1", None, "*"],
|
||||
)
|
||||
def test_cymru_query_name_returns_none_for_non_routable_or_invalid_input(ip):
|
||||
assert cymru_query_name(ip, ORIGIN_ZONE, ORIGIN6_ZONE) is None
|
||||
|
||||
|
||||
def test_parse_cymru_asname_extracts_name_and_country():
|
||||
name, country = parse_cymru_asname("15169 | US | arin | 2000-03-30 | GOOGLE, US")
|
||||
assert name == "GOOGLE, US"
|
||||
assert country == "US"
|
||||
|
||||
|
||||
def test_parse_cymru_asname_handles_a_malformed_record():
|
||||
assert parse_cymru_asname("garbage without pipes") == (None, None)
|
||||
|
||||
|
||||
def test_parse_cymru_asname_handles_an_empty_record():
|
||||
assert parse_cymru_asname("") == (None, None)
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_enrich_hops_dedupes_repeated_ips_and_skips_none_ips(monkeypatch):
|
||||
calls: list[str] = []
|
||||
|
||||
async def fake_lookup_hop(ip, **kwargs):
|
||||
calls.append(ip)
|
||||
return {"asn": "AS15169", "as_name": "GOOGLE", "country": "US"}
|
||||
|
||||
monkeypatch.setattr("netsleuth.probes.hop_asn.lookup_hop", fake_lookup_hop)
|
||||
|
||||
hop_a = TraceHop(ttl=1, ip="8.8.8.8")
|
||||
hop_b = TraceHop(ttl=2, ip="8.8.8.8")
|
||||
hop_c = TraceHop(ttl=3, ip=None)
|
||||
trace = TraceResult(target="8.8.8.8", hops=[hop_a, hop_b, hop_c])
|
||||
|
||||
await enrich_hops(
|
||||
[trace], origin_zone=ORIGIN_ZONE, origin6_zone=ORIGIN6_ZONE, asn_zone="asn.cymru.com", timeout=1.0
|
||||
)
|
||||
|
||||
assert calls == ["8.8.8.8"]
|
||||
assert hop_a.asn == hop_b.asn == "AS15169"
|
||||
assert hop_c.asn is None
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_enrich_hops_fills_reverse_dns_only_when_absent(monkeypatch):
|
||||
async def fake_lookup_hop(ip, **kwargs):
|
||||
return {"reverse_dns": "router.example.net"}
|
||||
|
||||
monkeypatch.setattr("netsleuth.probes.hop_asn.lookup_hop", fake_lookup_hop)
|
||||
|
||||
hop_with_rdns = TraceHop(ttl=1, ip="1.1.1.1", reverse_dns="already.set")
|
||||
hop_without_rdns = TraceHop(ttl=2, ip="9.9.9.9")
|
||||
trace = TraceResult(target="1.1.1.1", hops=[hop_with_rdns, hop_without_rdns])
|
||||
|
||||
await enrich_hops(
|
||||
[trace], origin_zone=ORIGIN_ZONE, origin6_zone=ORIGIN6_ZONE, asn_zone="asn.cymru.com", timeout=1.0
|
||||
)
|
||||
|
||||
assert hop_with_rdns.reverse_dns == "already.set"
|
||||
assert hop_without_rdns.reverse_dns == "router.example.net"
|
||||
|
||||
|
||||
def _trace_with_countries(*countries: str | None) -> TraceResult:
|
||||
hops = [TraceHop(ttl=i + 1, ip=f"1.1.1.{i}", country=c) for i, c in enumerate(countries)]
|
||||
return TraceResult(target="1.1.1.1", hops=hops)
|
||||
|
||||
|
||||
def test_path_asn_findings_fires_on_two_consecutive_foreign_hops():
|
||||
trace = _trace_with_countries("RU", "DE", "DE", "US")
|
||||
findings = path_asn_findings(trace, "RU")
|
||||
assert [f.id for f in findings] == ["path.detour_country"]
|
||||
assert "DE" in findings[0].value
|
||||
|
||||
|
||||
def test_path_asn_findings_silent_on_a_single_foreign_hop():
|
||||
trace = _trace_with_countries("RU", "DE", "US")
|
||||
assert path_asn_findings(trace, "RU") == []
|
||||
|
||||
|
||||
def test_path_asn_findings_silent_when_country_is_unknown_throughout():
|
||||
trace = _trace_with_countries(None, None, None)
|
||||
assert path_asn_findings(trace, "RU") == []
|
||||
Loading…
Add table
Reference in a new issue