mokhtar/nxdns
1
0
Fork 0
Code Pull Requests Actions Packages Releases 13 Activity

milestone 21: abp list exceptions and a regex rule kind

Browse Source
This commit is contained in:
mokhtar
2026-08-13 19:14:47 +02:00
parent b340521716
commit 2ab7c1f1de
51 changed files with 4016 additions and 465 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/tutorial/first-run.md
+42 -25
View File
@@ -9,13 +9,13 @@ delete the directory.
Follow the steps in order. Each one says what it did.
Every command below was executed on x86_64 Linux with Zig 0.16.0, Node.js
24.14.1, dig 9.20.26 and curl 8.21.0. Steps 4 to 11, 13 and 14 were re-run end
to end for this revision, and the transcripts are that run's output with the
24.14.1, dig 9.20.26 and curl 8.21.0. Steps 2, 4 to 11, 13 and 14 were re-run
end to end for this revision, and the transcripts are that run's output with the
tutorial directory substituted. Two things were not re-run: the browser page in
step 12 — its endpoints were exercised, the page itself was not opened — and the
two build commands in steps 1 and 2, which had already produced the binary under
test. The ZON block at the end of step 14 was checked with `nxdns check
--config` rather than started.
`npm` build in step 1, whose `web/dist` was already on disk and is the one the
binary under test embeds. The ZON block at the end of step 14 was checked with
`nxdns check --config` rather than started.
## What you need
@@ -109,7 +109,7 @@ zig-out/bin/nxdns import ~/nxdns-tutorial/config.zon --data-dir ~/nxdns-tutorial
```
```
info(migrations): config.db migrated from schema version 0 to 2
info(migrations): config.db migrated from schema version 0 to 4
imported /home/you/nxdns-tutorial/config.zon
```
@@ -127,7 +127,7 @@ zig-out/bin/nxdns run --data-dir ~/nxdns-tutorial/data
```
info(querylog_schema): created querylog database '/home/you/nxdns-tutorial/data/querylog.db'
info(blocklist_manager): blocklist snapshot generation 1: 0 of 0 sources loaded, 199 bytes
info(blocklist_manager): blocklist snapshot generation 1: 0 of 0 sources loaded, 231 bytes
info(nxdns): authority: database
info(nxdns): nxdns <version> serving on udp [::1]:15353 udp 127.0.0.1:15353 tcp [::1]:15353 tcp 127.0.0.1:15353; 1 upstream(s); blocklist generation 1
info(web_server): web interface listening on 127.0.0.1:8080
@@ -146,8 +146,8 @@ dig @127.0.0.1 -p 15353 example.com A +noall +answer
```
```
example.com. 90 IN A 172.66.147.243
example.com. 90 IN A 104.20.23.154
example.com. 229 IN A 172.66.147.243
example.com. 229 IN A 104.20.23.154
```
nxdns had no answer cached, so it forwarded the query to
@@ -220,20 +220,27 @@ curl -s -X POST http://127.0.0.1:8080/api/blocklists/update
```
```json
{"sources":[{"id":1,"state":"ok","loaded":true,"last_attempt":1786473715,"last_success":1786473715,"url":"https://raw.githubusercontent.com/StevenBlack/hosts/master/hosts","last_error":"","domains":99559,"wildcards":0,"skipped_regex":0}]}
{"sources":[{"id":1,"state":"ok","loaded":true,"last_attempt":1786629237,"last_success":1786629238,"url":"https://raw.githubusercontent.com/StevenBlack/hosts/master/hosts","last_error":"","domains":97648,"wildcards":0,"exceptions":0,"skipped_regex":0}]}
```
The three zeros are the parts of this list that a hosts file cannot have.
`wildcards` counts entries covering a name and its subdomains, `exceptions`
counts the `@@` lines an Adblock Plus list uses to lift a name another list
blocks, and `skipped_regex` counts the regex lines nxdns declines to take from a
downloaded list. Only the Adblock Plus syntax writes any of them; a hosts file is
one name per line.
The download is about 3 MB and takes a few seconds. Watch the first terminal
until this appears:
```
info(blocklist_manager): blocklist snapshot generation 6: 1 of 1 sources loaded, 3096006 bytes
info(blocklist_manager): blocklist snapshot generation 6: 1 of 1 sources loaded, 2523973 bytes
```
`1 of 1 sources loaded` is the line to wait for. nxdns builds each blocklist
snapshot in full and swaps it in atomically, so queries keep being answered from
the previous snapshot the whole time the new one is being built. After this, the
domain count in the JSON above — 99559 on the day this was run — is live.
domain count in the JSON above — 97648 on the day this was run — is live.
From here on, the list is on disk under `~/nxdns-tutorial/data/blocklists`.
Restarting nxdns does not re-download it.
@@ -259,16 +266,18 @@ dig @127.0.0.1 -p 15353 wikipedia.org A +noall +answer
```
```
wikipedia.org. 17 IN A 185.15.58.224
wikipedia.org. 130 IN A 185.15.58.224
```
One thing to know before you try other names: a blocklist entry blocks exactly
the name it names. `doubleclick.net` on the list does not block
`ads.doubleclick.net`; that name is blocked because the list happens to contain
it too. Blocklist entries do not walk up the parent chain — only rules you write
yourself can, with a wildcard pattern such as `*.doubleclick.net`. So when you
pick a domain to test, pick one that is literally in the file.
`www.google-analytics.com` is another that is.
One thing to know before you try other names: an entry in a hosts list blocks
exactly the name it names. `doubleclick.net` is on this list, and so are
`ad.doubleclick.net` and `www.google-analytics.com`. `ads.doubleclick.net` is
not on it, and nxdns does not block it — the entry for the parent says nothing
about the child. Two things do walk up the parent chain, and neither is in play
here: a rule you write yourself, with a wildcard pattern such as
`*.doubleclick.net`, and an Adblock Plus list's `||doubleclick.net^`, which
covers the name and everything under it. This list is a hosts file, so when you
pick a domain to test against it, pick one that is literally in the file.
## 12. Open the web interface
@@ -291,7 +300,7 @@ nxdns catches SIGINT and SIGTERM, stops serving and exits 0.
Start it again with the same command as in step 6 and read the first log lines:
```
info(blocklist_manager): blocklist snapshot generation 1: 1 of 1 sources loaded, 3096006 bytes
info(blocklist_manager): blocklist snapshot generation 1: 1 of 1 sources loaded, 2523973 bytes
info(nxdns): authority: database
```
@@ -314,8 +323,12 @@ zig-out/bin/nxdns run --data-dir ~/nxdns-tutorial/data --config ~/nxdns-tutorial
```
reconciled '/home/you/nxdns-tutorial/config.zon': sources +0 ~0 -1; group_sources +0 ~0 -1;
info(blocklist_manager): blocklist snapshot generation 1: 0 of 0 sources loaded, 199 bytes
info(blocklist_manager): blocklist snapshot generation 1: 0 of 0 sources loaded, 231 bytes
info(nxdns): authority: file (/home/you/nxdns-tutorial/config.zon)
info(nxdns): nxdns <version> serving on udp [::1]:15353 udp 127.0.0.1:15353 tcp [::1]:15353 tcp 127.0.0.1:15353; 1 upstream(s); blocklist generation 1
info(blocklist_manager): pruned orphaned blocklist file 1.allow
info(blocklist_manager): pruned orphaned blocklist file 1.wild
info(blocklist_manager): pruned orphaned blocklist file 1.list
```
**Read that first line.** The blocklist source is gone. That is not a bug — it is
@@ -324,8 +337,12 @@ source, and in file mode the file is the complete statement of what the
configuration is, so anything the database holds that the file does not name is
removed at every start. The reconcile said so in one line before doing it.
The compiled list is still on disk and the query log is untouched; what changed
is the configuration, and it now matches the file exactly.
The three `pruned` lines are the rest of that removal: with the row gone, the
compiled files it owned belong to nobody, so the sweep that runs at every start
deletes them. The three names are the three bodies one source compiles into —
exact domains, wildcards, and the exceptions an Adblock Plus list can lift. The
query log is untouched; what changed is the configuration, and it now matches
the file exactly.
Neither mode is the "advanced" one. Database mode suits a box someone
administers through the web interface. File mode suits a file kept in git and
@@ -345,7 +362,7 @@ Ctrl-C to stop it.
## What you have now
A resolver that answers real queries, a real blocklist of about 99000 domains
A resolver that answers real queries, a real blocklist of about 98000 domains
attached to the default group, a query log, and a web interface — all inside one
directory you can delete: