Polish and Publish post: DNS

1 files changed, 266 insertions, 0 deletions

diff --git a/_posts/2025-01-12-2025-01-12-migrating-dns-servers.md b/_posts/2025-01-12-2025-01-12-migrating-dns-servers.md
new file mode 100644
index 0000000..e64e2bc
--- /dev/null
+++ b/_posts/2025-01-12-2025-01-12-migrating-dns-servers.md
@@ -0,0 +1,266 @@
+---
+layout: post
+title: Migrating DNS servers
+lang: en
+categories: tech
+date: 2025-01-12 16:49 +0100
+---
+
+## Preface
+
+[As I posted on Mastodon](https://furry.engineer/@uvok/113780013806190576), 
+
+* the DNS registrar (where you register the domain at),
+* the DNS hoster / provider (which is responsible for answering DNS requests for
+  your domain) and
+* the (web) hosting service
+
+can be three different entities, even though for most beginner projects these
+are the same.
+
+I either bought or migrated my domain over to [INWX](https://www.inwx.de/en)
+quite some while ago. However, their web interface for editing records is a bit
+clunky. And while they do provide an API, it's a bit awkward (XML based, and
+yes, there are ready-made packages for various programming languages). [^1]
+
+With [DN42](https://blog.uvokchee.de/dn42.html), I already used
+[PowerDNS](https://doc.powerdns.com/authoritative/) a lot. I actually just use
+edit-zone and edit the zone[^2] in an editor, that's my preferred way. Definitely
+preferred to some web interface. It would be nice to do this to my main domains
+as well. Also, this is another nice "have a technical task to procrastinate
+other stuff" thing ;3. Well, I like playing with tech. This is by no means
+essential.
+
+## Glossary
+
+Quick and dirty glossary, for the purpose of reading this article. It's not
+100% accurate and complete, but should be enough to understand the article.
+
+* **Domain Name**: Basically "the name of a server" (what you see in the address
+  bar of the browser, e.g. "blog.uvokchee.de").
+* **DNS**: The "Domain Name System". Very simplified, translates domain names
+  to IP addresses (which computers use to connect to each other).
+  To be more precise, it's like a lookup in a database (-> DNS record).
+* **TLD**: Top-Level Domain. It's "the last part of the domain name" (e.g.
+  ".de").
+* **Registry**: These run the TLDs.
+* **Registrar**: These sell you domain names from the registries. As an "end
+  user", you can't talk to a registry yourself.
+* **DNSSEC**: DNS is a plain-text, insecure protocol, from way back when the
+  internet consisted of a handful of trustworthy people. DNSSEC tries to prevent
+  malicious parties from sending wrong information in response to DNS requests.
+  It does so by signing the responses with a cryptographic key, of which the
+  public part is stored in the registry.
+* **NSEC, NSEC3**: Basically these are "negative replies", i.e. "this domain
+  does not exist". These are signed as well. If not present, malicious parties
+  could send back replies to sites they don't want to be reachable as "doesn't
+  exist" all the time.
+* **Zone**: A zone file contains all the -> DNS records for a specific domain.
+* **DNS record**: Like "a row in the database" which belongs to your domain. Can
+  be an address, but also things like text, PGP keys, pointer to other domain
+  names, etc..
+* **TTL**: Time To Live. The time (in seconds) a reply to a DNS request may be
+  cached by a server. For example, a TTL of 3600 seconds means that a (caching)
+  DNS server can cache a record for 1 hour before checking with the
+  -> authoritative server again.
+* **Authoritative Server:** The DNS server that holds the official and most
+  up-to-date records for a specific domain.
+    * **Primary**: The primary source of truth.
+    * **Secondary**: Mirrors the information from the primary. There can be more
+      than one.
+* **Recursion**: DNS is hierarchical. There are fixed "root servers" which serve
+  the nameservers for all the TLDs. The TLDs provide the nameservers for the
+  domains "below them". A recursive server "walks along this path" to answer
+  requests. At some point, it arrives at the authoritative server.
+* **Bogus reply**: A reply with invalid signature, either because it has been
+  tampered with, or because of wrong configuration.
+* **AXFR**: A request type for a -> Domain zone transfer.
+* **Domain Zone Transfer**: The process of copying the contents of a zone from
+  one DNS server to another.
+
+## Migration and DNSSEC
+
+So, why not migrate my domain over to my PowerDNS setup? Well, I want some
+availability, and I don't trust myself enough not to fuck things up. When my
+server is shut down, DNS requests should still be answered. But hey, that's
+where secondaries come in! So, I manage the zone on my server, as primary, and
+let another provider mirror its contents. These will actually "serve the zone".
+
+Now, only… there's the "problem" of DNSSEC, in conjunction with the DNS
+propagation time and TTLs. If I simply set the nameservers (in the de zone) to
+a newly created zone by PowerDNS, the DNSSEC keys will be either not present,
+or different. I can't set *additional* keys at INWX easily, I can only switch
+to manual keys (by switching off DNSSEC first). Which would lead to a downtime
+of up to one day, because people requesting records from my domain (such as
+myself) would receive bogus replies in that time. I want to avoid that.
+
+## Steps for migration
+
+But, I think I found a solution for that. This are the steps which I had to take
+for the registrar INWX and my uvokchee.de domain. The process might be different
+for other TLDs and other registrars.
+
+1. Download the zone data from the INWX web interface
+1. Create the zone in PowerDNS and set various settings
+
+       #!/bin/bash
+       zone="example.com"
+       pdnsutil create-zone $zone
+       pdnsutil set-kind $zone primary
+       pdnsutil secure-zone $zone
+       pdnsutil set-presigned $zone
+       pdnsutil set-meta $zone ALLOW-AXFR-FROM <ip of secondaries>
+       # optional? see below for a discussion
+       pdnsutil set-nsec3 $zone $parameters 
+1. Filter and import the zone. PowerDNS will actually complain about the NSEC3
+   records otherwise once you open the zone in your editor.
+
+       # any other regex that filters only NSEC3 records, but not RRSIG NSEC3,
+       # work as well
+       grep -vw "IN NSEC3" ./dl-zone.txt > imp-zone.txt
+       pdnsutil load-zone $zone ./imp-zone.txt
+
+   This should keep the RRSIGs for the NSEC3 records. This doesn't help with
+   the errors I receive later, though…
+
+1. Set up the secondaries. I went with [Hurricane
+   Electric](https://dns.he.net/), but I had to write them an e-mail. I couldn't
+   add the zone as secondary myself, because the web interface requires the
+   nameservers in the parent zone (?) already to be set to HE, but I wanted a
+   zero-downtime migration.
+1. Wait for the DNS provider to AXFR.
+1. (optional) check with [dnsviz](https://dnsviz.net/) whether you get any
+   errors. You should set the nameserver on the "Analyze" tab, and then set the
+   "Additional trusted keys:" to what `dig example.com DNSKEY` tells you. At
+   this point, I got warnings/errors regarding the "Denial of existence" /
+   NSEC3 records, and some errors because the RRSIGs can't be checked. (Once
+   you set a manual nameserver, the tool doesn't seem to check for the parent
+   zone anymore? I'm unclear on that).
+1. Try to set the new nameservers (nsx.he.net) in the registry (e.g. DENIC) via
+   the INWX web interface ("external nameservers").
+1. Receive an "UPDATE FAILED". Read the error message.
+
+       ERROR: 53300102912 Nameserver error [ERROR: 118 Inconsistent set of NS
+       RRs (NS, IP, NS host names) (ns5.he.net, 2001:470:500::2, ['ns.inwx.de',
+       'ns2.inwx.de', 'ns3.inwx.eu'])]
+   
+   Realize you fucked up.
+   You need to set the new nameservers within your own zone first. This seems
+   to be a requirement for DENIC at any case.
+1. Add the NS entries to the HE nameservers within your zone in the INWX web
+   interface. [^4]
+1. Download and import the zone file again. Wait for the AXFR.
+1. Retry sending the nameservers to the registry. Wait until success.
+1. At this point, I checked again with dnsviz. With the default options,
+   everything was fine. However, when I enabled the "Denial of existence:"
+   option, I received some *extremely scary looking error messages*. I can only
+   guess this is because PowerDNS can't generate these records? I *have no
+   clue*. Any useful information will be gladly accepted. I choose to ignore it.
+
+        NSEC3 proving non-existence of l2v1y.7t5is.uvokchee.de/A: No RRSIG covering the RRset was returned in the response. See RFC 4035, Sec. 3.1.1. (216.218.130.2, 216.218.131.2, 216.218.132.2, 2001:470:100::2, 2001:470:200::2, 2001:470:300::2, UDP_-_EDNS0_4096_D_KN)
+        NSEC3 proving non-existence of uvokchee.de/CNAME: No RRSIG covering the RRset was returned in the response. See RFC 4035, Sec. 3.1.1. (216.218.130.2, 216.218.131.2, 216.218.132.2, 2001:470:100::2, 2001:470:200::2, 2001:470:300::2, UDP_-_EDNS0_4096_D_KN)
+
+    As of 2015-01-11, I read the [PowerDNS
+    docs](https://doc.powerdns.com/authoritative/dnssec/migration.html#from-existing-dnssec-non-powerdns-setups-pre-signed)
+    again. Specifically, I need to set `pdnsutil set-nsec3 $zone <whatever
+    NSEC3PARAM says>`. Then, PowerDNS gives back SERVFAILs, because, of course,
+    PowerDNS can't sign the reply, because it doesn't have the keys to sign the
+    NSEC3 replies (??? I guess ???). So now, instead of sending back an unsigned
+    reply, it sends back an error.
+
+    I imported the RRSIGs for these NSEC3 records, though, at least in a second
+    attempt. I tried all combinations of `(set-nsec3|unset-nsec3)` and `(import
+    NSEC3-RRSIG|don't import NSEC3-RRSIG)`. All variants failed. I have no clue
+    how this is supposed to work cleanly. [^7]
+
+1. Wait at least 24 hours (TTLs, DNS propagation time).
+1. Let PowerDNS output its own keys it generated for the zone. Unfortunately,
+   `pdnsutil export-zone-dnskey $zone $keynr` *does not output a completely
+   valid record*, nor does `pdnsutil export-zone-ds $zone`. These outputs are
+   missing the TTLs in the second column, at least with PowerDNS 4.7.3 in the
+   Debian stable repos. You have to add those yourself. I saw some tools like
+   dnsviz break when you enter the records as-is.
+
+   **Important**, I had to temporarily run `unset-presigned $zone` (see
+   below) before running `pdnsutil export-zone-ds`, so PowerDNS actually outputs
+   the hash of the new key(s)! Otherwise it will only show the current keys
+   signature. Otherwise, you may also use online tools that convert the DNSKEY
+   to a DS record.
+1. Have a copy of the *current* DS / DNSKEY records as well (`dig` is your
+   friend).
+1. Set the DNSSEC from "auto" to "manual" in the INWX web interface.
+     * Delete all keys from the domain.
+     * Add DNSSEC again for the domain, in manual mode.
+     * Add DNSKEY and DS records of the old keys.
+     * Add key, insert DNSKEY and DS records emitted by pdnsutil for the new
+       key.
+
+   As far as I know, this will be sent to the parent zone. [^8]
+1. After I got an email from INWX confirming the new DNSSEC entries, I ran
+   dnsviz again and nearly got an heart attack, because there were a lot of
+   errors and red exclamation marks. Actually, the DNSSEC was still valid (i.e.
+   not returning bogus records). The tool only complained about
+
+       The DS RRset for the zone included algorithm 13 (ECDSAP256SHA256), but no
+       RRSIG with algorithm 13 covering the RRset was returned in the response.
+
+   I *think* dnsviz expects both `RRSIG`s, i.e. both keys must sign the replies?
+   That shouldn't have an effect on the reachability, otherwise the tool would
+   report BOGUS records?
+1. Wait at least 24 hours (TTLs, DNS propagation time). - Depending on the
+   previous DS records TTL.
+1. Switch to automatic signing by PowerDNS. I guess at this point you must or should
+      * stop PowerDNS, [^5]
+      * `pdnsutil unset-presigned $zone`,
+      * `pdnsutil unset-nsec3 $zone`,
+      * edit the zone (clearing out the RRSIGs you imported. If I understood
+        the PowerDNS docs correctly, there will be trouble [^6] if you don't. Oh,
+        don't forget the NSEC3PARAM record, any leftover (C)DNSKEYS and (C)DS,
+        while also increasing the serial (otherwise the AXFR might not take
+        place), [^9]
+      * `pdnsutil rectify-zone $zone`
+      * start PowerDNS,
+      * (optional) notify the secondaries
+1. At this point, you should be done. And I *guess* at this point you could
+   already delete the other DS/DNSKEY via the registrar?
+
+By the way, I'll choose to use NSEC instead of NSEC3. NSEC3 seems like too much
+of a headache for me, also, domain names are not secrets.
+
+## Further references
+
+* [PowerDNS docs](https://doc.powerdns.com/authoritative/)
+* IETF Draft: [Changing DNS Operators for DNSSEC signed
+  Zones](https://datatracker.ietf.org/doc/html/draft-koch-dnsop-dnssec-operator-change-06)
+
+## Footnotes
+
+[^1]: Okay, it's not like I edit my records that often.
+
+[^2]:
+    For the purposes of this blog article, I think you can freely substitute
+    zone with domain in your head, I probably won't use the words correctly 100% of
+    the time either.
+
+[^4]:
+    I think that shouldn't to any damage. DNS recursion should always ask the parent
+    zone?
+
+[^5]:
+    Avoiding a spurious notify or AXFR directly after editing? No idea if needed.
+
+[^6]: such as duplicate RRSIG replies.
+
+[^7]:
+    There are two GitHub issues similar to this:
+    [#9263](https://github.com/PowerDNS/pdns/issues/9263) and
+    [#8892](https://github.com/PowerDNS/pdns/issues/8892).
+     
+[^8]:
+    Actually, I have no idea why INWX wants me to enter both the DS and the
+    DNSKEY. Only the DS gets entered in the DENIC servers.
+
+[^9]:
+    e.g. `:g/\<\(RRSIG\|NSEC3PARAM\|DS\|CDS\|DNSKEY\|CDNSKEY\)\>/d` in vim.
+<!-- vim: set ft=markdown tw=80 ai tabstop=4 shiftwidth=4 expandtab: -->
+