--- layout: post title: Further learnings from running my AS date: 2024-03-17 14:42 +0100 last_modified_at: 2024-07-08 20:17 +02:00 lang: en categories: ["tech"] description: What you can learn when you run your own Autonomous System --- You can think of this as the second part to [Running my own AS]({% post_url 2023-08-18-networking-adventure-my-own-ipv6-prefix-and-as %}). So, it's been about 6 months since I started running my own AS. Recently, I tried to dive into RPKI/ROA validation. I had quite some head scratching trying to understand some of this stuff, getting confused over obsolete documentation, and so on. This post might get updated to reflect my ongoing learning. I'll probably also throw the bird config into my [Ansible repo](https://git.uvok.de/ansible/) at a later time (yadda yadda work is exhausting, I need the weekend to relax, yadda yadda). I've gotten a second prefix, this time paying for it, at a provider which offers RPKI for their prefixes. This is not strictly necessary for me, but it… kinda "feels better", and gives me the chance to experiment with stuff. Or so I thought, turns out I get a "management GUI" where I can enter the prefix, an AS number, and that's it, I don't see what's going on in the background. (The actual signing; I can see the verification results on [bgp.tools](https://bgp.tools/)). Oh well, dealing with crypto myself would probably only have lead to more head-scratching. So, first off: Why do I need ROA/RPKI at all, in the BGP landscape? Basically to prevent foreign AS announcing prefixes they don't own. This happened in the past, e.g. for censoring YouTube in authoritarian states. Where does the information what one can announce come from? This happens via the Regional Internet Registries (e.g. RIPE). Their databases contain all the AS and IP prefixes, with the IP prefix entries containing information about which AS (singular?) is allowed to announce them. RPKI is… uh… the information which prefixes should origin from which AS, cryptographically signed. Entered by the prefix owners (I think). I lack the detailed knowledge here. "SecureBGP" is another way to solve the problem of AS announcing prefixes they don't own, but that's a different protocol, and I haven't looked into it at all. So, to use or check this information, you need an additional piece of software, the validator. There are probably roughly a dozen around, some of them stopped being maintained. I'm not even gonna try listing them here, the info will be outdated at some later point anyway. The basic operation seems to be "grab the RIR database entry dumps, scan them, and provide a way to send requests for validation", using the "RTR protocol" (RPKI to router protocol). In my case, bird2 provides an rpki protocol block, and using the function `roa_check`, the imported routes are checked against the RPKI database. There are basically three states, VALID, INVALID and UNKNOWN (the latter meaning: no matching entry, validator is down?). Then you can decide what to do with "faulty" routes. I went with keeping them at first, tagging them with a community, sending all these routes to the bgp.tools route collector (for a nice analysis), and setting up a second table which goes into the kernel. The "incoming" and the "used" table are connected via the pipe protocol, and this is where I do the filtering, i.e. reject the invalid routes. Aside: On can *also* export the RPKI/ROA info in a compatible format to a single file and use that as input for bird, but I don't know how reloading works in that case. I think you have to do an explicit `birdc configure`, meaning all the routes you currently have get flushed. With rpki protocol and the right configuration (`import table` and `rpki reload`), this happens "automagically" (?), without losing the current routes. Some things I stumbled upon when setting stuff up: The software [StayRTR](https://github.com/bgp/stayrtr) (which has a very cute logo) uses a pre-build database ("VRR export"?) when started up without any further config, fetching it from the internet. Startup is nearly instant. When I started setting up [Fort Validator](https://github.com/NICMx/FORT-validator), I was confused because "it's not working". Well, I had to tune the default config, so the log is emitted on info level, and there I could see the software was grabbing all the RIR database data. Which took some time. After that, it started working as well. Also, if you use an internet search machine for this topic, you'll see slides that StayRTR and rpki-client are used together. This made me a bit mad, because I didn't see why. I was using StayRTR + bird2 just fine together. Turns out, StayRTR needs the aforementioned VRR exports. It grabs these from the internet by default, but you can just as well build this yourself using rpki-client. Apparently. Haven't tried that. The "general advice" seems to be to run at least two different RPKI validators, using different software, and ideally in geographically diverse locations. I opted to ignore the geographical advice, this is a hobby, I don't wanna rent VPS all over the world. Anyway, that's it for this time. Futher info: - [Philip Smith's Internet Development Site / bgp4all](https://bgp4all.com/pfs/hints/rpki) - [DN42 wiki](https://dn42.eu/howto/Bird2#route-origin-authorization), scroll down for RPKI example configuration. - [APNIC: Deploying RPKI at Comcast](https://blog.apnic.net/2022/04/08/deploying-rpki-at-comcast/) **Update 2024-04-29** - I can actually "safely" discard everything with an AS path length >15. I took a look at the routes, and there are (only/mostly?) routes with *lots of* path prepending in that size range. Maybe I could go *even lower*, but I got tired of making the birdc filter (`show route where bgp_path !~ [...]`) longer and longer, and then scrolling through the route output. - Another "oops!". I thought giving my hosting provider a higher BGP local preference was a good idea, as all the other transit providers are connected via GRE tunnel. Only I wondered why *on my other VPS* suddenly a huge amount of routes preferred to go via IBGP. I facepalmed when I realized what I did. (BGP local pref attribute is sent via IBGP sessions). **Update 2024-06-26** - Ooops! Again. I should not accept *any* default route if I want to do filtering. Assuming my upstream/transit *doesn't* do proper RPKI/bogon filtering. Instead, I need to add a default blackhole. (The filtered routes will not be in my routing table, and if I don't want to route to them, I need to prevent the kernel from using the default upstream route). **Update 2024-07-08** - Another oops! I just realized I also have to export *my prefix* as unreachable into the kernel! Otherwise, due to the way I configured the routing in Linux (`ip rule add ... from table xxx`), when a package arrives from `eth0` which addresses a host that is not present in my AS, the default route will be taken. And the gateway will throw the packet back at me. And I will throw it back an the gateway. And so on.