1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
|
---
layout: post
title: Further learnings from running my AS
date: 2024-03-17 14:42 +0100
last_modified_at: 2024-07-08 20:17 +02:00
lang: en
categories: ["tech"]
description: What you can learn when you run your own Autonomous System
---
You can think of this as the second part to
[Running my own AS]({% post_url 2023-08-18-networking-adventure-my-own-ipv6-prefix-and-as %}).
So, it's been about 6 months since I started running my own AS. Recently, I
tried to dive into RPKI/ROA validation. I had quite some head scratching trying
to understand some of this stuff, getting confused over obsolete documentation,
and so on. This post might get updated to reflect my ongoing learning.
I'll probably also throw the bird config into my [Ansible
repo](https://git.uvok.de/ansible/) at a later time (yadda yadda work is
exhausting, I need the weekend to relax, yadda yadda).
I've gotten a second prefix, this time paying for it, at a provider which offers
RPKI for their prefixes. This is not strictly necessary for me, but it… kinda
"feels better", and gives me the chance to experiment with stuff. Or so I
thought, turns out I get a "management GUI" where I can enter the prefix, an AS
number, and that's it, I don't see what's going on in the background. (The
actual signing; I can see the verification results on
[bgp.tools](https://bgp.tools/)). Oh well, dealing with crypto myself would
probably only have lead to more head-scratching.
So, first off: Why do I need ROA/RPKI at all, in the BGP landscape? Basically
to prevent foreign AS announcing prefixes they don't own. This happened in the
past, e.g. for censoring YouTube in authoritarian states. Where does the
information what one can announce come from? This happens via the
Regional Internet Registries (e.g. RIPE). Their databases contain all
the AS and IP prefixes, with the IP prefix entries containing information about
which AS (singular?) is allowed to announce them. RPKI is… uh… the information
which prefixes should origin from which AS, cryptographically signed. Entered by
the prefix owners (I think). I lack the detailed knowledge here. "SecureBGP" is
another way to solve the problem of AS announcing prefixes they don't own, but
that's a different protocol, and I haven't looked into it at all.
So, to use or check this information, you need an additional piece of software,
the validator. There are probably roughly a dozen around, some of them stopped
being maintained. I'm not even gonna try listing them here, the info will be
outdated at some later point anyway. The basic operation seems to be "grab the
RIR database entry dumps, scan them, and provide a way to send requests for
validation", using the "RTR protocol" (RPKI to router protocol).
In my case, bird2 provides an rpki protocol block, and using the function
`roa_check`, the imported routes are checked against the RPKI database. There
are basically three states, VALID, INVALID and UNKNOWN (the latter meaning: no
matching entry, validator is down?). Then you can decide what to do with
"faulty" routes. I went with keeping them at first, tagging them with a
community, sending all these routes to the bgp.tools route collector (for a nice
analysis), and setting up a second table which goes into the kernel. The
"incoming" and the "used" table are connected via the pipe protocol, and this is
where I do the filtering, i.e. reject the invalid routes.
Aside: On can *also* export the RPKI/ROA info in a compatible format to a single
file and use that as input for bird, but I don't know how reloading works in
that case. I think you have to do an explicit `birdc configure`, meaning all the
routes you currently have get flushed. With rpki protocol and the right
configuration (`import table` and `rpki reload`), this happens "automagically"
(?), without losing the current routes.
Some things I stumbled upon when setting stuff up: The software
[StayRTR](https://github.com/bgp/stayrtr) (which has a very cute logo) uses a
pre-build database ("VRR export"?) when started up without any further config,
fetching it from the internet. Startup is nearly instant. When I started setting
up [Fort Validator](https://github.com/NICMx/FORT-validator), I was confused
because "it's not working". Well, I had to tune the default config, so the log
is emitted on info level, and there I could see the software was grabbing all
the RIR database data. Which took some time. After that, it started working as
well.
Also, if you use an internet search machine for this topic, you'll see slides
that StayRTR and rpki-client are used together. This made me a bit mad, because
I didn't see why. I was using StayRTR + bird2 just fine together. Turns out,
StayRTR needs the aforementioned VRR exports. It grabs these from the internet
by default, but you can just as well build this yourself using rpki-client.
Apparently. Haven't tried that.
The "general advice" seems to be to run at least two different RPKI validators,
using different software, and ideally in geographically diverse locations. I
opted to ignore the geographical advice, this is a hobby, I don't wanna rent VPS
all over the world.
Anyway, that's it for this time.
Futher info:
- [Philip Smith's Internet Development Site / bgp4all](https://bgp4all.com/pfs/hints/rpki)
- [DN42 wiki](https://dn42.eu/howto/Bird2#route-origin-authorization), scroll
down for RPKI example configuration.
- [APNIC: Deploying RPKI at Comcast](https://blog.apnic.net/2022/04/08/deploying-rpki-at-comcast/)
**Update 2024-04-29**
- I can actually "safely" discard everything with an AS path length >15.
I took a look at the routes, and there are (only/mostly?) routes with
*lots of* path prepending in that size range. Maybe I could go
*even lower*, but I got tired of making the birdc filter
(`show route where bgp_path !~ [...]`) longer and longer, and then scrolling
through the route output.
- Another "oops!". I thought giving my hosting provider a higher BGP local preference
was a good idea, as all the other transit providers are connected via GRE tunnel.
Only I wondered why *on my other VPS* suddenly a huge amount of routes preferred
to go via IBGP. I facepalmed when I realized what I did. (BGP local pref attribute
is sent via IBGP sessions).
**Update 2024-06-26**
- Ooops! Again. I should not accept *any* default route if I want to do filtering.
Assuming my upstream/transit *doesn't* do proper RPKI/bogon filtering.
Instead, I need to add a default blackhole.
(The filtered routes will not be in my routing table, and if I don't want to route
to them, I need to prevent the kernel from using the default upstream route).
**Update 2024-07-08**
- Another oops! I just realized I also have to export *my prefix* as unreachable
into the kernel! Otherwise, due to the way I configured the routing in Linux
(`ip rule add ... from <myprefix> table xxx`), when a package arrives from
`eth0` which addresses a host that is not present in my AS, the default route
will be taken. And the gateway will throw the packet back at me. And I will
throw it back an the gateway. And so on.
|
