summaryrefslogtreecommitdiff
path: root/_posts/2026-06-07-migration-time-again.md
blob: 3d9362242dd151317ee1565fff53c0bd4bd6e3df (plain)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
---
layout: post
title: Server migration time (again)
lang: en
categories: tech
date: 2026-06-07 13:44 +0200
---

It's a long weekend in Germany (for me) again. Thursday was the "Feast of Corpus
Christi" (sarcastically translated as "Happy Cadaver Day" sometimes), and I took
Friday as a vacation day. Plenty of time for "projects". This is an attempt to
write up a digest of the thoughts I… let's say spit out on
[Mastodon](https://woof.tech/@uvok) over the previous days.

## HomeAssistant

> It actually began a bit earlier. I bought a used car back in April. My old car
> was threatening to get some expensive repairs. And with the gas prices going
> up, I said "fuck it" and took a look the the various portals for used cars. I
> might've panicked/overreacted a bit, and decided for a Seat Mii Electric
> shortly before my Brussels vacation (on which I also wanted to write a short
> blog article as well, some time), as in, getting fear of all the used cars
> being sold very quickly and me being left with nothing.  In retrospect, there
> is indeed a very high demand for used electric cars, so it was probably a good
> decision.  Anyway, I couldn't be happier. I never had *so much fun* driving
> before!
>
> Anyway, this is one of these cars which already have an eSIM built-in. It's
> disabled by default, but if you activate it (via customer support), you can
> use the SEAT Connect app and see various status reports, and get telemetry
> data (like how much you drove, how much energy you used). In a furry EV chat,
> some furries even had their cars integrated into HomeAssistant.

This reminded me… I do have an HomeAssistant, which was still running HA
2024.something. Since it only runs in my own network, I don't see a problem
with that, actually. I never really updated because I keep on reading about
breakages by upgrades. My setup basically is running this in Proxmox:

- 1 VM with Zigbee2MQTT (Z2M)
- 1 VM with HA OS (including the MQTT server)

With so much time on my hands, I decided "just clone the existing VM, import the
new HA OS disk into the new VM, backup and restore, and see if it works. If so,
I'll use it, otherwise I'll revert.". Yeah, only no. That didn't work out that
easily. I ended up with unavailable values in the dashboard of the new
installation.

Since the MQTT server runs inside HA, Z2M will only connect to the broker of the
old installation. So I had to push an item onto the mental stack, "Create an LXC
container running Mosquitto". So I did that.

Only, wait. The default Proxmox LXC Alpine template is stupid. It doesn't have
Dropbear installed. So inserting the SSH key in the container config wizard
*doesn't do anything*. So I looked up Distrobuilder to create an LXC template
which does have Dropbear installed -- and enabled -- by default.

Only, then I had to remember how I set up my internal certificates, since I
wanted to secure the broker with TLS.  Thus, I had to (stack.push) set up
uacme/ualpn first on that container first.  Which put some "mental load" on me
trying to keep in mind the end goal.

Anyway, I got this sorted out in the end, and now I have an up-to-date HA
running, only… uhh… I think my Z2M is still not up to date, because I would need
to setup a newer node version first.

Why did I upgrade HA OS again… oh right, I was poked to integrate my SEAT car
into it. Only, Volkswagen[^1] decided to change/shutdown their API, or
something. This seems to be a long-going battle, which started with VW simply
changing an OAuth endpoint, but in the meantime they probably changed more. At
least I got basic info on my desktop PC using
[CarConnectivity](https://github.com/tillsteinbach/CarConnectivity-connector-seatcupra)

[^1]: A Seat Mii is basically a Volkswagen e-up

At this point, I realized I still have NetBox running, and should probably
document the VMs there as well. *sighs*

## Uptime Kuma

Apropos of Node.
For "monitoring", I use [Uptime Kuma](https://uptime.kuma.pet/) (UK). I haven't
upgraded that in some time as well, and they actually had a major upgrade. After
figuring out which repo I need for getting an appropriate node version[^2]<sup>,
</sup>[^3], I was able to update UK successfully.

[^2]: Apparently, NodeSource is the way to go?
[^3]: Maybe I should just run it in Podman… But eh, Proxmox isn't naturally
      suited for that.

During that, I actually decided to approach another problem I had again and
again, my local network certificates (step-ca) expiring, because I made mistakes
in my automations[^4]. So I created an HTTPS monitor, only to figure out that UK
only notifies of expiries via notifications, but doesn't show that in the
dashboard.  Oh well, I can live with that.

[^4]: Specifically, the certificate always fails to update after every OpenWRT
      upgrade. Either I forgot to create the directories in the image builder,
      or I forgot to set `chmod +x` in some script, or I forgot to include the
      step-ca root into the trusted certificates (which is always ugly, because
      every distribution does it differently, and it depends whether you have
      OpenSSL installed, which brings `update-ca-certificates`. Only, on OWRT, I
      don't want to install that, so apparently you have to append your cert to
      the certificate store/bundle manually.

Another problem I encountered was that my UK runs in a "VM VLAN", and I
configured my router (running OpenWRT) to not allow incoming (to the router)
connections from that VLAN. So UK can't directly query the certificate expiry
that way.  I ended up with a manual/push monitor, where the notifier script runs
on Proxmox itself. Ergh. I find it ugly, but it works.

## Blog migration!

For quite some time now, I wanted all my uvok.de / uvokchee.de services running
on the same server. But… dependencies! Take this blog for example. It's a Jekyll
site. Build via Buildbot. From a git repo hosted with gitolite. And it also
runs Hatsu, for making my blog ActivityPub-connectable. You see where this is
going. This is how it went:

- So, I started with the gitolite repos. Luckily, they provide [helpful
  documentation](https://gitolite.com/gitolite/install.html#moving-servers) for
  exactly that. Nice! That went relatively smoothly.
- Aside: It's really annoying you can't just do a `su - user` anymore to login
  as a different user, if you expect to have systemctl working. No, you have to
  do a `machinectl shell user@ /bin/bash`.
- Then I set up buildbot again. Only I decided I might as well build the blog
  inside Podman.
  * No more worries about system ruby version. No more installing the same gems
    every time. This is gonna go great.
  * "This should be simple." Just write a
    [Containerfile](https://git.uvok.de/blog/tree/_ci/Containerfile) with the
    needed gems installed and run Podman for building the Jekyll site.
  * I actually started with creating the image on my local machine, pushing it
    to the Hetzner server, only to find out it doesn't work. Well, yes, amd64
    binaries don't run on an arm64 machine. *facepalm*
  * Wait, why does Podman emit warnings?
    I was getting messages about Podman not being able to connect to the user
    session, and falling back to cgroupfs.  Do I need `enable-linger` for
    buildbot? Do I need to enable the Podman socket? No, that didn't change
    anything, either.
  * Using `podman --remote` in the build step didn't work at all. I got a
    permission denied.
  * What's a `DockerLatentWorker`, buildbot? Oh, you need buildbot-worker
    installed into the container for that. Not what I want.
  * Oh, I had `PrivateTmp` and `PrivateDevices` specified in the service file.
    Apparently this lead to the permission problems.
  * Oh, and apparently, it's easier (and more secure?) to just use a user unit,
    instead using a system unit with `User=` and `Group=`. So I moved that
    around, and removed all the sandboxing directives from the unit file[^7].
  * And, for completeness: No, I won't blame systemd here, I doubt it would have
    gone better under SysVinit.
- Uh, so… where was I again? Oh yeah, building the blog.
  So, a forced buildbot build (so, manual action) works now. I'll worry about
  the push hooks later. [^5]
- Next up, the webserver.
  I thought, while I'm at it, I might as well get rid of Certbot, and
  let Caddy handle the certificates automatically. While still running Nginx in
  the backend, because apparently, you're more likely to find DokuWiki and
  Nextcloud snippets for Nginx than for Caddy.
  - At this point it occurred to me, "Oh, I still have Authelia set up." I
    ended up migrating that to Caddy completely. Luckily, the Authelia docs have
    snippets for that.
- So, of course, to go live with Caddy, as in, getting the TLS certificates, I
  need to point the domains to the new server. So, a DNS update is in order as
  well.
- I ended up migrating Hatsu[^6] as well, and update it while I'm at it. Hatsu
  is written in Rust.
  * You know what, I'll use Podman for that as well! So I don't need to install
    the rust/cargo toolchain on the system.
  * *types `podman run ... cargo build`*
  * … Hey, why doesn't my VPS react anymore? Shit.
  * *logs into the Hetzner console and reboots/shutdowns the VPS*
  * *types `podman run -m 2g --cpus 1 `*
  * Ah, now it compiles without crashing my server!

[^5]: That was always a pain point in the past. I used the scripts in the
      examples of the buildbot repo, which depend on twisted, so I had to make
      sure the required modules were installed inside whatever happens in
      gitolite's update hook. Very ugly. Maybe I'll look up a simple curl call.

[^6]: The service offering ActivityPub integration.

[^7]: I really wish there was a built-in unit generator in systemd, I always end
      up having to do a web search for a template.

So, after several hours of work, wanting to bang my head against the wall, and
head scratching later: YAY! My blog is now on the other server! And Hatsu works as well!

I still have "get my Funkwhale running again" on my maybe-todo-list. To be
honest, I'm not sure if it's worth the effort. I uploaded some guitar pieces I
played there, but… my heart is not really in it.

I might look into what else I can run in Podman in my homelab, but I am not
convinced running it directly on the Proxmox host is a good idea. Especially
with all the networking stuff. I'm glad I got the bridging with VLAN figured out
in the first place.
At the same time, the machine might be too weak to take another VM on which I
can run it. It's a "just-for-fun" project anyway. It's not like I run anything
mission-critical on that.

Oh, and I still need to migrate the Gemini version of my blog! Not sure if I can
and should do it this weekend as well. Weekend is for relaxation, after all. :)
I ended up philosophizing about how dissatisfying "administration stuff" is for
me, because at the end of the day, you don't *see* the result of your work, as
in, you don't have a physical artifact.