Installer reference

install.sh is a single bash script that turns a fresh Ubuntu server into a Kwerft cluster. It also joins servers to a cluster, installs remote clusters in agent mode, rescues a locked firewall and uninstalls. Every stage is idempotent: running the same command again repairs or upgrades the install. For a first install, start with Getting started.

Run it

The script runs as root. kwerft.dev/install.sh forwards to the script in the public ehilzinger/kwerft-install repository on GitHub:

curl -fsSL https://kwerft.dev/install.sh | sudo bash -s -- --domain ops.example.com --email ops@example.com --yes

Or download it first, read it, and run it:

curl -fsSLO https://kwerft.dev/install.sh
sudo bash install.sh --help

The whole script is a set of functions called on its last line, so a download that breaks off halfway never runs half a script.

Pin a version

https://kwerft.dev/install.sh is always the latest stable release (v0.4.0 at the time of writing). Every release, release candidates included, also has its own copy at https://kwerft.dev/<version>/install.sh:

curl -fsSL https://kwerft.dev/v0.4.0/install.sh | sudo bash -s -- --domain ops.example.com --yes

A pinned script installs its own version. The release candidates of v0.5.0, which bring Clusters & nodes, the Hetzner Cloud integration and several of the flags below, are at https://kwerft.dev/v0.5.0-rc.2/install.sh.

--version picks the Kwerft release (console image and Helm chart) independently of the script; it takes 0.4.0 or v0.4.0. The installer checks that the release is published before it changes anything and stops with exit code 50 if it is not. Prefer the script of the release you install, since flags and stages change between releases.

The console image is ghcr.io/ehilzinger/kwerft (amd64 and arm64) and the chart oci://ghcr.io/ehilzinger/charts/kwerft; both are public, so no registry login is needed.

Options

Flags marked v0.5.0 exist from the v0.5.0 release candidates on.

Install

Join an existing cluster

You rarely type these yourself: the console’s join command (Clusters & nodes › a cluster › Nodes › Join a server) and node pools pass them.

Connect a new cluster to a console (agent mode)

v0.5.0. Installs Kwerft without its own console; the cluster is managed from another console. The console’s Adopt a cluster dialog shows the whole command. See Clusters & nodes.

A server keeps the mode it was installed in: --agent on a console server, or a plain install on an agent server, stops with exit code 2.

Development

Maintenance

General

Environment variables

Options with a value can also come from the environment; flags win. This suits cloud-init and CI:

Switches such as --lite, --yes or --agent have no environment variable; pass them as flags.

curl -fsSL https://kwerft.dev/install.sh \
  | sudo KWERFT_DOMAIN=ops.example.com bash -s -- --yes

Config file

--config FILE reads a YAML file for unattended installs. Flags still win over it. Nested keys can be written as a block or inline (dns: { solver: hetzner, tokenFile: /root/dns.token }).

domain: ops.example.com
email: ops@example.com
appsDomain: apps.example.com
dns:
  solver: hetzner
  tokenFile: /root/dns.token
  records: true
hcloud:
  tokenFile: /root/hcloud.token

An apps domain or DNS solver given here replaces what was chosen in Settings; without them, Settings stays as it is. The token files stay where they are; the cluster keeps a copy that the console never shows. See Domains & TLS and Clusters & nodes.

Stages

The installer runs these stages in order and prints one line per stage. Completed stages are recorded in /var/lib/kwerft/stages/.

In join mode the stages are Preflight, System, Firewall, Join cluster and Registry mirror.

--dry-run lists the stages it would run without running any.

The installer’s own log is /var/log/kwerft/install.log. When a stage fails, the installer names it, its exit code and the log, and tells you to re-run the same command.

Re-run to repair or upgrade

Run the same command again at any time:

• Repair: stages that always run converge the server back to the expected state; completed one-time stages are skipped.
• Upgrade: the script of a newer release upgrades Kwerft and the components pinned in it (Cilium, cert-manager, Traefik, monitoring). k3s is installed once and not upgraded by a re-run. Upgrades with automatic rollback are planned before the public beta.
• Settings are kept: without --domain, the console’s hostname stays what Settings chose. --acme-server must be given on every run, or certificates come from Let’s Encrypt production again.
• The firewall stays up: reloading the firewall replaces Kwerft’s base rules in one step and keeps the console’s rules, so there is no moment without a firewall.

The setup token

The handoff stage creates a single-use setup token in /etc/kwerft/setup-token (readable by root only), valid for 24 hours; only its hash goes into the cluster. If the file is lost or the token expired before anyone used it, a re-run creates a new one. Once the owner account exists, a re-run removes any leftover token and prints the console’s address instead.

Exit codes

Exit codes are a stable contract for automation:

Firewall rescue

If a firewall change locked you out of SSH, open the server’s console from Hetzner (VNC for Cloud servers, KVM for dedicated ones) and run:

curl -fsSL https://kwerft.dev/install.sh | sudo bash -s -- --reset-firewall

It removes Kwerft’s nftables table and pauses the console’s firewall rules on this node, so they are not put back. Run your install command again to restore the baseline, and delete /var/lib/kwerft/firewall/paused to let the console manage this node’s firewall again. See Network › Server firewall.

Uninstall

curl -fsSL https://kwerft.dev/install.sh | sudo bash -s -- --uninstall --yes

This removes k3s with every workload on the server and the data on its local volumes, Kwerft’s firewall table, the registry mirror and AppArmor profile, /var/lib/kwerft, /etc/kwerft and Kwerft’s sysctl, module and SSH files. The logs in /var/log/kwerft stay. Without --yes it asks first.

A Hetzner Cloud Firewall or Load Balancer that Kwerft created stays in your Hetzner project. Turn them off in Settings before you uninstall, or delete them in the Hetzner Console afterwards.

There are no backups yet (they are planned before the public beta): uninstalling deletes your apps’ data.

Pinned components

Each release pins the versions it installs at the top of the script. v0.4.0 and v0.5.0-rc.2 pin: